ArvindSridhar/DistributedDeepLearning 2
Exploring strategies to intelligently and efficiently train deep learning models on distributed datasets in parallel, merge/ensemble the results, and perform prediction on a test set. Inspired by my 2018 summer internship at Pivotal Software's Greenplum Advanced Analytics team.
ArvindSridhar/AbstractiveTextSummarization 1
Abstractive text summarization using pointer-generated networks and coverage along with a baseline bi-directional RNN encoder/decoder model
Text Summarization Project
Summer Internship 2018: Created ML/deep learning algorithms for large distributed datasets, along with associated utility functions, to enable parallel model training atop Pivotal's Greenplum database
ArvindSridhar/AutonomousDriving-DataCollection 0
Full stack web application to conduct online experiments detecting danger scenarios while driving. Built with Node.js, Angular, Python, MongoDB, HTML/CSS/JS
Full bash_it theme repo for iTerm2, includes themes such as bobby
Frontend prototype of app enabling students to order food from Bellarmine's cafeteria. Developed during high school.
CalHacks 2017: App that enables you to network with others at events using selfies, making it more memorable. React Native, Expo.io, Python (Flask)
pull request commentgreenplum-db/gpdb
Use 'ss' command in absence of 'netstat' to find active tcp port
Why do we shell out when we can get the same information with import socket?
comment created time in 14 hours
pull request commentgreenplum-db/gpdb
@ashwinstar is there any news on this PR? Is general design OK or should we change something? WAL-G part is nearing completion, btw. Currently, we support only 7+, but 6 would be awesome.
comment created time in 16 hours
PR opened greenplum-db/gpdb
PR #10003 added an extra ResCleanUpLock() call into ResLockRelease(). However ResCleanUpLock() requires a HeldExclusive ResQueueLock, an assertion failure would occur if this one ResCleanUpLock() called.
#0 0x00007f0ed3e3f337 in raise () from /lib64/libc.so.6
#1 0x00007f0ed3e40a28 in abort () from /lib64/libc.so.6
#2 0x00000000006c8aa9 in ExceptionalCondition (conditionName=0x13a6080 "!(LWLockHeldExclusiveByMe((&MainLWLockArray[38 + 4].lock)))", errorType=<optimized out>, fileName=<optimized out>, lineNumber=<optimized out>) at assert.c:66
#3 0x0000000000c7fb74 in ResCleanUpLock (lock=lock@entry=0x7f0eb93a3110, proclock=proclock@entry=0x7f0eb9aee010, hashcode=hashcode@entry=699557762, wakeupNeeded=wakeupNeeded@entry=0 '\000') at resqueue.c:978
#4 0x0000000000c7ff8b in ResLockRelease (locktag=locktag@entry=0x7fff8a5aa880, resPortalId=0) at resqueue.c:572
#5 0x0000000000c82fd2 in ResLockPortal (portal=portal@entry=0x7f2cd88, qDesc=qDesc@entry=0x803dd98) at resscheduler.c:701
#6 0x0000000000aec117 in PortalStart (portal=portal@entry=0x7f2cd88, params=params@entry=0x0, eflags=eflags@entry=0, snapshot=snapshot@entry=0x0, ddesc=ddesc@entry=0x0) at pquery.c:713
#7 0x0000000000ae7411 in exec_simple_query (query_string=0x7dff158 "SELECT 1") at postgres.c:1781
#8 0x0000000000ae9a38 in PostgresMain (argc=<optimized out>, argv=argv@entry=0x7e1e810, dbname=<optimized out>, username=<optimized out>) at postgres.c:5242
#9 0x0000000000a66f93 in BackendRun (port=0x7e142d0) at postmaster.c:4811
#10 BackendStartup (port=0x7e142d0) at postmaster.c:4468
#11 ServerLoop () at postmaster.c:1948
#12 0x0000000000a67f6d in PostmasterMain (argc=argc@entry=6, argv=argv@entry=0x7ddca20) at postmaster.c:1518
#13 0x00000000006f859a in main (argc=6, argv=0x7ddca20) at main.c:245
Fix this by adusting ResQueueLock acquire position.
Here are some reminders before you submit the pull request
- [ ] Add tests for the change
- [ ] Document changes
- [ ] Communicate in the mailing list if needed
- [ ] Pass
make installcheck - [ ] Review a PR in return to support the community
pr created time in a day
PR opened greenplum-db/gpdb
Inside targetid_get_partition(), the parentInfo->ri_partition_hash named "Partition Result Relation Hash" was created under the TopMemoryContext; CloseResultRelInfo() only freezes it without calling hash_destroy().
Long running backend processes (e.g. pgpool connections) might leak a large amount of memory into the partition result hash context, blowing up vmem memory usage.
Fix this by creating the context under the estate->es_query_cxt.
Here are some reminders before you submit the pull request
- [ ] Add tests for the change
- [ ] Document changes
- [ ] Communicate in the mailing list if needed
- [ ] Pass
make installcheck - [ ] Review a PR in return to support the community
pr created time in a day
PR opened greenplum-db/gpdb
This commit ensures that check_node_deprecated_tables_walker recurses into
all possible expressions that can contain range tables. Earlier, we only
dealt with CTEs, but not sublinks for instance. So the following view
would not have been detected:
CREATE VIEW dep_rel_sublink AS SELECT dbid FROM gp_segment_configuration
WHERE 0 < ALL (SELECT fsedbid FROM pg_filespace_entry);
This commit also makes the walker implementation more canonical and
removes the pre-existing recursion logic for CTEs(which we now get for
free with expression_tree_walker). Some additional minor improvements
along the way.
Please refer to #12168, which introduces a similar walker to check for deprecated columns.
pr created time in a day
PR opened greenplum-db/gpdb
This is a back patch of PR #12153 from 7. Cherry-pick was clean, this PR is only for visibility.
The before_read_command fault location is executed by not only regular backends but also special backends such as FTS probe backends, fault injection backends, walsender, etc. Several tests made use of this fault to cause PANIC in QD, after executing a command. This strategy caused intermittent failures in CI because the fault would trigger unexpectedly from other backends listed above. This patch defines a new fault in exec_simple_query() that is guaranteed to be triggered only by query executing backends. This should resolve some of the intermittent failures in CI.
pr created time in 2 days
PR opened greenplum-db/gpdb
PgPortIsActive would rely on netstat to find active tcp port. In many
modern Centos systems it has been replaced by ss. So now in absence of
netstat we use ss.
Here are some reminders before you submit the pull request
- [ ] Add tests for the change
- [ ] Document changes
- [ ] Communicate in the mailing list if needed
- [ ] Pass
make installcheck - [ ] Review a PR in return to support the community
pr created time in 2 days
pull request commentgreenplum-db/gpdb
[6X FIX]Fix a bug of concurrently executing UPDATE/DELETE with VACUUM on a partition ao table
I will get to reviewing this PR tomorrow. Meanwhile, lets add README file to document current partition locking semantics based on knowledge we have gained as part of this PR.
HI, Ashwin, can you tell clearly where we should add README file to?
comment created time in 2 days
Pull request review commentgreenplum-db/gpdb
Fix an assert failure caused by unwanted motion node
adjust_modifytable_flow(PlannerInfo *root, ModifyTable *node, List *is_split_upd all_subplans_replicated = false; /* Master-only table */- if (subplan->flow->flotype == FLOW_PARTITIONED ||- subplan->flow->flotype == FLOW_REPLICATED ||- (subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1))+ if (subplan->flow->locustype != CdbLocusType_General &&+ (subplan->flow->flotype == FLOW_PARTITIONED ||+ subplan->flow->flotype == FLOW_REPLICATED ||+ (subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1)))
Had a short discussion with @gfphoenix78 . Maybe we avoid the gather motion using the logic of this line
subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1
Also master does not have the issue so maybe we can refer the master code logic some although there seem to be big code refactor of the related code. Also do we need to add the test to the master branch to avoid regression in the future?
comment created time in 2 days
startedrguo12/awesome-causality-algorithms
started time in 2 days
issue commentgreenplum-db/gpdb
Failed to complete obtain psql count Master gp_segment_configuration Script Exiting!
Can you tell us how to reproduce this error?
comment created time in 2 days
PR opened greenplum-db/gpdb
Here are some reminders before you submit the pull request
- [ ] Add tests for the change
- [ ] Document changes
- [ ] Communicate in the mailing list if needed
- [ ] Pass
make installcheck - [ ] Review a PR in return to support the community
pr created time in 2 days
Pull request review commentgreenplum-db/gpdb
Improve performance of DELETEs on randomly distributed tables in ORCA.
CDistributionSpecRandom::CDistributionSpecRandom() BOOL CDistributionSpecRandom::Matches(const CDistributionSpec *pds) const {- if (Edt() != pds->Edt())+ if (pds->Edt() == CDistributionSpec::EdtRouted) {- return false;+ // This follows a 2x2 decision matrix:+ // If both gp_segment_ids are null, then it does not match+ // If one gp_segment_id is null and the other is not, it doesn't match+ // If both gp_segment_ids are not null, they match iff they are equal+ const CDistributionSpecRouted *pdsRouted =+ dynamic_cast<const CDistributionSpecRouted *>(pds);+ GPOS_ASSERT(pdsRouted != nullptr);++ const BOOL localNull = m_gp_segment_id == nullptr;+ const BOOL pdsNull =+ pdsRouted == nullptr || pdsRouted->Pcr() == nullptr;++ if (localNull || pdsNull)+ {+ return false;+ }+ else+ {+ GPOS_ASSERT(!localNull);+ GPOS_ASSERT(!pdsNull);+ return m_gp_segment_id->Id() == pdsRouted->Pcr()->Id();+ } }+ else if (pds->Edt() == CDistributionSpec::EdtRandom)+ {+ const CDistributionSpecRandom *pdsRandom =+ dynamic_cast<const CDistributionSpecRandom *>(pds);+ GPOS_ASSERT(pdsRandom != nullptr); - const CDistributionSpecRandom *pdsRandom =- dynamic_cast<const CDistributionSpecRandom *>(pds);-- return pdsRandom->IsDuplicateSensitive() == m_is_duplicate_sensitive;+ return pdsRandom->IsDuplicateSensitive() == m_is_duplicate_sensitive;
In case of DML queries, the motion may be duplicate sensitive when you have const table get. If a motion is duplicate sensitive, it is converted into result node with hash filter in dxl to planned stmt. Hash filter allows only those rows to pass which match the hash else filter them out.
Example below: Select 1 can be run anywhere, and while inserting the hash filter allows the rows which match the gp_segment_id based on the hash to be inserted into that segment, all other rows are discarded on that segment
create table tool (a int, b int);
bchaudhary=# explain insert into tool select 1, 1;
LOG: statement: explain insert into tool select 1, 1;
LOG: 2021-06-17 13:56:06:321336 PDT,THD000,TRACE,"
Physical plan:
+--CPhysicalDML (Insert, "tool"), Source Columns: ["?column?" (1), "?column?" (2)], Action: ("ColRef_0003" (3)) rows:1 width:1 rebinds:1 cost:0.015659 origin: [Grp:6, GrpExpr:2]
+--CPhysicalComputeScalar rows:1 width:1 rebinds:1 cost:0.000034 origin: [Grp:9, GrpExpr:3]
|--CPhysicalMotionHashDistribute HASHED: [ CScalarIdent "?column?" (1), nulls colocated, duplicate sensitive ], opfamilies: (1977,1.0), rows:1 width:1 rebinds:1 cost:0.000030 origin: [Grp:5, GrpExpr:2]
| +--CPhysicalComputeScalar rows:1 width:1 rebinds:1 cost:0.000009 origin: [Grp:5, GrpExpr:1]
| |--CPhysicalConstTableGet Columns: ["" (0)] Values: [(1)] rows:1 width:1 rebinds:1 cost:0.000001 origin: [Grp:0, GrpExpr:1]
| +--CScalarProjectList origin: [Grp:4, GrpExpr:0]
| |--CScalarProjectElement "?column?" (1) origin: [Grp:2, GrpExpr:0]
| | +--CScalarConst (1) origin: [Grp:1, GrpExpr:0]
| +--CScalarProjectElement "?column?" (2) origin: [Grp:3, GrpExpr:0]
| +--CScalarConst (1) origin: [Grp:1, GrpExpr:0]
+--CScalarProjectList origin: [Grp:8, GrpExpr:0]
+--CScalarProjectElement "ColRef_0003" (3) origin: [Grp:7, GrpExpr:0]
+--CScalarConst (1) origin: [Grp:1, GrpExpr:0]
",
QUERY PLAN
------------------------------------------------------------------
Insert (cost=0.00..0.02 rows=1 width=8)
-> Result (cost=0.00..0.00 rows=1 width=12)
-> Result (cost=0.00..0.00 rows=1 width=8)
-> Result (cost=0.00..0.00 rows=1 width=8)
-> Result (cost=0.00..0.00 rows=1 width=1)
comment created time in 2 days
issue closedgreenplum-db/gpdb
Greenplum Version
postgres --gp-version
postgres (Greenplum Database) 6.16.2 build commit:950f103fe23180ed0148552ef08cd62cbbf5a681
OS Version
CentOS Linux release 7.9.2009 (Core)
Linux mdw 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Problem description
From the Greenplum 6.16 documentation, using the CREATE FUNCTION Use polymorphic types to return an ENUM array example generates a server panic when the function is used in a SELECT statement. The problem is reliably reproducible.
Please note: This behavior is NOT exhibited with the planner.
Reproduction steps
CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');
CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
RETURNS TABLE (ae anyenum, aa anyarray) AS $$
SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;
SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
Here is the master server stack
2021-06-16 02:15:56.065475 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd3,seg-1,,,,sx1,"LOG","00000","statement: CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');",,,,,,"CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');",0,,"postgres.c",1639,
2021-06-16 02:15:56.084572 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd4,seg-1,,,,sx1,"LOG","00000","statement: CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
RETURNS TABLE (ae anyenum, aa anyarray) AS $$
SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;",,,,,,"CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
RETURNS TABLE (ae anyenum, aa anyarray) AS $$
SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;",0,,"postgres.c",1639,
2021-06-16 02:15:56.094923 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd5,seg-1,,,,sx1,"LOG","00000","statement: SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",,,,,,"SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",0,,"postgres.c",1639,
2021-06-16 02:15:56.298684 EDT,,,p16957,th0,,,2021-06-16 02:15:21 EDT,0,con10,cmd5,seg-1,,,,,"PANIC","XX000","Unexpected internal error: Master process received signal SIGSEGV",,,,,,,0,,,,"1 0x7f099562e630 libpthread.so.0 <symbol not found> + 0x9562e630
2 0xfcec66 postgres _ZN5gpdxl16CTranslatorUtils23ConvertToCDXLLogicalTVFEPN4gpos11CMemoryPoolEPN5gpopt11CMDAccessorEPNS_12CIdGeneratorEPK13RangeTblEntry (CRefCount.h:103)
3 0xfef497 postgres _ZN5gpdxl21CTranslatorQueryToDXL17TranslateTVFToDXLEPK13RangeTblEntryjj (CTranslatorQueryToDXL.cpp:3577)
4 0xfe88f6 postgres _ZN5gpdxl21CTranslatorQueryToDXL22TranslateFromExprToDXLEP8FromExpr (CTranslatorQueryToDXL.cpp:2922)
5 0xfe9d7a postgres _ZN5gpdxl21CTranslatorQueryToDXL21TranslateGroupingSetsEP8FromExprP4ListS4_bPN4gpos8CHashMapIijXadL_ZNS5_9HashValueIiEEjPKT_EEXadL_ZNS5_6EqualsIiEEbSA_SA_EEXadL_ZNS5_13CleanupDeleteIiEEvPS8_EEXadL_ZNSC_IjEEvSD_EEEESF_ (CTranslatorQueryToDXL.cpp:613)
6 0xfea1bd postgres _ZN5gpdxl21CTranslatorQueryToDXL25TranslateSelectQueryToDXLEv (CTranslatorQueryToDXL.cpp:557)
7 0xfeb946 postgres _ZN5gpdxl21CTranslatorQueryToDXL19TranslateQueryToDXLEv (CTranslatorQueryToDXL.cpp:659)
8 0x10004f4 postgres _ZN9COptTasks12OptimizeTaskEPv (discriminator 3)
9 0xcf12d6 postgres _ZN4gpos5CTask7ExecuteEv (CTask.cpp:130)
10 0xcf2268 postgres _ZN4gpos7CWorker7ExecuteEPNS_5CTaskE (CWorker.cpp:82)
11 0xcf079d postgres _ZN4gpos14CAutoTaskProxy7ExecuteEPNS_5CTaskE (CAutoTaskProxy.cpp:301)
12 0xcf2e61 postgres gpos_exec (_api.cpp:246)
13 0xffe6a1 postgres _ZN9COptTasks7ExecuteEPFPvS0_ES0_ (COptTasks.cpp:261)
14 0xfff8ce postgres _ZN9COptTasks18GPOPTOptimizedPlanEP5QueryP11SOptContext (COptTasks.cpp:775)
15 0x1001b91 postgres GPOPTOptimizedPlan (CGPOptimizer.cpp:60)
16 0x9d3243 postgres optimize_query (orca.c:46)
17 0x9c48cd postgres standard_planner (planner.c:240)
18 0x9c4c25 postgres planner (planner.c:200)
19 0xa8ac01 postgres <symbol not found> (postgres.c:961)
20 0xa8d59f postgres PostgresMain (postgres.c:5242)
"
2021-06-16 02:15:56.298876 EDT,,,p15522,th-1744660352,,,,0,,,seg-1,,,,,"LOG","00000","server process (PID 16957) was terminated by signal 11: Segmentation fault","Failed process was running: SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",,,,,,0,,"postmaster.c",3970,
closed time in 2 days
edespinoissue commentgreenplum-db/gpdb
Merged fix to 6X (and master) in 2f45387.
comment created time in 2 days
issue closedgreenplum-db/gpdb
create function return enum as array core dump when orca is open
Greenplum version or build
PostgreSQL 9.4.24 (Greenplum Database 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c Open Source) on x86_64-unknown-li
OS version and uname -a
CentOS Linux release 7.3.1611 (Core) Linux node129 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
autoconf options used ( config.status --config )
null
Installation information ( pg_config )
BINDIR = /home/gpadmin/greenplum-db-6.16.0/bin DOCDIR = /home/gpadmin/greenplum-db-6.16.0/share/doc/postgresql HTMLDIR = /home/gpadmin/greenplum-db-6.16.0/share/doc/postgresql INCLUDEDIR = /home/gpadmin/greenplum-db-6.16.0/include PKGINCLUDEDIR = /home/gpadmin/greenplum-db-6.16.0/include/postgresql INCLUDEDIR-SERVER = /home/gpadmin/greenplum-db-6.16.0/include/postgresql/server LIBDIR = /home/gpadmin/greenplum-db-6.16.0/lib PKGLIBDIR = /home/gpadmin/greenplum-db-6.16.0/lib/postgresql LOCALEDIR = /home/gpadmin/greenplum-db-6.16.0/share/locale MANDIR = /home/gpadmin/greenplum-db-6.16.0/man SHAREDIR = /home/gpadmin/greenplum-db-6.16.0/share/postgresql SYSCONFDIR = /home/gpadmin/greenplum-db-6.16.0/etc/postgresql PGXS = /home/gpadmin/greenplum-db-6.16.0/lib/postgresql/pgxs/src/makefiles/pgxs.mk CONFIGURE = '--disable-debug-extensions' '--disable-tap-tests' '--enable-orca' '--with-zstd' '--with-gssapi' '--with-libxml' '--with-perl' '--with-python' '--with-openssl' '--with-pam' '--with-ldap' '--with-extra-version= Open Source' '--prefix=/usr/local/greenplum-db-6.16.0' '--mandir=/usr/local/greenplum-db-6.16.0/man' 'CC=gcc' 'CFLAGS=-O3 -fargument-noalias-global -fno-omit-frame-pointer -g' CC = gcc CPPFLAGS = -D_GNU_SOURCE -I/usr/include/libxml2 -I/home/gpadmin/greenplum-db-6.16.0/include CFLAGS = -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -fno-aggressive-loop-optimizations -Wno-unused-but-set-variable -Wno-address -O3 -fargument-noalias-global -fno-omit-frame-pointer -g -std=gnu99 -Werror=uninitialized -Werror=implicit-function-declaration -I/home/gpadmin/greenplum-db-6.16.0/include CFLAGS_SL = -fPIC LDFLAGS = -Wl,--as-needed -Wl,-rpath,'/usr/local/greenplum-db-6.16.0/lib',--enable-new-dtags -L/home/gpadmin/greenplum-db-6.16.0/lib LDFLAGS_EX = LDFLAGS_SL = LIBS = -lpgcommon -lpgport -lxerces-c -lxml2 -lpam -lrt -lgssapi_krb5 -lzstd -lrt -lcrypt -ldl -lm -L/home/gpadmin/greenplum-db-6.16.0/lib VERSION = PostgreSQL 9.4.24
Expected behavior
CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet'); CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement ) RETURNS TABLE (ae anyenum, aa anyarray) AS $$ SELECT $1, array[$2, $3] $$ LANGUAGE SQL STABLE; SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
execute all sql should ok
Actual behavior
coredump
Step to reproduce the behavior
postgres=# select version();
version
-------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
PostgreSQL 9.4.24 (Greenplum Database 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c Open Source) on x86_64-unknown-li
nux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Apr 22 2021 21:21:23
(1 row)
postgres=# CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');
CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
RETURNS TABLE (ae anyenum, aa anyarray) AS $$
SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;
SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
CREATE TYPE
postgres=# CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
postgres-# RETURNS TABLE (ae anyenum, aa anyarray) AS $$
postgres$# SELECT $1, array[$2, $3]
postgres$# $$ LANGUAGE SQL STABLE;
CREATE FUNCTION
postgres=# SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> \q
note: if set optimizer to off , execute sql is ok
coredump stack
(gdb) bt
#0 0x00007f2f051ca23b in raise () from /lib64/libpthread.so.0
#1 0x0000000000bf4bc0 in StandardHandlerForSigillSigsegvSigbus_OnMainThread (processName=<optimized out>, postgres_signal_arg=11)
at elog.c:5548
#2 <signal handler called>
#3 operator new (line=398, filename=0x1271fc8 "CTranslatorUtils.cpp", mp=0x5, size=48)
at ../../../../src/backend/gporca/libgpos/include/gpos/memory/CMemoryPool.h:280
#4 gpdxl::CTranslatorUtils::ResolvePolymorphicTypes (mp=mp@entry=0x35e4ec8, mdid_array=mdid_array@entry=0x37f7cc8,
arg_types_list=<optimized out>, funcexpr=funcexpr@entry=0x37cf880) at CTranslatorUtils.cpp:398
#5 0x0000000000fc5fd6 in gpdxl::CTranslatorUtils::ConvertToCDXLLogicalTVF (mp=0x35e4ec8, md_accessor=0x7ffd0d43a280,
id_generator=0x37b47f8, rte=rte@entry=0x37cf690) at CTranslatorUtils.cpp:317
#6 0x0000000000fe6807 in gpdxl::CTranslatorQueryToDXL::TranslateTVFToDXL (this=0x37b46a8, rte=0x37cf690, rt_index=1)
at CTranslatorQueryToDXL.cpp:3576
#7 0x0000000000fdfc66 in gpdxl::CTranslatorQueryToDXL::TranslateFromExprToDXL (this=this@entry=0x37b46a8,
from_expr=from_expr@entry=0x37cfb90) at CTranslatorQueryToDXL.cpp:2922
#8 0x0000000000fe10ea in TranslateSelectProjectJoinToDXL (group_clause=0x0, output_attno_to_colid_mapping=0x37b8b38,
sort_group_attno_to_colid_mapping=0x37b8138, from_expr=0x37cfb90, target_list=0x37cfc28, this=0x37b46a8)
at CTranslatorQueryToDXL.cpp:608
#9 gpdxl::CTranslatorQueryToDXL::TranslateGroupingSets (this=this@entry=0x37b46a8, from_expr=0x37cfb90, target_list=0x37cfc28,
group_clause=0x0, has_aggs=<optimized out>, sort_grpref_to_colid_mapping=sort_grpref_to_colid_mapping@entry=0x37b8138,
output_attno_to_colid_mapping=0x37b8638) at CTranslatorQueryToDXL.cpp:2218
#10 0x0000000000fe152d in gpdxl::CTranslatorQueryToDXL::TranslateSelectQueryToDXL (this=this@entry=0x37b46a8)
at CTranslatorQueryToDXL.cpp:557
#11 0x0000000000fe2cb6 in gpdxl::CTranslatorQueryToDXL::TranslateQueryToDXL (this=0x37b46a8) at CTranslatorQueryToDXL.cpp:659
#12 0x0000000000ff7864 in COptTasks::OptimizeTask (ptr=0x7ffd0d43ad90) at COptTasks.cpp:559
#13 0x0000000000ce8466 in gpos::CTask::Execute (this=this@entry=0x3798fe8) at CTask.cpp:130
#14 0x0000000000ce93f8 in gpos::CWorker::Execute (this=0x7ffd0d43ac40, task=task@entry=0x3798fe8) at CWorker.cpp:81
#15 0x0000000000ce792d in gpos::CAutoTaskProxy::Execute (this=this@entry=0x7ffd0d43ac70, task=task@entry=0x3798fe8)
at CAutoTaskProxy.cpp:286
#16 0x0000000000ce9ff1 in gpos_exec (params=params@entry=0x7ffd0d43ad00) at _api.cpp:246
#17 0x0000000000ff5a11 in COptTasks::Execute (func=func@entry=0xff7630 <COptTasks::OptimizeTask(void*)>,
func_arg=func_arg@entry=0x7ffd0d43ad90) at COptTasks.cpp:247
#18 0x0000000000ff6c3e in COptTasks::GPOPTOptimizedPlan (query=query@entry=0x36e0330,
gpopt_context=gpopt_context@entry=0x7ffd0d43ad90) at COptTasks.cpp:774
---Type <return> to continue, or q <return> to quit---
#19 0x0000000000ff8f01 in GPOPTOptimizedPlan (had_unexpected_failure=0x7ffd0d43ae0f, query=0x36e0330) at CGPOptimizer.cpp:58
#20 GPOPTOptimizedPlan (query=query@entry=0x36e0330, had_unexpected_failure=had_unexpected_failure@entry=0x7ffd0d43ae0f)
at CGPOptimizer.cpp:214
#21 0x00000000009d0d93 in optimize_query (parse=parse@entry=0x35c05b8, boundParams=boundParams@entry=0x0) at orca.c:138
#22 0x00000000009c241d in standard_planner (parse=0x35c05b8, cursorOptions=0, boundParams=0x0) at planner.c:238
#23 0x00000000009c2775 in planner (parse=parse@entry=0x35c05b8, cursorOptions=cursorOptions@entry=0,
boundParams=boundParams@entry=0x0) at planner.c:200
#24 0x0000000000a885b1 in pg_plan_query (boundParams=0x0, cursorOptions=0, querytree=0x35c05b8) at postgres.c:959
#25 pg_plan_queries (cursorOptions=0, boundParams=0x0, querytrees=<optimized out>) at postgres.c:1018
#26 exec_simple_query (
query_string=0x35bf038 "SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);")
at postgres.c:1748
#27 0x0000000000a8af4f in PostgresMain (argc=<optimized out>, argv=argv@entry=0x359ea30, dbname=<optimized out>,
username=<optimized out>) at postgres.c:5242
#28 0x00000000006b1a05 in BackendRun (port=0x35c9e40) at postmaster.c:4811
#29 BackendStartup (port=0x35c9e40) at postmaster.c:4468
#30 ServerLoop () at postmaster.c:1948
#31 0x0000000000a10a12 in PostmasterMain (argc=argc@entry=6, argv=argv@entry=0x359ca20) at postmaster.c:1518
#32 0x00000000006b5791 in main (argc=6, argv=0x359ca20) at main.c:245
closed time in 2 days
shenfowangissue commentgreenplum-db/gpdb
create function return enum as array core dump when orca is open
Merged fix to 6X (and master) in https://github.com/greenplum-db/gpdb/commit/2f45387e41c5a01cd18905e57c43662730b97e6c, thanks for the report and contribution!
comment created time in 2 days
push eventgreenplum-db/gpdb
commit sha 2888fcc2e0a5882794352d982bd75b331f590b93
Fix array overflow for function argument types in orca In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args. Fixes GitHub issue #11880 (cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)
push time in 2 days
PR merged greenplum-db/gpdb
Forward port of https://github.com/greenplum-db/gpdb/pull/12051
In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args.
Fixes GitHub issue #11880
(cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)
pr closed time in 2 days
PR opened greenplum-db/gpupgrade
Draft PR for now, until I have a bit more time to add final touches.
If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.
pr created time in 2 days
PR closed greenplum-db/gpupgrade
Draft PR for now, until I have a bit more time to add final touches.
If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.
pr closed time in 2 days
PR opened greenplum-db/gpupgrade
Draft PR for now, until I have a bit more time to add final touches.
If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.
pr created time in 2 days
push eventgreenplum-db/gpdb
commit sha fa12b30ac135dbb8e649a125fef496a0f9a9d565
addmirrors/recoverseg : change -B/-b help desc gprecoverseg and gpaddmirrors have changed behavior for -B/-b options. This commit is to edit the help files to show that change.
push time in 2 days
PR merged greenplum-db/gpdb
gprecoverseg and gpaddmirrors have changed behavior for -B/-b options. This commit is to edit the help files to show that change.
gprecoverseg changes: 736bd3fdff, d7c854f8ae, 54261fb073, 0f30f1dd3c gpaddmirrors changes: 489b89c4ad
pr closed time in 2 days
PR opened greenplum-db/gpdb
Forward port of https://github.com/greenplum-db/gpdb/pull/12051
In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args.
Fixes GitHub issue #11880
(cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)
pr created time in 2 days
push eventgreenplum-db/gpdb
commit sha 2f45387e41c5a01cd18905e57c43662730b97e6c
Fix array overflow for function argument types in orca In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args. Fixes GitHub issue #11880
push time in 2 days
PR merged greenplum-db/gpdb
in function CTranslatorUtils::ResolvePolymorphicTypes(), the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args).
Fixes GitHub issue #11880
Here are some reminders before you submit the pull request
- [ ] Add tests for the change
- [ ] Document changes
- [ ] Communicate in the mailing list if needed
- [ ] Pass
make installcheck - [ ] Review a PR in return to support the community
pr closed time in 2 days
Pull request review commentgreenplum-db/gpdb
[6X FIX]Fix a bug of concurrently executing UPDATE/DELETE with VACUUM on a partition ao table
+src/backend/storage/lmgr/README-partition
Thank you for starting this README. Is there a way we can represent the information using tabular format or something capturing information about different Operations (I/U/D/DDLs), GDD enabled, GDD disabled, with ORCA, without ORCA. Clearly flagging any deviations from upstream we have in the table as well.
@asimrp would you be interested to work with @bbbearxyz to help out to get it in that shape, I think you were also wondering about it with me
comment created time in 2 days
push eventgreenplum-db/gpdb
commit sha 9ed94bca9faa0dc11db9138bd0695c26f560c3a8
Fix stale sequence reference causing duplicate nextval() (#12029) (#12122) In the current sequence design, all sequence allocations originating from dispatcher or executor ultimately flow through the dispatcher. This led to a situation where it was possible to reference a `SeqTable` object containing cached sequence numbers that belonged to a different requestor. This meant that it was possible to generate duplicate sequence values. Fix is to maintain separate sequence caches on dispatcher that distinguishes between sequences allocated to dispatcher and those allocated to executors. When the dispatcher sequence is mantained separately, it is able to freely consume its own sequence numbers from its cache without fear of duplicating values allocated to an executor. (cherry picked from commit d8747002b9a0e7d73c377cb56539c08bc0b7c47f)
push time in 3 days
PR merged greenplum-db/gpdb
In the current sequence design, all sequence allocations originating
from dispatcher or executor ultimately flow through the dispatcher.
This led to a situation where it was possible to reference a SeqTable
object containing cached sequence numbers that belonged to a different
requestor. This meant that it was possible to generate duplicate
sequence values.
Fix is to maintain separate sequence caches on dispatcher that distinguishes between sequences allocated to dispatcher and those allocated to executors. When the dispatcher sequence is mantained separately, it is able to freely consume its own sequence numbers from its cache without fear of duplicating values allocated to an executor.
(cherry picked from commit d8747002b9a0e7d73c377cb56539c08bc0b7c47f)
Notes:
In 6X this bug was somewhat hidden by the fact that CACHE default value is 1.
Only merge conflict is that in 6X had to explicitly call out ctl.hash = tag_hash;
pr closed time in 3 days