profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/ArvindSridhar/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Arvind Sridhar ArvindSridhar UC Berkeley Berkeley, CA www.arvindpsridhar.com Student at UC Berkeley passionate about deep learning, algorithms, and app development. SWE/ML intern @ Uber ATG, Nuro, Citadel, Argo AI.

ArvindSridhar/DistributedDeepLearning 2

Exploring strategies to intelligently and efficiently train deep learning models on distributed datasets in parallel, merge/ensemble the results, and perform prediction on a test set. Inspired by my 2018 summer internship at Pivotal Software's Greenplum Advanced Analytics team.

ArvindSridhar/AbstractiveTextSummarization 1

Abstractive text summarization using pointer-generated networks and coverage along with a baseline bi-directional RNN encoder/decoder model

blchu/mlab-intuit-fa18 1

Text Summarization Project

ArvindSridhar/ApacheMadlib 0

Summer Internship 2018: Created ML/deep learning algorithms for large distributed datasets, along with associated utility functions, to enable parallel model training atop Pivotal's Greenplum database

ArvindSridhar/AutonomousDriving-DataCollection 0

Full stack web application to conduct online experiments detecting danger scenarios while driving. Built with Node.js, Angular, Python, MongoDB, HTML/CSS/JS

ArvindSridhar/BashItTheme 0

Full bash_it theme repo for iTerm2, includes themes such as bobby

ArvindSridhar/BCP-Dining 0

Frontend prototype of app enabling students to order food from Bellarmine's cafeteria. Developed during high school.

ArvindSridhar/BriefMeet 0

CalHacks 2017: App that enables you to network with others at events using selfies, making it more memorable. React Native, Expo.io, Python (Flask)

pull request commentgreenplum-db/gpdb

Use 'ss' command in absence of 'netstat' to find active tcp port

Why do we shell out when we can get the same information with import socket?

sambitesh

comment created time in 14 hours

pull request commentgreenplum-db/gpdb

Backport PITR to 6X

@ashwinstar is there any news on this PR? Is general design OK or should we change something? WAL-G part is nearing completion, btw. Currently, we support only 7+, but 6 would be awesome.

x4m

comment created time in 16 hours

PR opened greenplum-db/gpdb

Fix a resource queue lock issue

PR #10003 added an extra ResCleanUpLock() call into ResLockRelease(). However ResCleanUpLock() requires a HeldExclusive ResQueueLock, an assertion failure would occur if this one ResCleanUpLock() called.


#0  0x00007f0ed3e3f337 in raise () from /lib64/libc.so.6
#1  0x00007f0ed3e40a28 in abort () from /lib64/libc.so.6
#2  0x00000000006c8aa9 in ExceptionalCondition (conditionName=0x13a6080 "!(LWLockHeldExclusiveByMe((&MainLWLockArray[38 + 4].lock)))", errorType=<optimized out>, fileName=<optimized out>, lineNumber=<optimized out>) at assert.c:66
#3  0x0000000000c7fb74 in ResCleanUpLock (lock=lock@entry=0x7f0eb93a3110, proclock=proclock@entry=0x7f0eb9aee010, hashcode=hashcode@entry=699557762, wakeupNeeded=wakeupNeeded@entry=0 '\000') at resqueue.c:978
#4  0x0000000000c7ff8b in ResLockRelease (locktag=locktag@entry=0x7fff8a5aa880, resPortalId=0) at resqueue.c:572
#5  0x0000000000c82fd2 in ResLockPortal (portal=portal@entry=0x7f2cd88, qDesc=qDesc@entry=0x803dd98) at resscheduler.c:701
#6  0x0000000000aec117 in PortalStart (portal=portal@entry=0x7f2cd88, params=params@entry=0x0, eflags=eflags@entry=0, snapshot=snapshot@entry=0x0, ddesc=ddesc@entry=0x0) at pquery.c:713
#7  0x0000000000ae7411 in exec_simple_query (query_string=0x7dff158 "SELECT 1") at postgres.c:1781
#8  0x0000000000ae9a38 in PostgresMain (argc=<optimized out>, argv=argv@entry=0x7e1e810, dbname=<optimized out>, username=<optimized out>) at postgres.c:5242
#9  0x0000000000a66f93 in BackendRun (port=0x7e142d0) at postmaster.c:4811
#10 BackendStartup (port=0x7e142d0) at postmaster.c:4468
#11 ServerLoop () at postmaster.c:1948
#12 0x0000000000a67f6d in PostmasterMain (argc=argc@entry=6, argv=argv@entry=0x7ddca20) at postmaster.c:1518
#13 0x00000000006f859a in main (argc=6, argv=0x7ddca20) at main.c:245

Fix this by adusting ResQueueLock acquire position.

Here are some reminders before you submit the pull request

  • [ ] Add tests for the change
  • [ ] Document changes
  • [ ] Communicate in the mailing list if needed
  • [ ] Pass make installcheck
  • [ ] Review a PR in return to support the community
+5 -4

0 comment

1 changed file

pr created time in a day

PR opened greenplum-db/gpdb

Create Partition ResultRel Hash table under the per query context

Inside targetid_get_partition(), the parentInfo->ri_partition_hash named "Partition Result Relation Hash" was created under the TopMemoryContext; CloseResultRelInfo() only freezes it without calling hash_destroy().

Long running backend processes (e.g. pgpool connections) might leak a large amount of memory into the partition result hash context, blowing up vmem memory usage.

Fix this by creating the context under the estate->es_query_cxt.

Here are some reminders before you submit the pull request

  • [ ] Add tests for the change
  • [ ] Document changes
  • [ ] Communicate in the mailing list if needed
  • [ ] Pass make installcheck
  • [ ] Review a PR in return to support the community
+2 -1

0 comment

1 changed file

pr created time in a day

PR opened greenplum-db/gpdb

Fix walker that detects deprecated tables version: 5X_STABLE

This commit ensures that check_node_deprecated_tables_walker recurses into all possible expressions that can contain range tables. Earlier, we only dealt with CTEs, but not sublinks for instance. So the following view would not have been detected:

CREATE VIEW dep_rel_sublink AS SELECT dbid FROM gp_segment_configuration
	WHERE 0 < ALL (SELECT fsedbid FROM pg_filespace_entry);

This commit also makes the walker implementation more canonical and removes the pre-existing recursion logic for CTEs(which we now get for free with expression_tree_walker). Some additional minor improvements along the way.

Please refer to #12168, which introduces a similar walker to check for deprecated columns.

+16 -40

0 comment

2 changed files

pr created time in a day

PR opened greenplum-db/gpdb

[6X back patch] Avoid before_read_command fault usage where possible

This is a back patch of PR #12153 from 7. Cherry-pick was clean, this PR is only for visibility.

The before_read_command fault location is executed by not only regular backends but also special backends such as FTS probe backends, fault injection backends, walsender, etc. Several tests made use of this fault to cause PANIC in QD, after executing a command. This strategy caused intermittent failures in CI because the fault would trigger unexpectedly from other backends listed above. This patch defines a new fault in exec_simple_query() that is guaranteed to be triggered only by query executing backends. This should resolve some of the intermittent failures in CI.

+62 -35

0 comment

10 changed files

pr created time in 2 days

PR opened greenplum-db/gpdb

Use 'ss' command in absence of 'netstat' to find active tcp port

PgPortIsActive would rely on netstat to find active tcp port. In many modern Centos systems it has been replaced by ss. So now in absence of netstat we use ss.

Here are some reminders before you submit the pull request

  • [ ] Add tests for the change
  • [ ] Document changes
  • [ ] Communicate in the mailing list if needed
  • [ ] Pass make installcheck
  • [ ] Review a PR in return to support the community
+54 -4

0 comment

2 changed files

pr created time in 2 days

pull request commentgreenplum-db/gpdb

[6X FIX]Fix a bug of concurrently executing UPDATE/DELETE with VACUUM on a partition ao table

I will get to reviewing this PR tomorrow. Meanwhile, lets add README file to document current partition locking semantics based on knowledge we have gained as part of this PR.

HI, Ashwin, can you tell clearly where we should add README file to?

bbbearxyz

comment created time in 2 days

Pull request review commentgreenplum-db/gpdb

Fix an assert failure caused by unwanted motion node

 adjust_modifytable_flow(PlannerInfo *root, ModifyTable *node, List *is_split_upd 				all_subplans_replicated = false;  				/* Master-only table */-				if (subplan->flow->flotype == FLOW_PARTITIONED ||-					subplan->flow->flotype == FLOW_REPLICATED ||-					(subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1))+				if (subplan->flow->locustype != CdbLocusType_General &&+					(subplan->flow->flotype == FLOW_PARTITIONED ||+					 subplan->flow->flotype == FLOW_REPLICATED ||+					 (subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1)))

Had a short discussion with @gfphoenix78 . Maybe we avoid the gather motion using the logic of this line

subplan->flow->flotype == FLOW_SINGLETON && subplan->flow->segindex != -1

Also master does not have the issue so maybe we can refer the master code logic some although there seem to be big code refactor of the related code. Also do we need to add the test to the master branch to avoid regression in the future?

gfphoenix78

comment created time in 2 days

startedrguo12/awesome-causality-algorithms

started time in 2 days

issue commentgreenplum-db/gpdb

Failed to complete obtain psql count Master gp_segment_configuration Script Exiting!

Can you tell us how to reproduce this error?

Mr-Diaolvyu

comment created time in 2 days

PR opened greenplum-db/gpdb

gpcheckcat: Make opt block similar to 6X

Here are some reminders before you submit the pull request

  • [ ] Add tests for the change
  • [ ] Document changes
  • [ ] Communicate in the mailing list if needed
  • [ ] Pass make installcheck
  • [ ] Review a PR in return to support the community
+6 -3

0 comment

1 changed file

pr created time in 2 days

Pull request review commentgreenplum-db/gpdb

Improve performance of DELETEs on randomly distributed tables in ORCA.

 CDistributionSpecRandom::CDistributionSpecRandom() BOOL CDistributionSpecRandom::Matches(const CDistributionSpec *pds) const {-	if (Edt() != pds->Edt())+	if (pds->Edt() == CDistributionSpec::EdtRouted) 	{-		return false;+		// This follows a 2x2 decision matrix:+		// If both gp_segment_ids are null, then it does not match+		// If one gp_segment_id is null and the other is not, it doesn't match+		// If both gp_segment_ids are not null, they match iff they are equal+		const CDistributionSpecRouted *pdsRouted =+			dynamic_cast<const CDistributionSpecRouted *>(pds);+		GPOS_ASSERT(pdsRouted != nullptr);++		const BOOL localNull = m_gp_segment_id == nullptr;+		const BOOL pdsNull =+			pdsRouted == nullptr || pdsRouted->Pcr() == nullptr;++		if (localNull || pdsNull)+		{+			return false;+		}+		else+		{+			GPOS_ASSERT(!localNull);+			GPOS_ASSERT(!pdsNull);+			return m_gp_segment_id->Id() == pdsRouted->Pcr()->Id();+		} 	}+	else if (pds->Edt() == CDistributionSpec::EdtRandom)+	{+		const CDistributionSpecRandom *pdsRandom =+			dynamic_cast<const CDistributionSpecRandom *>(pds);+		GPOS_ASSERT(pdsRandom != nullptr); -	const CDistributionSpecRandom *pdsRandom =-		dynamic_cast<const CDistributionSpecRandom *>(pds);--	return pdsRandom->IsDuplicateSensitive() == m_is_duplicate_sensitive;+		return pdsRandom->IsDuplicateSensitive() == m_is_duplicate_sensitive;

In case of DML queries, the motion may be duplicate sensitive when you have const table get. If a motion is duplicate sensitive, it is converted into result node with hash filter in dxl to planned stmt. Hash filter allows only those rows to pass which match the hash else filter them out.

Example below: Select 1 can be run anywhere, and while inserting the hash filter allows the rows which match the gp_segment_id based on the hash to be inserted into that segment, all other rows are discarded on that segment

create table tool (a int, b int);
bchaudhary=# explain insert into tool select 1, 1;
LOG:  statement: explain insert into tool select 1, 1;
LOG:  2021-06-17 13:56:06:321336 PDT,THD000,TRACE,"
Physical plan:
+--CPhysicalDML (Insert, "tool"), Source Columns: ["?column?" (1), "?column?" (2)], Action: ("ColRef_0003" (3))   rows:1   width:1  rebinds:1   cost:0.015659   origin: [Grp:6, GrpExpr:2]
   +--CPhysicalComputeScalar   rows:1   width:1  rebinds:1   cost:0.000034   origin: [Grp:9, GrpExpr:3]
      |--CPhysicalMotionHashDistribute HASHED: [ CScalarIdent "?column?" (1), nulls colocated, duplicate sensitive ], opfamilies: (1977,1.0),   rows:1   width:1  rebinds:1   cost:0.000030   origin: [Grp:5, GrpExpr:2]
      |  +--CPhysicalComputeScalar   rows:1   width:1  rebinds:1   cost:0.000009   origin: [Grp:5, GrpExpr:1]
      |     |--CPhysicalConstTableGet Columns: ["" (0)] Values: [(1)]   rows:1   width:1  rebinds:1   cost:0.000001   origin: [Grp:0, GrpExpr:1]
      |     +--CScalarProjectList   origin: [Grp:4, GrpExpr:0]
      |        |--CScalarProjectElement "?column?" (1)   origin: [Grp:2, GrpExpr:0]
      |        |  +--CScalarConst (1)   origin: [Grp:1, GrpExpr:0]
      |        +--CScalarProjectElement "?column?" (2)   origin: [Grp:3, GrpExpr:0]
      |           +--CScalarConst (1)   origin: [Grp:1, GrpExpr:0]
      +--CScalarProjectList   origin: [Grp:8, GrpExpr:0]
         +--CScalarProjectElement "ColRef_0003" (3)   origin: [Grp:7, GrpExpr:0]
            +--CScalarConst (1)   origin: [Grp:1, GrpExpr:0]
",
                            QUERY PLAN
------------------------------------------------------------------
 Insert  (cost=0.00..0.02 rows=1 width=8)
   ->  Result  (cost=0.00..0.00 rows=1 width=12)
         ->  Result  (cost=0.00..0.00 rows=1 width=8)
               ->  Result  (cost=0.00..0.00 rows=1 width=8)
                     ->  Result  (cost=0.00..0.00 rows=1 width=1)
danhoffman699

comment created time in 2 days

issue closedgreenplum-db/gpdb

ORCA Panics when using the CREATE FUNCTION documentation example for "Use polymorphic types to return an ENUM array"

Greenplum Version

postgres --gp-version
postgres (Greenplum Database) 6.16.2 build commit:950f103fe23180ed0148552ef08cd62cbbf5a681

OS Version

  CentOS Linux release 7.9.2009 (Core)
  Linux mdw 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Problem description

From the Greenplum 6.16 documentation, using the CREATE FUNCTION Use polymorphic types to return an ENUM array example generates a server panic when the function is used in a SELECT statement. The problem is reliably reproducible.

Please note: This behavior is NOT exhibited with the planner.

Reproduction steps

CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');
CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement ) 
    RETURNS TABLE (ae anyenum, aa anyarray) AS $$
    SELECT $1, array[$2, $3] 
$$ LANGUAGE SQL STABLE;

SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);

Here is the master server stack

2021-06-16 02:15:56.065475 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd3,seg-1,,,,sx1,"LOG","00000","statement: CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');",,,,,,"CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');",0,,"postgres.c",1639,
2021-06-16 02:15:56.084572 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd4,seg-1,,,,sx1,"LOG","00000","statement: CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
    RETURNS TABLE (ae anyenum, aa anyarray) AS $$
    SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;",,,,,,"CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
    RETURNS TABLE (ae anyenum, aa anyarray) AS $$
    SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;",0,,"postgres.c",1639,
2021-06-16 02:15:56.094923 EDT,"gpadmin","dev",p16957,th-1744660352,"[local]",,2021-06-16 02:15:21 EDT,0,con10,cmd5,seg-1,,,,sx1,"LOG","00000","statement: SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",,,,,,"SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",0,,"postgres.c",1639,
2021-06-16 02:15:56.298684 EDT,,,p16957,th0,,,2021-06-16 02:15:21 EDT,0,con10,cmd5,seg-1,,,,,"PANIC","XX000","Unexpected internal error: Master process received signal SIGSEGV",,,,,,,0,,,,"1    0x7f099562e630 libpthread.so.0 <symbol not found> + 0x9562e630
2    0xfcec66 postgres _ZN5gpdxl16CTranslatorUtils23ConvertToCDXLLogicalTVFEPN4gpos11CMemoryPoolEPN5gpopt11CMDAccessorEPNS_12CIdGeneratorEPK13RangeTblEntry (CRefCount.h:103)
3    0xfef497 postgres _ZN5gpdxl21CTranslatorQueryToDXL17TranslateTVFToDXLEPK13RangeTblEntryjj (CTranslatorQueryToDXL.cpp:3577)
4    0xfe88f6 postgres _ZN5gpdxl21CTranslatorQueryToDXL22TranslateFromExprToDXLEP8FromExpr (CTranslatorQueryToDXL.cpp:2922)
5    0xfe9d7a postgres _ZN5gpdxl21CTranslatorQueryToDXL21TranslateGroupingSetsEP8FromExprP4ListS4_bPN4gpos8CHashMapIijXadL_ZNS5_9HashValueIiEEjPKT_EEXadL_ZNS5_6EqualsIiEEbSA_SA_EEXadL_ZNS5_13CleanupDeleteIiEEvPS8_EEXadL_ZNSC_IjEEvSD_EEEESF_ (CTranslatorQueryToDXL.cpp:613)
6    0xfea1bd postgres _ZN5gpdxl21CTranslatorQueryToDXL25TranslateSelectQueryToDXLEv (CTranslatorQueryToDXL.cpp:557)
7    0xfeb946 postgres _ZN5gpdxl21CTranslatorQueryToDXL19TranslateQueryToDXLEv (CTranslatorQueryToDXL.cpp:659)
8    0x10004f4 postgres _ZN9COptTasks12OptimizeTaskEPv (discriminator 3)
9    0xcf12d6 postgres _ZN4gpos5CTask7ExecuteEv (CTask.cpp:130)
10   0xcf2268 postgres _ZN4gpos7CWorker7ExecuteEPNS_5CTaskE (CWorker.cpp:82)
11   0xcf079d postgres _ZN4gpos14CAutoTaskProxy7ExecuteEPNS_5CTaskE (CAutoTaskProxy.cpp:301)
12   0xcf2e61 postgres gpos_exec (_api.cpp:246)
13   0xffe6a1 postgres _ZN9COptTasks7ExecuteEPFPvS0_ES0_ (COptTasks.cpp:261)
14   0xfff8ce postgres _ZN9COptTasks18GPOPTOptimizedPlanEP5QueryP11SOptContext (COptTasks.cpp:775)
15   0x1001b91 postgres GPOPTOptimizedPlan (CGPOptimizer.cpp:60)
16   0x9d3243 postgres optimize_query (orca.c:46)
17   0x9c48cd postgres standard_planner (planner.c:240)
18   0x9c4c25 postgres planner (planner.c:200)
19   0xa8ac01 postgres <symbol not found> (postgres.c:961)
20   0xa8d59f postgres PostgresMain (postgres.c:5242)
"
2021-06-16 02:15:56.298876 EDT,,,p15522,th-1744660352,,,,0,,,seg-1,,,,,"LOG","00000","server process (PID 16957) was terminated by signal 11: Segmentation fault","Failed process was running: SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);",,,,,,0,,"postmaster.c",3970,

closed time in 2 days

edespino

issue closedgreenplum-db/gpdb

create function return enum as array core dump when orca is open

Greenplum version or build

PostgreSQL 9.4.24 (Greenplum Database 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c Open Source) on x86_64-unknown-li

OS version and uname -a

CentOS Linux release 7.3.1611 (Core) Linux node129 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

autoconf options used ( config.status --config )

null

Installation information ( pg_config )

BINDIR = /home/gpadmin/greenplum-db-6.16.0/bin DOCDIR = /home/gpadmin/greenplum-db-6.16.0/share/doc/postgresql HTMLDIR = /home/gpadmin/greenplum-db-6.16.0/share/doc/postgresql INCLUDEDIR = /home/gpadmin/greenplum-db-6.16.0/include PKGINCLUDEDIR = /home/gpadmin/greenplum-db-6.16.0/include/postgresql INCLUDEDIR-SERVER = /home/gpadmin/greenplum-db-6.16.0/include/postgresql/server LIBDIR = /home/gpadmin/greenplum-db-6.16.0/lib PKGLIBDIR = /home/gpadmin/greenplum-db-6.16.0/lib/postgresql LOCALEDIR = /home/gpadmin/greenplum-db-6.16.0/share/locale MANDIR = /home/gpadmin/greenplum-db-6.16.0/man SHAREDIR = /home/gpadmin/greenplum-db-6.16.0/share/postgresql SYSCONFDIR = /home/gpadmin/greenplum-db-6.16.0/etc/postgresql PGXS = /home/gpadmin/greenplum-db-6.16.0/lib/postgresql/pgxs/src/makefiles/pgxs.mk CONFIGURE = '--disable-debug-extensions' '--disable-tap-tests' '--enable-orca' '--with-zstd' '--with-gssapi' '--with-libxml' '--with-perl' '--with-python' '--with-openssl' '--with-pam' '--with-ldap' '--with-extra-version= Open Source' '--prefix=/usr/local/greenplum-db-6.16.0' '--mandir=/usr/local/greenplum-db-6.16.0/man' 'CC=gcc' 'CFLAGS=-O3 -fargument-noalias-global -fno-omit-frame-pointer -g' CC = gcc CPPFLAGS = -D_GNU_SOURCE -I/usr/include/libxml2 -I/home/gpadmin/greenplum-db-6.16.0/include CFLAGS = -Wall -Wmissing-prototypes -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -fno-aggressive-loop-optimizations -Wno-unused-but-set-variable -Wno-address -O3 -fargument-noalias-global -fno-omit-frame-pointer -g -std=gnu99 -Werror=uninitialized -Werror=implicit-function-declaration -I/home/gpadmin/greenplum-db-6.16.0/include CFLAGS_SL = -fPIC LDFLAGS = -Wl,--as-needed -Wl,-rpath,'/usr/local/greenplum-db-6.16.0/lib',--enable-new-dtags -L/home/gpadmin/greenplum-db-6.16.0/lib LDFLAGS_EX = LDFLAGS_SL = LIBS = -lpgcommon -lpgport -lxerces-c -lxml2 -lpam -lrt -lgssapi_krb5 -lzstd -lrt -lcrypt -ldl -lm -L/home/gpadmin/greenplum-db-6.16.0/lib VERSION = PostgreSQL 9.4.24

Expected behavior

CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet'); CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement ) RETURNS TABLE (ae anyenum, aa anyarray) AS $$ SELECT $1, array[$2, $3] $$ LANGUAGE SQL STABLE; SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);

execute all sql should ok

Actual behavior

coredump

Step to reproduce the behavior

postgres=# select version();
                                                                                                      version

-------------------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------
 PostgreSQL 9.4.24 (Greenplum Database 6.16.0 build commit:5650be2b79197fed564dca8d734d10f2a76b876c Open Source) on x86_64-unknown-li
nux-gnu, compiled by gcc (GCC) 6.4.0, 64-bit compiled on Apr 22 2021 21:21:23
(1 row)

postgres=# CREATE TYPE rainbow AS ENUM('red','orange','yellow','green','blue','indigo','violet');
CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
RETURNS TABLE (ae anyenum, aa anyarray) AS $$
SELECT $1, array[$2, $3]
$$ LANGUAGE SQL STABLE;
SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
CREATE TYPE
postgres=# CREATE FUNCTION return_enum_as_array( anyenum, anyelement, anyelement )
postgres-# RETURNS TABLE (ae anyenum, aa anyarray) AS $$
postgres$# SELECT $1, array[$2, $3]
postgres$# $$ LANGUAGE SQL STABLE;
CREATE FUNCTION
postgres=# SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!> \q

note: if set optimizer to off , execute sql is ok

coredump stack

(gdb) bt
#0  0x00007f2f051ca23b in raise () from /lib64/libpthread.so.0
#1  0x0000000000bf4bc0 in StandardHandlerForSigillSigsegvSigbus_OnMainThread (processName=<optimized out>, postgres_signal_arg=11)
    at elog.c:5548
#2  <signal handler called>
#3  operator new (line=398, filename=0x1271fc8 "CTranslatorUtils.cpp", mp=0x5, size=48)
    at ../../../../src/backend/gporca/libgpos/include/gpos/memory/CMemoryPool.h:280
#4  gpdxl::CTranslatorUtils::ResolvePolymorphicTypes (mp=mp@entry=0x35e4ec8, mdid_array=mdid_array@entry=0x37f7cc8,
    arg_types_list=<optimized out>, funcexpr=funcexpr@entry=0x37cf880) at CTranslatorUtils.cpp:398
#5  0x0000000000fc5fd6 in gpdxl::CTranslatorUtils::ConvertToCDXLLogicalTVF (mp=0x35e4ec8, md_accessor=0x7ffd0d43a280,
    id_generator=0x37b47f8, rte=rte@entry=0x37cf690) at CTranslatorUtils.cpp:317
#6  0x0000000000fe6807 in gpdxl::CTranslatorQueryToDXL::TranslateTVFToDXL (this=0x37b46a8, rte=0x37cf690, rt_index=1)
    at CTranslatorQueryToDXL.cpp:3576
#7  0x0000000000fdfc66 in gpdxl::CTranslatorQueryToDXL::TranslateFromExprToDXL (this=this@entry=0x37b46a8,
    from_expr=from_expr@entry=0x37cfb90) at CTranslatorQueryToDXL.cpp:2922
#8  0x0000000000fe10ea in TranslateSelectProjectJoinToDXL (group_clause=0x0, output_attno_to_colid_mapping=0x37b8b38,
    sort_group_attno_to_colid_mapping=0x37b8138, from_expr=0x37cfb90, target_list=0x37cfc28, this=0x37b46a8)
    at CTranslatorQueryToDXL.cpp:608
#9  gpdxl::CTranslatorQueryToDXL::TranslateGroupingSets (this=this@entry=0x37b46a8, from_expr=0x37cfb90, target_list=0x37cfc28,
    group_clause=0x0, has_aggs=<optimized out>, sort_grpref_to_colid_mapping=sort_grpref_to_colid_mapping@entry=0x37b8138,
    output_attno_to_colid_mapping=0x37b8638) at CTranslatorQueryToDXL.cpp:2218
#10 0x0000000000fe152d in gpdxl::CTranslatorQueryToDXL::TranslateSelectQueryToDXL (this=this@entry=0x37b46a8)
    at CTranslatorQueryToDXL.cpp:557
#11 0x0000000000fe2cb6 in gpdxl::CTranslatorQueryToDXL::TranslateQueryToDXL (this=0x37b46a8) at CTranslatorQueryToDXL.cpp:659
#12 0x0000000000ff7864 in COptTasks::OptimizeTask (ptr=0x7ffd0d43ad90) at COptTasks.cpp:559
#13 0x0000000000ce8466 in gpos::CTask::Execute (this=this@entry=0x3798fe8) at CTask.cpp:130
#14 0x0000000000ce93f8 in gpos::CWorker::Execute (this=0x7ffd0d43ac40, task=task@entry=0x3798fe8) at CWorker.cpp:81
#15 0x0000000000ce792d in gpos::CAutoTaskProxy::Execute (this=this@entry=0x7ffd0d43ac70, task=task@entry=0x3798fe8)
    at CAutoTaskProxy.cpp:286
#16 0x0000000000ce9ff1 in gpos_exec (params=params@entry=0x7ffd0d43ad00) at _api.cpp:246
#17 0x0000000000ff5a11 in COptTasks::Execute (func=func@entry=0xff7630 <COptTasks::OptimizeTask(void*)>,
    func_arg=func_arg@entry=0x7ffd0d43ad90) at COptTasks.cpp:247
#18 0x0000000000ff6c3e in COptTasks::GPOPTOptimizedPlan (query=query@entry=0x36e0330,
    gpopt_context=gpopt_context@entry=0x7ffd0d43ad90) at COptTasks.cpp:774
---Type <return> to continue, or q <return> to quit---
#19 0x0000000000ff8f01 in GPOPTOptimizedPlan (had_unexpected_failure=0x7ffd0d43ae0f, query=0x36e0330) at CGPOptimizer.cpp:58
#20 GPOPTOptimizedPlan (query=query@entry=0x36e0330, had_unexpected_failure=had_unexpected_failure@entry=0x7ffd0d43ae0f)
    at CGPOptimizer.cpp:214
#21 0x00000000009d0d93 in optimize_query (parse=parse@entry=0x35c05b8, boundParams=boundParams@entry=0x0) at orca.c:138
#22 0x00000000009c241d in standard_planner (parse=0x35c05b8, cursorOptions=0, boundParams=0x0) at planner.c:238
#23 0x00000000009c2775 in planner (parse=parse@entry=0x35c05b8, cursorOptions=cursorOptions@entry=0,
    boundParams=boundParams@entry=0x0) at planner.c:200
#24 0x0000000000a885b1 in pg_plan_query (boundParams=0x0, cursorOptions=0, querytree=0x35c05b8) at postgres.c:959
#25 pg_plan_queries (cursorOptions=0, boundParams=0x0, querytrees=<optimized out>) at postgres.c:1018
#26 exec_simple_query (
    query_string=0x35bf038 "SELECT * FROM return_enum_as_array('red'::rainbow, 'green'::rainbow, 'blue'::rainbow);")
    at postgres.c:1748
#27 0x0000000000a8af4f in PostgresMain (argc=<optimized out>, argv=argv@entry=0x359ea30, dbname=<optimized out>,
    username=<optimized out>) at postgres.c:5242
#28 0x00000000006b1a05 in BackendRun (port=0x35c9e40) at postmaster.c:4811
#29 BackendStartup (port=0x35c9e40) at postmaster.c:4468
#30 ServerLoop () at postmaster.c:1948
#31 0x0000000000a10a12 in PostmasterMain (argc=argc@entry=6, argv=argv@entry=0x359ca20) at postmaster.c:1518
#32 0x00000000006b5791 in main (argc=6, argv=0x359ca20) at main.c:245

closed time in 2 days

shenfowang

issue commentgreenplum-db/gpdb

create function return enum as array core dump when orca is open

Merged fix to 6X (and master) in https://github.com/greenplum-db/gpdb/commit/2f45387e41c5a01cd18905e57c43662730b97e6c, thanks for the report and contribution!

shenfowang

comment created time in 2 days

push eventgreenplum-db/gpdb

shenfowang

commit sha 2888fcc2e0a5882794352d982bd75b331f590b93

Fix array overflow for function argument types in orca In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args. Fixes GitHub issue #11880 (cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)

view details

push time in 2 days

PR merged greenplum-db/gpdb

Fix array overflow for function argument types in orca area: gporca

Forward port of https://github.com/greenplum-db/gpdb/pull/12051

In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args.

Fixes GitHub issue #11880

(cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)

+46 -15

0 comment

5 changed files

chrishajas

pr closed time in 2 days

PR opened greenplum-db/gpupgrade

Reviewers
version: use -q when ssh'ing for remote version

Draft PR for now, until I have a bit more time to add final touches.

If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.

+44 -5

0 comment

5 changed files

pr created time in 2 days

PR closed greenplum-db/gpupgrade

Reviewers
version: use -q when ssh'ing for remote version

Draft PR for now, until I have a bit more time to add final touches.

If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.

+44 -5

0 comment

5 changed files

kalensk

pr closed time in 2 days

PR opened greenplum-db/gpupgrade

Reviewers
version: use -q when ssh'ing for remote version

Draft PR for now, until I have a bit more time to add final touches.

If the remote machine uses motd banners or other messages this will get mingled with the output of the version command leading to errors. Suppress motd banner messages by using the ssh -q parameter.

+44 -5

0 comment

5 changed files

pr created time in 2 days

push eventgreenplum-db/gpdb

Divyesh Vanjare

commit sha fa12b30ac135dbb8e649a125fef496a0f9a9d565

addmirrors/recoverseg : change -B/-b help desc gprecoverseg and gpaddmirrors have changed behavior for -B/-b options. This commit is to edit the help files to show that change.

view details

push time in 2 days

PR merged greenplum-db/gpdb

Reviewers
addmirrors/recoverseg : change -B/-b help desc

gprecoverseg and gpaddmirrors have changed behavior for -B/-b options. This commit is to edit the help files to show that change.

gprecoverseg changes: 736bd3fdff, d7c854f8ae, 54261fb073, 0f30f1dd3c gpaddmirrors changes: 489b89c4ad

+31 -13

0 comment

2 changed files

divyeshddv

pr closed time in 2 days

PR opened greenplum-db/gpdb

Fix array overflow for function argument types in orca area: gporca

Forward port of https://github.com/greenplum-db/gpdb/pull/12051

In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args.

Fixes GitHub issue #11880

(cherry picked from commit 2f45387e41c5a01cd18905e57c43662730b97e6c)

+46 -15

0 comment

5 changed files

pr created time in 2 days

push eventgreenplum-db/gpdb

shenfowang

commit sha 2f45387e41c5a01cd18905e57c43662730b97e6c

Fix array overflow for function argument types in orca In function “CTranslatorUtils::ResolvePolymorphicTypes()”, the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args). Then copy the first 'num_args' function argument types for arrary arg_types, the copying number can not exceed num_args. Fixes GitHub issue #11880

view details

push time in 2 days

PR merged greenplum-db/gpdb

Reviewers
Fix array overflow for function argument types in orca area: gporca

in function CTranslatorUtils::ResolvePolymorphicTypes(), the size of array arg_types is num_args, and it is smaller than total_args, this causes a memory corruption and a backend crash. Fixed by correcting the array size "total_args"(num_args + num_return_args).

Fixes GitHub issue #11880

Here are some reminders before you submit the pull request

  • [ ] Add tests for the change
  • [ ] Document changes
  • [ ] Communicate in the mailing list if needed
  • [ ] Pass make installcheck
  • [ ] Review a PR in return to support the community
+46 -15

9 comments

5 changed files

shenfowang

pr closed time in 2 days

Pull request review commentgreenplum-db/gpdb

[6X FIX]Fix a bug of concurrently executing UPDATE/DELETE with VACUUM on a partition ao table

+src/backend/storage/lmgr/README-partition

Thank you for starting this README. Is there a way we can represent the information using tabular format or something capturing information about different Operations (I/U/D/DDLs), GDD enabled, GDD disabled, with ORCA, without ORCA. Clearly flagging any deviations from upstream we have in the table as well.

@asimrp would you be interested to work with @bbbearxyz to help out to get it in that shape, I think you were also wondering about it with me

bbbearxyz

comment created time in 2 days

push eventgreenplum-db/gpdb

David Kimura

commit sha 9ed94bca9faa0dc11db9138bd0695c26f560c3a8

Fix stale sequence reference causing duplicate nextval() (#12029) (#12122) In the current sequence design, all sequence allocations originating from dispatcher or executor ultimately flow through the dispatcher. This led to a situation where it was possible to reference a `SeqTable` object containing cached sequence numbers that belonged to a different requestor. This meant that it was possible to generate duplicate sequence values. Fix is to maintain separate sequence caches on dispatcher that distinguishes between sequences allocated to dispatcher and those allocated to executors. When the dispatcher sequence is mantained separately, it is able to freely consume its own sequence numbers from its cache without fear of duplicating values allocated to an executor. (cherry picked from commit d8747002b9a0e7d73c377cb56539c08bc0b7c47f)

view details

push time in 3 days

PR merged greenplum-db/gpdb

[6X Backport] Fix stale sequence reference causing duplicate nextval() (#12029) version: 6X_STABLE

In the current sequence design, all sequence allocations originating from dispatcher or executor ultimately flow through the dispatcher. This led to a situation where it was possible to reference a SeqTable object containing cached sequence numbers that belonged to a different requestor. This meant that it was possible to generate duplicate sequence values.

Fix is to maintain separate sequence caches on dispatcher that distinguishes between sequences allocated to dispatcher and those allocated to executors. When the dispatcher sequence is mantained separately, it is able to freely consume its own sequence numbers from its cache without fear of duplicating values allocated to an executor.

(cherry picked from commit d8747002b9a0e7d73c377cb56539c08bc0b7c47f)

Notes: In 6X this bug was somewhat hidden by the fact that CACHE default value is 1. Only merge conflict is that in 6X had to explicitly call out ctl.hash = tag_hash;

+64 -4

0 comment

3 changed files

dgkimura

pr closed time in 3 days