Re: Decoding speculative insert with toast leaks memory

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Petr Jelinek <petr.jelinek@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>
Date: 2021-06-16T14:48:24Z
Lists: pgsql-hackers
Amit Kapila <amit.kapila16@gmail.com> writes:
> Pushed!

skink reports that this has valgrind issues:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2021-06-15%2020%3A49%3A26

2021-06-16 01:20:13.344 UTC [2198271][4/0:0] LOG:  received replication command: IDENTIFY_SYSTEM
2021-06-16 01:20:13.384 UTC [2198271][4/0:0] LOG:  received replication command: START_REPLICATION SLOT "sub2" LOGICAL 0/0 (proto_version '1', publication_names '"pub2"')
2021-06-16 01:20:13.454 UTC [2198271][4/0:0] LOG:  starting logical decoding for slot "sub2"
2021-06-16 01:20:13.454 UTC [2198271][4/0:0] DETAIL:  Streaming transactions committing after 0/157C828, reading WAL from 0/157C7F0.
2021-06-16 01:20:13.488 UTC [2198271][4/0:0] LOG:  logical decoding found consistent point at 0/157C7F0
2021-06-16 01:20:13.488 UTC [2198271][4/0:0] DETAIL:  There are no running transactions.
...
==2198271== VALGRINDERROR-BEGIN
==2198271== Conditional jump or move depends on uninitialised value(s)
==2198271==    at 0x80EF890: rel_sync_cache_relation_cb (pgoutput.c:833)
==2198271==    by 0x666AEB: LocalExecuteInvalidationMessage (inval.c:595)
==2198271==    by 0x53A423: ReceiveSharedInvalidMessages (sinval.c:90)
==2198271==    by 0x666026: AcceptInvalidationMessages (inval.c:683)
==2198271==    by 0x53FBDD: LockRelationOid (lmgr.c:136)
==2198271==    by 0x1D3943: relation_open (relation.c:56)
==2198271==    by 0x26F21F: table_open (table.c:43)
==2198271==    by 0x66D97F: ScanPgRelation (relcache.c:346)
==2198271==    by 0x674644: RelationBuildDesc (relcache.c:1059)
==2198271==    by 0x674BE8: RelationClearRelation (relcache.c:2568)
==2198271==    by 0x675064: RelationFlushRelation (relcache.c:2736)
==2198271==    by 0x6750A6: RelationCacheInvalidateEntry (relcache.c:2797)
==2198271==  Uninitialised value was created by a heap allocation
==2198271==    at 0x6AC308: MemoryContextAlloc (mcxt.c:826)
==2198271==    by 0x68A8D9: DynaHashAlloc (dynahash.c:283)
==2198271==    by 0x68A94B: element_alloc (dynahash.c:1675)
==2198271==    by 0x68AA58: get_hash_entry (dynahash.c:1284)
==2198271==    by 0x68B23E: hash_search_with_hash_value (dynahash.c:1057)
==2198271==    by 0x68B3D4: hash_search (dynahash.c:913)
==2198271==    by 0x80EE855: get_rel_sync_entry (pgoutput.c:681)
==2198271==    by 0x80EEDA5: pgoutput_truncate (pgoutput.c:530)
==2198271==    by 0x4E48A2: truncate_cb_wrapper (logical.c:797)
==2198271==    by 0x4EFDDB: ReorderBufferCommit (reorderbuffer.c:1777)
==2198271==    by 0x4E1DBE: DecodeCommit (decode.c:637)
==2198271==    by 0x4E1F31: DecodeXactOp (decode.c:245)
==2198271== 
==2198271== VALGRINDERROR-END

			regards, tom lane



Commits

  1. Fix ABI break introduced by commit 4daa140a2f.

  2. Fix valgrind issue in pgoutput.c.

  3. Fix decoding of speculative aborts.

  4. pgoutput: Fix memory leak due to RelationSyncEntry.map.

  5. Fix initialization of RelationSyncEntry for streaming transactions.

  6. Implement streaming mode in ReorderBuffer.