Horton

Re: [HACKERS] WAL logging problem in 9.4.3?

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: noah@leadboat.com
Cc: robertmhaas@gmail.com, pgsql-hackers@postgresql.org, 9erthalion6@gmail.com, andrew.dunstan@2ndquadrant.com, hlinnaka@iki.fi, michael@paquier.xyz
Date: 2020-02-19T08:31:36Z
Lists: pgsql-hackers 
Sorry, just one fix. (omitting some typos, though..)

At Wed, 19 Feb 2020 17:29:08 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> At Tue, 18 Feb 2020 23:44:52 -0800, Noah Misch <noah@leadboat.com> wrote in 
> > I think attached v35nm is ready for commit to master.  Would anyone like to
> > talk me out of back-patching this?  I would not enjoy back-patching it, but
> > it's hard to justify lack of back-patch for a data-loss bug.
> > 
> > Notable changes since v34:
> > 
> > - Separate a few freestanding fixes into their own patches.
> 
> All of the three patches look fine.
> 
> > On Mon, Jan 27, 2020 at 07:28:31PM +0900, Kyotaro Horiguchi wrote:
> > > --- a/src/backend/catalog/storage.c
> > > +++ b/src/backend/catalog/storage.c
> > > @@ -388,13 +388,7 @@ RelationPreTruncate(Relation rel)
> > >  	/* Record largest maybe-unsynced block of files under tracking  */
> > >  	pending = hash_search(pendingSyncHash, &(rel->rd_smgr->smgr_rnode.node),
> > >  						  HASH_FIND, NULL);
> > > -	if (pending)
> > > -	{
> > > -		BlockNumber nblocks = smgrnblocks(rel->rd_smgr, MAIN_FORKNUM);
> > > -
> > > -		if (pending->max_truncated < nblocks)
> > > -			pending->max_truncated = nblocks;
> > > -	}
> > > +	pending->is_truncated = true;
> > 
> > - Fix this crashing when "pending" is NULL, as it is in this test case:
> > 
> >   begin;
> >   create temp table t ();
> >   create table t2 ();  -- cause pendingSyncHash to exist
> >   truncate t;
> >   rollback;
> 
> That's terrible... Thanks for fixint it.
> 
> > - Fix the "deleted while still in use" problem that Thomas Munro reported, by
> >   removing the heap_create() change.  Restoring the saved rd_createSubid had
> >   made obsolete the heap_create() change.  check-world now passes with
> >   wal_level=minimal and CLOBBER_CACHE_ALWAYS.
> 
> Ok, as in the previous mail.
> 
> > - Set rd_droppedSubid in RelationForgetRelation(), not
> >   RelationClearRelation().  RelationForgetRelation() knows it is processing a
> >   drop, but RelationClearRelation() could only infer that from circumstantial
> >   evidence.  This seems more future-proof to me.
> 
> Agreed. Different from RelationClearRelatoin, RelationForgetRelation
> is called only for "drop"ing the relation.
> 
> > - When reusing an index build, instead of storing the dropped relid in the
> >   IndexStmt and opening the dropped relcache entry in ATExecAddIndex(), store
> >   the subid fields in the IndexStmt.  This is less code, and I felt
> >   RelationIdGetRelationCache() invited misuse.
> 
> Hmm. I'm not sure that index_create having the new subid parameters is
> good. And the last if(OidIsValid) clause handles storage persistence
> so I did that there. But I don't strongly against it.

Hmm. I'm not sure that index_create having the new subid parameters is
good. And the last if(OidIsValid) clause in AtExecAddIndex handles
storage persistence so I did that there. But I don't strongly against
it.

> Please give a bit more time to look it.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Add perl2host call missing from a new test file.

  2. Skip WAL for new relfilenodes, under wal_level=minimal.

  3. Revert "Skip WAL for new relfilenodes, under wal_level=minimal."

  4. Back-patch log_newpage_range().

  5. During heap rebuild, lock any TOAST index until end of transaction.

  6. In log_newpage_range(), heed forkNum and page_std arguments.

  7. Back-patch src/test/recovery and PostgresNode from 9.6 to 9.5.

  8. Reduce pg_ctl's reaction time when waiting for postmaster start/stop.

  9. Accelerate end-of-transaction dropping of relations

  10. Redesign the planner's handling of index-descent cost estimation.

  11. Make TRUNCATE do truncate-in-place when processing a relation that was created