Horton

Re: WAL logging problem in 9.4.3?

Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>

View thread View on postgr.es
From: Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp>
To: tgl@sss.pgh.pa.us
Cc: alvherre@2ndquadrant.com, michael.paquier@gmail.com, david@pgmasters.net, hlinnaka@iki.fi, simon@2ndquadrant.com, andres@anarazel.de, masao.fujii@gmail.com, kleptog@svana.org, pgsql-hackers@postgresql.org
Date: 2017-04-13T06:29:35Z
Lists: pgsql-hackers 
I'd like to put a supplimentary explanation.

At Tue, 11 Apr 2017 17:38:12 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in <20170411.173812.133964522.horiguchi.kyotaro@lab.ntt.co.jp>
> Sorry, what I have just sent was broken.
> 
> At Tue, 11 Apr 2017 17:33:41 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in <20170411.173341.257028732.horiguchi.kyotaro@lab.ntt.co.jp>
> > At Tue, 11 Apr 2017 09:56:06 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI <horiguchi.kyotaro@lab.ntt.co.jp> wrote in <20170411.095606.245908357.horiguchi.kyotaro@lab.ntt.co.jp>
> > > Hello, thank you for looking this.
> > > 
> > > At Fri, 07 Apr 2017 20:38:35 -0400, Tom Lane <tgl@sss.pgh.pa.us> wrote in <27309.1491611915@sss.pgh.pa.us>
> > > > Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > > > > Interesting.  I wonder if it's possible that a relcache invalidation
> > > > > would cause these values to get lost for some reason, because that would
> > > > > be dangerous.
> > > > 
> > > > > I suppose the rationale is that this shouldn't happen because any
> > > > > operation that does things this way must hold an exclusive lock on the
> > > > > relation.  But that doesn't guarantee that the relcache entry is
> > > > > completely stable,
> > > > 
> > > > It ABSOLUTELY is not safe.  Relcache flushes can happen regardless of
> > > > how strong a lock you hold.
> > > > 
> > > > 			regards, tom lane
> > > 
> > > Ugh. Yes, relcache invalidation happens anytime and it resets the

The pending locations are not stored in relcache hash so the
problem here is not invalidation but that Relation objects are
created as necessary, anywhere. Even if no invalidation happens,
the same thing will happen in a bit different form.

> > > added values. pg_stat_info deceived me that it can store
> > > transient values. But I  came up with another thought.
> > > 
> > > The reason I proposed it was I thought that hash_search for every
> > > buffer is not good. Instead, like pg_stat_info, we can link the
> > 
> > buffer => buffer modification
> > 
> > > pending-sync hash entry to Relation. This greately reduces the
> > > frequency of hash-searching.
> > > 
> > > I'll post new patch in this way soon.
> > 
> > Here it is.
> 
> It contained tariling space and missing test script.  This is the
> correct patch.
> 
> > - Relation has new members no_pending_sync and pending_sync that
> >   works as instant cache of an entry in pendingSync hash.
> > 
> > - Commit-time synchronizing is restored as Michael's patch.
> > 
> > - If relfilenode is replaced, pending_sync for the old node is
> >   removed. Anyway this is ignored on abort and meaningless on
> >   commit.
> > 
> > - TAP test is renamed to 012 since some new files have been added.
> > 
> > Accessing pending sync hash occured on every calling of
> > HeapNeedsWAL() (per insertion/update/freeze of a tuple) if any of
> > accessing relations has pending sync.  Almost of them are
> > eliminated as the result.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Add perl2host call missing from a new test file.

  2. Skip WAL for new relfilenodes, under wal_level=minimal.

  3. Revert "Skip WAL for new relfilenodes, under wal_level=minimal."

  4. Back-patch log_newpage_range().

  5. During heap rebuild, lock any TOAST index until end of transaction.

  6. In log_newpage_range(), heed forkNum and page_std arguments.

  7. Back-patch src/test/recovery and PostgresNode from 9.6 to 9.5.

  8. Reduce pg_ctl's reaction time when waiting for postmaster start/stop.

  9. Accelerate end-of-transaction dropping of relations

  10. Redesign the planner's handling of index-descent cost estimation.

  11. Make TRUNCATE do truncate-in-place when processing a relation that was created