Horton

Re: [HACKERS] WAL logging problem in 9.4.3?

Noah Misch <noah@leadboat.com>

View thread View on postgr.es
From: Noah Misch <noah@leadboat.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: robertmhaas@gmail.com, pgsql-hackers@postgresql.org, 9erthalion6@gmail.com, andrew.dunstan@2ndquadrant.com, hlinnaka@iki.fi, michael@paquier.xyz
Date: 2020-02-19T07:44:52Z
Lists: pgsql-hackers

Attachments

I think attached v35nm is ready for commit to master.  Would anyone like to
talk me out of back-patching this?  I would not enjoy back-patching it, but
it's hard to justify lack of back-patch for a data-loss bug.

Notable changes since v34:

- Separate a few freestanding fixes into their own patches.

On Mon, Jan 27, 2020 at 07:28:31PM +0900, Kyotaro Horiguchi wrote:
> --- a/src/backend/catalog/storage.c
> +++ b/src/backend/catalog/storage.c
> @@ -388,13 +388,7 @@ RelationPreTruncate(Relation rel)
>  	/* Record largest maybe-unsynced block of files under tracking  */
>  	pending = hash_search(pendingSyncHash, &(rel->rd_smgr->smgr_rnode.node),
>  						  HASH_FIND, NULL);
> -	if (pending)
> -	{
> -		BlockNumber nblocks = smgrnblocks(rel->rd_smgr, MAIN_FORKNUM);
> -
> -		if (pending->max_truncated < nblocks)
> -			pending->max_truncated = nblocks;
> -	}
> +	pending->is_truncated = true;

- Fix this crashing when "pending" is NULL, as it is in this test case:

  begin;
  create temp table t ();
  create table t2 ();  -- cause pendingSyncHash to exist
  truncate t;
  rollback;

- Fix the "deleted while still in use" problem that Thomas Munro reported, by
  removing the heap_create() change.  Restoring the saved rd_createSubid had
  made obsolete the heap_create() change.  check-world now passes with
  wal_level=minimal and CLOBBER_CACHE_ALWAYS.

- Set rd_droppedSubid in RelationForgetRelation(), not
  RelationClearRelation().  RelationForgetRelation() knows it is processing a
  drop, but RelationClearRelation() could only infer that from circumstantial
  evidence.  This seems more future-proof to me.

- When reusing an index build, instead of storing the dropped relid in the
  IndexStmt and opening the dropped relcache entry in ATExecAddIndex(), store
  the subid fields in the IndexStmt.  This is less code, and I felt
  RelationIdGetRelationCache() invited misuse.

Commits

  1. Add perl2host call missing from a new test file.

  2. Skip WAL for new relfilenodes, under wal_level=minimal.

  3. Revert "Skip WAL for new relfilenodes, under wal_level=minimal."

  4. Back-patch log_newpage_range().

  5. During heap rebuild, lock any TOAST index until end of transaction.

  6. In log_newpage_range(), heed forkNum and page_std arguments.

  7. Back-patch src/test/recovery and PostgresNode from 9.6 to 9.5.

  8. Reduce pg_ctl's reaction time when waiting for postmaster start/stop.

  9. Accelerate end-of-transaction dropping of relations

  10. Redesign the planner's handling of index-descent cost estimation.

  11. Make TRUNCATE do truncate-in-place when processing a relation that was created