Horton

Re: WAL logging problem in 9.4.3?

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Fujii Masao <masao.fujii@gmail.com>, Martijn van Oosterhout <kleptog@svana.org>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2015-07-06T15:21:23Z
Lists: pgsql-hackers 
On 2015-07-06 11:14:40 -0400, Tom Lane wrote:
> BEGIN;
> CREATE TABLE test (i int primary key);
> INSERT INTO test VALUES(-1);
> \copy test from /tmp/num.csv with csv
> COMMIT;
> SELECT COUNT(*) FROM test;
> 
> The COUNT() correctly says 11 rows, but after crash-and-recover,
> only the row with -1 is there.  This is because the INSERT writes
> out an INSERT+INIT WAL record, which we happily replay, clobbering
> the data added later by COPY.

ISTM any WAL logged action that touches a relfilenode essentially needs
to disable further optimization based on the knowledge that the relation
is new.

> We might have to give up on this COPY optimization :-(.

A crazy, not well though through, bandaid for the INSERT+INIT case would
be to force COPY to use a new page when using the SKIP_WAL codepath.

> I'm not sure what would be a safe rule for deciding that we can skip
> WAL logging in this situation, but I am pretty sure that it would
> require keeping information we don't currently keep about what's
> happened earlier in the transaction.

It'd not be impossible to add more state to the relcache entry for the
relation. Whether it's likely that we'd find all the places that'd need
updating that state, I'm not sure.

Commits

  1. Add perl2host call missing from a new test file.

  2. Skip WAL for new relfilenodes, under wal_level=minimal.

  3. Revert "Skip WAL for new relfilenodes, under wal_level=minimal."

  4. Back-patch log_newpage_range().

  5. During heap rebuild, lock any TOAST index until end of transaction.

  6. In log_newpage_range(), heed forkNum and page_std arguments.

  7. Back-patch src/test/recovery and PostgresNode from 9.6 to 9.5.

  8. Reduce pg_ctl's reaction time when waiting for postmaster start/stop.

  9. Accelerate end-of-transaction dropping of relations

  10. Redesign the planner's handling of index-descent cost estimation.

  11. Make TRUNCATE do truncate-in-place when processing a relation that was created