Horton

Re: WAL logging problem in 9.4.3?

Fujii Masao <masao.fujii@gmail.com>

View thread View on postgr.es
From: Fujii Masao <masao.fujii@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Martijn van Oosterhout <kleptog@svana.org>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2015-07-06T13:43:15Z
Lists: pgsql-hackers 
On Sat, Jul 4, 2015 at 2:26 AM, Andres Freund <andres@anarazel.de> wrote:
> On 2015-07-03 19:02:29 +0200, Andres Freund wrote:
>> Maybe I'm just daft right now (35C outside, 32 inside, so ...), but I'm
>> right now missing how the whole "skip wal logging if relation has just
>> been truncated" optimization can ever actually be crashsafe unless we
>> use a new relfilenode (which we don't!).

Agreed... When I ran the following test scenario, I found that
the loaded data disappeared after the crash recovery.

1. start PostgreSQL server with wal_level = minimal
2. execute the following SQL statements
\copy (SELECT num FROM generate_series(1,10) num) to /tmp/num.csv with csv
BEGIN;
CREATE TABLE test (i int primary key);
TRUNCATE TABLE test;
\copy test from /tmp/num.csv with csv
COMMIT;
SELECT COUNT(*) FROM test;  -- returns 10

3. shutdown the server with immediate mode
4. restart the server
5. execute the following SQL statement after crash recovery ends
SELECT COUNT(*) FROM test;  -- returns 0..

In #2, 10 rows were copied and the transaction was committed.
The subsequent statement of "select count(*)" obviously returned 10.
However, after crash recovery, in #5, the same statement returned 0.
That is, the loaded (+ committed) 10 data was lost after the crash.

> We actually used to use a different relfilenode, but optimized that
> away: cab9a0656c36739f59277b34fea8ab9438395869
>
> commit cab9a0656c36739f59277b34fea8ab9438395869
> Author: Tom Lane <tgl@sss.pgh.pa.us>
> Date:   Sun Aug 23 19:23:41 2009 +0000
>
>     Make TRUNCATE do truncate-in-place when processing a relation that was created
>     or previously truncated in the current (sub)transaction.  This is safe since
>     if the (sub)transaction later rolls back, we'd just discard the rel's current
>     physical file anyway.  This avoids unreasonable growth in the number of
>     transient files when a relation is repeatedly truncated.  Per a performance
>     gripe a couple weeks ago from Todd Cook.
>
> to me the reasoning here looks flawed.

Before this commit, when I ran the above test scenario, no data loss happened.

Regards,

-- 
Fujii Masao

Commits

  1. Add perl2host call missing from a new test file.

  2. Skip WAL for new relfilenodes, under wal_level=minimal.

  3. Revert "Skip WAL for new relfilenodes, under wal_level=minimal."

  4. Back-patch log_newpage_range().

  5. During heap rebuild, lock any TOAST index until end of transaction.

  6. In log_newpage_range(), heed forkNum and page_std arguments.

  7. Back-patch src/test/recovery and PostgresNode from 9.6 to 9.5.

  8. Reduce pg_ctl's reaction time when waiting for postmaster start/stop.

  9. Accelerate end-of-transaction dropping of relations

  10. Redesign the planner's handling of index-descent cost estimation.

  11. Make TRUNCATE do truncate-in-place when processing a relation that was created