Horton

Re: snapbuild woes

Petr Jelinek <petr.jelinek@2ndquadrant.com>

View thread View on postgr.es
From: Petr Jelinek <petr.jelinek@2ndquadrant.com>
To: Erik Rijkers <er@xs4all.nl>
Cc: Andres Freund <andres@anarazel.de>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Robert Haas <robertmhaas@gmail.com>, Craig Ringer <craig@2ndquadrant.com>
Date: 2017-02-22T12:07:46Z
Lists: pgsql-hackers 

On 22/02/17 11:29, Erik Rijkers wrote:
> On 2017-02-22 03:05, Petr Jelinek wrote:
>>
>> So to summarize attached patches:
>> 0001 - Fixes performance issue where we build tons of snapshots that we
>> don't need which kills CPU.
>>
>> 0002 - Disables the use of ondisk historical snapshots for initial
>> consistent snapshot export as it may result in corrupt data. This
>> definitely needs backport.
>>
>> 0003 - Fixes bug where we might never reach snapshot on busy server due
>> to race condition in xl_running_xacts logging. The original use of extra
>> locking does not seem to be enough in practice. Once we have agreed fix
>> for this it's probably worth backpatching. There are still some comments
>> that need updating, this is more of a PoC.
>>
> 
> I am not not entirely sure what to expect.  Should a server with these 3
> patches do initial data copy or not?  The sgml seems to imply there is
> not inital data copy.  But my test does copy something.
> 

Not by itself (without the copy patch), those fixes are for snapshots.

> 
> With
>> 0001-Skip-unnecessary-snapshot-builds.patch
>> 0002-Don-t-use-on-disk-snapshots-for-snapshot-export-in-l.patch
>> 0003-Fix-xl_running_xacts-usage-in-snapshot-builder.patch
> 
> the consistent (but wrong) end state is always that only one of the four
> pgbench tables, pgbench_history, is replicated (always correctly).
> 
> Below is the output from the test (I've edited the lines for email)
> (below, a,b,t,h stand for: pgbench_accounts, pgbench_branches,
> pgbench_tellers, pgbench_history)
> (master on port 6972, replica on port 6973.)
> 
> port
> 6972 a,b,t,h: 100000      1     10    347
> 6973 a,b,t,h:      0      0      0    347
> 
> a,b,t,h: a68efc81a  2c27f7ba5  128590a57  1e4070879   master
> a,b,t,h: d41d8cd98  d41d8cd98  d41d8cd98  1e4070879   replica NOK
> 
> The md5-initstrings are from a md5 of the whole content of each table
> (an ordered select *)
> 
> I repeated this a few times: of course, the number of rows in
> pgbench_history varies a bit but otherwise it is always the same: 3
> empty replica tables, pgbench_history replicated correctly.
> 

That's actually correct behaviour without the initial copy patch, it
replicates changes, but since the 3 tables only get updates there is
nothing to replicate as there is no data downstream. However inserts
will of course work fine even without data downstream.

-- 
  Petr Jelinek                  http://www.2ndQuadrant.com/
  PostgreSQL Development, 24x7 Support, Training & Services

Commits

  1. Fix thinko introduced in 2bef06d516460 et al.

  2. Avoid superfluous work for commits during logical slot creation.

  3. Fix race condition leading to hanging logical slot creation.

  4. Don't use on-disk snapshots for exported logical decoding snapshot.

  5. Preserve required !catalog tuples while computing initial decoding snapshot.