Re: logical decoding / rewrite map vs. maxAllocatedDescs

Hi,

On 2018-08-11 01:55:43 +0200, Tomas Vondra wrote:
> On 08/10/2018 11:59 PM, Tomas Vondra wrote:
> > 
> > ...
> > 
> > I suspect there's some other ingredient, e.g. some manipulation with the
> > subscription. Or maybe it's not needed at all and I'm just imagining things.
> > 
> 
> Indeed, the manipulation with the subscription seems to be the key here.
> I pretty reliably get the 'could not read block' error when doing this:
> 
> 1) start the insert pgbench
> 
>    pgbench -n -c 4 -T 300 -p 5433 -f insert.sql test
> 
> 2) start the vacuum full pgbench
> 
>    pgbench -n -f vacuum.sql -T 300 -p 5433 test
> 
> 3) try to create a subscription, but with small amount of conflicting
> data so that the sync fails like this:
> 
>   LOG:  logical replication table synchronization worker for
>   subscription "s", table "t" has started
>   ERROR:  duplicate key value violates unique constraint "t_pkey"
>   DETAIL:  Key (a)=(5997542) already exists.
>   CONTEXT:  COPY t, line 1
>   LOG:  worker process: logical replication worker for subscription
>   16458 sync 16397 (PID 31983) exited with exit code 1
> 
> 4) At this point the insert pgbench (at least some clients) should have
> failed with the error. If not, rinse and repeat.
> 
> This kinda explains why I've been seeing the error only occasionally,
> because it only happened when I forgotten to clean the table on the
> subscriber while recreating the subscription.

I'll try to reproduce this.  If you're also looking, I suspect a good
first hint would be to just change the ERROR into a PANIC and look at
the backtrace from the generated core file.

To the point that I wonder if we shouldn't just change the ERROR into a
PANIC on master (but not REL_11_STABLE), so the buildfarm gives us
feedback.  I don't think the problem can fundamentally be related to
subscriptions, given the error occurs before any subscriptions are
created in the schedule.

Greetings,

Andres Freund