Horton

Re: Minimal logical decoding on standbys

bdrouvotAWS <bdrouvot@amazon.com>

View thread
From: "Drouvot, Bertrand" <bdrouvot@amazon.com>
To: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Ibrar Ahmed <ibrar.ahmad@gmail.com>, Amit Khandekar <amitdkhan.pg@gmail.com>, <fabriziomello@gmail.com>, tushar <tushar.ahuja@enterprisedb.com>, Rahila Syed <rahila.syed@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-06-30T08:49:29Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce the log level in 035_standby_logical_decoding.pl.

  2. 035_standby_logical_decoding: Add missing waits for replication

  3. For cascading replication, wake physical and logical walsenders separately

  4. Handle logical slot conflicts on standby

  5. Support invalidating replication slots due to horizon and wal_level

  6. Prevent use of invalidated logical slot in CreateDecodingContext()

  7. Replace replication slot's invalidated_at LSN with an enum

  8. Pass down table relation into more index relation functions

  9. Assert only valid flag bits are passed to visibilitymap_set()

  10. Remove unused _bt_delitems_delete() argument.

  11. Add xl_btree_delete optimization.

Hi,

On 2/25/22 10:34 AM, Drouvot, Bertrand wrote:
> Hi,
>
> On 10/28/21 11:07 PM, Andres Freund wrote:
>> Hi,
>>
>> On 2021-10-28 16:24:22 -0400, Robert Haas wrote:
>>> On Wed, Oct 27, 2021 at 2:56 AM Drouvot, Bertrand 
>>> <bdrouvot@amazon.com> wrote:
>>>> So you have in mind to check for XLogLogicalInfoActive() first, and 
>>>> if true, then open the relation and call
>>>> RelationIsAccessibleInLogicalDecoding()?
>>> I think 0001 is utterly unacceptable. We cannot add calls to
>>> table_open() in low-level functions like this. Suppose for example
>>> that _bt_getbuf() calls _bt_log_reuse_page() which with 0001 applied
>>> would call get_rel_logical_catalog(). _bt_getbuf() will have acquired
>>> a buffer lock on the page. The idea that it's safe to call
>>> table_open() while holding a buffer lock cannot be taken seriously.
>> Yes - that's pretty clearly a deadlock hazard. It shouldn't too hard 
>> to fix, I
>> think. Possibly a bit more verbose than nice, but...
>>
>> Alternatively we could propagate the information whether a relcache 
>> entry is
>> for a catalog from the table to the index. Then we'd not need to 
>> change the
>> btree code to pass the table down.
>
> +1 for the idea of propagating to the index. If that sounds good to 
> you too, I can try to have a look at it.
>
> Thanks Robert and Andres for the feedbacks you have done on the 
> various sub-patches.
>
> I've now in mind to work sub patch by sub patch (starting with 0001 
> then) and move to the next one once we agree that the current one is 
> "ready".
>
> I think that could help us to get this new feature moving forward more 
> "easily", what do you think?
>
> Thanks
>
> Bertrand
>
I'm going to re-create a CF entry for it, as:

- It seems there is a clear interest for the feature (given the time 
already spend on it and the number of people that worked on)

- I've in mind to resume working on it

- It would give more visibility in case others want to jump in

Hope that makes sense,

Thanks,

Bertrand