Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

Peter Smith <smithpb2250@gmail.com>

From: Peter Smith <smithpb2250@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Önder Kalacı <onderkalaci@gmail.com>, vignesh C <vignesh21@gmail.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>, Marco Slot <marco.slot@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-03-12T21:14:03Z
Lists: pgsql-hackers
On Sat, Mar 11, 2023 at 6:05 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Mar 10, 2023 at 7:58 PM Önder Kalacı <onderkalaci@gmail.com> wrote:
> >
> >
> > I think one option could be to drop some cases altogether, but not sure we'd want that.
> >
> > As a semi-related question: Are you aware of any setting that'd make pg_stat_all_indexes
> > reflect the changes sooner? It is hard to debug what is the bottleneck in the tests, but
> > I have a suspicion that there might be several poll_query_until() calls on
> > pg_stat_all_indexes, which might be the reason?
> >
>
> Yeah, I also think poll_query_until() calls on pg_stat_all_indexes is
> the main reason for these tests taking more time. When I commented
> those polls, it drastically reduces the test time. On looking at
> pgstat_report_stat(), it seems we don't report stats sooner than 1s
> and as most of this patch's test relies on stats, it leads to taking
> more time. I don't have a better idea to verify this patch without
> checking whether the index scan is really used by referring to
> pg_stat_all_indexes. I think trying to reduce the poll calls may help
> in reducing the test timings further. Some ideas on those lines are as
> follows:

If the reason for the stats polling was only to know if some index is
chosen or not, I was wondering if you can just convey the same
information to the TAP test via some conveniently placed (DEBUG?)
logging.

This way the TAP test can do a 'wait_for_log' instead of the
'poll_query_until'. It will probably generate lots of extra logging
but it still might be lots faster that current code because it won't
incur the 1s overheads of the stats.

------
Kind Regards,
Peter Smith.
Fujitsu Australia.



Commits

  1. Add the testcases for 89e46da5e5.

  2. Allow the use of indexes other than PK and REPLICA IDENTITY on the subscriber.

  3. Rework query relation permission checking

  4. Generalize ri_RootToPartitionMap to use for non-partition children

  5. Add wait_for_subscription_sync for TAP tests.

  6. Logical replication