Re: [PATCH] Use indexes on the subscriber when REPLICA IDENTITY is full on the publisher

Amit Kapila <amit.kapila16@gmail.com>

From: Amit Kapila <amit.kapila16@gmail.com>
To: Peter Smith <smithpb2250@gmail.com>
Cc: Önder Kalacı <onderkalaci@gmail.com>, vignesh C <vignesh21@gmail.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, "wangw.fnst@fujitsu.com" <wangw.fnst@fujitsu.com>, Marco Slot <marco.slot@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-03-09T06:05:21Z
Lists: pgsql-hackers
On Thu, Mar 9, 2023 at 10:37 AM Peter Smith <smithpb2250@gmail.com> wrote:
>
> On Thu, Mar 9, 2023 at 3:49 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Mar 8, 2023 at 8:44 PM Önder Kalacı <onderkalaci@gmail.com> wrote:
> > >
> > >>
> > >> I felt that once you remove the create publication/subscription/wait
> > >> for sync steps, the test execution might become faster and save some
> > >> time in the local execution, cfbot and the various machines in
> > >> buildfarm. If the execution time will not reduce, then no need to
> > >> change.
> > >>
> > >
> > > So, as I noted earlier, there are different schemas. As far as I count, there are at least
> > > 7 different table definitions. I think all tables having the same name are maybe confusing?
> > >
> > > Even if I try to group the same table definitions, and avoid create publication/subscription/wait
> > > for sync steps, the total execution time of the test drops only ~5%. As far as I test, that does not
> > > seem to be the bottleneck for the tests.
> > >
> > > Well, I'm really not sure if it is really worth doing that. I think having each test independent of each
> > > other is really much easier to follow.
> > >
> >
> > This new test takes ~9s on my machine whereas most other tests in
> > subscription/t take roughly 2-5s. I feel we should try to reduce the
> > test timing without sacrificing much of the functionality or code
> > coverage.  I think if possible we should try to reduce setup/teardown
> > cost for each separate test by combining them where possible. I have a
> > few comments on tests which also might help to optimize these tests.
> >
>
> To avoid culling useful tests just because they take too long to run I
> have often thought we should separate some of the useful (but costly)
> subscription tests from the mainstream other tests. Then they won't
> cost any extra time for the build-farm, but at least we can still run
> them on-demand using PG_TEST_EXTRA [1] approach if we really want to.
>

I don't think that is relevant here. It is mostly about removing
duplicate work we are doing in tests. I don't see anything in the
tests that should require a long time to complete.

-- 
With Regards,
Amit Kapila.



Commits

  1. Add the testcases for 89e46da5e5.

  2. Allow the use of indexes other than PK and REPLICA IDENTITY on the subscriber.

  3. Rework query relation permission checking

  4. Generalize ri_RootToPartitionMap to use for non-partition children

  5. Add wait_for_subscription_sync for TAP tests.

  6. Logical replication