Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Sort DO_SUBSCRIPTION_REL dump objects independent of OIDs.

  2. Sort dump objects independent of OIDs, for the 7 holdout object types.

  1. pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    vignesh C <vignesh21@gmail.com> — 2025-12-15T18:05:35Z

    Hi,

While verifying upgrade of subscriber instance, I noticed pg_dump
crash caused by incomplete sorting logic for DO_SUBSCRIPTION_REL
objects in DOTypeNameCompare(). When multiple subscription–relation
entries belong to the same subscription, the comparison does not
establish a complete ordering. In this case, the comparison falls
through to the generic assertion path. The attached patch fixes this
by extending the comparison for DO_SUBSCRIPTION_REL objects to include
deterministic ordering keys. After the subscription name comparison,
entries are ordered by the referenced table's schema name and then by
table name.

This issue has started failing after commit:
commit 0decd5e89db9f5edb9b27351082f0d74aae7a9b6
Sort dump objects independent of OIDs, for the 7 holdout object types.

This can be reproduced by having logical replication setup with
subscription subscribing to few tables.

Thanks,
Vignesh

  2. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    Noah Misch <noah@leadboat.com> — 2025-12-15T18:30:18Z

    On Mon, Dec 15, 2025 at 11:35:35PM +0530, vignesh C wrote:
> While verifying upgrade of subscriber instance, I noticed pg_dump
> crash caused by incomplete sorting logic for DO_SUBSCRIPTION_REL
> objects in DOTypeNameCompare(). When multiple subscription–relation
> entries belong to the same subscription, the comparison does not
> establish a complete ordering. In this case, the comparison falls
> through to the generic assertion path. The attached patch fixes this
> by extending the comparison for DO_SUBSCRIPTION_REL objects to include
> deterministic ordering keys. After the subscription name comparison,
> entries are ordered by the referenced table's schema name and then by
> table name.
> 
> This issue has started failing after commit:
> commit 0decd5e89db9f5edb9b27351082f0d74aae7a9b6
> Sort dump objects independent of OIDs, for the 7 holdout object types.
> 
> This can be reproduced by having logical replication setup with
> subscription subscribing to few tables.

That makes sense.  Thanks.  Do you have commands we could add to
src/test/regress/sql/subscription.sql to cover this code?

  3. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    vignesh C <vignesh21@gmail.com> — 2025-12-16T11:52:36Z

    On Tue, 16 Dec 2025 at 00:00, Noah Misch <noah@leadboat.com> wrote:
>
> On Mon, Dec 15, 2025 at 11:35:35PM +0530, vignesh C wrote:
> > While verifying upgrade of subscriber instance, I noticed pg_dump
> > crash caused by incomplete sorting logic for DO_SUBSCRIPTION_REL
> > objects in DOTypeNameCompare(). When multiple subscription–relation
> > entries belong to the same subscription, the comparison does not
> > establish a complete ordering. In this case, the comparison falls
> > through to the generic assertion path. The attached patch fixes this
> > by extending the comparison for DO_SUBSCRIPTION_REL objects to include
> > deterministic ordering keys. After the subscription name comparison,
> > entries are ordered by the referenced table's schema name and then by
> > table name.
> >
> > This issue has started failing after commit:
> > commit 0decd5e89db9f5edb9b27351082f0d74aae7a9b6
> > Sort dump objects independent of OIDs, for the 7 holdout object types.
> >
> > This can be reproduced by having logical replication setup with
> > subscription subscribing to few tables.
>
> That makes sense.  Thanks.  Do you have commands we could add to
> src/test/regress/sql/subscription.sql to cover this code?

This dumping of subscription relation is specific to upgrading to
preserve the subscription relation. So I felt we will not be able to
add tests to subscription.sql, instead how about adding one more table
to 004_subscription.pl where subscription upgrade tests are verified
like the attached patch.

Regards,
Vignesh

  4. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    Noah Misch <noah@leadboat.com> — 2025-12-16T19:24:03Z

    On Tue, Dec 16, 2025 at 05:22:36PM +0530, vignesh C wrote:
> On Tue, 16 Dec 2025 at 00:00, Noah Misch <noah@leadboat.com> wrote:
> > On Mon, Dec 15, 2025 at 11:35:35PM +0530, vignesh C wrote:
> > > This issue has started failing after commit:
> > > commit 0decd5e89db9f5edb9b27351082f0d74aae7a9b6
> > > Sort dump objects independent of OIDs, for the 7 holdout object types.

> > That makes sense.  Thanks.  Do you have commands we could add to
> > src/test/regress/sql/subscription.sql to cover this code?
> 
> This dumping of subscription relation is specific to upgrading to
> preserve the subscription relation. So I felt we will not be able to
> add tests to subscription.sql, instead how about adding one more table
> to 004_subscription.pl where subscription upgrade tests are verified
> like the attached patch.

That's a good way to test it.

> --- a/src/bin/pg_dump/pg_dump_sort.c
> +++ b/src/bin/pg_dump/pg_dump_sort.c
> @@ -454,6 +454,20 @@ DOTypeNameCompare(const void *p1, const void *p2)
>  		if (cmpval != 0)
>  			return cmpval;
>  	}
> +	else if (obj1->objType == DO_SUBSCRIPTION_REL)
> +	{
> +		SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
> +		SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
> +
> +		/* Sort by schema name (subscription name was already considered) */

Let's change the values getSubscriptionRelations() stores in
SubrRelInfo.obj.namespace and SubrRelInfo.obj.name to be more like
DO_PUBLICATION_REL:

		pubrinfo[j].dobj.namespace = tbinfo->dobj.namespace;
		pubrinfo[j].dobj.name = tbinfo->dobj.name;
		pubrinfo[j].publication = pubinfo;

DO_SUBSCRIPTION_REL (new in v17) is gratuitously different:

		subrinfo[i].dobj.name = pg_strdup(subinfo->dobj.name);
		subrinfo[i].tblinfo = tblinfo;
		subrinfo[i].subinfo = subinfo;

What do you think?  This would change sort order from (subname, rel) to (rel,
subname).  Historically, we've avoided churning dump order, in case folks diff
dump output over time.  I bet that practice is less common for binary dumps.
Since DO_SUBSCRIPTION_REL is only for binary dumps, let's just change it.

> +		cmpval = strcmp(srobj1->tblinfo->dobj.namespace->dobj.name,
> +						srobj2->tblinfo->dobj.namespace->dobj.name);

The subrinfo change will make this comparison go away.  If it had stayed, it
should be ready for namespace==NULL like pgTypeNameCompare() is.  (I think
pgTypeNameCompare() could drop that defense, because the getTypes() call to
findNamespace() will pg_fatal() on absence.  Until it does drop that defense,
the rest of pg_dump_sort.c should handle namespace==NULL.)

> --- a/src/bin/pg_upgrade/t/004_subscription.pl
> +++ b/src/bin/pg_upgrade/t/004_subscription.pl

> -# Wait till the table tab_upgraded1 reaches 'ready' state
> +# Wait till the tables tab_upgraded and tab_upgraded1 reaches 'ready' state

s/reaches/reach/ there.

To find other text needing edits, I searched this file for tab_upgraded.  The
following comment still implies "all tables" encompasses just two:

# ------------------------------------------------------
# Check that pg_upgrade is successful when all tables are in ready or in
# init state (tab_upgraded1 table is in ready state and tab_upgraded2 table is
# in init state) along with retaining the replication origin's remote lsn,
# subscription's running status, failover option, and retain_dead_tuples
# option.
# ------------------------------------------------------

  5. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    vignesh C <vignesh21@gmail.com> — 2025-12-17T04:41:58Z

    On Wed, 17 Dec 2025 at 00:54, Noah Misch <noah@leadboat.com> wrote:
>
> On Tue, Dec 16, 2025 at 05:22:36PM +0530, vignesh C wrote:
> > On Tue, 16 Dec 2025 at 00:00, Noah Misch <noah@leadboat.com> wrote:
> > > On Mon, Dec 15, 2025 at 11:35:35PM +0530, vignesh C wrote:
> > > > This issue has started failing after commit:
> > > > commit 0decd5e89db9f5edb9b27351082f0d74aae7a9b6
> > > > Sort dump objects independent of OIDs, for the 7 holdout object types.
>
> > > That makes sense.  Thanks.  Do you have commands we could add to
> > > src/test/regress/sql/subscription.sql to cover this code?
> >
> > This dumping of subscription relation is specific to upgrading to
> > preserve the subscription relation. So I felt we will not be able to
> > add tests to subscription.sql, instead how about adding one more table
> > to 004_subscription.pl where subscription upgrade tests are verified
> > like the attached patch.
>
> That's a good way to test it.
>
> > --- a/src/bin/pg_dump/pg_dump_sort.c
> > +++ b/src/bin/pg_dump/pg_dump_sort.c
> > @@ -454,6 +454,20 @@ DOTypeNameCompare(const void *p1, const void *p2)
> >               if (cmpval != 0)
> >                       return cmpval;
> >       }
> > +     else if (obj1->objType == DO_SUBSCRIPTION_REL)
> > +     {
> > +             SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
> > +             SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
> > +
> > +             /* Sort by schema name (subscription name was already considered) */
>
> Let's change the values getSubscriptionRelations() stores in
> SubrRelInfo.obj.namespace and SubrRelInfo.obj.name to be more like
> DO_PUBLICATION_REL:
>
>                 pubrinfo[j].dobj.namespace = tbinfo->dobj.namespace;
>                 pubrinfo[j].dobj.name = tbinfo->dobj.name;
>                 pubrinfo[j].publication = pubinfo;
>
> DO_SUBSCRIPTION_REL (new in v17) is gratuitously different:
>
>                 subrinfo[i].dobj.name = pg_strdup(subinfo->dobj.name);
>                 subrinfo[i].tblinfo = tblinfo;
>                 subrinfo[i].subinfo = subinfo;

Modified

> What do you think?  This would change sort order from (subname, rel) to (rel,
> subname).  Historically, we've avoided churning dump order, in case folks diff
> dump output over time.

Although grouping by publication and subscription would be cleaner,
publication relations have been ordered first by relation and then by
publication. Retaining the same ordering for subscription relations
preserves consistency.

> I bet that practice is less common for binary dumps.
> Since DO_SUBSCRIPTION_REL is only for binary dumps, let's just change it.
>
> > +             cmpval = strcmp(srobj1->tblinfo->dobj.namespace->dobj.name,
> > +                                             srobj2->tblinfo->dobj.namespace->dobj.name);
>
> The subrinfo change will make this comparison go away.  If it had stayed, it
> should be ready for namespace==NULL like pgTypeNameCompare() is.  (I think
> pgTypeNameCompare() could drop that defense, because the getTypes() call to
> findNamespace() will pg_fatal() on absence.  Until it does drop that defense,
> the rest of pg_dump_sort.c should handle namespace==NULL.)
>
> > --- a/src/bin/pg_upgrade/t/004_subscription.pl
> > +++ b/src/bin/pg_upgrade/t/004_subscription.pl
>
> > -# Wait till the table tab_upgraded1 reaches 'ready' state
> > +# Wait till the tables tab_upgraded and tab_upgraded1 reaches 'ready' state
>
> s/reaches/reach/ there.

Modified

> To find other text needing edits, I searched this file for tab_upgraded.  The
> following comment still implies "all tables" encompasses just two:
>
> # ------------------------------------------------------
> # Check that pg_upgrade is successful when all tables are in ready or in
> # init state (tab_upgraded1 table is in ready state and tab_upgraded2 table is
> # in init state) along with retaining the replication origin's remote lsn,
> # subscription's running status, failover option, and retain_dead_tuples
> # option.
> # ------------------------------------------------------

Updated the comments

The attached v3 version patch has the changes for the same.

Regards,
Vignesh

  6. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    Noah Misch <noah@leadboat.com> — 2025-12-17T19:51:33Z

    On Wed, Dec 17, 2025 at 10:11:58AM +0530, vignesh C wrote:
> The attached v3 version patch has the changes for the same.

The "tag" variable needed a change to compensate for the subrinfo->dobj.name
change.  I plan to push the attached version.

  7. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    vignesh C <vignesh21@gmail.com> — 2025-12-18T06:49:21Z

    On Thu, 18 Dec 2025 at 01:21, Noah Misch <noah@leadboat.com> wrote:
>
> On Wed, Dec 17, 2025 at 10:11:58AM +0530, vignesh C wrote:
> > The attached v3 version patch has the changes for the same.
>
> The "tag" variable needed a change to compensate for the subrinfo->dobj.name
> change.  I plan to push the attached version.

Thanks, the changes look good.

Regards,
Vignesh

  8. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    Chao Li <li.evan.chao@gmail.com> — 2025-12-18T08:35:14Z

    
> On Dec 18, 2025, at 03:51, Noah Misch <noah@leadboat.com> wrote:
> 
> On Wed, Dec 17, 2025 at 10:11:58AM +0530, vignesh C wrote:
>> The attached v3 version patch has the changes for the same.
> 
> The "tag" variable needed a change to compensate for the subrinfo->dobj.name
> change.  I plan to push the attached version.
> <DO_SUBSCRIPTION_REL-v4.patch>

v4 looks solid. A couple of nitpicks:

1
```
+		SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
+		SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
```

These two temp pointers can be const, like:
```
const SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
const SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
```

2
```
+		/* Sort by subscription name, since (namespace, name) match the rel */
```

This comment is correct, but sounds a little insider-ish. Maybe:

/* Tiebreak by subscription name; (namespace, name) already identify the table */

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  9. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    vignesh C <vignesh21@gmail.com> — 2025-12-18T12:05:45Z

    On Thu, 18 Dec 2025 at 14:05, Chao Li <li.evan.chao@gmail.com> wrote:
>
>
>
> > On Dec 18, 2025, at 03:51, Noah Misch <noah@leadboat.com> wrote:
> >
> > On Wed, Dec 17, 2025 at 10:11:58AM +0530, vignesh C wrote:
> >> The attached v3 version patch has the changes for the same.
> >
> > The "tag" variable needed a change to compensate for the subrinfo->dobj.name
> > change.  I plan to push the attached version.
> > <DO_SUBSCRIPTION_REL-v4.patch>
>
> v4 looks solid. A couple of nitpicks:
>
> 1
> ```
> +               SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
> +               SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
> ```
>
> These two temp pointers can be const, like:
> ```
> const SubRelInfo *srobj1 = *(SubRelInfo *const *) p1;
> const SubRelInfo *srobj2 = *(SubRelInfo *const *) p2;
> ```

I felt the way it is handled in the patch is ok and consistent with
the other variables used in this function.

> 2
> ```
> +               /* Sort by subscription name, since (namespace, name) match the rel */
> ```
>
> This comment is correct, but sounds a little insider-ish. Maybe:
>
> /* Tiebreak by subscription name; (namespace, name) already identify the table */

Similarly here too, it is inline with similar comments of other enums
in this function.

Regards,
Vignesh

  10. Re: pg_dump crash due to incomplete ordering of DO_SUBSCRIPTION_REL objects

    Noah Misch <noah@leadboat.com> — 2025-12-18T21:10:02Z

    On Thu, Dec 18, 2025 at 05:35:45PM +0530, vignesh C wrote:
> On Thu, 18 Dec 2025 at 14:05, Chao Li <li.evan.chao@gmail.com> wrote:
> > > On Dec 18, 2025, at 03:51, Noah Misch <noah@leadboat.com> wrote:
> > > I plan to push the attached version.
> > > <DO_SUBSCRIPTION_REL-v4.patch>

Pushed as d49936f etc.

> > 2
> > ```
> > +               /* Sort by subscription name, since (namespace, name) match the rel */
> > ```
> >
> > This comment is correct, but sounds a little insider-ish. Maybe:
> >
> > /* Tiebreak by subscription name; (namespace, name) already identify the table */
> 
> Similarly here too, it is inline with similar comments of other enums
> in this function.

Exactly.  For cosmetics, consistency with nearby code is the stronger rule.