Re: BF mamba failure

Hi,

On Wed, Jul 08, 2026 at 08:59:35AM +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 01:13:59PM +0300, Kouber Saparev wrote:
> > We had this issue again, the database just stopped - PostgreSQL 17.6.
> > 
> > FATAL,XX000,"trying to drop stats entry already dropped: kind=relation
> > dboid=16420 objoid=3885511363 refcount=1 generation=0"
> > 
> > There is no such object present in the database.
> > 
> > Do you think the issue might be fixed in PostgreSQL 18.4?... Or PostgreSQL
> > 19 eventually?
> 
> Well, honestly, hard to say.  That's a case where a reproducible
> scenario would be useful, as it seems to ve very workload dependent in
> terms of handling of the relation entries..  (Aka I have not dug in
> details how to do that.)
> 
> As one example, the buildfarm seems to be quite silent, but it would
> help if we need something like a wraparound for better
> reproducibility.

So given that Kouber said that "we have around 150 entries added and deleted
per second in pg_class, and around 800 in pg_attribute. So something is actively
creating and dropping tables all the time.", OID reused is likely the culprit (
doable in months at this rate).

To try to simulate an OID reuse, I created an extension to manually set the next
OID and was able to reproduce the issue on 17.6, that way:

Primary: create table popo (a int);
Standby: select count(*) from popo; (keep this session open)
Primary: SELECT oid FROM pg_class WHERE relname = 'popo'; (17044 here)
Primary: drop table popo;
Primary: checkpoint;
Primary: vacuum;
Primary: SELECT set_next_oid(17044::oid);
Primary: create table popo2 (a int);
Primary: SELECT oid FROM pg_class WHERE relname = 'popo2'; (verify it reused 17044)
Primary: drop table popo2;

Produces:

2026-07-08 10:30:22.566 UTC [2096038] FATAL:  trying to drop stats entry already dropped: kind=relation dboid=5 objoid=17044 refcount=1 generation=0

So, the standby backend holds a local reference to the pgstat entry. When the
first DROP is replayed on the standby, the entry is marked dropped=true and refcount
goes from 2 to 1 so it can't be freed.
Then when the second DROP (for popo2, same OID) is replayed, pgstat_drop_entry_internal()
finds the entry with dropped=true and errors out.

The good news is that it does not reproduce on the 17 STABLE branch, I guess it's
due to 850b9218c8e being backpatched to stable branches. So a fix will be in the
next minor versions.

Michael, I wonder if that would make sense to add this "set_next_oid" kind of
thing to a contrib module to test this kind of OID reuse issue. Same idea as
xid_wraparound.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com