Re: WIP: Avoid creation of the free space map for small tables

John Naylor <john.naylor@2ndquadrant.com>

From: John Naylor <john.naylor@2ndquadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Mithun Cy <mithun.cy@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2019-01-21T21:23:18Z
Lists: pgsql-hackers
On Mon, Jan 21, 2019 at 6:32 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> So we won't allow transfer of FSM files if their size is below
> HEAP_FSM_CREATION_THRESHOLD.  What will be its behavior in link mode?
> It seems that the old files will remain there. Will it create any
> problem when we try to create the files via the new server, can you
> once test this case?

I tried upgrading in --link mode, and on the new cluster, enlarging
the table past the threshold causes a new FSM to be created as
expected.

> Also, another case to think in this regard is the upgrade for standby
> servers, if you read below paragraph from the user manual [1], you
> will see what I am worried about?
>
> "What this does is to record the links created by pg_upgrade's link
> mode that connect files in the old and new clusters on the primary
> server. It then finds matching files in the standby's old cluster and
> creates links for them in the standby's new cluster. Files that were
> not linked on the primary are copied from the primary to the standby.
> (They are usually small.)"
>
> [1] - https://www.postgresql.org/docs/devel/pgupgrade.html

Trying this, I ran into a couple problems. I'm probably doing
something wrong, but I can't help but think there's a pg_upgrade
bug/feature I'm unaware of:

I set up my test to have primary directory data1 and for the secondary
standby/data1. I instructed pg_upgrade to upgrade data1 into data1u,
and I tried the rsync recipe in the docs quoted above, and the
upgraded standby wouldn't go into recovery. While debugging that, I
found surprisingly that pg_upgrade also went further and upgraded
standby/data1 into standby/data1u. I tried deleting standby/data1u
before running the rsync command and still nothing. Because the
upgraded secondary is non-functional, I can't really answer your
question.

Not sure if this is normal, but the pg_upgraded new cluster no longer
had the replication slot. Re-adding it didn't allow my upgraded
secondary to go into recovery, either. (I made sure to copy the
recovery settings, so that can't be the problem)


--
John Naylor                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Commits

  1. Improve code comments in b0eaa4c51b.

  2. During pg_upgrade, conditionally skip transfer of FSMs.

  3. Add more tests for FSM.

  4. Doc: Update the documentation for FSM behavior for small tables.

  5. Make FSM test portable.

  6. Avoid creation of the free space map for small heap relations, take 2.

  7. Move page initialization from RelationAddExtraBlocks() to use, take 2.

  8. Avoid creation of the free space map for small heap relations.

  9. In bootstrap mode, don't allow the creation of files if they don't already