Horton

Re: making relfilenodes 56 bits

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-09-06T08:40:28Z
Lists: pgsql-hackers 
On Sun, Sep 4, 2022 at 9:27 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Sat, Sep 3, 2022 at 5:11 PM Amit Kapila <amit.kapila16@gmail.com> wrote:

> > Isn't this happening because we are passing "--clean
> > --create"/"--create" options to pg_restore in create_new_objects()? If
> > so, then I think one idea to decouple would be to not use those
> > options. Perform drop/create separately via commands (for create, we
> > need to generate the command as we are generating while generating the
> > dump in custom format), then rewrite the conflicting tables, and
> > finally restore the dump.
>
> Hmm, you are right.  So I think something like this is possible to do,
> I will explore this more. Thanks for the idea.

I have explored this area more and also tried to come up with a
working prototype, so while working on this I realized that we would
have almost to execute all the code which is getting generated as part
of the dumpDatabase() and dumpACL() which is basically,

1. UPDATE pg_catalog.pg_database SET datistemplate = false
2. DROP DATABASE
3. CREATE DATABASE with all the database properties like ENCODING,
LOCALE_PROVIDER, LOCALE, LC_COLLATE, LC_CTYPE, ICU_LOCALE,
COLLATION_VERSION, TABLESPACE
4. COMMENT ON DATABASE
5. Logic inside dumpACL()

I feel duplicating logic like this is really error-prone, but I do not
find any clear way to reuse the code as dumpDatabase() has a high
dependency on the Archive handle and generating the dump file.

So currently I have implemented most of this logic except for a few
e.g. dumpACL(), comments on the database, etc.  So before we go too
far in this direction I wanted to know the opinions of others.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. Fix typos in comments, code and documentation

  2. Increase width of RelFileNumbers from 32 bits to 56 bits.

  3. Move RelFileNumber declarations to common/relpath.h.

  4. Include RelFileLocator fields individually in BufferTag.

  5. Convert macros to static inline functions (buf_internals.h)

  6. Remove the restriction that the relmap must be 512 bytes.

  7. Change internal RelFileNode references to RelFileNumber or RelFileLocator.

  8. Reorder subskiplsn in pg_subscription to avoid alignment issues.

  9. Fix DROP {DATABASE,TABLESPACE} on Windows.

  10. Restructure smgr API as per recent proposal. smgr no longer depends on