Horton

Re: Non-text mode for pg_dumpall

Mahendra Singh Thalor <mahi6run@gmail.com>

View thread
From: Mahendra Singh Thalor <mahi6run@gmail.com>
To: jian he <jian.universality@gmail.com>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, Guillaume Lelarge <guillaume@lelarge.info>, Nathan Bossart <nathandbossart@gmail.com>, Magnus Hagander <magnus@hagander.net>, Tom Lane <tgl@sss.pgh.pa.us>, Andrew Dunstan <andrew@dunslane.net>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Dilip Kumar <dilipbalaut@gmail.com>
Date: 2025-01-23T10:35:31Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add non-text output formats to pg_dumpall

  2. Improve pg_dump/pg_dumpall help synopses and terminology

  3. Non text modes for pg_dumpall, correspondingly change pg_restore

  4. Doc: manually break lines in wide UUID examples.

On Thu, 23 Jan 2025 at 14:59, jian he <jian.universality@gmail.com> wrote:
>
> hi.
> The four patches attached are to solve the
> TODO1: We need to think for --exclude-database=PATTERN for pg_restore.
> it is based on your v11_pg_dumpall-with-directory-tar-custom-format-21-jan.patch
>
>
> 0001. pg_dumpall --exclude-database=PATTERN already works,
> main function resolve pattern matching is expand_dbname_patterns.
> make it an extern function, so pg_restore --exclude-database can also use it.

Hi Jian,
We can't use the same expand_dbname_patterns function pg_restore.

In the 1st patch, by mistake I also used this function but then I
realised that we should not use this function due to some limitation
for pg_restore.

While doing pg_dumpall, we have all the existence database names in
the pg_database catalog but while restoring, we don't have all
databases in the catalog.
Actually, we will read dbnames from map.dat file to skip matching
patterns for restore.

Ex: let say we have a fresh server with postgres and template1
databases. Now we want to restore one backup
and inside the map.dat file, we have dbname=db_123 and dbname=db_234.
If we want to use --exclude-database=db_123, then
your patch will not work as this db hasn't been created.

Please cross verify again and let me know your feedback.
I think, as of now, mine v11 patch is working as per expectation.

>
> 0002 cosmetic code changes not in pg_restore.c
> 0003 cosmetic code changes in pg_restore.c
>
>
> 0004 fully implement pg_restore --exclude-database=PATTERN
> similar to pg_dumpall.c
> declare two file static variables:
> static SimpleStringList database_exclude_names = {NULL, NULL};
> static SimpleStringList db_exclude_patterns = {NULL, NULL};
> I also deleted the function is_full_pattern.
>
>
> I use
> $BIN10/pg_restore --exclude-database=*x* --exclude-database=*s*
> --exclude-database=*t* --verbose --file=test.sql x1.dump
> the verbose message to verify my changes.

-- 
Thanks and Regards
Mahendra Singh Thalor
EnterpriseDB: http://www.enterprisedb.com