Horton

Re: Can we get rid of repeated queries from pg_dump?

hubert depesz lubaczewski <depesz@depesz.com>

View thread View on postgr.es
From: hubert depesz lubaczewski <depesz@depesz.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Laurenz Albe <laurenz.albe@cybertec.at>, pgsql-general <pgsql-general@lists.postgresql.org>
Date: 2021-08-30T14:45:51Z
Lists: pgsql-hackers, pgsql-general 
On Mon, Aug 30, 2021 at 10:11:22AM -0400, Tom Lane wrote:
> I don't suppose you could send me a schema-only dump of that
> database, off-list?  I'm now quite curious.

Asked the owners for their permission.

> > The thing is - even though it was called 1804 times, dump contains data only
> > about 107 functions (pg_restore -l schema.dump | grep -c FUNCTION), so it kinda
> > seems that 94% of these calls is not needed.

> Hm.  It's not doing that for *every* row in pg_proc, at least.
> I speculate that it is collecting and then not printing the info
> about functions that are in extensions --- can you check on
> how many there are of those?
> (Actually, if you've got a whole lot of objects inside extensions,
> maybe that explains the 5000 calls?)

Well, not sure if that's a lot, but:
there are 15 extensions, including plpgsql.

SELECT
    count(*)
FROM
    pg_catalog.pg_depend
WHERE
    refclassid = 'pg_catalog.pg_extension'::pg_catalog.regclass
    AND deptype = 'e';


return 2110 objects:

SELECT
    classid::regclass,
    count(*)
FROM
    pg_catalog.pg_depend
WHERE
    refclassid = 'pg_catalog.pg_extension'::pg_catalog.regclass
    AND deptype = 'e'
GROUP BY
    1
ORDER BY
    1;

         classid         │ count 
─────────────────────────┼───────
 pg_type                 │    31
 pg_proc                 │  1729
 pg_class                │    61
 pg_foreign_data_wrapper │     1
 pg_cast                 │    30
 pg_language             │     1
 pg_opclass              │    73
 pg_operator             │   111
 pg_opfamily             │    73
(9 rows)

Best regards,

depesz

Commits

  1. In pg_dump, avoid doing per-table queries for RLS policies.

  2. Cache the results of format_type() queries in pg_dump.