Horton
Menu

Re: BUG #16722: PG hanging on COPY when table has close to 2^32 toasts in the table.

tomohiro hiramitsu <hiramit.tm@gmail.com>

View thread
From: tomohiro hiramitsu <hiramit.tm@gmail.com>
To: Fujii Masao <masao.fujii@oss.nttdata.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, Kasahara Tatsuhito <kasahara.tatsuhito@gmail.com>, Magnus Hagander <magnus@hagander.net>, Andres Freund <andres@anarazel.de>, skoposov@ed.ac.uk, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Date: 2021-03-22T05:43:21Z
Lists: pgsql-bugs

Attachments

 On Fri, Mar 19, 2021 at 1:32 AM Fujii Masao <masao.fujii@oss.nttdata.com>
wrote:
> Thanks for updating the patch! I applied the following and cosmetic
changes
> to the patch. Attached is the updated version of the patch.

Thanks for fixing the patch!
I think your fix is fine.


+#define GETNEWOID_WARN_THRESHOLD 1000000
+#define GETNEWOID_WARN_MAX_INTERVAL 100000000

I think it is better to use the following equation to determine the value
of GETNEWOID_WARN_MAX_INTERVAL.

  GETNEWOID_WARN_MAX_INTERVAL = 1000000 * 2^n

Example 1:
GETNEWOID_WARN_MAX_INTERVAL = 100000000 /* (Current setting) */

DETAIL:  OID candidates were checked "1000000"  times, but no unused OID is
yet found.
DETAIL:  OID candidates were checked "2000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "4000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "8000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "16000000"  times, but no unused OID
is yet found. (*=2)
DETAIL:  OID candidates were checked "32000000"  times, but no unused OID
is yet found. (*=2)
DETAIL:  OID candidates were checked "64000000"  times, but no unused OID
is yet found. (*=2) <---GETNEWOID_WARN_MAX_INTERVAL=100000000
DETAIL:  OID candidates were checked "164000000"  times, but no unused OID
is yet found. (+= GETNEWOID_WARN_MAX_INTERVAL)
DETAIL:  OID candidates were checked "264000000"  times, but no unused OID
is yet found. (+= GETNEWOID_WARN_MAX_INTERVAL)
DETAIL:  OID candidates were checked "364000000"  times, but no unused OID
is yet found. (+= GETNEWOID_WARN_MAX_INTERVAL)

Example 2:
GETNEWOID_WARN_MAX_INTERVAL= 128000000 /* (1000000 * 2^7) */

DETAIL:  OID candidates were checked "1000000"  times, but no unused OID is
yet found.
DETAIL:  OID candidates were checked "2000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "4000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "8000000"  times, but no unused OID is
yet found. (*=2)
DETAIL:  OID candidates were checked "16000000"  times, but no unused OID
is yet found. (*=2)
DETAIL:  OID candidates were checked "32000000"  times, but no unused OID
is yet found. (*=2)
DETAIL:  OID candidates were checked "64000000"  times, but no unused OID
is yet found. (*=2)
DETAIL:  OID candidates were checked "128000000"  times, but no unused OID
is yet found. (*=2) <---GETNEWOID_WARN_MAX_INTERVAL=128000000
DETAIL:  OID candidates were checked "256000000"  times, but no unused OID
is yet found. (+= GETNEWOID_WARN_MAX_INTERVAL)
DETAIL:  OID candidates were checked "384000000"  times, but no unused OID
is yet found. (+= GETNEWOID_WARN_MAX_INTERVAL)

I like example 2 better for the notification timing after the threshold is
exceeded.

Best regards,
--
Tomohiro Hiramitsu
NTT Open Source Software Center

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Log when GetNewOidWithIndex() fails to find unused OID many times.

  2. Add tests for bytea LIKE operator