Thread
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix assertion failure in pgbench when handling multiple pipeline sync messages.
- 6914a330f019 15.14 landed
- 1d3ded521a0b 16.10 landed
- 398e07162eca 17.6 landed
- fee46ab4f244 18.0 landed
- 12efa48978c6 19 (unreleased) landed
-
Assertion failure in pgbench
Fujii Masao <masao.fujii@gmail.com> — 2025-07-30T15:51:58Z
Hi, I encountered the following assertion failure in pgbench on the current master: Assertion failed: (res == ((void*)0)), function discardUntilSync, file pgbench.c, line 3515. Abort trap: 6 This can be reliably reproduced with the following steps: ------------------------ $ psql -c "ALTER SYSTEM SET default_transaction_isolation TO 'serializable'" $ psql -c "SELECT pg_reload_conf()" $ pgbench -i $ cat test.sql \set aid random(1, 100000 * :scale) \set bid random(1, 1 * :scale) \set tid random(1, 10 * :scale) \set delta random(-5000, 5000) \startpipeline BEGIN; UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid; SELECT abalance FROM pgbench_accounts WHERE aid = :aid; UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid; UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid; INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP); END; \endpipeline $ pgbench -f test.sql -c 10 -j 10 -T 60 -M extended ------------------------ Even without a custom script, shutting down the server with immediate mode while running "pgbench -c 10 -j 10 -T 60" could trigger the same assertion, though not always reliably. /* receive PGRES_PIPELINE_SYNC and null following it */ for (;;) { PGresult *res = PQgetResult(st->con); if (PQresultStatus(res) == PGRES_PIPELINE_SYNC) { PQclear(res); res = PQgetResult(st->con); Assert(res == NULL); break; } PQclear(res); } The failure occurs in this code. This code assumes that PGRES_PIPELINE_SYNC is always followed by a NULL. However, it seems that another PGRES_PIPELINE_SYNC can appear consecutively, which violates that assumption and causes the assertion to fail. Thought? Regards. -- Fujii Masao -
Re: Assertion failure in pgbench
Tatsuo Ishii <ishii@postgresql.org> — 2025-07-31T01:43:23Z
> Hi, > > I encountered the following assertion failure in pgbench on the current master: > > Assertion failed: (res == ((void*)0)), function discardUntilSync, > file pgbench.c, line 3515. > Abort trap: 6 > > > This can be reliably reproduced with the following steps: > > ------------------------ > $ psql -c "ALTER SYSTEM SET default_transaction_isolation TO 'serializable'" > > $ psql -c "SELECT pg_reload_conf()" > > $ pgbench -i > > $ cat test.sql > \set aid random(1, 100000 * :scale) > \set bid random(1, 1 * :scale) > \set tid random(1, 10 * :scale) > \set delta random(-5000, 5000) > \startpipeline > BEGIN; > UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid; > SELECT abalance FROM pgbench_accounts WHERE aid = :aid; > UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid; > UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid; > INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES > (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP); > END; > \endpipeline > > $ pgbench -f test.sql -c 10 -j 10 -T 60 -M extended > ------------------------ > > Even without a custom script, shutting down the server with > immediate mode while running "pgbench -c 10 -j 10 -T 60" could > trigger the same assertion, though not always reliably. > > > /* receive PGRES_PIPELINE_SYNC and null following it */ > for (;;) > { > PGresult *res = PQgetResult(st->con); > > if (PQresultStatus(res) == PGRES_PIPELINE_SYNC) > { > PQclear(res); > res = PQgetResult(st->con); > Assert(res == NULL); > break; > } > PQclear(res); > } > > The failure occurs in this code. This code assumes that PGRES_PIPELINE_SYNC > is always followed by a NULL. However, it seems that another > PGRES_PIPELINE_SYNC can appear consecutively, which violates that assumption > and causes the assertion to fail. Thought? Yes. When an error occurs and an error response message returned from backend, pgbench will send one more sync message, then sends ROLLBACK if necessary. I think the code above should be changed to call PQgetResult repeatably until it returns NULL. Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp -
Re: Assertion failure in pgbench
Tatsuo Ishii <ishii@postgresql.org> — 2025-07-31T02:03:06Z
>> Hi, >> >> I encountered the following assertion failure in pgbench on the current master: >> >> Assertion failed: (res == ((void*)0)), function discardUntilSync, >> file pgbench.c, line 3515. >> Abort trap: 6 >> >> >> This can be reliably reproduced with the following steps: >> >> ------------------------ >> $ psql -c "ALTER SYSTEM SET default_transaction_isolation TO 'serializable'" >> >> $ psql -c "SELECT pg_reload_conf()" >> >> $ pgbench -i >> >> $ cat test.sql >> \set aid random(1, 100000 * :scale) >> \set bid random(1, 1 * :scale) >> \set tid random(1, 10 * :scale) >> \set delta random(-5000, 5000) >> \startpipeline >> BEGIN; >> UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = :aid; >> SELECT abalance FROM pgbench_accounts WHERE aid = :aid; >> UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = :tid; >> UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = :bid; >> INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES >> (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP); >> END; >> \endpipeline >> >> $ pgbench -f test.sql -c 10 -j 10 -T 60 -M extended >> ------------------------ >> >> Even without a custom script, shutting down the server with >> immediate mode while running "pgbench -c 10 -j 10 -T 60" could >> trigger the same assertion, though not always reliably. >> >> >> /* receive PGRES_PIPELINE_SYNC and null following it */ >> for (;;) >> { >> PGresult *res = PQgetResult(st->con); >> >> if (PQresultStatus(res) == PGRES_PIPELINE_SYNC) >> { >> PQclear(res); >> res = PQgetResult(st->con); >> Assert(res == NULL); >> break; >> } >> PQclear(res); >> } >> >> The failure occurs in this code. This code assumes that PGRES_PIPELINE_SYNC >> is always followed by a NULL. However, it seems that another >> PGRES_PIPELINE_SYNC can appear consecutively, which violates that assumption >> and causes the assertion to fail. Thought? > > Yes. When an error occurs and an error response message returned from > backend, pgbench will send one more sync message, then sends ROLLBACK > if necessary. I think the code above should be changed to call > PQgetResult repeatably until it returns NULL. Correction. That would not be a proper fix. Just removing inner PQgetResult and the Assert is enough? Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp -
Re: Assertion failure in pgbench
Stepan Neretin <slpmcf@gmail.com> — 2025-07-31T02:14:29Z
On Thu, Jul 31, 2025 at 9:03 AM Tatsuo Ishii <ishii@postgresql.org> wrote: > >> Hi, > >> > >> I encountered the following assertion failure in pgbench on the current > master: > >> > >> Assertion failed: (res == ((void*)0)), function discardUntilSync, > >> file pgbench.c, line 3515. > >> Abort trap: 6 > >> > >> > >> This can be reliably reproduced with the following steps: > >> > >> ------------------------ > >> $ psql -c "ALTER SYSTEM SET default_transaction_isolation TO > 'serializable'" > >> > >> $ psql -c "SELECT pg_reload_conf()" > >> > >> $ pgbench -i > >> > >> $ cat test.sql > >> \set aid random(1, 100000 * :scale) > >> \set bid random(1, 1 * :scale) > >> \set tid random(1, 10 * :scale) > >> \set delta random(-5000, 5000) > >> \startpipeline > >> BEGIN; > >> UPDATE pgbench_accounts SET abalance = abalance + :delta WHERE aid = > :aid; > >> SELECT abalance FROM pgbench_accounts WHERE aid = :aid; > >> UPDATE pgbench_tellers SET tbalance = tbalance + :delta WHERE tid = > :tid; > >> UPDATE pgbench_branches SET bbalance = bbalance + :delta WHERE bid = > :bid; > >> INSERT INTO pgbench_history (tid, bid, aid, delta, mtime) VALUES > >> (:tid, :bid, :aid, :delta, CURRENT_TIMESTAMP); > >> END; > >> \endpipeline > >> > >> $ pgbench -f test.sql -c 10 -j 10 -T 60 -M extended > >> ------------------------ > >> > >> Even without a custom script, shutting down the server with > >> immediate mode while running "pgbench -c 10 -j 10 -T 60" could > >> trigger the same assertion, though not always reliably. > >> > >> > >> /* receive PGRES_PIPELINE_SYNC and null following it */ > >> for (;;) > >> { > >> PGresult *res = PQgetResult(st->con); > >> > >> if (PQresultStatus(res) == PGRES_PIPELINE_SYNC) > >> { > >> PQclear(res); > >> res = PQgetResult(st->con); > >> Assert(res == NULL); > >> break; > >> } > >> PQclear(res); > >> } > >> > >> The failure occurs in this code. This code assumes that > PGRES_PIPELINE_SYNC > >> is always followed by a NULL. However, it seems that another > >> PGRES_PIPELINE_SYNC can appear consecutively, which violates that > assumption > >> and causes the assertion to fail. Thought? > > > > Yes. When an error occurs and an error response message returned from > > backend, pgbench will send one more sync message, then sends ROLLBACK > > if necessary. I think the code above should be changed to call > > PQgetResult repeatably until it returns NULL. > > Correction. That would not be a proper fix. Just removing inner > PQgetResult and the Assert is enough? > > Best regards, > -- > Tatsuo Ishii > SRA OSS K.K. > English: http://www.sraoss.co.jp/index_en/ > Japanese:http://www.sraoss.co.jp > > Hi, Tatsuo. Do you understand why there is an assertion error in the immediate shutdown case? Best Regards, Stepan Neretin -
Re: Assertion failure in pgbench
Tatsuo Ishii <ishii@postgresql.org> — 2025-07-31T02:56:07Z
> Hi, Tatsuo. > Do you understand why there is an assertion error in the immediate shutdown > case? No. I was not able to reproduce the case so far. Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp
-
Re: Assertion failure in pgbench
Fujii Masao <masao.fujii@gmail.com> — 2025-07-31T15:25:14Z
On Thu, Jul 31, 2025 at 11:03 AM Tatsuo Ishii <ishii@postgresql.org> wrote: > > Yes. When an error occurs and an error response message returned from > > backend, pgbench will send one more sync message, then sends ROLLBACK > > if necessary. I think the code above should be changed to call > > PQgetResult repeatably until it returns NULL. I was thinking the same. The attached patch implements that approach, and it seems to eliminate the assertion failure. > Correction. That would not be a proper fix. Just removing inner > PQgetResult and the Assert is enough? Could you explain why you think repeatedly calling PQgetResult until it returns NULL isn't the right fix? Regards, -- Fujii Masao
-
Re: Assertion failure in pgbench
Fujii Masao <masao.fujii@gmail.com> — 2025-07-31T15:30:25Z
On Thu, Jul 31, 2025 at 11:56 AM Tatsuo Ishii <ishii@postgresql.org> wrote: > > > Hi, Tatsuo. > > Do you understand why there is an assertion error in the immediate shutdown > > case? > > No. I was not able to reproduce the case so far. I would be mistaken here, as I haven't been able to reproduce the issue (i.e., assertion failure by immediate shutdown) since my earlier post. Sorry for the noise. Regards, -- Fujii Masao
-
Re: Assertion failure in pgbench
Tatsuo Ishii <ishii@postgresql.org> — 2025-08-01T00:03:21Z
> On Thu, Jul 31, 2025 at 11:03 AM Tatsuo Ishii <ishii@postgresql.org> wrote: >> > Yes. When an error occurs and an error response message returned from >> > backend, pgbench will send one more sync message, then sends ROLLBACK >> > if necessary. I think the code above should be changed to call >> > PQgetResult repeatably until it returns NULL. > > I was thinking the same. The attached patch implements that approach, > and it seems to eliminate the assertion failure. The patch looks good to me and I confirmed it fixes the the assertion failure. >> Correction. That would not be a proper fix. Just removing inner >> PQgetResult and the Assert is enough? > > Could you explain why you think repeatedly calling PQgetResult > until it returns NULL isn't the right fix? Sorry, that was my thinko. I should have had more coffee. -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp
-
Re: Assertion failure in pgbench
Fujii Masao <masao.fujii@gmail.com> — 2025-08-01T11:55:10Z
On Fri, Aug 1, 2025 at 9:03 AM Tatsuo Ishii <ishii@postgresql.org> wrote: > > > On Thu, Jul 31, 2025 at 11:03 AM Tatsuo Ishii <ishii@postgresql.org> wrote: > >> > Yes. When an error occurs and an error response message returned from > >> > backend, pgbench will send one more sync message, then sends ROLLBACK > >> > if necessary. I think the code above should be changed to call > >> > PQgetResult repeatably until it returns NULL. > > > > I was thinking the same. The attached patch implements that approach, > > and it seems to eliminate the assertion failure. > > The patch looks good to me and I confirmed it fixes the the assertion > failure. Thanks for the review and testing! I've updated the commit message and attached a revised version of the patch. This assertion failure can occur when retriable errors (like serialization errors) happen while using pipeline mode. Since this issue exists from v15 onward, the fix should be back-patched to v15. I’ve also attached a version of the patch that applies cleanly to v15 and v16, as the master patch doesn’t apply cleanly to those branches. Unless there are any objections, I plan to commit this and back-patch to v15. Regards, -- Fujii Masao
-
Re: Assertion failure in pgbench
Fujii Masao <masao.fujii@gmail.com> — 2025-08-03T01:56:05Z
On Fri, Aug 1, 2025 at 8:55 PM Fujii Masao <masao.fujii@gmail.com> wrote: > Unless there are any objections, I plan to commit this and > back-patch to v15. I've pushed the patch. Thanks! Regards, -- Fujii Masao
-
Re: Assertion failure in pgbench
Tatsuo Ishii <ishii@postgresql.org> — 2025-08-04T10:06:03Z
> On Fri, Aug 1, 2025 at 8:55 PM Fujii Masao <masao.fujii@gmail.com> wrote: >> Unless there are any objections, I plan to commit this and >> back-patch to v15. > > I've pushed the patch. Thanks! Thanks. Great! -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp