Re: Suggestion to add --continue-client-on-abort option to pgbench

On Tue, 17 Jun 2025 03:47:00 +0000
"Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com> wrote:

> Dear Nagata-san,
> 
> > > > 2. The exit-on-abort option and continue-on-error option are mutually
> > exclusive.
> > > > Therefore, I've updated the patch to throw a FATAL error when two options
> > are
> > > > set simultaneously. Corresponding explanation was also added.
> > 
> > I don't think that's right since "abort" and "error" are different concept in pgbench.
> > (Here, "abort" refers to the termination of a client, not a transaction abort.)
> > 
> > The --exit-on-abort option forces to exit pgbench immediately when any client is
> > aborted
> > due to some error. When the --continue-on-error option is not set, SQL errors
> > other than
> > deadlock or serialization error cause a client to be aborted. On the other hand,
> > when the option
> > is set, clients are not aborted due to any SQL errors; instead they continue to run
> > after them.
> > However, clients can still be aborted for other reasons, such as connection
> > failures or
> > meta-command errors (e.g., \set x 1/0). In these cases, the --exit-on-abort option
> > remains
> > useful even when --continue-on-error is enabled.
> 
> To clarify: another approach is that allow --continue-on-error option to continue
> running even when clients meet such errors. Which one is better?

It might be worth discussing which types of errors this option should allow pgbench
to continue after. On my understand the current patch's goal is to allow only SQL
level errors like comstraint violations. It seems good because this could simulate
behaviour of applications that ignore or retry such errors (although they are not
retried in the current patch). Perhaps, it makes sense to allow to continue after
some network errors because it would enable benchmarks usign a cluster system as a
cloud service that could report a temporary error during a failover.

It might be worth discussing which types of errors this option should allow pgbench to
continue after.

As I understand it, the current patch aims to allow continuation only after SQL-level
errors, such as constraint violations. That seems reasonable, as it can simulate the
behavior of applications that ignore or retry such errors (even though retries are not
implemented in the current patch).

Perhaps it also makes sense to allow continuation after certain network errors, as this
would enable benchmarking with cluster systems or cloud services, which might report
temporary errors during a failover. We would need additional work to properly detect
and handle network errors, though.

However, I'm not sure it's reasonable to allow continuation after other types of errors,
such as misuse of meta-commands or unexpected errors during their execution, since these
wouldn't simulate any real application behavior and would more likely indicate a failure
in the benchmarking process itself.

Best regards,
Yugo Nagata

-- 
Yugo Nagata <nagata@sraoss.co.jp>