Thread

1. Infinite Autovacuum loop caused by failing virtual generated column expression

   SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-04-10T20:18:36Z

   Hi Hackers,
   
   PG19 added support for stats on virtual generated columns [1]. Creating
   extended statistics on a virtual generated column whose expression can
   raise an error leads to ANALYZE failing repeatedly, and autovacuum retrying
   indefinitely. This floods the server logs and also wastes resources. Vacuum
   analyze on that column (without extended stats) succeeds.
   
   In order to avoid retry storms, I think we have two options. (1)
   skipping the offending row from the sample, (2) skipping the extended stats
   computation for that table with a warning message. At least this avoid
   autovacuum infinite retry. Attached a draft patch for the option (2).
   Thoughts?
   
   
   Repro:
   
   CREATE TABLE t (
           id int PRIMARY KEY,
           a int,
           gen int GENERATED ALWAYS AS (100 / a) VIRTUAL
       );
       INSERT INTO t VALUES (1, 10), (2, 5), (3, 0);
   
       -- This succeeds (per-column stats don't evaluate the expression for
   every row)
       ANALYZE t;
   
       -- Add extended statistics referencing the virtual gen col
       CREATE STATISTICS t_stat ON a, gen FROM t;
   
       -- This fails
       ANALYZE t;
       -- ERROR:  division by zero
   
   -- this succeeds
   ANALYZE t(gen)
   
   [1]:
   https://www.postgresql.org/message-id/flat/20250422181006.dd6f9d1d81299f5b2ad55e1a%40sraoss.co.jp
   
   Thanks,
   Satya
   

2. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   Dean Rasheed <dean.a.rasheed@gmail.com> — 2026-04-11T16:33:13Z

   On Fri, 10 Apr 2026 at 21:19, SATYANARAYANA NARLAPURAM
   <satyanarlapuram@gmail.com> wrote:
   >
   > PG19 added support for stats on virtual generated columns [1]. Creating extended statistics on a virtual generated column whose expression can raise an error leads to ANALYZE failing repeatedly, and autovacuum retrying indefinitely. This floods the server logs and also wastes resources. Vacuum analyze on that column (without extended stats) succeeds.
   >
   
   True, though this is nothing new. The same thing can happen with
   expression statistics on an expression that raises an error, which has
   been possible since PG14.
   
   > In order to avoid retry storms, I think we have two options. (1) skipping the offending row from the sample, (2) skipping the extended stats computation for that table with a warning message. At least this avoid autovacuum infinite retry. Attached a draft patch for the option (2). Thoughts?
   >
   
   I'm not sure. The default retry interval is 1 minute, so it won't
   exactly be a flood of messages. Also, if the error only occurs for a
   small subset of rows, it's possible that retrying might succeed.
   
   Regards,
   Dean
   
   
   
   

3. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   Yugo Nagata <nagata@sraoss.co.jp> — 2026-04-14T06:24:25Z

   On Sat, 11 Apr 2026 17:33:13 +0100
   Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
   
   > On Fri, 10 Apr 2026 at 21:19, SATYANARAYANA NARLAPURAM
   > <satyanarlapuram@gmail.com> wrote:
   > >
   > > PG19 added support for stats on virtual generated columns [1]. Creating extended statistics on a virtual generated column whose expression can raise an error leads to ANALYZE failing repeatedly, and autovacuum retrying indefinitely. This floods the server logs and also wastes resources. Vacuum analyze on that column (without extended stats) succeeds.
   > >
   > 
   > True, though this is nothing new. The same thing can happen with
   > expression statistics on an expression that raises an error, which has
   > been possible since PG14.
   
   Yes, this issue is not new, and I’m not aware of a way to prevent it a priori.
   
   > 
   > > In order to avoid retry storms, I think we have two options. (1) skipping the offending row from the sample, (2) skipping the extended stats computation for that table with a warning message. At least this avoid autovacuum infinite retry. Attached a draft patch for the option (2). Thoughts?
   > >
   > 
   > I'm not sure. The default retry interval is 1 minute, so it won't
   > exactly be a flood of messages. Also, if the error only occurs for a
   > small subset of rows, it's possible that retrying might succeed.
   
   I think it would be good to skip ANALYZE for the extended statistics that cause
   errors and just emit a warning, rather than aborting ANALYZE for the entire table.
   It seems reasonable to treat this as the user’s responsibility to notice the warning
   and address the underlying issue.
   
   Regards,
   Yugo Nagata
   
   -- 
   Yugo Nagata <nagata@sraoss.co.jp>
   
   
   
   

4. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-04-14T07:16:42Z

   Hi
   
   On Mon, Apr 13, 2026 at 11:24 PM Yugo Nagata <nagata@sraoss.co.jp> wrote:
   
   > On Sat, 11 Apr 2026 17:33:13 +0100
   > Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
   >
   > > On Fri, 10 Apr 2026 at 21:19, SATYANARAYANA NARLAPURAM
   > > <satyanarlapuram@gmail.com> wrote:
   > > >
   > > > PG19 added support for stats on virtual generated columns [1].
   > Creating extended statistics on a virtual generated column whose expression
   > can raise an error leads to ANALYZE failing repeatedly, and autovacuum
   > retrying indefinitely. This floods the server logs and also wastes
   > resources. Vacuum analyze on that column (without extended stats) succeeds.
   > > >
   > >
   > > True, though this is nothing new. The same thing can happen with
   > > expression statistics on an expression that raises an error, which has
   > > been possible since PG14.
   >
   > Yes, this issue is not new, and I’m not aware of a way to prevent it a
   > priori.
   >
   > >
   > > > In order to avoid retry storms, I think we have two options. (1)
   > skipping the offending row from the sample, (2) skipping the extended stats
   > computation for that table with a warning message. At least this avoid
   > autovacuum infinite retry. Attached a draft patch for the option (2).
   > Thoughts?
   > > >
   > >
   > > I'm not sure. The default retry interval is 1 minute, so it won't
   > > exactly be a flood of messages. Also, if the error only occurs for a
   > > small subset of rows, it's possible that retrying might succeed.
   >
   > I think it would be good to skip ANALYZE for the extended statistics that
   > cause
   > errors and just emit a warning, rather than aborting ANALYZE for the
   > entire table.
   > It seems reasonable to treat this as the user’s responsibility to notice
   > the warning
   > and address the underlying issue.
   >
   
   Yugo, thanks for the comments. Could you please review the v1 patch when you
   get a chance. It is in the direction you suggested.
   
   Thanks,
   Satya
   

5. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   Yugo Nagata <nagata@sraoss.co.jp> — 2026-04-28T09:14:20Z

   On Tue, 14 Apr 2026 00:16:42 -0700
   SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:
   
   > Hi
   > 
   > On Mon, Apr 13, 2026 at 11:24 PM Yugo Nagata <nagata@sraoss.co.jp> wrote:
   > 
   > > On Sat, 11 Apr 2026 17:33:13 +0100
   > > Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
   > >
   > > > On Fri, 10 Apr 2026 at 21:19, SATYANARAYANA NARLAPURAM
   > > > <satyanarlapuram@gmail.com> wrote:
   > > > >
   > > > > PG19 added support for stats on virtual generated columns [1].
   > > Creating extended statistics on a virtual generated column whose expression
   > > can raise an error leads to ANALYZE failing repeatedly, and autovacuum
   > > retrying indefinitely. This floods the server logs and also wastes
   > > resources. Vacuum analyze on that column (without extended stats) succeeds.
   > > > >
   > > >
   > > > True, though this is nothing new. The same thing can happen with
   > > > expression statistics on an expression that raises an error, which has
   > > > been possible since PG14.
   > >
   > > Yes, this issue is not new, and I’m not aware of a way to prevent it a
   > > priori.
   > >
   > > >
   > > > > In order to avoid retry storms, I think we have two options. (1)
   > > skipping the offending row from the sample, (2) skipping the extended stats
   > > computation for that table with a warning message. At least this avoid
   > > autovacuum infinite retry. Attached a draft patch for the option (2).
   > > Thoughts?
   > > > >
   > > >
   > > > I'm not sure. The default retry interval is 1 minute, so it won't
   > > > exactly be a flood of messages. Also, if the error only occurs for a
   > > > small subset of rows, it's possible that retrying might succeed.
   > >
   > > I think it would be good to skip ANALYZE for the extended statistics that
   > > cause
   > > errors and just emit a warning, rather than aborting ANALYZE for the
   > > entire table.
   > > It seems reasonable to treat this as the user’s responsibility to notice
   > > the warning
   > > and address the underlying issue.
   > >
   > 
   > Yugo, thanks for the comments. Could you please review the v1 patch when you
   > get a chance. It is in the direction you suggested.
   
   I've looked into the patch and have some comments.
   
   The child ResourceOwner is created and released in BuildRelationExtStatistics(),
   but I don't think it is necessary if we add other PG_TRY block in make_build_data()
   and compute_expr_stats(). For example in make_build_data():
   
   +                       PG_TRY();
   +                       {
   +                               datum = ExecEvalExpr(exprstate,
   +                                                                        GetPerTupleExprContext(estate),
   +                                                                        &isnull);
   +                               }
   +                       PG_CATCH();
   +                       {
   +                               ExecDropSingleTupleTableSlot(slot);
   +                               FreeExecutorState(estate);
   +                               PG_RE_THROW();
   +                       }
   +                       PG_END_TRY();
   
   Also, we could add tests for extended statistics that do not involve virtual generated
   columns, since those are not the cause root of the issue. In addition, it might be useful
   to verify that non-skipped extended statistics are still computed successfully.
   For example:
   
   +CREATE TABLE expr_err (a int);
   +INSERT INTO expr_err VALUES (1), (2), (3);
   +CREATE STATISTICS expr_err_s1 ON ((a/0)) FROM expr_err;
   +CREATE STATISTICS expr_err_s2 ON (a/0),(a+1) FROM expr_err;
   +CREATE STATISTICS expr_err_s3 ON ((a+1)) FROM expr_err;
   +ANALYZE expr_err;  -- should warn, not fail
   +SELECT statistics_name from pg_stats_ext x
   +    WHERE tablename = 'expr_err' ORDER BY ROW(x.*);
   
   Regards,
   Yugo Nagata
   
   -- 
   Yugo Nagata <nagata@sraoss.co.jp>
   
   
   
   

6. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> — 2026-05-03T18:04:59Z

   Hi,
   
   On Tue, Apr 28, 2026 at 2:14 AM Yugo Nagata <nagata@sraoss.co.jp> wrote:
   
   > On Tue, 14 Apr 2026 00:16:42 -0700
   > SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:
   >
   > > Hi
   > >
   > > On Mon, Apr 13, 2026 at 11:24 PM Yugo Nagata <nagata@sraoss.co.jp>
   > wrote:
   > >
   > > > On Sat, 11 Apr 2026 17:33:13 +0100
   > > > Dean Rasheed <dean.a.rasheed@gmail.com> wrote:
   > > >
   > > > > On Fri, 10 Apr 2026 at 21:19, SATYANARAYANA NARLAPURAM
   > > > > <satyanarlapuram@gmail.com> wrote:
   > > > > >
   > > > > > PG19 added support for stats on virtual generated columns [1].
   > > > Creating extended statistics on a virtual generated column whose
   > expression
   > > > can raise an error leads to ANALYZE failing repeatedly, and autovacuum
   > > > retrying indefinitely. This floods the server logs and also wastes
   > > > resources. Vacuum analyze on that column (without extended stats)
   > succeeds.
   > > > > >
   > > > >
   > > > > True, though this is nothing new. The same thing can happen with
   > > > > expression statistics on an expression that raises an error, which
   > has
   > > > > been possible since PG14.
   > > >
   > > > Yes, this issue is not new, and I’m not aware of a way to prevent it a
   > > > priori.
   > > >
   > > > >
   > > > > > In order to avoid retry storms, I think we have two options. (1)
   > > > skipping the offending row from the sample, (2) skipping the extended
   > stats
   > > > computation for that table with a warning message. At least this avoid
   > > > autovacuum infinite retry. Attached a draft patch for the option (2).
   > > > Thoughts?
   > > > > >
   > > > >
   > > > > I'm not sure. The default retry interval is 1 minute, so it won't
   > > > > exactly be a flood of messages. Also, if the error only occurs for a
   > > > > small subset of rows, it's possible that retrying might succeed.
   > > >
   > > > I think it would be good to skip ANALYZE for the extended statistics
   > that
   > > > cause
   > > > errors and just emit a warning, rather than aborting ANALYZE for the
   > > > entire table.
   > > > It seems reasonable to treat this as the user’s responsibility to
   > notice
   > > > the warning
   > > > and address the underlying issue.
   > > >
   > >
   > > Yugo, thanks for the comments. Could you please review the v1 patch when
   > you
   > > get a chance. It is in the direction you suggested.
   >
   > I've looked into the patch and have some comments.
   >
   > The child ResourceOwner is created and released in
   > BuildRelationExtStatistics(),
   > but I don't think it is necessary if we add other PG_TRY block in
   > make_build_data()
   > and compute_expr_stats(). For example in make_build_data():
   >
   > +                       PG_TRY();
   > +                       {
   > +                               datum = ExecEvalExpr(exprstate,
   > +
   > GetPerTupleExprContext(estate),
   > +
   > &isnull);
   > +                               }
   > +                       PG_CATCH();
   > +                       {
   > +                               ExecDropSingleTupleTableSlot(slot);
   > +                               FreeExecutorState(estate);
   > +                               PG_RE_THROW();
   > +                       }
   > +                       PG_END_TRY();
   >
   
   Thanks, for reviewing the patch. Agreed, please find the updated patch.
   
   
   > Also, we could add tests for extended statistics that do not involve
   > virtual generated
   > columns, since those are not the cause root of the issue. In addition, it
   > might be useful
   > to verify that non-skipped extended statistics are still computed
   > successfully.
   > For example:
   >
   > +CREATE TABLE expr_err (a int);
   > +INSERT INTO expr_err VALUES (1), (2), (3);
   > +CREATE STATISTICS expr_err_s1 ON ((a/0)) FROM expr_err;
   > +CREATE STATISTICS expr_err_s2 ON (a/0),(a+1) FROM expr_err;
   > +CREATE STATISTICS expr_err_s3 ON ((a+1)) FROM expr_err;
   > +ANALYZE expr_err;  -- should warn, not fail
   > +SELECT statistics_name from pg_stats_ext x
   > +    WHERE tablename = 'expr_err' ORDER BY ROW(x.*);
   >
   
   Added these tests as well in the v2 patch.
   
    Thanks,
   Satya
   

7. Re: Infinite Autovacuum loop caused by failing virtual generated column expression

   Yugo Nagata <nagata@sraoss.co.jp> — 2026-05-12T10:30:11Z

   On Sun, 3 May 2026 11:04:59 -0700
   SATYANARAYANA NARLAPURAM <satyanarlapuram@gmail.com> wrote:
   
   Thank you for updating the patch!
   
   There are a few comments on v2 patch.
   
   In compute_expr_stats(),
   
   +		PG_CATCH();
   +		{
   +			ExecDropSingleTupleTableSlot(slot);
   +			FreeExecutorState(estate);
   +			PG_RE_THROW();
    		}
   
   Should we switch the context to expr_context before releasing
   slot and estate? Also, should we call MemoryContextDelete(expr_context)
   to release palloc'ed memory in the loop?
   (In fact, the error would not be cached in compute_expr_stats(), though,
    since it would be cached in make_build_data() if any.)
   
   Should we switch to expr_context before releasing slot and estate? 
   Also, should we call MemoryContextDelete(expr_context) to release
   memory allocated by palloc() in the loop?
   
   (In practice, errors would not catched in compute_expr_stats() itself,
    since they would already be caught in make_build_data() if any.)
   
   +       /*
   +        * Wrap expression evaluation and stats computation in PG_TRY so
   +        * that errors from evaluating expressions (e.g. division by zero
   +        * in virtual generated columns) don't cause ANALYZE to fail
   +        * entirely.  Skip the statistics object and issue a WARNING
   +        * instead.
   
   How about rewriting the comments to reflect the more general case?
   
   Extended statistics involving virtual generated columns are a somewhat
   special case, while errors in expression statistics seem more common
   in practice. The same applies to the commit message as well.
   
   
   Regards,
   Yugo Nagata
   
   -- 
   Yugo Nagata <nagata@sraoss.co.jp>