Horton

Thread

Commits

  1. Go back to returning int from ereport auxiliary functions.

  2. Improve the internal implementation of ereport().

  3. Re-implement the ereport() macro using __VA_ARGS__.

  4. Add missing errcode() in a few ereport calls.

  1. Missing errcode() in ereport

    Masahiko Sawada <masahiko.sawada@2ndquadrant.com> — 2020-03-17T08:37:51Z

    Hi,

In PageIsVerified() we report a WARNING as follows:

        ereport(WARNING,
                (ERRCODE_DATA_CORRUPTED,
                 errmsg("page verification failed, calculated checksum
%u but expected %u",
                        checksum, p->pd_checksum)));

However the error message won't have sql error code due to missing
errcode(). As far as I can see there are four places:

$ git grep "(ERRCODE" | grep -v errcode
contrib/adminpack/adminpack.c:
(ERRCODE_DUPLICATE_FILE,
contrib/adminpack/adminpack.c:                          (ERRCODE_DUPLICATE_FILE,
contrib/adminpack/adminpack.c:
 (ERRCODE_UNDEFINED_FILE,
src/backend/storage/page/bufpage.c:
(ERRCODE_DATA_CORRUPTED,
src/pl/plpgsql/src/pl_exec.c:           else if
(ERRCODE_IS_CATEGORY(sqlerrstate) &&

Attached patch add errcode() to these places.

Regards,

-- 
Masahiko Sawada            http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  2. Re: Missing errcode() in ereport

    Amit Kapila <amit.kapila16@gmail.com> — 2020-03-17T08:59:47Z

    On Tue, Mar 17, 2020 at 2:08 PM Masahiko Sawada
<masahiko.sawada@2ndquadrant.com> wrote:
>
> Hi,
>
> In PageIsVerified() we report a WARNING as follows:
>
>         ereport(WARNING,
>                 (ERRCODE_DATA_CORRUPTED,
>                  errmsg("page verification failed, calculated checksum
> %u but expected %u",
>                         checksum, p->pd_checksum)));
>
> However the error message won't have sql error code due to missing
> errcode(). As far as I can see there are four places:
>
> $ git grep "(ERRCODE" | grep -v errcode
> contrib/adminpack/adminpack.c:
> (ERRCODE_DUPLICATE_FILE,
> contrib/adminpack/adminpack.c:                          (ERRCODE_DUPLICATE_FILE,
> contrib/adminpack/adminpack.c:
>  (ERRCODE_UNDEFINED_FILE,
> src/backend/storage/page/bufpage.c:
> (ERRCODE_DATA_CORRUPTED,
> src/pl/plpgsql/src/pl_exec.c:           else if
> (ERRCODE_IS_CATEGORY(sqlerrstate) &&
>
> Attached patch add errcode() to these places.
>

+1.  This looks like an oversight to me.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  3. Re: Missing errcode() in ereport

    Julien Rouhaud <rjuju123@gmail.com> — 2020-03-17T09:26:57Z

    On Tue, Mar 17, 2020 at 10:00 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Mar 17, 2020 at 2:08 PM Masahiko Sawada
> <masahiko.sawada@2ndquadrant.com> wrote:
> >
> > Hi,
> >
> > In PageIsVerified() we report a WARNING as follows:
> >
> >         ereport(WARNING,
> >                 (ERRCODE_DATA_CORRUPTED,
> >                  errmsg("page verification failed, calculated checksum
> > %u but expected %u",
> >                         checksum, p->pd_checksum)));
> >
> > However the error message won't have sql error code due to missing
> > errcode(). As far as I can see there are four places:
> >
> > $ git grep "(ERRCODE" | grep -v errcode
> > contrib/adminpack/adminpack.c:
> > (ERRCODE_DUPLICATE_FILE,
> > contrib/adminpack/adminpack.c:                          (ERRCODE_DUPLICATE_FILE,
> > contrib/adminpack/adminpack.c:
> >  (ERRCODE_UNDEFINED_FILE,
> > src/backend/storage/page/bufpage.c:
> > (ERRCODE_DATA_CORRUPTED,
> > src/pl/plpgsql/src/pl_exec.c:           else if
> > (ERRCODE_IS_CATEGORY(sqlerrstate) &&
> >
> > Attached patch add errcode() to these places.
> >
>
> +1.  This looks like an oversight to me.

good catch!  And patch LGTM.

  4. Re: Missing errcode() in ereport

    Michael Paquier <michael@paquier.xyz> — 2020-03-17T09:58:39Z

    On Tue, Mar 17, 2020 at 10:26:57AM +0100, Julien Rouhaud wrote:
> On Tue, Mar 17, 2020 at 10:00 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>> +1.  This looks like an oversight to me.
> 
> good catch!  And patch LGTM.

Definitely an oversight.  All stable branches down to 9.5 have
mistakes in the same area, with nothing extra by grepping around.
Amit, I guess that you will take care of it?
--
Michael

  5. Re: Missing errcode() in ereport

    Amit Kapila <amit.kapila16@gmail.com> — 2020-03-17T10:16:17Z

    On Tue, Mar 17, 2020 at 3:28 PM Michael Paquier <michael@paquier.xyz> wrote:
>
> On Tue, Mar 17, 2020 at 10:26:57AM +0100, Julien Rouhaud wrote:
> > On Tue, Mar 17, 2020 at 10:00 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >> +1.  This looks like an oversight to me.
> >
> > good catch!  And patch LGTM.
>
> Definitely an oversight.  All stable branches down to 9.5 have
> mistakes in the same area, with nothing extra by grepping around.
> Amit, I guess that you will take care of it?
>

Yes, I will unless I see any objections in a day or so.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  6. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-17T14:09:18Z

    Amit Kapila <amit.kapila16@gmail.com> writes:
> On Tue, Mar 17, 2020 at 3:28 PM Michael Paquier <michael@paquier.xyz> wrote:
>> Definitely an oversight.  All stable branches down to 9.5 have
>> mistakes in the same area, with nothing extra by grepping around.
>> Amit, I guess that you will take care of it?

> Yes, I will unless I see any objections in a day or so.

No need to wait, it's a pretty obvious thinko.

We might want to spend some effort thinking how to find or prevent
additional bugs of the same ilk ... but correcting the ones already
found needn't wait on that.

			regards, tom lane

  7. Re: Missing errcode() in ereport

    Amit Kapila <amit.kapila16@gmail.com> — 2020-03-18T03:31:28Z

    On Tue, Mar 17, 2020 at 7:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Amit Kapila <amit.kapila16@gmail.com> writes:
> > On Tue, Mar 17, 2020 at 3:28 PM Michael Paquier <michael@paquier.xyz> wrote:
> >> Definitely an oversight.  All stable branches down to 9.5 have
> >> mistakes in the same area, with nothing extra by grepping around.
> >> Amit, I guess that you will take care of it?
>
> > Yes, I will unless I see any objections in a day or so.
>
> No need to wait, it's a pretty obvious thinko.
>

Okay, I will push in some time.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  8. Re: Missing errcode() in ereport

    Amit Kapila <amit.kapila16@gmail.com> — 2020-03-18T04:57:37Z

    On Wed, Mar 18, 2020 at 9:01 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Mar 17, 2020 at 7:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Amit Kapila <amit.kapila16@gmail.com> writes:
> > > On Tue, Mar 17, 2020 at 3:28 PM Michael Paquier <michael@paquier.xyz> wrote:
> > >> Definitely an oversight.  All stable branches down to 9.5 have
> > >> mistakes in the same area, with nothing extra by grepping around.
> > >> Amit, I guess that you will take care of it?
> >
> > > Yes, I will unless I see any objections in a day or so.
> >
> > No need to wait, it's a pretty obvious thinko.
> >
>
> Okay, I will push in some time.
>

Pushed.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com

  9. Re: Missing errcode() in ereport

    Masahiko Sawada <masahiko.sawada@2ndquadrant.com> — 2020-03-19T03:31:44Z

    On Wed, 18 Mar 2020 at 13:57, Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Mar 18, 2020 at 9:01 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Tue, Mar 17, 2020 at 7:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >
> > > Amit Kapila <amit.kapila16@gmail.com> writes:
> > > > On Tue, Mar 17, 2020 at 3:28 PM Michael Paquier <michael@paquier.xyz> wrote:
> > > >> Definitely an oversight.  All stable branches down to 9.5 have
> > > >> mistakes in the same area, with nothing extra by grepping around.
> > > >> Amit, I guess that you will take care of it?
> > >
> > > > Yes, I will unless I see any objections in a day or so.
> > >
> > > No need to wait, it's a pretty obvious thinko.
> > >
> >
> > Okay, I will push in some time.
> >
>
> Pushed.

Thank you!


Regards,

--
Masahiko Sawada            http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  10. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-19T17:59:02Z

    Hi,

On 2020-03-17 10:09:18 -0400, Tom Lane wrote:
> We might want to spend some effort thinking how to find or prevent
> additional bugs of the same ilk ...

Yea, that'd be good.  Trying to help people new to postgres write their
first patches I found that ereport is very confusing to them - largely
because the syntax doesn't make much sense. Made worse by the compiler
error messages being terrible in many cases.

Not sure there's much we can do without changing ereport's "signature"
though :(

Regards,

Andres

  11. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-19T18:07:04Z

    Andres Freund <andres@anarazel.de> writes:
> On 2020-03-17 10:09:18 -0400, Tom Lane wrote:
>> We might want to spend some effort thinking how to find or prevent
>> additional bugs of the same ilk ...

> Yea, that'd be good.  Trying to help people new to postgres write their
> first patches I found that ereport is very confusing to them - largely
> because the syntax doesn't make much sense. Made worse by the compiler
> error messages being terrible in many cases.

> Not sure there's much we can do without changing ereport's "signature"
> though :(

Now that we can rely on having varargs macros, I think we could
stop requiring the extra level of parentheses, ie instead of

        ereport(ERROR,
                (errcode(ERRCODE_DIVISION_BY_ZERO),
                 errmsg("division by zero")));

it could be just

        ereport(ERROR,
                errcode(ERRCODE_DIVISION_BY_ZERO),
                errmsg("division by zero"));

(The old syntax had better still work, of course.  I'm not advocating
running around and changing existing calls.)

I'm not sure that this'd really move the goalposts much in terms of making
it any less confusing, but possibly it would improve the compiler errors?
Do you have any concrete examples of crummy error messages?

			regards, tom lane

  12. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-19T23:04:46Z

    Hi,

On 2020-03-19 14:07:04 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2020-03-17 10:09:18 -0400, Tom Lane wrote:
> >> We might want to spend some effort thinking how to find or prevent
> >> additional bugs of the same ilk ...
> 
> > Yea, that'd be good.  Trying to help people new to postgres write their
> > first patches I found that ereport is very confusing to them - largely
> > because the syntax doesn't make much sense. Made worse by the compiler
> > error messages being terrible in many cases.
> 
> > Not sure there's much we can do without changing ereport's "signature"
> > though :(
> 
> Now that we can rely on having varargs macros, I think we could
> stop requiring the extra level of parentheses, ie instead of
> 
>         ereport(ERROR,
>                 (errcode(ERRCODE_DIVISION_BY_ZERO),
>                  errmsg("division by zero")));
> 
> it could be just
> 
>         ereport(ERROR,
>                 errcode(ERRCODE_DIVISION_BY_ZERO),
>                 errmsg("division by zero"));
> 
> (The old syntax had better still work, of course.  I'm not advocating
> running around and changing existing calls.)

I think that'd be an improvement, because:

> I'm not sure that this'd really move the goalposts much in terms of making
> it any less confusing, but possibly it would improve the compiler errors?
> Do you have any concrete examples of crummy error messages?

ane of the ones I saw confuse people is just:
/home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: error: ‘ereport’ undeclared (first use in this function); did you mean ‘rresvport’?
 3727 |    ereport(FATAL,
      |    ^~~~~~~
      |    rresvport
/home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: note: each undeclared identifier is reported only once for each function it appears in

because the extra parens haven't been added.


I personally actually hit a variant of that on a semi-regular basis:
Closing the parens for "rest" early, as there's just so many parens in
ereports, especially if an errmsg argument is a function call
itself. Which leads to a variation of the above error message.  I know
how to address it, obviously, but I do find it somewhat annoying to deal
with.

Another one I've both seen and committed byself is converting an elog to
an ereport, and not adding an errcode() around the error code - which
silently looks like it works.

Greetings,

Andres Freund

  13. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-19T23:32:55Z

    Andres Freund <andres@anarazel.de> writes:
> On 2020-03-19 14:07:04 -0400, Tom Lane wrote:
>> Now that we can rely on having varargs macros, I think we could
>> stop requiring the extra level of parentheses,

> I think that'd be an improvement, because:
> ane of the ones I saw confuse people is just:
> /home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: error: ‘ereport’ undeclared (first use in this function); did you mean ‘rresvport’?
>  3727 |    ereport(FATAL,
>       |    ^~~~~~~
>       |    rresvport
> /home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: note: each undeclared identifier is reported only once for each function it appears in
> because the extra parens haven't been added.

Ah, so the macro isn't expanded at all if it appears to have more than
two parameters?  That seems odd, but I suppose some compilers might
work that way.  Switching to varargs would improve that for sure.

> Another one I've both seen and committed byself is converting an elog to
> an ereport, and not adding an errcode() around the error code - which
> silently looks like it works.

You mean not adding errmsg() around the error string?  Yeah, I can
believe that one easily.  It's sort of the same problem as forgetting
to wrap errcode() around the ERRCODE_ constant.

I think that at least some compilers will complain about side-effect-free
subexpressions of a comma expression.  Could we restructure things so
that the errcode/errmsg/etc calls form a standalone comma expression
rather than appearing to be arguments of a varargs function?  The
compiler's got no hope of realizing there's anything wrong when it
thinks it's passing an integer or string constant to a function it knows
nothing about.  But if it could see that nothing at all is being done with
the constant, maybe we'd get somewhere.

			regards, tom lane

  14. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-20T01:03:17Z

    I wrote:
> I think that at least some compilers will complain about side-effect-free
> subexpressions of a comma expression.  Could we restructure things so
> that the errcode/errmsg/etc calls form a standalone comma expression
> rather than appearing to be arguments of a varargs function?

Yeah, the attached quick-hack patch seems to work really well for this.
I find that this now works:

    ereport(ERROR,
            errcode(ERRCODE_DIVISION_BY_ZERO),
            errmsg("foo"));

where before it gave the weird error you quoted.  Also, these both
draw warnings about side-effect-free expressions:

    ereport(ERROR,
            ERRCODE_DIVISION_BY_ZERO,
            errmsg("foo"));

    ereport(ERROR,
            "%d", 42);

With gcc you just need -Wall to get such warnings, and clang
seems to give them by default.

As a nice side benefit, the backend gets a couple KB smaller from
removal of errfinish's useless dummy argument.

I think that we could now also change all the helper functions
(errmsg et al) to return void instead of a dummy value, but that
would make the patch a lot longer and probably not move the
goalposts much in terms of error detection.

It also looks like we could use the same trick to get plain elog()
to have the behavior of not evaluating its arguments when it's
not going to print anything.  I've not poked at that yet either.

			regards, tom lane

  15. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-20T01:59:08Z

    Hi,

On 2020-03-19 19:32:55 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2020-03-19 14:07:04 -0400, Tom Lane wrote:
> >> Now that we can rely on having varargs macros, I think we could
> >> stop requiring the extra level of parentheses,
>
> > I think that'd be an improvement, because:
> > ane of the ones I saw confuse people is just:
> > /home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: error: ‘ereport’ undeclared (first use in this function); did you mean ‘rresvport’?
> >  3727 |    ereport(FATAL,
> >       |    ^~~~~~~
> >       |    rresvport
> > /home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: note: each undeclared identifier is reported only once for each function it appears in
> > because the extra parens haven't been added.
>
> Ah, so the macro isn't expanded at all if it appears to have more than
> two parameters?  That seems odd, but I suppose some compilers might
> work that way.  Switching to varargs would improve that for sure.

Yea. Newer gcc versions do warn about a parameter mismatch count, which
helps.  I'm not sure what the preprocessor / compiler should do intead?

FWIW, clang also doesn't expand the macro.


> > Another one I've both seen and committed byself is converting an elog to
> > an ereport, and not adding an errcode() around the error code - which
> > silently looks like it works.
>
> You mean not adding errmsg() around the error string?  Yeah, I can
> believe that one easily.  It's sort of the same problem as forgetting
> to wrap errcode() around the ERRCODE_ constant.

Both of these, I think.


> Could we restructure things so that the errcode/errmsg/etc calls form
> a standalone comma expression rather than appearing to be arguments of
> a varargs function?  The compiler's got no hope of realizing there's
> anything wrong when it thinks it's passing an integer or string
> constant to a function it knows nothing about.  But if it could see
> that nothing at all is being done with the constant, maybe we'd get
> somewhere.

Worth a try. Not doing a pointless varargs call could also end up
reducing elog overhead a bit (right now we push a lot of 0 as vararg
arguments for no reason).

A quick hack suggests that it works:

/home/andres/src/postgresql/src/backend/tcop/postgres.c: In function ‘process_postgres_switches’:
/home/andres/src/postgresql/src/backend/tcop/postgres.c:3728:27: warning: left-hand operand of comma expression has no effect [-Wunused-value]
 3728 |      (ERRCODE_SYNTAX_ERROR,
      |                           ^
/home/andres/src/postgresql/src/include/utils/elog.h:126:4: note: in definition of macro ‘ereport_domain’
  126 |    rest; \
      |    ^~~~
/home/andres/src/postgresql/src/backend/tcop/postgres.c:3727:4: note: in expansion of macro ‘ereport’
 3727 |    ereport(FATAL,
      |    ^~~~~~~


and in an optimized build the resulting code indeed looks a bit better:

before:
   text	   data	    bss	    dec	    hex	filename
8462795	 176128	 204464	8843387	 86f07b	src/backend/postgres

Looking at FloatExceptionHandler as an example:

2860		/* We're not returning, so no need to save errno */
2861		ereport(ERROR,
   0x0000000000458bc0 <+0>:	push   %rbp
   0x0000000000458bc1 <+1>:	mov    $0xb2d,%edx
   0x0000000000458bc6 <+6>:	lea    0x214c9b(%rip),%rsi        # 0x66d868
   0x0000000000458bcd <+13>:	mov    %rsp,%rbp
   0x0000000000458bd0 <+16>:	push   %r13
   0x0000000000458bd2 <+18>:	xor    %r8d,%r8d
   0x0000000000458bd5 <+21>:	lea    0x2d9dc4(%rip),%rcx        # 0x7329a0 <__func__.41598>
   0x0000000000458bdc <+28>:	push   %r12
   0x0000000000458bde <+30>:	mov    $0x14,%edi
   0x0000000000458be3 <+35>:	callq  0x5a8c00 <errstart>
   0x0000000000458be8 <+40>:	lea    0x214e59(%rip),%rdi        # 0x66da48
   0x0000000000458bef <+47>:	xor    %eax,%eax
   0x0000000000458bf1 <+49>:	callq  0x5acb00 <errdetail>
   0x0000000000458bf6 <+54>:	mov    %eax,%r13d
   0x0000000000458bf9 <+57>:	lea    0x1bf7fb(%rip),%rdi        # 0x6183fb
   0x0000000000458c00 <+64>:	xor    %eax,%eax
   0x0000000000458c02 <+66>:	callq  0x5ac710 <errmsg>
   0x0000000000458c07 <+71>:	mov    $0x1020082,%edi
   0x0000000000458c0c <+76>:	mov    %eax,%r12d
   0x0000000000458c0f <+79>:	callq  0x5ac5b0 <errcode>
   0x0000000000458c14 <+84>:	mov    %eax,%edi
   0x0000000000458c16 <+86>:	mov    %r13d,%edx
   0x0000000000458c19 <+89>:	mov    %r12d,%esi
   0x0000000000458c1c <+92>:	xor    %eax,%eax
   0x0000000000458c1e <+94>:	callq  0x5ac020 <errfinish>

vs after:
   text	   data	    bss	    dec	    hex	filename
8395731	 176128	 204464	8776323	 85ea83	src/backend/postgres
2861		ereport(ERROR,
   0x0000000000449dd0 <+0>:	push   %rbp
   0x0000000000449dd1 <+1>:	xor    %r8d,%r8d
   0x0000000000449dd4 <+4>:	lea    0x2d8a65(%rip),%rcx        # 0x722840 <__func__.41598>
   0x0000000000449ddb <+11>:	mov    %rsp,%rbp
   0x0000000000449dde <+14>:	mov    $0xb2d,%edx
   0x0000000000449de3 <+19>:	lea    0x2137ee(%rip),%rsi        # 0x65d5d8
   0x0000000000449dea <+26>:	mov    $0x14,%edi
   0x0000000000449def <+31>:	callq  0x5992e0 <errstart>
   0x0000000000449df4 <+36>:	mov    $0x1020082,%edi
   0x0000000000449df9 <+41>:	callq  0x59cc80 <errcode>
   0x0000000000449dfe <+46>:	lea    0x1be496(%rip),%rdi        # 0x60829b
   0x0000000000449e05 <+53>:	xor    %eax,%eax
   0x0000000000449e07 <+55>:	callq  0x59cde0 <errmsg>
   0x0000000000449e0c <+60>:	lea    0x2139a5(%rip),%rdi        # 0x65d7b8
   0x0000000000449e13 <+67>:	xor    %eax,%eax
   0x0000000000449e15 <+69>:	callq  0x59d1d0 <errdetail>
   0x0000000000449e1a <+74>:	callq  0x59c700 <errfinish>


I wonder if it'd become a relevant backpatch pain if we started to have
some ereports() without the additional parens in 13+. Would it perhaps
make sense to backpatch just the part that removes the need for the
parents, but not the return type changes?

This is patch 0001.


We can also remove elog() support code now, because with __VA_ARGS__
support it's really just a wrapper for ereport(elevel,
errmsg(__VA_ARGS_)).  This is patch 0002.


I think it might also be a good idea to move __FILE__, __LINE__,
PG_FUNCNAME_MACRO, domain from being parameters to errstart to
errfinish. For elevel < ERROR its sad that we currently pass them even
if we don't emit the message.  This is patch 0003.


I wonder if its worth additionally ensuring that errcode, errmsg,
... are only called within errstart/errfinish? We could have errstart()
return the error being "prepared" and store it in a local variable, and
make errmsg() etc macros that internally pass that variable to the
"real" errmsg() etc. Like

#define errmsg(...) errmsg_impl(current_error, __VA_ARGS__)

and set up current_error in ereport_domain() roughly like
    do {
        ErrorData  *current_error_;
        if ((current_error_ = errstart(elevel, the, rest)) != NULL)
        {
            ...
            errfinish()
        }
        ...

probably not worth it?

Greetings,

Andres Freund

  16. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-20T02:12:04Z

    Hi,

On 2020-03-19 21:03:17 -0400, Tom Lane wrote:
> I wrote:
> > I think that at least some compilers will complain about side-effect-free
> > subexpressions of a comma expression.  Could we restructure things so
> > that the errcode/errmsg/etc calls form a standalone comma expression
> > rather than appearing to be arguments of a varargs function?
>
> Yeah, the attached quick-hack patch seems to work really well for
> this.

Heh, you're too fast. I just sent something similar, and a few followup
patches.

What is your thinking about pain around backpatching it might introduce?
We can't easily backpatch support for ereport-without-extra-parens, I
think, because it needs __VA_ARGS__?


> As a nice side benefit, the backend gets a couple KB smaller from
> removal of errfinish's useless dummy argument.

I don't think it's the removal of the dummy parameter itself that
constitutes most of the savings, but instead it's not having to push the
return value of errmsg(), errcode(), et al as vararg arguments.


> I think that we could now also change all the helper functions
> (errmsg et al) to return void instead of a dummy value, but that
> would make the patch a lot longer and probably not move the
> goalposts much in terms of error detection.

I did include that in my prototype patch. Agree that it's not necessary
for the error detection capability, but it seems misleading to leave the
return values around if they're not actually needed.


> It also looks like we could use the same trick to get plain elog()
> to have the behavior of not evaluating its arguments when it's
> not going to print anything.  I've not poked at that yet either.

I've included a patch for that. I think it's now sufficient to
#define elog(elevel, ...)  \
	ereport(elevel, errmsg(__VA_ARGS__))

which imo is quite nice, as it allows us to get rid of a lot of
duplication.

Greetings,

Andres Freund

  17. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-20T02:32:30Z

    Andres Freund <andres@anarazel.de> writes:
> I wonder if it'd become a relevant backpatch pain if we started to have
> some ereports() without the additional parens in 13+.

Yeah, it would be a nasty backpatch hazard.

> Would it perhaps
> make sense to backpatch just the part that removes the need for the
> parents, but not the return type changes?

I was just looking into that.  It would be pretty painless to do it
in v12, but before that we weren't requiring C99.  Having said that,
trolling the buildfarm database shows exactly zero active members
that don't report having __VA_ARGS__, in the branches where we test
that.  (And pg_config.h.win32 was assuming that MSVC had that, too.)

Could we get away with moving the compiler goalposts for the back
branches?  I dunno, but it's a fact that we aren't testing anymore
with any compilers that would complain about unconditional use of
__VA_ARGS__.  So it might be broken already and we wouldn't know it.
(I suspect the last buildfarm animal that would've complained about
this was pademelon, which I retired more than a year ago IIRC.)

> We can also remove elog() support code now, because with __VA_ARGS__
> support it's really just a wrapper for ereport(elevel,
> errmsg(__VA_ARGS_)).  This is patch 0002.

Yeah, something similar had occurred to me but I didn't write the patch
yet.  Note it should be errmsg_internal(); also, does the default
for errcode come out the same?

> I think it might also be a good idea to move __FILE__, __LINE__,
> PG_FUNCNAME_MACRO, domain from being parameters to errstart to
> errfinish. For elevel < ERROR its sad that we currently pass them even
> if we don't emit the message.  This is patch 0003.

Oh, that's a good idea that hadn't occurred to me.

> I wonder if its worth additionally ensuring that errcode, errmsg,
> ... are only called within errstart/errfinish?

Meh.  That's wrong at least for errcontext(), and I'm not sure it's
really worth anything to enforce it for the others.

I think the key decision we'd have to make to move forward on this
is to decide whether it's still project style to prefer the extra
parens, or whether we want new code to do without them going
forward.  If we don't want to risk requiring __VA_ARGS__ for the
old branches then I'd vote in favor of keeping the parens as
preferred style, at least till v11 is out of support.  If we do
change that in the back branches then there'd be reason to prefer
to go without parens.  New coders might still be confused about
why there are all these calls with the useless parens, though.

			regards, tom lane

  18. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-20T04:56:37Z

    Hi,

On 2020-03-19 22:32:30 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I wonder if it'd become a relevant backpatch pain if we started to have
> > some ereports() without the additional parens in 13+.
> 
> Yeah, it would be a nasty backpatch hazard.
> 
> > Would it perhaps
> > make sense to backpatch just the part that removes the need for the
> > parents, but not the return type changes?
> 
> I was just looking into that.  It would be pretty painless to do it
> in v12, but before that we weren't requiring C99.  Having said that,
> trolling the buildfarm database shows exactly zero active members
> that don't report having __VA_ARGS__, in the branches where we test
> that.  (And pg_config.h.win32 was assuming that MSVC had that, too.)
> 
> Could we get away with moving the compiler goalposts for the back
> branches?  I dunno, but it's a fact that we aren't testing anymore
> with any compilers that would complain about unconditional use of
> __VA_ARGS__.  So it might be broken already and we wouldn't know it.

FWIW, I did grep for unprotected uses, and  didn't find anything.


> (I suspect the last buildfarm animal that would've complained about
> this was pademelon, which I retired more than a year ago IIRC.)

I guess a query that searches the logs backwards for animals without
__VA_ARGS__ would be a good idea?


> > We can also remove elog() support code now, because with __VA_ARGS__
> > support it's really just a wrapper for ereport(elevel,
> > errmsg(__VA_ARGS_)).  This is patch 0002.
> 
> Yeah, something similar had occurred to me but I didn't write the patch
> yet.  Note it should be errmsg_internal();

Oh, right.


> also, does the default for errcode come out the same?

I think so - there's no distinct code setting sqlerrcode in
elog_start/finish. That already relied on errstart().


> > I wonder if its worth additionally ensuring that errcode, errmsg,
> > ... are only called within errstart/errfinish?
> 
> Meh.  That's wrong at least for errcontext(), and I'm not sure it's
> really worth anything to enforce it for the others.

Yea, I'm not convinced either. Especially after changing the err* return
types to void, as that presumably will cause errors with a lot of
incorrect parens, e.g. about a function with void return type as an
argument to errmsg().


> I think the key decision we'd have to make to move forward on this
> is to decide whether it's still project style to prefer the extra
> parens, or whether we want new code to do without them going
> forward.  If we don't want to risk requiring __VA_ARGS__ for the
> old branches then I'd vote in favor of keeping the parens as
> preferred style, at least till v11 is out of support.

Agreed.


> If we do change that in the back branches then there'd be reason to
> prefer to go without parens.  New coders might still be confused about
> why there are all these calls with the useless parens, though.

That seems to be an acceptable pain, from my POV.

Greetings,

Andres Freund

  19. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-20T14:40:21Z

    Andres Freund <andres@anarazel.de> writes:
> On 2020-03-19 22:32:30 -0400, Tom Lane wrote:
>> Could we get away with moving the compiler goalposts for the back
>> branches?  I dunno, but it's a fact that we aren't testing anymore
>> with any compilers that would complain about unconditional use of
>> __VA_ARGS__.  So it might be broken already and we wouldn't know it.

> FWIW, I did grep for unprotected uses, and  didn't find anything.

Yeah, I also grepped the v11 branch for that.

>> (I suspect the last buildfarm animal that would've complained about
>> this was pademelon, which I retired more than a year ago IIRC.)

> I guess a query that searches the logs backwards for animals without
> __VA_ARGS__ would be a good idea?

I did a more wide-ranging scan, looking at the 9.4 branch as far back
as 2015.  Indeed, pademelon is the only animal that ever reported
not having __VA_ARGS__ in that timeframe.

So I've got mixed emotions about this.  On the one hand, it seems
quite unlikely that anyone would ever object if we started requiring
__VA_ARGS__ in the back branches.  On the other hand, it's definitely
not project policy to change requirements like that in minor releases.
Also the actual benefit might not be much.  If anyone did mistakenly
back-patch a fix that included a paren-free ereport, the buildfarm
would point out the error soon enough.

I thought for a little bit about making the back branches define ereport
with "..." if HAVE__VA_ARGS and otherwise not, but if we did that then
the buildfarm would *not* complain about paren-free ereports in the back
branches.  I think that would inevitably lead to there soon being some,
so that we'd effectively be requiring __VA_ARGS__ anyway.  (I suppose
I could resurrect pademelon ... but who knows whether that old war
horse will keep working for another four years till v11 is retired.)

On balance I'm leaning towards keeping the parens as preferred style
for now, adjusting v12 so that the macro will allow paren omission
but we don't break ABI, and not touching the older branches.  But
if there's a consensus to require __VA_ARGS__ in the back branches,
I won't stand in the way.

			regards, tom lane

  20. Re: Missing errcode() in ereport

    Alvaro Herrera <alvherre@2ndquadrant.com> — 2020-03-20T14:42:02Z

    On 2020-Mar-19, Tom Lane wrote:

> I think the key decision we'd have to make to move forward on this
> is to decide whether it's still project style to prefer the extra
> parens, or whether we want new code to do without them going
> forward.  If we don't want to risk requiring __VA_ARGS__ for the
> old branches then I'd vote in favor of keeping the parens as
> preferred style, at least till v11 is out of support.  If we do
> change that in the back branches then there'd be reason to prefer
> to go without parens.  New coders might still be confused about
> why there are all these calls with the useless parens, though.

It seems fine to accept new code in pg14 without the extra parens.  All
existing committers are (or should be) used to the style with the
parens, so it's unlikely that we'll mess up when backpatching bugfixes;
and even if we do, the buildfarm would alert us to that soon enough.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  21. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-23T21:24:49Z

    I wrote:
> On balance I'm leaning towards keeping the parens as preferred style
> for now, adjusting v12 so that the macro will allow paren omission
> but we don't break ABI, and not touching the older branches.

Hearing no objections, I started to review Andres' patchset with
that plan in mind.  I noted two things that I don't agree with:

1. I think we should write the ereport macro as

        if (errstart(...)) \
            __VA_ARGS__, errfinish(...); \

as I had it, not

        if (errstart(...)) \
        { \
            __VA_ARGS__; \
            errfinish(...); \
        } \

as per Andres.  The reason is that I don't have as much faith in the
latter construct producing warnings for no-op expressions.

2. We cannot postpone the passing of the "domain" argument as Andres'
0003 patch proposes, because that has to be available to the auxiliary
error functions so they do message translation properly.  Maybe it'd
be possible to finagle things to postpone translation to the very end,
but the provisions for errcontext messages being translated in a different
domain would make that pretty ticklish.  Frankly I don't think it'd be
worth the complication.  There is a clear benefit in delaying the passing
of filename (since we can skip that strchr() call) but beyond that it
seems pretty marginal.

Other than that it looks pretty good.  I wrote some documentation
adjustments and re-split the patch into 0001, which is proposed for
back-patch into v12, and 0002 which would have to be HEAD only.

One thing I'm not totally sure about is whether we can rely on
this change:

-extern void errfinish(int dummy,...);
+extern void errfinish(void);

being fully ABI-transparent.  One would think that as long as errfinish
doesn't inspect its argument(s) it doesn't matter whether any are passed,
but maybe somewhere there's an architecture where the possible presence
of varargs arguments makes for a significant difference in the calling
convention?  We could leave that change out of the v12 patch if we're
worried about it.

			regards, tom lane

  22. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-23T21:44:46Z

    Hi,

On 2020-03-23 17:24:49 -0400, Tom Lane wrote:
> I wrote:
> > On balance I'm leaning towards keeping the parens as preferred style
> > for now, adjusting v12 so that the macro will allow paren omission
> > but we don't break ABI, and not touching the older branches.
> 
> Hearing no objections, I started to review Andres' patchset with
> that plan in mind.  I noted two things that I don't agree with:
> 
> 1. I think we should write the ereport macro as
> 
>         if (errstart(...)) \
>             __VA_ARGS__, errfinish(...); \
> 
> as I had it, not
> 
>         if (errstart(...)) \
>         { \
>             __VA_ARGS__; \
>             errfinish(...); \
>         } \
> 
> as per Andres.  The reason is that I don't have as much faith in the
> latter construct producing warnings for no-op expressions.

Hm. I don't think it'll be better, but I don't have a problem going with
yours either.


> 2. We cannot postpone the passing of the "domain" argument as Andres'
> 0003 patch proposes, because that has to be available to the auxiliary
> error functions so they do message translation properly.

Ah, good point.


I wondered before whether there's a way we could move the elevel check
in errstart to the macro. For it to be a win we'd presumably have to
have a "synthesized" log_level variable, basically
min(log_min_messages, client_min_messages, ERROR).

Probably not worth it.


> Maybe it'd be possible to finagle things to postpone translation to
> the very end, but the provisions for errcontext messages being
> translated in a different domain would make that pretty ticklish.
> Frankly I don't think it'd be worth the complication.  There is a
> clear benefit in delaying the passing of filename (since we can skip
> that strchr() call) but beyond that it seems pretty marginal.

Fair enough.


> Other than that it looks pretty good.  I wrote some documentation
> adjustments and re-split the patch into 0001, which is proposed for
> back-patch into v12, and 0002 which would have to be HEAD only.

Cool.


> One thing I'm not totally sure about is whether we can rely on
> this change:
> 
> -extern void errfinish(int dummy,...);
> +extern void errfinish(void);
> 
> being fully ABI-transparent.  One would think that as long as errfinish
> doesn't inspect its argument(s) it doesn't matter whether any are passed,
> but maybe somewhere there's an architecture where the possible presence
> of varargs arguments makes for a significant difference in the calling
> convention?  We could leave that change out of the v12 patch if we're
> worried about it.

I would vote for leaving that out of the v12 patch. I'm less worried
about odd architectures, and more about sanitizers and/or compilers
emitting "security enhanced" code checking signatures match etc
("control flow integrity").

Greetings,

Andres Freund

  23. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-23T21:57:28Z

    Andres Freund <andres@anarazel.de> writes:
> I wondered before whether there's a way we could move the elevel check
> in errstart to the macro. For it to be a win we'd presumably have to
> have a "synthesized" log_level variable, basically
> min(log_min_messages, client_min_messages, ERROR).
> Probably not worth it.

Yeah, I don't really agree with that idea.  The whole business of which
elevels will trigger output is fundamentally policy, not mechanism,
and you do not want policy decisions embedded into ABI so that there
are hundreds of copies of them.  It's a loss for code size and a worse
loss if you ever want to change the behavior.

> On 2020-03-23 17:24:49 -0400, Tom Lane wrote:
>> One thing I'm not totally sure about is whether we can rely on
>> this change:
>> 
>> -extern void errfinish(int dummy,...);
>> +extern void errfinish(void);
>> 
>> being fully ABI-transparent.  One would think that as long as errfinish
>> doesn't inspect its argument(s) it doesn't matter whether any are passed,
>> but maybe somewhere there's an architecture where the possible presence
>> of varargs arguments makes for a significant difference in the calling
>> convention?  We could leave that change out of the v12 patch if we're
>> worried about it.

> I would vote for leaving that out of the v12 patch. I'm less worried
> about odd architectures, and more about sanitizers and/or compilers
> emitting "security enhanced" code checking signatures match etc
> ("control flow integrity").

Yeah, good point.  Let's skinny the v12 patch down to just macro
and docs changes, then, not touching elog.c at all.

I made a commitfest entry for this just to see what the cfbot
thinks of it, but if there are not objections we should go ahead
and push in a day or so, IMO.

			regards, tom lane

BTW, the CF app is showing me as the first author, even though
I definitely put you in first.  Guess it doesn't care about
author ordering ... is that a bug to be fixed?

  24. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-24T19:38:34Z

    On 2020-03-23 17:24:49 -0400, Tom Lane wrote:
> Hearing no objections, I started to review Andres' patchset with
> that plan in mind.

Thanks for pushing the first part!

  25. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-24T20:29:51Z

    Andres Freund <andres@anarazel.de> writes:
> On 2020-03-23 17:24:49 -0400, Tom Lane wrote:
>> Hearing no objections, I started to review Andres' patchset with
>> that plan in mind.

> Thanks for pushing the first part!

I pushed all of it, actually.

			regards, tom lane

  26. Re: Missing errcode() in ereport

    Thomas Munro <thomas.munro@gmail.com> — 2020-03-25T05:27:56Z

    On Wed, Mar 25, 2020 at 9:30 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2020-03-23 17:24:49 -0400, Tom Lane wrote:
> >> Hearing no objections, I started to review Andres' patchset with
> >> that plan in mind.
>
> > Thanks for pushing the first part!
>
> I pushed all of it, actually.

I think this caused anole to say:

"reloptions.c", line 1362: error #2042: operand types are incompatible
("void" and "int")
  errdetail_internal("%s", _(optenum->detailmsg)) : 0));

  27. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-25T05:39:57Z

    Thomas Munro <thomas.munro@gmail.com> writes:
> I think this caused anole to say:

> "reloptions.c", line 1362: error #2042: operand types are incompatible
> ("void" and "int")
>   errdetail_internal("%s", _(optenum->detailmsg)) : 0));

Yeah, I was just looking at that :-(

We could revert the change to have these functions return void,
or we could run around and change the places with this usage
pattern to use "(void) 0" instead of just "0".  The latter would
be somewhat painful if only minority compilers warn, though.
Also, I don't think that having to change ereport usage was part
of the agreed-to plan here ... so I'm leaning to the former.

			regards, tom lane

  28. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-25T16:18:22Z

    I wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
>> I think this caused anole to say:
>> "reloptions.c", line 1362: error #2042: operand types are incompatible
>> ("void" and "int")
>> errdetail_internal("%s", _(optenum->detailmsg)) : 0));

> Yeah, I was just looking at that :-(

> We could revert the change to have these functions return void,
> or we could run around and change the places with this usage
> pattern to use "(void) 0" instead of just "0".  The latter would
> be somewhat painful if only minority compilers warn, though.
> Also, I don't think that having to change ereport usage was part
> of the agreed-to plan here ... so I'm leaning to the former.

Done that way.

			regards, tom lane

  29. Re: Missing errcode() in ereport

    Craig Ringer <craig@2ndquadrant.com> — 2020-03-30T03:54:03Z

    On Fri, 20 Mar 2020, 01:59 Andres Freund, <andres@anarazel.de> wrote:

> Hi,
>
> On 2020-03-17 10:09:18 -0400, Tom Lane wrote:
> > We might want to spend some effort thinking how to find or prevent
> > additional bugs of the same ilk ...
>
> Yea, that'd be good.  Trying to help people new to postgres write their
> first patches I found that ereport is very confusing to them - largely
> because the syntax doesn't make much sense. Made worse by the compiler
> error messages being terrible in many cases.


Very much agreed.

I'd have found it helpful to just have the docs explain clearly how it
works by chaining the comma operator using functions with ignored return
values.

That would also help people understand how they can make parts of an
ereport conditional, e.g. only set errdetail() if there additional info is
currently available W/O duplicating the rest of the ereport .


Not sure there's much we can do without changing ereport's "signature"
> though :(
>
> Regards,
>
> Andres
>
>
>

  30. Re: Missing errcode() in ereport

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-03-30T03:57:50Z

    Craig Ringer <craig@2ndquadrant.com> writes:
> I'd have found it helpful to just have the docs explain clearly how it
> works by chaining the comma operator using functions with ignored return
> values.

Want to write some text?

> That would also help people understand how they can make parts of an
> ereport conditional, e.g. only set errdetail() if there additional info is
> currently available W/O duplicating the rest of the ereport .

There are examples of that in the tree, of course, but maybe an
example in the docs wouldn't be bad either.

			regards, tom lane

  31. Re: Missing errcode() in ereport

    Andres Freund <andres@anarazel.de> — 2020-03-30T04:07:34Z

    Hi,

On 2020-03-30 11:54:03 +0800, Craig Ringer wrote:
> On Fri, 20 Mar 2020, 01:59 Andres Freund, <andres@anarazel.de> wrote:
> > On 2020-03-17 10:09:18 -0400, Tom Lane wrote:
> > > We might want to spend some effort thinking how to find or prevent
> > > additional bugs of the same ilk ...
> >
> > Yea, that'd be good.  Trying to help people new to postgres write their
> > first patches I found that ereport is very confusing to them - largely
> > because the syntax doesn't make much sense. Made worse by the compiler
> > error messages being terrible in many cases.
> 
> 
> Very much agreed.
> 
> I'd have found it helpful to just have the docs explain clearly how it
> works by chaining the comma operator using functions with ignored return
> values.

IDK, that seems like it'll be a bit too implementation specific.


> That would also help people understand how they can make parts of an
> ereport conditional, e.g. only set errdetail() if there additional info is
> currently available W/O duplicating the rest of the ereport .

Well, they can do whatever they can in any function argument list. I
don't see why the ereport docs are a good place to explain ... ? ... :
... ? Or am I misunderstanding what you suggest here?


> Not sure there's much we can do without changing ereport's "signature"

I think the changes we just did already improved the situation a good
bit. Both by obviating the need for the additional parentheses, as well
as making parameters outside of err*() trigger warnings, and also
generally better error/warning messages.


I still think we might want to do a bigger change of the logging
APIs. See the messages leading up to
https://postgr.es/m/20190805214952.dmnn2oetdquixpp4%40alap3.anarazel.de
and the message linked from there.

Greetings,

Andres Freund