Horton
Menu

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. xml2: Fix failure with xslt_process() under -fsanitize=undefined

  2. Fix misuse of "volatile" in xml.c

  3. xml2: Improve error handling of libxml2 calls

  4. Improve error handling of libxml2 calls in xml.c

  1. BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    PG Bug reporting form <noreply@postgresql.org> — 2025-06-01T19:05:25Z

    The following bug has been logged on the website:

Bug reference:      18943
Logged by:          Karavaev Alexey
Email address:      maralist86@mail.ru
PostgreSQL version: 17.5
Operating system:   alt workstation 10.4 x86_64
Description:        

Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177
without checking for NULL, but it is usually checked for this function.
The function 'xmlBufferCreate' may return NULL, and in some cases the code
xmlStrdup(buf->content) will throw an error.

  2. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-02T05:21:24Z

    On Sun, Jun 01, 2025 at 07:05:25PM +0000, PG Bug reporting form wrote:
> Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177
> without checking for NULL, but it is usually checked for this function.
> The function 'xmlBufferCreate' may return NULL, and in some cases the code
> xmlStrdup(buf->content) will throw an error.

Hmm.  There are quite a few things going on here, because xpath.c
lacks in a couple of code paths a PgXmlErrorContext and we need
TRY/CATCH blocks to be able to free any memory allocated in the
context of libxml.  I think that this means a couple of calls to
pg_xml_init() and pg_xml_done() calls painted in the right places.

Another one is pgxml_result_to_text().  As cstring_to_text() does a
palloc(), it could be possible that we miss the allocation done for
xpresstr().

While going through the code, xslt_process() is actually a bit
incorrect with the handling of resstr?  If the palloc() used for the
result fails, we would lose the xmlFree() for resstr, meaning that
we'd better call cstring_to_text_with_len() in the try/catch block.

With all that, I get the attached.

Thoughts or comments?
--
Michael

  3. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-03T04:18:13Z

    On Mon, Jun 02, 2025 at 02:21:24PM +0900, Michael Paquier wrote:
> With all that, I get the attached.
> 
> Thoughts or comments?

I'll try to have a second look at this patch in a couple of weeks.
For now, I have added it to the next comment fest:
https://commitfest.postgresql.org/patch/5794/

If somebody would like to chime in, feel free.
--
Michael

  4. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-06-03T17:15:33Z

    Michael Paquier <michael@paquier.xyz> writes:
> I'll try to have a second look at this patch in a couple of weeks.
> For now, I have added it to the next comment fest:
> https://commitfest.postgresql.org/patch/5794/
> If somebody would like to chime in, feel free.

Thanks for taking this on --- contrib/xml2 is really a mess so far as
error handling goes.  Your patch looks like an improvement, although
I do have one concern: the routines in xml.c that use an xmlerrcxt
seem to check xmlerrcxt->err_occurred pretty frequently, eg xmlStrdup
is used like this:

            doc->encoding = xmlStrdup((const xmlChar *) "UTF-8");
            if (doc->encoding == NULL || xmlerrcxt->err_occurred)
                xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
                            "could not allocate XML document");

Not sure if that's needed here.

There's more that could be looked at, if you feel like it:

xml_encode_special_chars seems to need a PG_TRY block to avoid
possibly leaking "tt".  I also wonder if it's really not possible
for xmlEncodeSpecialChars to fail and return NULL.  (xmltext()
doesn't believe that either, but maybe it's wrong too.)

The usage of cleanup_workspace seems quite a mess: one caller
uses a PG_TRY block to ensure it's called, but the rest don't.
I also find it confusing that pgxml_xpath returns a value that is
also referenced in the workspace and cleanup_workspace is responsible
for freeing.  I wonder if there's a better way to do that.

In the end of xslt_process, I wonder why

	if (resstr)
		xmlFree((xmlChar *) resstr);

isn't done before the pg_xml_done call.

			regards, tom lane

  5. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-05T04:52:46Z

    On Tue, Jun 03, 2025 at 01:15:33PM -0400, Tom Lane wrote:
> Thanks for taking this on --- contrib/xml2 is really a mess so far as
> error handling goes.  Your patch looks like an improvement, although
> I do have one concern: the routines in xml.c that use an xmlerrcxt
> seem to check xmlerrcxt->err_occurred pretty frequently, eg xmlStrdup
> is used like this:
> 
>             doc->encoding = xmlStrdup((const xmlChar *) "UTF-8");
>             if (doc->encoding == NULL || xmlerrcxt->err_occurred)
>                 xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>                             "could not allocate XML document");
> 
> Not sure if that's needed here.

Yes, I was also wondering a bit about this part a couple of days ago
while looking at the module's code and xml.c.  Looking at cacd42d62cb2
and its thread (particularly [1]), I think that we need to bite the
bullet or we are going to miss some error context.

PgXmlErrorContext can remain private in xml.c as we can reuse
pg_xml_error_occurred() to do the job for out-of-core code.  

> There's more that could be looked at, if you feel like it:

Well, now that you mention these things, I do feel like it while I
have my hands on this area of the code.

> xml_encode_special_chars seems to need a PG_TRY block to avoid
> possibly leaking "tt".  I also wonder if it's really not possible
> for xmlEncodeSpecialChars to fail and return NULL.  (xmltext()
> doesn't believe that either, but maybe it's wrong too.)

Oh, missed this call.  xmlEncodeSpecialChars() relies on
xmlEscapeText(), which can return NULL on allocation failures based
on a check in the upstream code (first argument of the routine is not
used, only second is).  So xmltext() and xml_encode_special_chars()
are both wrong; we need a try/catch block for the edge case where
cstring_to_text_with_len() or cstring_to_text() could fail an
allocation or we would leak what could have been allocated by libxml.

> The usage of cleanup_workspace seems quite a mess: one caller
> uses a PG_TRY block to ensure it's called, but the rest don't.
> I also find it confusing that pgxml_xpath returns a value that is
> also referenced in the workspace and cleanup_workspace is responsible
> for freeing.  I wonder if there's a better way to do that.

Yes, that's not optimal.  This comes down to the fact that we need
workspace->res to not be free'd by libxml until the result of these
SQL functions is generated, and even with my previous patch we could
leak it if one of the allocations done for the function results fail.

I've been wondering about a few approaches, like adding the error
context to the workspace, but that did not feel right as we require
the error context before the try/catch block, and the workspace and
its internals allocated by libxml need to be fully handled in the
scope of the try/catch.  So I've finished with the attached, where the
workspace and its cleanup routine gain volatile declarations, keeping
pg_xml_done() outside the workspace logic.  This is a rather
mechanical change if done this way with the try/catch blocks moved one
level higher, but as long as we need to hold on the result inside
the workspace, I'm feeling that this is a cleaner approach in the
long-run for xml2, because it's impossible to miss what's in the scope
of the catch cleanup with the cleanup policy enforced in the
definition of cleanup_workspace().

> In the end of xslt_process, I wonder why
> 
> 	if (resstr)
> 		xmlFree((xmlChar *) resstr);
> 
> isn't done before the pg_xml_done call.

Good point.  Fixed.

All that is rather unlikely going to be a problem in practice, so I
don't really feel a strong reason to backpatch any of that.  v18 would
be OK, but we could just also wait for v19 based on how these are
unlikely going to be problems in the field.

Or it's worth worrying about a backpatch of the xml.c code paths which
are in the core backend?  The xmltext() case is isolated, so this part
is not invasive.

[1]: https://www.postgresql.org/message-id/23388.1311118974%40sss.pgh.pa.us
--
Michael

  6. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Jim Jones <jim.jones@uni-muenster.de> — 2025-06-05T09:47:01Z

    On 05.06.25 11:38, Jim Jones wrote:
> 
> 
> Hi Michael
> 
> Am Do., 5. Juni 2025 um 10:11 Uhr schrieb Michael Paquier
> <michael@paquier.xyz <mailto:michael@paquier.xyz>>:
> 
>     On Tue, Jun 03, 2025 at 01:15:33PM -0400, Tom Lane wrote:
>     > Thanks for taking this on --- contrib/xml2 is really a mess so far as
>     > error handling goes.  Your patch looks like an improvement, although
>     > I do have one concern: the routines in xml.c that use an xmlerrcxt
>     > seem to check xmlerrcxt->err_occurred pretty frequently, eg xmlStrdup
>     > is used like this:
>     >
>     >             doc->encoding = xmlStrdup((const xmlChar *) "UTF-8");
>     >             if (doc->encoding == NULL || xmlerrcxt->err_occurred)
>     >                 xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>     >                             "could not allocate XML document");
>     >
>     > Not sure if that's needed here.
> 
>     Yes, I was also wondering a bit about this part a couple of days ago
>     while looking at the module's code and xml.c.  Looking at cacd42d62cb2
>     and its thread (particularly [1]), I think that we need to bite the
>     bullet or we are going to miss some error context.
> 
>     PgXmlErrorContext can remain private in xml.c as we can reuse
>     pg_xml_error_occurred() to do the job for out-of-core code. 
> 
>     > There's more that could be looked at, if you feel like it:
> 
>     Well, now that you mention these things, I do feel like it while I
>     have my hands on this area of the code.
> 
>     > xml_encode_special_chars seems to need a PG_TRY block to avoid
>     > possibly leaking "tt".  I also wonder if it's really not possible
>     > for xmlEncodeSpecialChars to fail and return NULL.  (xmltext()
>     > doesn't believe that either, but maybe it's wrong too.)
> 
>     Oh, missed this call.  xmlEncodeSpecialChars() relies on
>     xmlEscapeText(), which can return NULL on allocation failures based
>     on a check in the upstream code (first argument of the routine is not
>     used, only second is).  So xmltext() and xml_encode_special_chars()
>     are both wrong; we need a try/catch block for the edge case where
>     cstring_to_text_with_len() or cstring_to_text() could fail an
>     allocation or we would leak what could have been allocated by libxml.
> 
> 
> Yeah, xmlEscapeText() does return NULL in some cases[1,2] and
> xmlEncodeSpecialChars() doesn't mind.
> 
> Taking a further look at xml.c I am wondering if other functions might
> also need some attention in this regard:
> 
> * xmlTextWriterStartElement [3]
> * xmlTextWriterWriteAttribute [4]
> * xmlTextWriterWriteRaw [5]
> * xmlTextWriterEndAttribute [6]
> 
> We're assuming they never fail. Perhaps something like this?
>  ...
>  nbytes = xmlTextWriterStartElement(writer, (xmlChar *) xexpr->name);
>  if (nbytes == -1 || xmlerrcxt->err_occurred)
>     xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>                         "could not allocate xmlTextWriterStartElement");
> 
> Best regards, Jim
> 
> 1 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlIO.c#L205 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlIO.c#L205>
> 2 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlIO.c#L284 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlIO.c#L284>
> 3 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L967 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L967>
> 4 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1950 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1950>
> 5 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1323 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1323>
> 6 -
> https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1839 <https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/xmlwriter.c#L1839> 
> 

Oups .. just replied to Michael.
Sorry!

Jim

  7. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Jim Jones <jim.jones@uni-muenster.de> — 2025-06-05T14:15:19Z

    On 05.06.25 11:47, Jim Jones wrote:
> Taking a further look at xml.c I am wondering if other functions might
> also need some attention in this regard:
> 
> * xmlTextWriterStartElement [3]
> * xmlTextWriterWriteAttribute [4]
> * xmlTextWriterWriteRaw [5]
> * xmlTextWriterEndAttribute [6]
> 
> We're assuming they never fail. Perhaps something like this?
>  ...
>  nbytes = xmlTextWriterStartElement(writer, (xmlChar *) xexpr->name);
>  if (nbytes == -1 || xmlerrcxt->err_occurred)
>     xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>                         "could not allocate xmlTextWriterStartElement");
> 

There is also a further xmlXPathCastNodeToString() call in xml.c at
xml_xmlnodetoxmltype() - it calls xmlNodeGetContent() and it can return
NULL.

xmlChar    *str;
str = xmlXPathCastNodeToString(cur);

PG_TRY();
{
  /* Here we rely on XML having the same representation as TEXT */
  char	   *escaped = escape_xml((char *) str);

  result = (xmltype *) cstring_to_text(escaped);
  pfree(escaped);
}
PG_FINALLY();
{
  xmlFree(str);
}
PG_END_TRY();

The function pgxmlNodeSetToText() also calls xmlXPathCastNodeToString(),
but apparently xmlBufferAdd() can handle NULL values.[1]

Best regards, Jim

1 -
https://github.com/GNOME/libxml2/blob/2b6b3945f2df548b56f2c73c490dda9781f92eb2/buf.c#L989

  8. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-06T05:54:40Z

    On Thu, Jun 05, 2025 at 04:15:19PM +0200, Jim Jones wrote:
> On 05.06.25 11:47, Jim Jones wrote:
>> Taking a further look at xml.c I am wondering if other functions might
>> also need some attention in this regard:
>> 
>> * xmlTextWriterStartElement [3]
>> * xmlTextWriterWriteAttribute [4]
>> * xmlTextWriterWriteRaw [5]
>> * xmlTextWriterEndAttribute [6]

It seems to me that you mean xmlTextWriterEndElement() and not
xmlTextWriterEndAttribute() for the last one.

I've been looking at the rest of the callers (this took some time),
like:
- xmlTextWriterWriteBase64, OK.
- xmlTextWriterWriteBinHex, OK.
- xmlNewStringInputStream, which is intentional in xmlPgEntityLoader()
- xmlAddChildList, where we expect valid input.
- xmlXPathCompiledEval, where valid input is expected.
- xmlXPathNewContext, which is incorrect, could fail an allocation.
- xmlReadMemory, looks OK.
- xmlBufferWriteChar, which could fail on OOM if they need to grow
memory, but let's leave these as they are; I suspect that
xmlBufferCreate() would fail anyway.

>> We're assuming they never fail. Perhaps something like this?
>>  ...
>>  nbytes = xmlTextWriterStartElement(writer, (xmlChar *) xexpr->name);
>>  if (nbytes == -1 || xmlerrcxt->err_occurred)
>>     xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>>                         "could not allocate xmlTextWriterStartElement");

Oh.  These can return XML_ERR_NO_MEMORY as well, with more error
patterns..

> There is also a further xmlXPathCastNodeToString() call in xml.c at
> xml_xmlnodetoxmltype() - it calls xmlNodeGetContent() and it can return
> NULL.

And it's documented as a routine that returns an allocated string, so
yes, we would miss allocation failures but we should not.  I think
that we should move the call of xmlXPathCastNodeToString() inside the
PG_TRY block and rely on the xmlerrcxt given by the caller to log an
error if an allocation fails, marking xmlChar *str as volatile to free
it in the finally block if required.

> The function pgxmlNodeSetToText() also calls xmlXPathCastNodeToString(),
> but apparently xmlBufferAdd() can handle NULL values.[1]

Yeah, the patch I have posted upthread gets that done better.

What do you think?
--
Michael

  9. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Jim Jones <jim.jones@uni-muenster.de> — 2025-06-06T10:22:30Z

    On 06.06.25 07:54, Michael Paquier wrote:
> On Thu, Jun 05, 2025 at 04:15:19PM +0200, Jim Jones wrote:
>> On 05.06.25 11:47, Jim Jones wrote:
>>> Taking a further look at xml.c I am wondering if other functions might
>>> also need some attention in this regard:
>>>
>>> * xmlTextWriterStartElement [3]
>>> * xmlTextWriterWriteAttribute [4]
>>> * xmlTextWriterWriteRaw [5]
>>> * xmlTextWriterEndAttribute [6]
> 
> It seems to me that you mean xmlTextWriterEndElement() and not
> xmlTextWriterEndAttribute() for the last one.

+1
The return value of the xmlTextWriter* functions are now properly evaluated.

> - xmlBufferWriteChar, which could fail on OOM if they need to grow
> memory, but let's leave these as they are; I suspect that
> xmlBufferCreate() would fail anyway.
> 

I also think we're safe in this case. xmlBufferAdd() can return
XML_ERR_ARGUMENT if the xmlBuffer* or the xmlChar* are NULL, which
cannot happen in this part of the code. XML_ERR_NO_MEMORY is also
unlikely to happen, since xmlBufferWriteChar() and xmlBufferWriteCHAR()
call xmlBufferAdd() with a -1 length.


>>> We're assuming they never fail. Perhaps something like this?
>>>  ...
>>>  nbytes = xmlTextWriterStartElement(writer, (xmlChar *) xexpr->name);
>>>  if (nbytes == -1 || xmlerrcxt->err_occurred)
>>>     xml_ereport(xmlerrcxt, ERROR, ERRCODE_OUT_OF_MEMORY,
>>>                         "could not allocate xmlTextWriterStartElement");
> 
> Oh.  These can return XML_ERR_NO_MEMORY as well, with more error
> patterns..

+1

> 
>> There is also a further xmlXPathCastNodeToString() call in xml.c at
>> xml_xmlnodetoxmltype() - it calls xmlNodeGetContent() and it can return
>> NULL.
> 
> And it's documented as a routine that returns an allocated string, so
> yes, we would miss allocation failures but we should not.  I think
> that we should move the call of xmlXPathCastNodeToString() inside the
> PG_TRY block and rely on the xmlerrcxt given by the caller to log an
> error if an allocation fails, marking xmlChar *str as volatile to free
> it in the finally block if required.

+1

> 
>> The function pgxmlNodeSetToText() also calls xmlXPathCastNodeToString(),
>> but apparently xmlBufferAdd() can handle NULL values.[1]
> 
> Yeah, the patch I have posted upthread gets that done better.
> 
> What do you think?

Going through xml.c once again, I stumbled upon xmlAddChildList(), which
returns the last child or NULL in case of error. [1]

...
xmlAddChildList(root, content_nodes);


So, perhaps this?

if (xmlAddChildList(root, content_nodes) == NULL ||
    xmlerrcxt->err_occurred)
  xml_ereport(xmlerrcxt,
     ERROR, ERRCODE_OUT_OF_MEMORY,
     "could not add content nodes to root element");



-- 
Jim


1-
https://github.com/GNOME/libxml2/blob/c6206c93872fc91d98fbc61215c5618b629bf915/tree.c#L2976

  10. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-08T02:19:29Z

    On Fri, Jun 06, 2025 at 12:22:30PM +0200, Jim Jones wrote:
> So, perhaps this?
> 
> if (xmlAddChildList(root, content_nodes) == NULL ||
>     xmlerrcxt->err_occurred)
>   xml_ereport(xmlerrcxt,
>      ERROR, ERRCODE_OUT_OF_MEMORY,
>      "could not add content nodes to root element");

ERRCODE_INTERNAL_ERROR would be more adapted, I'm only seeing error
code paths caused by inconsistencies in the nodes.

I have updated the patches with the attached, splitting the parts for
contrib/xml2/ and the backend into two parts.  These touch error paths
that are very unlikely going to be hit in practice, so let's do all
that once v19 opens for business only on HEAD.
--
Michael

  11. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Jim Jones <jim.jones@uni-muenster.de> — 2025-06-08T08:33:17Z

    On 08.06.25 04:19, Michael Paquier wrote:
> ERRCODE_INTERNAL_ERROR would be more adapted, I'm only seeing error
> code paths caused by inconsistencies in the nodes.

+1

> I have updated the patches with the attached, splitting the parts for
> contrib/xml2/ and the backend into two parts.  These touch error paths
> that are very unlikely going to be hit in practice, so let's do all
> that once v19 opens for business only on HEAD.

Out of curiosity, why aren't we applying this to v18?

Overall, LGTM.

Thanks!

-- 
Jim

  12. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-06-08T15:40:11Z

    Jim Jones <jim.jones@uni-muenster.de> writes:
> Out of curiosity, why aren't we applying this to v18?

Our risk-aversion level rises steadily over the course of a release
cycle, and is pretty high post beta1.  While I think the problems
we're trying to fix here are real, they are very low-probability
(I don't recall hearing any field reports traceable to this).
And you have to remember there is also some risk of introducing
new bugs.  On balance it's not a change I would back-patch, and
at this point v18 is pretty close to being a stable branch so
it's not getting fixes we wouldn't back-patch, unless that's
because they are in new-in-18 code.

tl;dr: I agree with Michael's choice to hold it for v19.
It's a judgment call of course, but I think it's the right one.

			regards, tom lane

  13. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Jim Jones <jim.jones@uni-muenster.de> — 2025-06-08T17:00:25Z

    On 08.06.25 17:40, Tom Lane wrote:
> Our risk-aversion level rises steadily over the course of a release
> cycle, and is pretty high post beta1.  While I think the problems
> we're trying to fix here are real, they are very low-probability
> (I don't recall hearing any field reports traceable to this).
> And you have to remember there is also some risk of introducing
> new bugs.  On balance it's not a change I would back-patch, and
> at this point v18 is pretty close to being a stable branch so
> it's not getting fixes we wouldn't back-patch, unless that's
> because they are in new-in-18 code.

Makes sense. Thanks for the clarification!

Best regards, Jim

  14. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-06-08T22:27:56Z

    On Sun, Jun 08, 2025 at 07:00:25PM +0200, Jim Jones wrote:
> On 08.06.25 17:40, Tom Lane wrote:
>> Our risk-aversion level rises steadily over the course of a release
>> cycle, and is pretty high post beta1.  While I think the problems
>> we're trying to fix here are real, they are very low-probability
>> (I don't recall hearing any field reports traceable to this).
>> And you have to remember there is also some risk of introducing
>> new bugs.  On balance it's not a change I would back-patch, and
>> at this point v18 is pretty close to being a stable branch so
>> it's not getting fixes we wouldn't back-patch, unless that's
>> because they are in new-in-18 code.

That's something that can be measured with a kind of risk/reward
ratio.  Here is the reward for the end-user is low, as we have no
reports of the current code in the field.  The risk is in introducing
new issues.  And the code is incorrect, so we should fix it. 

I've made similar choices in the past around the same time in the
release cycle not backpatching stuff that was an issue in backbranches
still minimal enough to not have to worry about, like 84e4570da923
(there are a few others).
--
Michael

  15. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-07-01T06:49:09Z

    On Mon, Jun 09, 2025 at 07:27:56AM +0900, Michael Paquier wrote:
> I've made similar choices in the past around the same time in the
> release cycle not backpatching stuff that was an issue in backbranches
> still minimal enough to not have to worry about, like 84e4570da923
> (there are a few others).

While doing a secondary review of all that, I have noticed one call to
pg_xml_done() missing at the end of xpath_string().  The rest was
pretty clean.  I have split the changes into two parts for clarity,
and applied each one of them separately: one for xml.c and one for the
contrib module xml2/.
--
Michael

  16. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Robin Haberkorn <haberkorn@b1-systems.de> — 2025-07-08T09:49:20Z

    Hello Michael!

On Sun Jun 8, 2025 at 05:19:29 GMT +03, Michael Paquier wrote:
> I have updated the patches with the attached, splitting the parts for
> contrib/xml2/ and the backend into two parts.  These touch error paths
> that are very unlikely going to be hit in practice, so let's do all
> that once v19 opens for business only on HEAD.

I know this has already been committed, but why are we still using
PG_XML_STRICTNESS_LEGACY in xpath.c? As we are always checking
pg_xml_error_occurred() this should no longer be necessary.

It's of course also still in xslt_proc.c, but I have submitted a
separate patch to the ongoing Commitfest, which will resolve that. [1]

Best regards,
Robin Haberkorn

[1] https://commitfest.postgresql.org/patch/5718/
btw. it's still looking for a rewiever.

-- 
Robin Haberkorn
Software Engineer

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / https://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt, HRB 3537

  17. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-07-08T11:23:30Z

    On Tue, Jul 08, 2025 at 09:49:20AM +0000, Robin Haberkorn wrote:
> I know this has already been committed, but why are we still using
> PG_XML_STRICTNESS_LEGACY in xpath.c? As we are always checking
> pg_xml_error_occurred() this should no longer be necessary.

Are you sure that you can do that?  When I looked at this code I was
under the impression that xmlReadMemory() can generate an error and
can go through the error context we set, so the legacy mode counted
for xpath_string() at least.

Perhaps we should just remove the pg_xml_error_occurred() then if
that's confusing, keeping the legacy mode.  Or do you have a different
idea in mind?
--
Michael

  18. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-08T13:36:37Z

    Michael Paquier <michael@paquier.xyz> writes:
> On Tue, Jul 08, 2025 at 09:49:20AM +0000, Robin Haberkorn wrote:
>> I know this has already been committed, but why are we still using
>> PG_XML_STRICTNESS_LEGACY in xpath.c? As we are always checking
>> pg_xml_error_occurred() this should no longer be necessary.

> Are you sure that you can do that?

The comment in xml_errorHandler() argues

     * Legacy error handling mode.  err_occurred is never set, we just add the
     * message to err_buf.  This mode exists because the xml2 contrib module
     * uses our error-handling infrastructure, but we don't want to change its
     * behaviour since it's deprecated anyway.  This is also why we don't
     * distinguish between notices, warnings and errors here --- the old-style
     * generic error handler wouldn't have done that either.

So switching to _ALL (or even _WELL_FORMED) mode would result in
nontrivial differences in the behavior of xpath.c's functions with
bad input.  Maybe that's a reasonable thing to do, but it's a
question of user-visible behavior not just code cleanliness.

			regards, tom lane

  19. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2025-07-09T00:45:07Z

    On Tue, Jul 08, 2025 at 09:36:37AM -0400, Tom Lane wrote:
> The comment in xml_errorHandler() argues

Yep.

> So switching to _ALL (or even _WELL_FORMED) mode would result in
> nontrivial differences in the behavior of xpath.c's functions with
> bad input.  Maybe that's a reasonable thing to do, but it's a
> question of user-visible behavior not just code cleanliness.

Yes, I don't see a huge advantage in doing the switch for this module.
If the gain in information in the error states grabbed from libxml2
makes it a win, that may be a different argument (I am fine to be
proved wrong), but I cannot get excited about that without more
data to claim it so.

I have quickly tested the change, and the xpath_string() path was one
area that immediately stood out, and we may report an incorrect error.
--
Michael

  20. Re: BUG #18943: Return value of a function 'xmlBufferCreate' is dereferenced at xpath.c:177 without checking for NUL

    Robin Haberkorn <haberkorn@b1-systems.de> — 2025-07-09T11:28:58Z

    On Wed Jul 9, 2025 at 03:45:07 GMT +03, Michael Paquier wrote:
> Yes, I don't see a huge advantage in doing the switch for this module.
> If the gain in information in the error states grabbed from libxml2
> makes it a win, that may be a different argument (I am fine to be
> proved wrong), but I cannot get excited about that without more
> data to claim it so.
>
Once switching to PG_XML_STRICTNESS_ALL, we should also theoretically
be able to receive warnings and notices, that would be silent otherwise.
I believe that getting rid of PG_XML_STRICTNESS_LEGACY might also be
desirable if we ever want to get xml2 into core.

But I notice that you did already change lots of PG_XML_STRICTNESS_LEGACY
into PG_XML_STRICTNESS_ALL.

> I have quickly tested the change, and the xpath_string() path was one
> area that immediately stood out, and we may report an incorrect error.

You are right. The test suite fails or hangs at least. We are probably
still missing some checks. So it wouldn't just be a matter of
replacing all remaining PG_XML_STRICTNESS_LEGACY.

-- 
Robin Haberkorn
Software Engineer

B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / https://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt, HRB 3537

  21. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    cca5507 <cca5507@qq.com> — 2026-03-08T14:12:09Z

    Hi,

It seems that there are 2 misuse of "volatile" in xml.c:

1) xmltext()

volatile xmlChar *xmlbuf = NULL; // -> xmlChar *volatile xmlbuf = NULL;

2) xml_xmlnodetoxmltype()

volatile xmlChar *str = NULL; // -> xmlChar *volatile str = NULL;

We want the pointer itself be volatile rather than what it points to.

--
Regards,
ChangAo Chen

  22. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    cca5507 <cca5507@qq.com> — 2026-03-08T15:05:32Z

    > It seems that there are 2 misuse of "volatile" in xml.c:
> 
> 1) xmltext()
> 
> volatile xmlChar *xmlbuf = NULL; // -> xmlChar *volatile xmlbuf = NULL;
> 
> 2) xml_xmlnodetoxmltype()
> 
> volatile xmlChar *str = NULL; // -> xmlChar *volatile str = NULL;
> 
> We want the pointer itself be volatile rather than what it points to.

Attach a small patch.

--
Regards,
ChangAo Chen

  23. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2026-03-09T07:31:37Z

    On Sun, Mar 08, 2026 at 11:05:32PM +0800, cca5507 wrote:
> Attach a small patch.

You are right, this needs to consider the pointer variable as
volatile, as done in your patch, and not treat as volatile what is
pointed at.  This comes from 2e947217474c, as of HEAD.  I'll take care
of it later.  The same business has been fixed in xml2 as of
93001888d85c. 
--
Michael

  24. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2026-03-09T22:10:13Z

    On Mon, Mar 09, 2026 at 04:31:37PM +0900, Michael Paquier wrote:
> You are right, this needs to consider the pointer variable as
> volatile, as done in your patch, and not treat as volatile what is
> pointed at.  This comes from 2e947217474c, as of HEAD.  I'll take care
> of it later.  The same business has been fixed in xml2 as of
> 93001888d85c. 

Fixed as of 6307b096e259.
--
Michael

  25. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    Alexander Lakhin <exclusion@gmail.com> — 2026-03-12T05:00:00Z

    Hello Michael,

Maybe you would like to fix in passing one more anomaly there:
create extension xml2;
select xslt_process('<aaa/>','<xsl:stylesheet version="1.0" 
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"></xsl:stylesheet>');

leads to:
varlena.c:199:2: runtime error: null pointer passed as argument 2, which is declared to never be null
     #0 0x640756666936 in cstring_to_text_with_len .../src/backend/utils/adt/varlena.c:199
     #1 0x7e46c0f4805e in xslt_process .../contrib/xml2/xslt_proc.c:149
     #2 0x640755a3ecbf in ExecInterpExpr .../src/backend/executor/execExprInterp.c:1001
     #3 0x640755a277aa in ExecInterpExprStillValid .../src/backend/executor/execExprInterp.c:2299
     #4 0x640755ef11e0 in ExecEvalExprSwitchContext ../../../../src/include/executor/executor.h:444
     #5 0x640755efd7b6 in evaluate_expr .../src/backend/optimizer/util/clauses.c:5724

for a build made with -fsanitize=undefined.

Best regards,
Alexander

  26. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2026-03-12T06:46:57Z

    On Thu, Mar 12, 2026 at 07:00:00AM +0200, Alexander Lakhin wrote:
> Hello Michael,
> 
> Maybe you would like to fix in passing one more anomaly there:
> create extension xml2;
> select xslt_process('<aaa/>','<xsl:stylesheet version="1.0"
> xmlns:xsl="http://www.w3.org/1999/XSL/Transform"></xsl:stylesheet>');
> 
> leads to:
> varlena.c:199:2: runtime error: null pointer passed as argument 2, which is declared to never be null
>     #0 0x640756666936 in cstring_to_text_with_len .../src/backend/utils/adt/varlena.c:199
>     #1 0x7e46c0f4805e in xslt_process .../contrib/xml2/xslt_proc.c:149
>     #2 0x640755a3ecbf in ExecInterpExpr .../src/backend/executor/execExprInterp.c:1001
>     #3 0x640755a277aa in ExecInterpExprStillValid .../src/backend/executor/execExprInterp.c:2299
>     #4 0x640755ef11e0 in ExecEvalExprSwitchContext ../../../../src/include/executor/executor.h:444
>     #5 0x640755efd7b6 in evaluate_expr .../src/backend/optimizer/util/clauses.c:5724
> 
> for a build made with -fsanitize=undefined.

Indeed, I can reproduce it locally.  This one is a super old
inconsistency, from what I can see.  This predates the introduction to
xml2 in contrib and even the use of cstring_to_text_with_len().  We've
never thought that xsltSaveResultToString() could return a NULL
xmlChar with a valid status code and a length of 0.  Back in the
day, before cstring_to_text_with_len(), that would be a memcpy with a
NULL pointer.

I am not sure if this is worth backpatching, so let's just use
something like the attached on HEAD.  This result cannot be NULL,
historically it has always been an empty string.

Opinions?
--
Michael

  27. Re: BUG #18943: Return value of a function 'xmlBufferCreate' isdereferenced at xpath.c:177 without checking for NUL

    Michael Paquier <michael@paquier.xyz> — 2026-03-13T07:12:28Z

    On Thu, Mar 12, 2026 at 03:46:57PM +0900, Michael Paquier wrote:
> I am not sure if this is worth backpatching, so let's just use
> something like the attached on HEAD.  This result cannot be NULL,
> historically it has always been an empty string.

Looking at the history of the tree, things like 46ab07ffda9d were
mentioned as worth a backpatch, so applied down to v14 for this one as
well.
--
Michael