Horton

Re: Regression with large XML data input

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Michael Paquier <michael@paquier.xyz>
Cc: Postgres hackers <pgsql-hackers@lists.postgresql.org>, Erik Wienhold <ewie@ewie.name>
Date: 2025-07-24T19:23:59Z
Lists: pgsql-hackers 
I wrote:
> BTW, further testing shows that the same failure occurs at
> f68d6aabb7e2^.  So AFAICS, the answer as to why the behavior
> changed there is that it didn't.

Oh, wait ... the plot thickens!  The above statement is true
when testing on my Mac with libxml2 2.13.8 from MacPorts.
With either HEAD or f68d6aabb7e2^, I get errors similar to
what Erik just showed:

ERROR:  invalid XML content
DETAIL:  line 1: Resource limit exceeded: Text node too long, try XML_PARSE_HUGE
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

However, when testing on RHEL8 with libxml2 2.9.7, indeed
I get "Huge input lookup" with our current code but no
failure with f68d6aabb7e2^.

The way I interpret these results is that in older libxml2 versions,
xmlParseBalancedChunkMemory is missing an XML_ERR_RESOURCE_LIMIT check
that does exist in newer versions.  So even if we were to do some kind
of reversion, it would only prevent the error in libxml2 versions that
lack that check.  And in those versions we'd probably be exposing
ourselves to resource-exhaustion problems.

On the whole I'm thinking more and more that we don't want to
touch this.  Our recommendation for processing multi-megabyte
chunks of XML should be "don't".  Unless we want to find or
write a replacement for libxml2 ... which we have discussed,
but so far nothing's happened.

			regards, tom lane

Commits

  1. Remove unnecessary complication around xmlParseBalancedChunkMemory.

  2. Avoid regression in the size of XML input that we will accept.

  3. Use xmlParseInNodeContext not xmlParseBalancedChunkMemory.

  4. Revert "Add support for parsing of large XML data (>= 10MB)"