Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Fix incorrect zero extension of Datum in JIT tuple deform code

  1. ERROR: XX000: could not find memoization table entry (reproducible)

    Emmanuel Touzery <emmanuel.touzery@plandela.si> — 2025-10-22T07:33:43Z

    Hello,

 we are getting this error:
ERROR:  XX000: could not find memoization table entry

 reproducibly, on two servers and two postgresql versions:
psql (PostgreSQL) 17.6 (Ubuntu 17.6-2.pgdg22.04+1)
psql (PostgreSQL) 18.0 (Ubuntu 18.0-1.pgdg24.04+3)

 However I can NOT reproduce the issue on my laptop (psql (PostgreSQL) 17.6, Fedora linux 42).

We can reproduce this issue everytime on these hosts with this postgresql dump (created with '-F c'):

​[https://res.public.onecdn.static.microsoft/assets/mail/file-icon/v2/generic_16x16.png]memo.dump<https://plandela-my.sharepoint.com/:u:/g/personal/emmanuel_touzery_plandela_si/EeXNT8zgImNNhFXAhEskPEwBitMBWDBmPDhrZac0uNvAQQ?e=UhOIBe>​

And also this SQL query:

SELECT DISTINCT ON(c.col3)
c.col3, c.col2 FROM
table1 c
INNER JOIN table2 tea ON
tea.col1 = c.col3 AND
tea.col2 <= c.col1 AND
tea.col3 >= c.col1 AND
tea.col4 = 't' AND tea.col5 = 'f'
WHERE
c.col1 >= '2025-10-01' AND
c.col1 <= '2025-10-31' AND
c.col3 = any(ARRAY[567,568,569,561,570,522,571,155,515,572,450,474,484,467,557,437,85,383,195,82,21,17,95,96,147,112,118,269,508,472,86,88,94,14,105,216,217,218,219,220,80,29,6,52,150,501,548,18,504,171,199,156,170,238,139,249,39,119,128,253,197,198,145,100,194,135,229,204,210,51,283,48,293,505,401,214,476,307,309,8,542,306,319,320,322,321,323,324,325,326,327,328,329,330,331,332,334,336,339,340,341,109,137,509,312,512,142,73,154,344,514,516,372,369,288,520,521,305,190,510,360,265,333,345,549,473,215,271,347,130,159,4,551,196,433,7,552,165,511,518,523,524,525,529,530,531,533,534,111,252,250,191,127,377,188,385,468,262,538,424,554,555,426,532,391,98,417,172,558,403,365,272,300,131,294,536,291,409,454,384,71,559,393,138,355,180,539,133,541,543,544,546,540,153,285,513,359,318,92,176,115,456,114,335,45,482,547,74,390,376,489,490,491,483,492,374,427,299,494,495,206,348,144,418,47,246,255,486,475,496,416,479,441,301,497,545,152,179,292,42,464,126,478,498,308,175,174,158,40,41,181,178,227,108,443,422,213,166,346,168,354,366,406,435,440,447,392,452,429,296,453,167,451,460,461,160,87,162,314,517,286,200,260,466,203,442,103,560,562,353,563,564,565,566,1,23,25,35,68]) AND
c.col4 <= 2204007 AND
c.col5 = 'f'
ORDER BY
c.col3, c.col1, c.col4 DESC;

When the issue occurs... With \set verbose, psql displays:

ERROR:  XX000: could not find memoization table entry
LOCATION:  cache_reduce_memory, nodeMemoize.c:484


Explain analyse fails with the same error, hower a simple "explain" on an host exhibiting the issue outputs:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Unique  (cost=296161.37..296734.58 rows=495 width=16)
   ->  Sort  (cost=296161.37..296447.98 rows=114642 width=16)
         Sort Key: c.col3, c.col1, c.col4 DESC
         ->  Nested Loop  (cost=0.45..286527.56 rows=114642 width=16)
               ->  Seq Scan on table2 tea  (cost=0.00..26.27 rows=1327 width=12)
                     Filter: (col4 AND (NOT col5))
               ->  Memoize  (cost=0.45..515.00 rows=84 width=16)
                     Cache Key: tea.col2, tea.col3, tea.col1
                     Cache Mode: binary
                     ->  Index Scan using table1_three on table1 c  (cost=0.44..514.99 rows=84 width=16)
                           Index Cond: ((col3 = tea.col1) AND (col3 = ANY ('{567,568,569,561,570,522,571,155,515,572,450,474,484,467,557,437,85,383,195,82,21,17,95,96,147,112,118,269,508,472,86,88,94,14,105,216,217,218,219,220,80,29,6,52,150,501,548,18,504,171,199,156,170,238,139,249,39,119,128,253,197,198,145,100,194,135,229,204,210,51,283,48,293,505,401,214,476,307,309,8,542,306,319,320,322,321,323,324,325,326,327,328,329,330,331,332,334,336,339,340,341,109,137,509,312,512,142,73,154,344,514,516,372,369,288,520,521,305,190,510,360,265,333,345,549,473,215,271,347,130,159,4,551,196,433,7,552,165,511,518,523,524,525,529,530,531,533,534,111,252,250,191,127,377,188,385,468,262,538,424,554,555,426,532,391,98,417,172,558,403,365,272,300,131,294,536,291,409,454,384,71,559,393,138,355,180,539,133,541,543,544,546,540,153,285,513,359,318,92,176,115,456,114,335,45,482,547,74,390,376,489,490,491,483,492,374,427,299,494,495,206,348,144,418,47,246,255,486,475,496,416,479,441,301,497,545,152,179,292,42,464,126,478,498,308,175,174,158,40,41,181,178,227,108,443,422,213,166,346,168,354,366,406,435,440,447,392,452,429,296,453,167,451,460,461,160,87,162,314,517,286,200,260,466,203,442,103,560,562,353,563,564,565,566,1,23,25,35,68}'::integer[])) AND (col1 >= tea.col2) AND (col1 <= tea.col3) AND (col1 >= '2025-10-01'::date) AND (col1 <= '2025-10-31'::date))
                           Filter: ((NOT col5) AND (col4 <= 2204007))
 JIT:
   Functions: 18
   Options: Inlining false, Optimization false, Expressions true, Deforming true
(15 rows)

Also relevant, in the log file we see the following:

2025-10-22 07:20:19.529 GMT,"postgres","flatman_test7",349111,"[local]",68f885b0.553b7,4,"SELECT",2025-10-22 07:20:16 GMT,75/232,0,ERROR,XX000,"could not find memoization table entry",,,,,,"SELECT DISTINCT ON(c.col3)
c.col3, c.col2 FROM
table1 c
INNER JOIN table2 tea ON
tea.col1 = c.col3 AND
tea.col2 <= c.col1 AND
tea.col3 >= c.col1 AND
tea.col4 = 't' AND tea.col5 = 'f'
WHERE
c.col1 >= '2025-10-01' AND
c.col1 <= '2025-10-31' AND
c.col3 = any(ARRAY[567,568,569,561,570,522,571,155,515,572,450,474,484,467,557,437,85,383,195,82,21,17,95,96,147,112,118,269,508,472,86,88,94,14,105,216,217,218,219,220,80,29,6,52,150,501,548,18,504,171,199,156,170,238,139,249,39,119,128,253,197,198,145,100,194,135,229,204,210,51,283
,48,293,505,401,214,476,307,309,8,542,306,319,320,322,321,323,324,325,326,327,328,329,330,331,332,334,336,339,340,341,109,137,509,312,512,142,73,154,344,514,516,372,369,288,520,521,305,190,510,360,265,333,345,549,473,215,271,347,130,159,4,551,196,433,7,552,165,511,518,523,524,525,529
,530,531,533,534,111,252,250,191,127,377,188,385,468,262,538,424,554,555,426,532,391,98,417,172,558,403,365,272,300,131,294,536,291,409,454,384,71,559,393,138,355,180,539,133,541,543,544,546,540,153,285,513,359,318,92,176,115,456,114,335,45,482,547,74,390,376,489,490,491,483,492,374,
427,299,494,495,206,348,144,418,47,246,255,486,475,496,416,479,441,301,497,545,152,179,292,42,464,126,478,498,308,175,174,158,40,41,181,178,227,108,443,422,213,166,346,168,354,366,406,435,440,447,392,452,429,296,453,167,451,460,461,160,87,162,314,517,286,200,260,466,203,442,103,560,5
62,353,563,564,565,566,1,23,25,35,68]) AND
c.col4 <= 2204007 AND
c.col5 = 'f'
ORDER BY
c.col3, c.col1, c.col4 DESC;",,,"psql","client backend",,0
2025-10-22 07:20:19.532 GMT,"postgres","flatman_test7",349111,"[local]",68f885b0.553b7,5,"SELECT",2025-10-22 07:20:16 GMT,75/0,0,LOG,00000,"temporary file: path ""base/pgsql_tmp/pgsql_tmp349111.0"", size 7593984",,,,,,,,,"psql","client backend",,0

It may be important that everytime we hit the issue, a temporary file is created.

Regards,

Emmanuel


[logo]
Emmanuel Touzery
Razvoj programske opreme
E: emmanuel.touzery@plandela.si
PLANDELA d.o.o.
Cesta Ljubljanske brigade 23a
1000 Ljubljana
www.plandela.si<https://www.plandela.si>


To sporočilo in morebitne priloge so poslovna skrivnost in namenjene izključno naslovniku, če ni navedeno drugače. Prosimo, da vsebino sporočila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. Če ste sporočilo prejeli pomotoma, vas prosimo, da takoj obvestite pošiljatelja, sporočilo in priloge pa nepovratno uničite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

  2. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T08:02:36Z

    On Wed, 22 Oct 2025 at 20:36, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
> When the issue occurs... With \set verbose, psql displays:
>
> ERROR:  XX000: could not find memoization table entry
> LOCATION:  cache_reduce_memory, nodeMemoize.c:484

Thanks for the recreation steps and dump file. I can recreate it here.
I'm just looking now.

David

  3. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T08:30:16Z

    On Wed, 22 Oct 2025 at 21:02, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Wed, 22 Oct 2025 at 20:36, Emmanuel Touzery
> <emmanuel.touzery@plandela.si> wrote:
> > When the issue occurs... With \set verbose, psql displays:
> >
> > ERROR:  XX000: could not find memoization table entry
> > LOCATION:  cache_reduce_memory, nodeMemoize.c:484
>
> Thanks for the recreation steps and dump file. I can recreate it here.
> I'm just looking now.

Are you able to confirm if you still get the error with:

SET jit_tuple_deforming TO off;

and if you do; does it still happen if you do:

SET jit TO off;

David

  4. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    Emmanuel Touzery <emmanuel.touzery@plandela.si> — 2025-10-22T08:48:12Z

    Hello,

Are you able to confirm if you still get the error with:

SET jit_tuple_deforming TO off;

and if you do; does it still happen if you do:

SET jit TO off;

I can confirm that seting jit_tuple_deforming to off is enough to fix the issue.

Thank you,

Emmanuel

To sporocilo in morebitne priloge so poslovna skrivnost in namenjene izkljucno naslovniku, ce ni navedeno drugace. Prosimo, da vsebino sporocila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. Ce ste sporocilo prejeli pomotoma, vas prosimo, da takoj obvestite posiljatelja, sporocilo in priloge pa nepovratno unicite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

  5. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T11:16:58Z

    On Wed, 22 Oct 2025 at 21:48, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
> I can confirm that seting jit_tuple_deforming to off is enough to fix the issue.

Thanks for checking that.

It does look like something weird is going on with the JIT deforming
code. I was playing around with the attached patch applied to look at
the tuple data. Rather surprisingly, the Datum value for negative ints
isn't the same for JIT deform vs normal deform.

Here's an example:

test=# set jit_above_cost=0;
SET
test=# create table a (a int);
CREATE TABLE
test=# insert into a values(-353400);
INSERT 0 1
test=# set jit=1;
SET
test=# explain analyze select * from a where a < 0;
NOTICE:  scan tuple = (a) (-353400) (4294613896)

test=# set jit=0;
SET
test=# explain analyze select * from a where a < 0;
NOTICE:  scan tuple = (a) (-353400) (18446744073709198216)

The format there is (<column>) (<logical value>) (<raw datum value>)

The lower 32 bits of these numbers are the same:

0000000000000000000000000000000011111111111110101001101110001000
1111111111111111111111111111111111111111111110101001101110001000

It's the upper 32 bits that are all 1s for no JIT and all zeros for JIT.

The reason this is causing Memoize indigestion is because when in
binary mode, the cache lookup is done with datum_image_hash() and
datum_image_eq(). I didn't check exactly, but I suspect that the value
we get when deforming the cache key's MinimalTuple has the Datum with
the non JIT representation.

To figure out why this is happening requires caffeination and staring
at llvmjit_deform.c for a long time. I suspect whatever is in there
that's meant to do what fetch_att() does isn't doing it the same way.

David

  6. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T11:27:40Z

    On Thu, 23 Oct 2025 at 00:16, David Rowley <dgrowleyml@gmail.com> wrote:
> To figure out why this is happening requires caffeination and staring
> at llvmjit_deform.c for a long time. I suspect whatever is in there
> that's meant to do what fetch_att() does isn't doing it the same way.

This code here:

LLVMValueRef v_tmp_loaddata;
LLVMTypeRef vartype = LLVMIntTypeInContext(lc, att->attlen * 8);
LLVMTypeRef vartypep = LLVMPointerType(vartype, 0);

v_tmp_loaddata =
LLVMBuildPointerCast(b, v_attdatap, vartypep, "");
v_tmp_loaddata = l_load(b, vartype, v_tmp_loaddata, "attr_byval");
v_tmp_loaddata = LLVMBuildZExt(b, v_tmp_loaddata, TypeDatum, "");

LLVMBuildStore(b, v_tmp_loaddata, v_resultp);

I don't speak this fluently by any means, but I think I see a zero
extension here up to the Datum width. That's not what fetch_att()
does.

Will look more tomorrow.

David

  7. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T21:04:55Z

    On Thu, 23 Oct 2025 at 00:27, David Rowley <dgrowleyml@gmail.com> wrote:
> v_tmp_loaddata = LLVMBuildZExt(b, v_tmp_loaddata, TypeDatum, "");

> Will look more tomorrow.

This seems to be a 1 liner fix. Basically, the zero extend should be a
signed extend. i.e, use LLVMBuildSExt().

Using [1] to get the extra debug output.

set jit_Above_Cost=0;
drop table if exists b;
create table b (a char, b smallint, c int);
insert into b values('-100'::char,-123,-1234);
insert into b values('123'::char,1234,12345);
set jit=1;
select * from b where c <> 0;
set jit=0;
select * from b where c <> 0;

With the patch, I get:

NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(124262785167352,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (124262785167320,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

SET
NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(124262785167352,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (124262785167320,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

And without, I get:

NOTICE:  scan tuple = (a,b,c) (-,-123,-1234) (135890207272952,65413,4294966062)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (135890207272920,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

SET
NOTICE:  scan tuple = (a,b,c) (-,-123,-1234)
(135890207272952,18446744073709551493,18446744073709550382)
NOTICE:  scan tuple = (a,b,c) (1,1234,12345) (135890207272920,1234,12345)
 a |  b   |   c
---+------+-------
 - | -123 | -1234
 1 | 1234 | 12345
(2 rows)

David

[1] https://www.postgresql.org/message-id/attachment/183414/debug_scan_data.diff

  8. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-22T21:13:57Z

    On Thu, 23 Oct 2025 at 10:04, David Rowley <dgrowleyml@gmail.com> wrote:
> This seems to be a 1 liner fix. Basically, the zero extend should be a
> signed extend. i.e, use LLVMBuildSExt().

The patch for the 1-liner.

David

  9. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-23T00:20:52Z

    On Thu, 23 Oct 2025 at 10:13, David Rowley <dgrowleyml@gmail.com> wrote:
> The patch for the 1-liner.

I've pushed the fix for this [1]. The only other instance I saw of
LLVMBuildZExt for TypeDatum is in llvmjit_expr.c for "case
EEOP_BOOL_NOT_STEP". I didn't adjust this as it wasn't clear to me
that it was a problem. I expect the value being zero extended there is
either a 0 or 1, so didn't want to risk touching that code if it isn't
broken.


Emmanuel,

You should be seeing it in the next minor releases due around
mid-November. In the meantime, I suggest turning off
jit_tuple_deforming.

Many thanks for reporting this and coming up with the steps and backup
to reproduce the issue.

David

[1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=10945148eec689cc114fe872d623c3c7c4f546e6

  10. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    Emmanuel Touzery <emmanuel.touzery@plandela.si> — 2025-10-23T06:30:22Z

    Hello,

 that's great news! Thank you!

 Maybe just a followup question: will the fix also be included in upcoming postgresql 17 minor releases? Seeing that it's not only PG18 that's affected.

 Thank you again!

Emmanuel

[logo]
Emmanuel Touzery
Razvoj programske opreme
E: emmanuel.touzery@plandela.si
PLANDELA d.o.o.
Cesta Ljubljanske brigade 23a
1000 Ljubljana
www.plandela.si<https://www.plandela.si>

________________________________
From: David Rowley <dgrowleyml@gmail.com>
Sent: Thursday, October 23, 2025 2:20 AM
To: Emmanuel Touzery <emmanuel.touzery@plandela.si>
Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Subject: Re: ERROR: XX000: could not find memoization table entry (reproducible)

On Thu, 23 Oct 2025 at 10:13, David Rowley <dgrowleyml@gmail.com> wrote:
> The patch for the 1-liner.

I've pushed the fix for this [1]. The only other instance I saw of
LLVMBuildZExt for TypeDatum is in llvmjit_expr.c for "case
EEOP_BOOL_NOT_STEP". I didn't adjust this as it wasn't clear to me
that it was a problem. I expect the value being zero extended there is
either a 0 or 1, so didn't want to risk touching that code if it isn't
broken.


Emmanuel,

You should be seeing it in the next minor releases due around
mid-November. In the meantime, I suggest turning off
jit_tuple_deforming.

Many thanks for reporting this and coming up with the steps and backup
to reproduce the issue.

David

[1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=10945148eec689cc114fe872d623c3c7c4f546e6

To sporo?ilo in morebitne priloge so poslovna skrivnost in namenjene izklju?no naslovniku, ?e ni navedeno druga?e. Prosimo, da vsebino sporo?ila in prilog varujete v skladu z veljavno zakonodajo in pogodbenimi obveznostmi. ?e ste sporo?ilo prejeli pomotoma, vas prosimo, da takoj obvestite po?iljatelja, sporo?ilo in priloge pa nepovratno uni?ite.

This e-mail and any attachments contain confidential information and is intended solely for the addressee unless stated otherwise. Please uphold strict confidentiality of the content of this message and attachments in accordance with the law and contractual obligations. If you have received this e-mail by mistake, please immediately notify the sender and irreversibly destroy this message and attachments.

  11. Re: ERROR: XX000: could not find memoization table entry (reproducible)

    David Rowley <dgrowleyml@gmail.com> — 2025-10-23T06:41:48Z

    On Thu, 23 Oct 2025 at 19:30, Emmanuel Touzery
<emmanuel.touzery@plandela.si> wrote:
>  Maybe just a followup question: will the fix also be included in upcoming postgresql 17 minor releases? Seeing that it's not only PG18 that's affected.

Yes, that's the plan. All the way back to PostgreSQL 13.

David