Re: remaining sql/json patches

amit <amitlangote09@gmail.com>

From: Amit Langote <amitlangote09@gmail.com>
To: Alexander Lakhin <exclusion@gmail.com>
Cc: jian he <jian.universality@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Himanshu Upadhyaya <upadhyaya.himanshu@gmail.com>, Erik Rijkers <er@xs4all.nl>, Andres Freund <andres@anarazel.de>, Andrew Dunstan <andrew@dunslane.net>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2024-04-05T07:09:29Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. SQL/JSON: Various improvements to SQL/JSON query function docs

  2. SQL/JSON: Fix some obsolete comments.

  3. SQL/JSON: Fix issues with DEFAULT .. ON ERROR / EMPTY

  4. JSON_TABLE: Add support for NESTED paths and columns

  5. Fix JsonExpr deparsing to emit QUOTES and WRAPPER correctly

  6. Fix typo introduced in 6185c9737

  7. Add basic JSON_TABLE() functionality

  8. Avoid splitting errmsg string to span multiple lines

  9. Add SQL/JSON query functions

  10. Implement various jsonpath methods

  11. Add soft error handling to some expression nodes

  12. Adjust populate_record_field() to handle errors softly

  13. Refactor code used by jsonpath executor to fetch variables

  14. Test EXPLAIN (FORMAT JSON) ... XMLTABLE

  15. Simplify productions for FORMAT JSON [ ENCODING name ]

  16. Add trailing commas to enum definitions

  17. doc: add missing <returnvalue> and whitespace

  18. Add more SQL/JSON constructor functions

  19. Rename a nonterminal used in SQL/JSON grammar

  20. Some refactoring to export json(b) conversion functions

  21. Don't include CaseTestExpr in JsonValueExpr.formatted_expr

  22. Code review for commit b6e1157e7d

  23. Pass constructName to transformJsonValueExpr()

  24. Unify JSON categorize type API and export for external use

  25. Make some indentation in gram.y consistent

  26. Allow most keywords to be used as column labels without requiring AS.

  27. Reduce size of backend scanner's tables.

  28. Use perfect hashing, instead of binary search, for keyword lookup.

Hi Alexander,

On Fri, Apr 5, 2024 at 3:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
>
> Hello Amit,
>
> 04.04.2024 15:02, Amit Langote wrote:
> > Pushed after fixing these and a few other issues.  I didn't include
> > the testing function you proposed in your other email.  It sounds
> > useful for testing locally but will need some work before we can
> > include it in the tree.
> >
> > I'll post the rebased 0002 tomorrow after addressing your comments.
>
> Please look at an assertion failure:
> TRAP: failed Assert("count <= tupdesc->natts"), File: "parse_relation.c", Line: 3048, PID: 1325146
>
> triggered by the following query:
> SELECT * FROM JSON_TABLE('0', '$' COLUMNS (js int PATH '$')),
>    COALESCE(row(1)) AS (a int, b int);
>
> Without JSON_TABLE() I get:
> ERROR:  function return row and query-specified return row do not match
> DETAIL:  Returned row contains 1 attribute, but query expects 2.

Thanks for the report.

Seems like it might be a pre-existing issue, because I can also
reproduce the crash with:

SELECT * FROM COALESCE(row(1)) AS (a int, b int);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

Backtrace:

#0  __pthread_kill_implementation (threadid=281472845250592,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x0000ffff806c4334 in __pthread_kill_internal (signo=6,
threadid=<optimized out>) at pthread_kill.c:78
#2  0x0000ffff8067c73c in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3  0x0000ffff80669034 in __GI_abort () at abort.c:79
#4  0x0000000000ad9d4c in ExceptionalCondition (conditionName=0xcbb368
"!(tupdesc->natts >= colcount)", errorType=0xcbb278 "FailedAssertion",
fileName=0xcbb2c8 "nodeFunctionscan.c",
    lineNumber=379) at assert.c:54
#5  0x000000000073edec in ExecInitFunctionScan (node=0x293d4ed0,
estate=0x293d51b8, eflags=16) at nodeFunctionscan.c:379
#6  0x0000000000724bc4 in ExecInitNode (node=0x293d4ed0,
estate=0x293d51b8, eflags=16) at execProcnode.c:248
#7  0x000000000071b1cc in InitPlan (queryDesc=0x292f5d78, eflags=16)
at execMain.c:1006
#8  0x0000000000719f6c in standard_ExecutorStart
(queryDesc=0x292f5d78, eflags=16) at execMain.c:252
#9  0x0000000000719cac in ExecutorStart (queryDesc=0x292f5d78,
eflags=0) at execMain.c:134
#10 0x0000000000945520 in PortalStart (portal=0x29399458, params=0x0,
eflags=0, snapshot=0x0) at pquery.c:527
#11 0x000000000093ee50 in exec_simple_query (query_string=0x29332d38
"SELECT * FROM COALESCE(row(1)) AS (a int, b int);") at
postgres.c:1175
#12 0x0000000000943cb8 in PostgresMain (argc=1, argv=0x2935d610,
dbname=0x2935d450 "postgres", username=0x2935d430 "amit") at
postgres.c:4297
#13 0x000000000087e978 in BackendRun (port=0x29356c00) at postmaster.c:4517
#14 0x000000000087e0bc in BackendStartup (port=0x29356c00) at postmaster.c:4200
#15 0x0000000000879638 in ServerLoop () at postmaster.c:1725
#16 0x0000000000878eb4 in PostmasterMain (argc=3, argv=0x292eeac0) at
postmaster.c:1398
#17 0x0000000000791db8 in main (argc=3, argv=0x292eeac0) at main.c:228

Backtrace looks a bit different with a query similar to yours:

SELECT * FROM generate_series(1, 1), COALESCE(row(1)) AS (a int, b int);
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>

#0  __pthread_kill_implementation (threadid=281472845250592,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x0000ffff806c4334 in __pthread_kill_internal (signo=6,
threadid=<optimized out>) at pthread_kill.c:78
#2  0x0000ffff8067c73c in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3  0x0000ffff80669034 in __GI_abort () at abort.c:79
#4  0x0000000000ad9d4c in ExceptionalCondition (conditionName=0xc903b0
"!(count <= tupdesc->natts)", errorType=0xc8f8c8 "FailedAssertion",
fileName=0xc8f918 "parse_relation.c",
    lineNumber=2649) at assert.c:54
#5  0x0000000000649664 in expandTupleDesc (tupdesc=0x293da188,
eref=0x293d7318, count=2, offset=0, rtindex=2, sublevels_up=0,
location=-1, include_dropped=true, colnames=0x0,
    colvars=0xffffc39253c8) at parse_relation.c:2649
#6  0x0000000000648d08 in expandRTE (rte=0x293d7390, rtindex=2,
sublevels_up=0, location=-1, include_dropped=true, colnames=0x0,
colvars=0xffffc39253c8) at parse_relation.c:2361
#7  0x0000000000849bd0 in build_physical_tlist (root=0x293d5318,
rel=0x293d88e8) at plancat.c:1681
#8  0x0000000000806ad0 in create_scan_plan (root=0x293d5318,
best_path=0x293cd888, flags=0) at createplan.c:605
#9  0x000000000080666c in create_plan_recurse (root=0x293d5318,
best_path=0x293cd888, flags=0) at createplan.c:389
#10 0x000000000080c4e8 in create_nestloop_plan (root=0x293d5318,
best_path=0x293d96f0) at createplan.c:4056
#11 0x0000000000807464 in create_join_plan (root=0x293d5318,
best_path=0x293d96f0) at createplan.c:1037
#12 0x0000000000806680 in create_plan_recurse (root=0x293d5318,
best_path=0x293d96f0, flags=1) at createplan.c:394
#13 0x000000000080658c in create_plan (root=0x293d5318,
best_path=0x293d96f0) at createplan.c:326
#14 0x0000000000816534 in standard_planner (parse=0x293d3728,
cursorOptions=256, boundParams=0x0) at planner.c:413
#15 0x00000000008162b4 in planner (parse=0x293d3728,
cursorOptions=256, boundParams=0x0) at planner.c:275
#16 0x000000000093e984 in pg_plan_query (querytree=0x293d3728,
cursorOptions=256, boundParams=0x0) at postgres.c:877
#17 0x000000000093eb04 in pg_plan_queries (querytrees=0x293d8018,
cursorOptions=256, boundParams=0x0) at postgres.c:967
#18 0x000000000093edc4 in exec_simple_query (query_string=0x29332d38
"SELECT * FROM generate_series(1, 1), COALESCE(row(1)) AS (a int, b
int);") at postgres.c:1142
#19 0x0000000000943cb8 in PostgresMain (argc=1, argv=0x2935d4f8,
dbname=0x2935d338 "postgres", username=0x2935d318 "amit") at
postgres.c:4297
#20 0x000000000087e978 in BackendRun (port=0x29356dd0) at postmaster.c:4517
#21 0x000000000087e0bc in BackendStartup (port=0x29356dd0) at postmaster.c:4200
#22 0x0000000000879638 in ServerLoop () at postmaster.c:1725
#23 0x0000000000878eb4 in PostmasterMain (argc=3, argv=0x292eeac0) at
postmaster.c:1398
#24 0x0000000000791db8 in main (argc=3, argv=0x292eeac0) at main.c:228

I suspect the underlying issue is the same, though I haven't figured
out what it is, except a guess that addRangeTableEntryForFunction()
might be missing something to handle this sanely.

Reproducible down to v12.

-- 
Thanks, Amit Langote