Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options
Tatsuo Ishii <ishii@postgresql.org>
From: Tatsuo Ishii <ishii@postgresql.org>
To: ojford@gmail.com
Cc: peter@eisentraut.org, li.evan.chao@gmail.com, krasiyan@gmail.com,
tgl@sss.pgh.pa.us, vik@postgresfriends.org, andrew@tao11.riddles.org.uk,
david@fetter.org, pgsql-hackers@lists.postgresql.org
Date: 2025-10-02T12:15:50Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Fix coding style with "else".
- 20628b62e46e 19 (unreleased) landed
-
Fix multi WinGetFuncArgInFrame/Partition calls with IGNORE NULLS.
- 2d7b247cb414 19 (unreleased) landed
-
Fix Coverity issue reported in commit 2273fa32bce.
- dd766a441d69 19 (unreleased) landed
-
Use ereport rather than elog in WinCheckAndInitializeNullTreatment.
- 5f3808646f67 19 (unreleased) landed
-
Avoid uninitialized-variable warnings from older compilers.
- 71540dcdcb22 19 (unreleased) cited
-
Fix Coverity issues reported in commit 25a30bbd423.
- 2273fa32bce7 19 (unreleased) landed
-
Improve EXPLAIN's display of window functions.
- 8b1b342544b6 18.0 cited
-
Automatically generate node support functions
- 964d01ae90c3 16.0 cited
Attachments
- v20-0001-Modify-parse-analysis-modules-to-accept-RESPECT-.patch (application/octet-stream) patch v20-0001
- v20-0002-Modify-get_windowfunc_expr_helper-to-handle-IGNO.patch (application/octet-stream) patch v20-0002
- v20-0003-Modify-eval_const_expressions_mutator-to-handle-.patch (application/octet-stream) patch v20-0003
- v20-0004-Modify-executor-and-window-functions-to-handle-I.patch (application/octet-stream) patch v20-0004
- v20-0005-Modify-documents-to-add-null-treatment-clause.patch (application/octet-stream) patch v20-0005
- v20-0006-Modify-window-function-regression-tests-to-test-.patch (application/octet-stream) patch v20-0006
> Thank you for the patch. (I noticed int8 is also missing). > > I have looked into the commit 964d01ae90c3 which was made by Peter. I > have quick read through the discussion to know why uint8/uint16 (and > int8) are missing in gen_node_support.pl. Unfortunately I have no > clear idea why these data types are missing in the script. > > Peter, > Maybe you wanted to limit the data types that are actually used at > that point? If so, probably we should only add uint8 support this time > (uint8 is only needed to implement $Subject for now). What do you > think? I decided not to include the fix to gen_node_support.pl for now and commit the patch without it. We could revisit it later on. So here is the commit message I would like to propose. For the technical part please look at the message. Non technical part: First of all the author is Oliver (no doubt). I would like to be listed as a co-author since I wrote the not null cache part. Next is reviewers. Actually the first effor to implement null treatment clause was back to 9.3 era (2013) at least. After that multiple trials to implemnt the feature happend but they had faded away. I think we don't need to include all of those who joined the old discussions as reviewers. So I started to check from the discussion: https://postgr.es/m/flat/CAGMVOdsbtRwE_4+v8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A@mail.gmail.com because it's refered to by the commit fest entry. Oliver and others, I love to hear your comment! BTW, Oliver's last patch made the CF bot to misunderstand the patch contents, which was not actually the main patch. So I attach the same patch as v20. ---------------------------------------------------------------------- Add IGNORE NULLS/RESPECT NULLS option to Window functions. Add IGNORE NULLS/RESPECT NULLS option (null treatment clause) to lead, lag, first_value, last_value and nth_value window functions. If unspecified, the default is RESPECT NULLS which includes NULL values in any result calculation. IGNORE NULLS ignores NULL values. Built-in window functions are modified to call new API WinCheckAndInitializeNullTreatment() to indicate whether they accept IGNORE NULLS/RESPECT NULLS option or not (the API can be called by user defined window functions as well). If WinGetFuncArgInPartition's allowNullTreatment argument is true and IGNORE NULLS option is given, WinGetFuncArgInPartition() or WinGetFuncArgInFrame() will return evaluated function's argument expression on specified non NULL row (if it exists) in the partition or the frame. When IGNORE NULLS option is given, window functions need to visit and evaluate same rows over and over again to look for non null rows. To mitigate the issue, 2-bit not null information array is created while executing window functions to remember whether the row has been already evaluated to NULL or NOT NULL. If already evaluated, we could skip some the evaluation work, thus we could get better performance. Author: Oliver Ford <ojford@gmail.com> Co-authored-by: Tatsuo Ishii <ishii@postgresql.org> Reviewed-by: Andrew Gierth <andrew@tao11.riddles.org.uk> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: David Fetter <david@fetter.org> Reviewed-by: Vik Fearing <vik@postgresfriends.org> Reviewed-by: "David G. Johnston" <david.g.johnston@gmail.com> Reviewed-by: Krasiyan Andreev <krasiyan@gmail.com> Reviewed-by: Chao Li <lic@highgo.com> Discussion: https://postgr.es/m/flat/CAGMVOdsbtRwE_4+v8zjH1d9xfovDeQAGLkP_B6k69_VoFEgX-A@mail.gmail.com ---------------------------------------------------------------------- Best regards, -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp