Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add special case fast-paths for strict functions

  2. Replace EEOP_DONE with special steps for return/no return

  1. Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2023-10-12T09:48:35Z

    The attached patch adds special-case expression steps for common sets of steps
in the executor to shave a few cycles off during execution, and make the JIT
generated code simpler.

* Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function calls of
  strict functions with 1 or 2 arguments (EEOP_FUNCEXPR_STRICT remains used for
  > 2 arguments).
* Adds EEOP_AGG_STRICT_INPUT_CHECK_ARGS_1 which is a special case for the
  common case of one arg aggs.
* Replace EEOP_DONE with EEOP_DONE_RETURN and EEOP_DONE_NO_RETURN to be able to
  skip extra setup for steps which are only interested in the side effects.

Stressing the EEOP_FUNCEXPR_STRICT_* steps specifically shows a 1.5%
improvement and pgbench over the branch shows a ~1% improvement in TPS (both
measured over 6 runs with outliers removed).

EEOP_FUNCEXPR_STRICT_* (10M iterations):
    master  : (7503.317, 7553.691, 7634.524) 
    patched : (7422.756, 7455.120, 7492.393)

pgbench:
    master  : (3653.83, 3792.97, 3863.70)
    patched : (3743.04, 3830.02, 3869.80)

This patch was extracted from a larger body of work from Andres [0] aiming at
providing the necessary executor infrastructure for making JIT expression
caching possible.  This patch, and more which are to be submitted, is however
separate in the sense that it is not part of the infrastructure, it's an
improvements on its own.

Thoughts?

--
Daniel Gustafsson

[0]: https://postgr.es/m/20191023163849.sosqbfs5yenocez3@alap3.anarazel.de

  2. Re: Special-case executor expression steps for common combinations

    Heikki Linnakangas <hlinnaka@iki.fi> — 2023-10-12T10:24:27Z

    On 12/10/2023 12:48, Daniel Gustafsson wrote:
> The attached patch adds special-case expression steps for common sets of steps
> in the executor to shave a few cycles off during execution, and make the JIT
> generated code simpler.
> 
> * Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function calls of
>    strict functions with 1 or 2 arguments (EEOP_FUNCEXPR_STRICT remains used for
>    > 2 arguments).
> * Adds EEOP_AGG_STRICT_INPUT_CHECK_ARGS_1 which is a special case for the
>    common case of one arg aggs.

Are these relevant when JITting? I'm a little sad if the JIT compiler 
cannot unroll these on its own. Is there something we could do to hint 
it, so that it could treat the number of arguments as a constant?

I understand that this can give a small boost in interpreter mode, so 
maybe we should do it in any case. But I'd like to know if we're missing 
a trick with the JITter, before we mask it with this.

> * Replace EEOP_DONE with EEOP_DONE_RETURN and EEOP_DONE_NO_RETURN to be able to
>    skip extra setup for steps which are only interested in the side effects.

I'm a little surprised if this makes a measurable performance 
difference, but sure, why not. It seems nice to be more explicit when 
you don't expect a return value.

-- 
Heikki Linnakangas
Neon (https://neon.tech)

  3. Re: Special-case executor expression steps for common combinations

    David Rowley <dgrowleyml@gmail.com> — 2023-10-12T12:04:21Z

    On Thu, 12 Oct 2023 at 22:54, Daniel Gustafsson <daniel@yesql.se> wrote:
> EEOP_FUNCEXPR_STRICT_* (10M iterations):
>     master  : (7503.317, 7553.691, 7634.524)
>     patched : (7422.756, 7455.120, 7492.393)
>
> pgbench:
>     master  : (3653.83, 3792.97, 3863.70)
>     patched : (3743.04, 3830.02, 3869.80)
>
> Thoughts?

Did any of these tests compile the expression with JIT?

If not, how does the performance compare for a query that JITs the expression?

David

  4. Re: Special-case executor expression steps for common combinations

    Andres Freund <andres@anarazel.de> — 2023-10-12T17:52:04Z

    Hi,

On 2023-10-12 13:24:27 +0300, Heikki Linnakangas wrote:
> On 12/10/2023 12:48, Daniel Gustafsson wrote:
> > The attached patch adds special-case expression steps for common sets of steps
> > in the executor to shave a few cycles off during execution, and make the JIT
> > generated code simpler.
> > 
> > * Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function calls of
> >    strict functions with 1 or 2 arguments (EEOP_FUNCEXPR_STRICT remains used for
> >    > 2 arguments).
> > * Adds EEOP_AGG_STRICT_INPUT_CHECK_ARGS_1 which is a special case for the
> >    common case of one arg aggs.
> 
> Are these relevant when JITting? I'm a little sad if the JIT compiler cannot
> unroll these on its own. Is there something we could do to hint it, so that
> it could treat the number of arguments as a constant?

I think it's mainly important for interpreted execution.


> >    skip extra setup for steps which are only interested in the side effects.
> 
> I'm a little surprised if this makes a measurable performance difference,
> but sure, why not. It seems nice to be more explicit when you don't expect a
> return value.

IIRC this is more interesting for JIT than the above, because it allows LLVM
to know that the return value isn't needed and thus doesn't need to be
computed.

Greetings,

Andres Freund

  5. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2023-10-19T10:11:13Z

    > On 12 Oct 2023, at 19:52, Andres Freund <andres@anarazel.de> wrote:
> On 2023-10-12 13:24:27 +0300, Heikki Linnakangas wrote:
>> On 12/10/2023 12:48, Daniel Gustafsson wrote:

>>> The attached patch adds special-case expression steps for common sets of steps
>>> in the executor to shave a few cycles off during execution, and make the JIT
>>> generated code simpler.
>>> 
>>> * Adds EEOP_FUNCEXPR_STRICT_1 and EEOP_FUNCEXPR_STRICT_2 for function calls of
>>>   strict functions with 1 or 2 arguments (EEOP_FUNCEXPR_STRICT remains used for
>>>> 2 arguments).
>>> * Adds EEOP_AGG_STRICT_INPUT_CHECK_ARGS_1 which is a special case for the
>>>   common case of one arg aggs.
>> 
>> Are these relevant when JITting? I'm a little sad if the JIT compiler cannot
>> unroll these on its own. Is there something we could do to hint it, so that
>> it could treat the number of arguments as a constant?
> 
> I think it's mainly important for interpreted execution.

Agreed.

>>>   skip extra setup for steps which are only interested in the side effects.
>> 
>> I'm a little surprised if this makes a measurable performance difference,
>> but sure, why not. It seems nice to be more explicit when you don't expect a
>> return value.

Right, performance benefits aside it does improve readability IMHO.

> IIRC this is more interesting for JIT than the above, because it allows LLVM
> to know that the return value isn't needed and thus doesn't need to be
> computed.

Correct, this is important to the JIT code which no longer has to perform two
Loads and one Store in order to get nothing, but can instead fastpath to
building a zero returnvalue.

--
Daniel Gustafsson

  6. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2024-06-20T15:22:41Z

    On 10/12/23 11:48 AM, Daniel Gustafsson wrote:
> Thoughts?

I have looked at the patch and it still applies, builds and passes the 
test cases and I personally think these optimizations are pretty much 
no-brainers that we should do and it is a pity nobody has had the time 
to review this patch.

1) The no-return case should help with the JIT, making jitted code faster.

2) The specialized strict steps helps with many common queries in the 
interpreted mode.

The code itself looks really good (great work!) but I have two comments 
on it.

1) I think the patch should be split into two. The two different 
optimizations are not related at all other than that they create 
specialized versions of expressions steps. Having them as separate makes 
the commit history easier to read for future developers.

2) We could generate functions which return void rather than NULL and 
therefore not have to do a return at all but I am not sure that small 
optimization and extra clarity would be worth the hassle. The current 
approach with adding Assert() is ok with me. Daniel, what do you think?

Andreas

  7. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2024-06-20T15:25:06Z

    On 6/20/24 5:22 PM, Andreas Karlsson wrote:
> On 10/12/23 11:48 AM, Daniel Gustafsson wrote:
>> Thoughts?
> 
> I have looked at the patch and it still applies, builds and passes the 
> test cases and I personally think these optimizations are pretty much 
> no-brainers that we should do and it is a pity nobody has had the time 
> to review this patch.

Forgot to write that I am planning to also try to do so benchmarks to 
see if I can reproduce the speedups. :)

Andreas

  8. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2024-07-04T16:26:18Z

    > On 20 Jun 2024, at 17:22, Andreas Karlsson <andreas@proxel.se> wrote:
> 
> On 10/12/23 11:48 AM, Daniel Gustafsson wrote:
>> Thoughts?
> 
> I have looked at the patch and it still applies, builds and passes the test cases and I personally think these optimizations are pretty much no-brainers that we should do and it is a pity nobody has had the time to review this patch.
> 
> 1) The no-return case should help with the JIT, making jitted code faster.
> 
> 2) The specialized strict steps helps with many common queries in the interpreted mode.
> 
> The code itself looks really good (great work!) but I have two comments on it.

Thanks for review!

> 1) I think the patch should be split into two. The two different optimizations are not related at all other than that they create specialized versions of expressions steps. Having them as separate makes the commit history easier to read for future developers.

That's a good point, the attached v2 splits it into two separate commits.

> 2) We could generate functions which return void rather than NULL and therefore not have to do a return at all but I am not sure that small optimization and extra clarity would be worth the hassle. The current approach with adding Assert() is ok with me. Daniel, what do you think?

I'm not sure that would move the needle enough to warrant the extra complexity.
It could be worth pursuing, but it can be done separately from this.

--
Daniel Gustafsson

  9. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2024-07-22T12:33:16Z

    On 7/4/24 6:26 PM, Daniel Gustafsson wrote:
>> 2) We could generate functions which return void rather than NULL and therefore not have to do a return at all but I am not sure that small optimization and extra clarity would be worth the hassle. The current approach with adding Assert() is ok with me. Daniel, what do you think?
> 
> I'm not sure that would move the needle enough to warrant the extra complexity.
> It could be worth pursuing, but it can be done separately from this.

Agreed.

I looked some more at the patch and have a suggestion for code style. 
Attaching the diff. Do with them what you wish, for me they make to code 
easier to read.

Andreas

  10. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2024-07-22T21:25:07Z

    I have bench marked the two patches now and failed to measure any 
speedup or slowdown from the first patch (removing return) but I think 
it is a good idea anyway.

For the second patch (optimize strict) I managed to measure a ~1% speed 
up for the following query "SELECT sum(x + y + 1) FROM t;" over one 
million rows.

I would say both patches are ready for committer modulo my proposed 
style fixes.

Andreas

  11. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2024-09-10T08:54:36Z

    > On 22 Jul 2024, at 23:25, Andreas Karlsson <andreas@proxel.se> wrote:
> 
> I have bench marked the two patches now and failed to measure any speedup or slowdown from the first patch (removing return) but I think it is a good idea anyway.
> 
> For the second patch (optimize strict) I managed to measure a ~1% speed up for the following query "SELECT sum(x + y + 1) FROM t;" over one million rows.

That's expected, this is mostly about refactoring the code to simplifying the
JITed code (and making tiny strides towards JIT expression caching).

> I would say both patches are ready for committer modulo my proposed style fixes.

I am a bit wary about removing the out_error label and goto since it may open
up for reports from static analyzers about control reaching the end of a
non-void function without a return. The other change has been incorporated.

The attached v3 is a rebase to handle executor changes done since v2, with the
above mentioned fix as well.  If there are no objections I think we should
apply this version.

--
Daniel Gustafsson

  12. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2024-09-13T13:01:03Z

    On 9/10/24 10:54 AM, Daniel Gustafsson wrote:
>> On 22 Jul 2024, at 23:25, Andreas Karlsson <andreas@proxel.se> wrote:
>>
>> I have bench marked the two patches now and failed to measure any speedup or slowdown from the first patch (removing return) but I think it is a good idea anyway.
>>
>> For the second patch (optimize strict) I managed to measure a ~1% speed up for the following query "SELECT sum(x + y + 1) FROM t;" over one million rows.
> 
> That's expected, this is mostly about refactoring the code to simplifying the
> JITed code (and making tiny strides towards JIT expression caching).

Yup! Expected and nice tiny speedup.

>> I would say both patches are ready for committer modulo my proposed style fixes.
> 
> I am a bit wary about removing the out_error label and goto since it may open
> up for reports from static analyzers about control reaching the end of a
> non-void function without a return. The other change has been incorporated.
> 
> The attached v3 is a rebase to handle executor changes done since v2, with the
> above mentioned fix as well.  If there are no objections I think we should
> apply this version.

Sounds good to me and in my opinion this is ready to be committed.

Andreas

  13. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-05T10:00:24Z

    > On 13 Sep 2024, at 15:01, Andreas Karlsson <andreas@proxel.se> wrote:
> 
> On 9/10/24 10:54 AM, Daniel Gustafsson wrote:
>>> On 22 Jul 2024, at 23:25, Andreas Karlsson <andreas@proxel.se> wrote:
>>> 
>>> I have bench marked the two patches now and failed to measure any speedup or slowdown from the first patch (removing return) but I think it is a good idea anyway.
>>> 
>>> For the second patch (optimize strict) I managed to measure a ~1% speed up for the following query "SELECT sum(x + y + 1) FROM t;" over one million rows.
>> That's expected, this is mostly about refactoring the code to simplifying the
>> JITed code (and making tiny strides towards JIT expression caching).
> 
> Yup! Expected and nice tiny speedup.
> 
>>> I would say both patches are ready for committer modulo my proposed style fixes.
>> I am a bit wary about removing the out_error label and goto since it may open
>> up for reports from static analyzers about control reaching the end of a
>> non-void function without a return. The other change has been incorporated.
>> The attached v3 is a rebase to handle executor changes done since v2, with the
>> above mentioned fix as well.  If there are no objections I think we should
>> apply this version.
> 
> Sounds good to me and in my opinion this is ready to be committed.

This fell off the ever-growing TODO again.  Re-reading it I still think it's a
good idea, it applied almost cleanly still and still gives a slight performance
improvement along with the more interesting refactoring which will make caching
of expressions easier down the line.

Attached is a rebased v4, unless there are objections I'll go ahead with this
version.

--
Daniel Gustafsson

  14. Re: Special-case executor expression steps for common combinations

    Andreas Karlsson <andreas@proxel.se> — 2025-03-08T16:15:23Z

    On 2/5/25 11:00 AM, Daniel Gustafsson wrote:
> This fell off the ever-growing TODO again.  Re-reading it I still think it's a
> good idea, it applied almost cleanly still and still gives a slight performance
> improvement along with the more interesting refactoring which will make caching
> of expressions easier down the line.
> 
> Attached is a rebased v4, unless there are objections I'll go ahead with this
> version.

+1 Still seems like a nice change to me too.

Andreas

  15. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2025-03-09T22:35:40Z

    > On 8 Mar 2025, at 17:15, Andreas Karlsson <andreas@proxel.se> wrote:
> 
> On 2/5/25 11:00 AM, Daniel Gustafsson wrote:
>> This fell off the ever-growing TODO again.  Re-reading it I still think it's a
>> good idea, it applied almost cleanly still and still gives a slight performance
>> improvement along with the more interesting refactoring which will make caching
>> of expressions easier down the line.
>> Attached is a rebased v4, unless there are objections I'll go ahead with this
>> version.
> 
> +1 Still seems like a nice change to me too.

Thanks, I actually had it staged to go in early next week.

--
Daniel Gustafsson

  16. Re: Special-case executor expression steps for common combinations

    Daniel Gustafsson <daniel@yesql.se> — 2025-03-12T08:57:25Z

    > On 9 Mar 2025, at 23:35, Daniel Gustafsson <daniel@yesql.se> wrote:
>> On 8 Mar 2025, at 17:15, Andreas Karlsson <andreas@proxel.se> wrote:

>> +1 Still seems like a nice change to me too.
> 
> Thanks, I actually had it staged to go in early next week.

..which was done yesterday, thanks for review!

--
Daniel Gustafsson