Horton

Thread

Commits

  1. Update out of date comments in pg_trgm

  2. Check for interrupts in pg_trgm word similarity

  3. Add strict_word_similarity to pg_trgm module

  1. Missing CFI in iterate_word_similarity()

    Robins Tharakan <tharakan@gmail.com> — 2022-08-02T02:41:10Z

    Hi,

For long strings, iterate_word_similarity() can run into long-running
tight-loops without honouring interrupts or statement_timeouts. For
example:

postgres=# set statement_timeout='1s';
SET
postgres=# select 1 where repeat('1.1',80000) %>> 'Lorem ipsum dolor sit amet';
?column?
----------
(0 rows)
Time: 29615.842 ms (00:29.616)

The associated perf report:

+ 99.98% 0.00% postgres postgres [.] ExecQual
+ 99.98% 0.00% postgres postgres [.] ExecEvalExprSwitchContext
+ 99.98% 0.00% postgres pg_trgm.so [.] strict_word_similarity_commutator_op
+ 99.98% 0.00% postgres pg_trgm.so [.] calc_word_similarity
+ 99.68% 99.47% postgres pg_trgm.so [.] iterate_word_similarity
0.21% 0.03% postgres postgres [.] pg_qsort
0.16% 0.00% postgres [kernel.kallsyms] [k] asm_sysvec_apic_timer_interrupt
0.16% 0.00% postgres [kernel.kallsyms] [k] sysvec_apic_timer_interrupt
0.16% 0.11% postgres [kernel.kallsyms] [k] __softirqentry_text_start
0.16% 0.00% postgres [kernel.kallsyms] [k] irq_exit_rcu

Adding CHECK_FOR_INTERRUPTS() ensures that such queries respond to
statement_timeout & Ctrl-C signals. With the patch applied, the
above query will interrupt more quickly:

postgres=# select 1 where repeat('1.1',80000) %>> 'Lorem ipsum dolor sit amet';
ERROR: canceling statement due to statement timeout
Time: 1000.768 ms (00:01.001)

Please find the patch attached. The patch does not show any performance
regressions when run against the above use-case. Thanks to SQLSmith
for indirectly leading me to this scenario.

-
Robins Tharakan
Amazon Web Services

  2. Re: Missing CFI in iterate_word_similarity()

    Daniel Gustafsson <daniel@yesql.se> — 2022-09-02T12:26:50Z

    > On 2 Aug 2022, at 04:41, Robins Tharakan <tharakan@gmail.com> wrote:

> For long strings, iterate_word_similarity() can run into long-running
> tight-loops without honouring interrupts or statement_timeouts.

> Adding CHECK_FOR_INTERRUPTS() ensures that such queries respond to
> statement_timeout & Ctrl-C signals. With the patch applied, the
> above query will interrupt more quickly:

Makes sense.  While this might be a bit of a theoretical issue given the
lengths required, the fix is still sane and any such query should honor
statement timeouts (especially in a trusted extension).

> Please find the patch attached. The patch does not show any performance
> regressions when run against the above use-case.

I wasn't able to find one either.

+       CHECK_FOR_INTERRUPTS();
+
        /* Get index of next trigram */
        int         trgindex = trg2indexes[i];

Placing code before declarations will generate a compiler warning, so the check
must go after trgindex is declared.  I've fixed that in the attached to get the
cfbot green.  Marking this ready for committer in the meantime.

Looking at this I also noticed that commit be8a7a68662 changed the check_only
param to instead use a flag value but didn't update all comments.  0002 fixes
that while in there.

--
Daniel Gustafsson		https://vmware.com/

  3. Re: Missing CFI in iterate_word_similarity()

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-09-02T12:57:00Z

    Daniel Gustafsson <daniel@yesql.se> writes:
> Placing code before declarations will generate a compiler warning, so the check
> must go after trgindex is declared.  I've fixed that in the attached to get the
> cfbot green.  Marking this ready for committer in the meantime.

I noticed the same thing, but sticking the CFI immediately after the
declaration didn't read well either.  I was considering moving it to
the bottom of the loop instead of that.  A possible objection is that
if there's ever a "continue;" in the loop, those iterations would bypass
the CFI; but we don't necessarily need a CFI every time.

			regards, tom lane

  4. Re: Missing CFI in iterate_word_similarity()

    Daniel Gustafsson <daniel@yesql.se> — 2022-09-02T13:06:34Z

    > On 2 Sep 2022, at 14:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Daniel Gustafsson <daniel@yesql.se> writes:
>> Placing code before declarations will generate a compiler warning, so the check
>> must go after trgindex is declared.  I've fixed that in the attached to get the
>> cfbot green.  Marking this ready for committer in the meantime.
> 
> I noticed the same thing, but sticking the CFI immediately after the
> declaration didn't read well either.  I was considering moving it to
> the bottom of the loop instead of that.  

I was contemplating that too, but kept it at the top after seeing quite a few
examples of that in other contrib modules (like amcheck/verify_nbtree.c and
pg_visibility/pg_visibility.c).  I don't have any strong feelings either way,
I'm happy to move it last.

> A possible objection is that
> if there's ever a "continue;" in the loop, those iterations would bypass
> the CFI; but we don't necessarily need a CFI every time.

Yeah, I don't think we need to worry about that.  If an added continue;
shortcuts the loop to the point where keeping the CFI last becomes a problem
then it's probably time to look at rewriting the loop.

--
Daniel Gustafsson		https://vmware.com/

  5. Re: Missing CFI in iterate_word_similarity()

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-09-02T13:16:13Z

    Daniel Gustafsson <daniel@yesql.se> writes:
>> On 2 Sep 2022, at 14:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I noticed the same thing, but sticking the CFI immediately after the
>> declaration didn't read well either.  I was considering moving it to
>> the bottom of the loop instead of that.  

> I was contemplating that too, but kept it at the top after seeing quite a few
> examples of that in other contrib modules (like amcheck/verify_nbtree.c and
> pg_visibility/pg_visibility.c).  I don't have any strong feelings either way,
> I'm happy to move it last.

You could keep it at the top, but then I'd be inclined to split up
the existing code:

        int            trgindex;

        CHECK_FOR_INTERRUPTS();

        /* Get index of next trigram */
        trgindex = trg2indexes[i];

        /* Update last position of this trigram */
        ...

What's annoying me about the one-liner fix is that it makes it
look like CFI is part of the "Get index" action.

			regards, tom lane

  6. Re: Missing CFI in iterate_word_similarity()

    Daniel Gustafsson <daniel@yesql.se> — 2022-09-02T13:22:06Z

    > On 2 Sep 2022, at 15:16, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> What's annoying me about the one-liner fix is that it makes it
> look like CFI is part of the "Get index" action.

Thats a good point, I'll split the code up to make it clearer.

--
Daniel Gustafsson		https://vmware.com/

  7. Re: Missing CFI in iterate_word_similarity()

    Daniel Gustafsson <daniel@yesql.se> — 2022-09-05T09:23:47Z

    > On 2 Sep 2022, at 15:22, Daniel Gustafsson <daniel@yesql.se> wrote:
> 
>> On 2 Sep 2022, at 15:16, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
>> What's annoying me about the one-liner fix is that it makes it
>> look like CFI is part of the "Get index" action.
> 
> Thats a good point, I'll split the code up to make it clearer.

Done that way and pushed, thanks!

--
Daniel Gustafsson		https://vmware.com/