Horton
Menu

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add EXPLAIN (IO) instrumentation for SeqScan

  2. Allocate separate DSM chunk for parallel Index[Only]Scan instrumentation

  1. [PATCH] Add RetrieveInstrumentation hook for CustomScan providers

    Siddharth Kothari <sidkot@google.com> — 2026-02-18T09:39:15Z

    Dear PostgreSQL Hackers,

This email proposes a patch to enhance the CustomScan provider interface.
The patch file,
0001-Add-RetrieveInstrumentationCustomScan-hook-for-Custo.patch, is
attached.

*Problem:*

CustomScan providers currently lack a standard method to aggregate
instrumentation data from parallel workers back to the leader process
before the Dynamic Shared Memory (DSM) segment is unlinked. This makes it
difficult to gather comprehensive performance metrics from parallelized
custom scans.

*Solution:*

This patch introduces a new optional hook, RetrieveInstrumentationCustomScan,
to the CustomExecMethods struct. This hook allows custom scan providers to
implement logic to collect and consolidate instrumentation from shared
memory or worker states during the parallel query cleanup phase. This hook
is invoked via the new ExecCustomScanRetrieveInstrumentation function,
called from ExecParallelRetrieveInstrumentation for T_CustomScanState nodes.
Since the hook is optional (checked for NULL before calling), it maintains
full backward compatibility.

*Testing & Compatibility:*

   - The patch compiles and passes all core regression tests (make
   check-world) on my x86_64 instance.
   - The changes are not platform-specific.
   - Regression Tests: This patch provides a new *capability* for custom
   scan providers. Since the hook's functionality is only realized when
   implemented by an extension, specific tests would naturally reside within
   that extension rather than in the core regression suite.

This patch does not directly address a specific item on the official TODO
list but enhances the extensibility framework.

I believe this patch is complete and ready for review. I look forward to
any feedback and am happy to make revisions. I will also add this patch to
the next CommitFest.

Thank you,

Siddharth Kothari

  2. Re: [PATCH] Add RetrieveInstrumentation hook for CustomScan providers

    Siddharth Kothari <sidkot@google.com> — 2026-04-08T05:27:16Z

    Hi everyone,

I’m just checking in to see if anyone has had a chance to look at this or
if there’s any further information I should provide to help with the
review. I have also added the patch to PG20-1 CF queue, the link is
https://commitfest.postgresql.org/patch/6524/.

Thanks,
Siddharth

On Wed, Feb 18, 2026 at 3:09 PM Siddharth Kothari <sidkot@google.com> wrote:

> Dear PostgreSQL Hackers,
>
> This email proposes a patch to enhance the CustomScan provider interface.
> The patch file,
> 0001-Add-RetrieveInstrumentationCustomScan-hook-for-Custo.patch, is
> attached.
>
> *Problem:*
>
> CustomScan providers currently lack a standard method to aggregate
> instrumentation data from parallel workers back to the leader process
> before the Dynamic Shared Memory (DSM) segment is unlinked. This makes it
> difficult to gather comprehensive performance metrics from parallelized
> custom scans.
>
> *Solution:*
>
> This patch introduces a new optional hook,
> RetrieveInstrumentationCustomScan, to the CustomExecMethods struct. This
> hook allows custom scan providers to implement logic to collect and
> consolidate instrumentation from shared memory or worker states during the
> parallel query cleanup phase. This hook is invoked via the new
> ExecCustomScanRetrieveInstrumentation function, called from
> ExecParallelRetrieveInstrumentation for T_CustomScanState nodes. Since
> the hook is optional (checked for NULL before calling), it maintains full
> backward compatibility.
>
> *Testing & Compatibility:*
>
>    - The patch compiles and passes all core regression tests (make
>    check-world) on my x86_64 instance.
>    - The changes are not platform-specific.
>    - Regression Tests: This patch provides a new *capability* for custom
>    scan providers. Since the hook's functionality is only realized when
>    implemented by an extension, specific tests would naturally reside within
>    that extension rather than in the core regression suite.
>
> This patch does not directly address a specific item on the official TODO
> list but enhances the extensibility framework.
>
> I believe this patch is complete and ready for review. I look forward to
> any feedback and am happy to make revisions. I will also add this patch to
> the next CommitFest.
>
> Thank you,
>
> Siddharth Kothari
>

  3. Re: [PATCH] Add RetrieveInstrumentation hook for CustomScan providers

    Siddharth Kothari <sidkot@google.com> — 2026-06-23T09:08:20Z

    Hi hackers,

Bumping this thread as this small patch is passing all CF checks but needs
a reviewer.

*The Problem:* Right now, if you write a parallel CustomScan extension, it
is impossible to get performance metrics from parallel workers. The data is
destroyed during DSM unlinking before the leader can grab it.

*The Fix:* This patch adds an optional RetrieveInstrumentation hook to
CustomExecMethods. It has zero overhead for existing extensions (it's a
simple NULL check) but unlocks metric aggregation for parallel custom scans.

Please let me know your thoughts.

Thanks,

Siddharth

On Wed, Apr 8, 2026 at 10:57 AM Siddharth Kothari <sidkot@google.com> wrote:

> Hi everyone,
>
> I’m just checking in to see if anyone has had a chance to look at this or
> if there’s any further information I should provide to help with the
> review. I have also added the patch to PG20-1 CF queue, the link is
> https://commitfest.postgresql.org/patch/6524/.
>
> Thanks,
> Siddharth
>
> On Wed, Feb 18, 2026 at 3:09 PM Siddharth Kothari <sidkot@google.com>
> wrote:
>
>> Dear PostgreSQL Hackers,
>>
>> This email proposes a patch to enhance the CustomScan provider interface.
>> The patch file,
>> 0001-Add-RetrieveInstrumentationCustomScan-hook-for-Custo.patch, is
>> attached.
>>
>> *Problem:*
>>
>> CustomScan providers currently lack a standard method to aggregate
>> instrumentation data from parallel workers back to the leader process
>> before the Dynamic Shared Memory (DSM) segment is unlinked. This makes it
>> difficult to gather comprehensive performance metrics from parallelized
>> custom scans.
>>
>> *Solution:*
>>
>> This patch introduces a new optional hook,
>> RetrieveInstrumentationCustomScan, to the CustomExecMethods struct. This
>> hook allows custom scan providers to implement logic to collect and
>> consolidate instrumentation from shared memory or worker states during the
>> parallel query cleanup phase. This hook is invoked via the new
>> ExecCustomScanRetrieveInstrumentation function, called from
>> ExecParallelRetrieveInstrumentation for T_CustomScanState nodes. Since
>> the hook is optional (checked for NULL before calling), it maintains full
>> backward compatibility.
>>
>> *Testing & Compatibility:*
>>
>>    - The patch compiles and passes all core regression tests (make
>>    check-world) on my x86_64 instance.
>>    - The changes are not platform-specific.
>>    - Regression Tests: This patch provides a new *capability* for custom
>>    scan providers. Since the hook's functionality is only realized when
>>    implemented by an extension, specific tests would naturally reside within
>>    that extension rather than in the core regression suite.
>>
>> This patch does not directly address a specific item on the official TODO
>> list but enhances the extensibility framework.
>>
>> I believe this patch is complete and ready for review. I look forward to
>> any feedback and am happy to make revisions. I will also add this patch to
>> the next CommitFest.
>>
>> Thank you,
>>
>> Siddharth Kothari
>>
>

  4. Re: [PATCH] Add RetrieveInstrumentation hook for CustomScan providers

    Siddharth Kothari <sidkot@google.com> — 2026-07-08T16:52:51Z

    Hi hackers,

Adding Melanie and Tomas to this thread. I noticed you both recently
committed similar changes to handle shared memory instrumentation
separation and retrieval for IndexScan/IndexOnlyScan (dd78e69cfc
<https://github.com/postgres/postgres/commit/dd78e69cfc337f93cfc0303ddf660262d7f1237e>)
and SeqScan (3b1117d6e2
<https://github.com/postgres/postgres/commit/3b1117d6e2e47d86cdbd978b79434c630cb0ef52>
).

My patch implements the exact same pattern for CustomScan states. It adds
an optional RetrieveInstrumentationCustomScan hook so that extensions can
aggregate their worker metrics before the DSM gets unlinked, bringing
custom scans to parity with the recent core scan instrumentation
improvements.

Could you please help review?

The CommitFest entry is also updated here:
https://commitfest.postgresql.org/patch/6524/

Thanks, Siddharth



On Tue, Jun 23, 2026 at 2:38 PM Siddharth Kothari <sidkot@google.com> wrote:

> Hi hackers,
>
> Bumping this thread as this small patch is passing all CF checks but needs
> a reviewer.
>
> *The Problem:* Right now, if you write a parallel CustomScan extension,
> it is impossible to get performance metrics from parallel workers. The data
> is destroyed during DSM unlinking before the leader can grab it.
>
> *The Fix:* This patch adds an optional RetrieveInstrumentation hook to
> CustomExecMethods. It has zero overhead for existing extensions (it's a
> simple NULL check) but unlocks metric aggregation for parallel custom scans.
>
> Please let me know your thoughts.
>
> Thanks,
>
> Siddharth
>
> On Wed, Apr 8, 2026 at 10:57 AM Siddharth Kothari <sidkot@google.com>
> wrote:
>
>> Hi everyone,
>>
>> I’m just checking in to see if anyone has had a chance to look at this or
>> if there’s any further information I should provide to help with the
>> review. I have also added the patch to PG20-1 CF queue, the link is
>> https://commitfest.postgresql.org/patch/6524/.
>>
>> Thanks,
>> Siddharth
>>
>> On Wed, Feb 18, 2026 at 3:09 PM Siddharth Kothari <sidkot@google.com>
>> wrote:
>>
>>> Dear PostgreSQL Hackers,
>>>
>>> This email proposes a patch to enhance the CustomScan provider
>>> interface. The patch file,
>>> 0001-Add-RetrieveInstrumentationCustomScan-hook-for-Custo.patch, is
>>> attached.
>>>
>>> *Problem:*
>>>
>>> CustomScan providers currently lack a standard method to aggregate
>>> instrumentation data from parallel workers back to the leader process
>>> before the Dynamic Shared Memory (DSM) segment is unlinked. This makes it
>>> difficult to gather comprehensive performance metrics from parallelized
>>> custom scans.
>>>
>>> *Solution:*
>>>
>>> This patch introduces a new optional hook,
>>> RetrieveInstrumentationCustomScan, to the CustomExecMethods struct.
>>> This hook allows custom scan providers to implement logic to collect and
>>> consolidate instrumentation from shared memory or worker states during the
>>> parallel query cleanup phase. This hook is invoked via the new
>>> ExecCustomScanRetrieveInstrumentation function, called from
>>> ExecParallelRetrieveInstrumentation for T_CustomScanState nodes. Since
>>> the hook is optional (checked for NULL before calling), it maintains full
>>> backward compatibility.
>>>
>>> *Testing & Compatibility:*
>>>
>>>    - The patch compiles and passes all core regression tests (make
>>>    check-world) on my x86_64 instance.
>>>    - The changes are not platform-specific.
>>>    - Regression Tests: This patch provides a new *capability* for
>>>    custom scan providers. Since the hook's functionality is only realized when
>>>    implemented by an extension, specific tests would naturally reside within
>>>    that extension rather than in the core regression suite.
>>>
>>> This patch does not directly address a specific item on the official
>>> TODO list but enhances the extensibility framework.
>>>
>>> I believe this patch is complete and ready for review. I look forward to
>>> any feedback and am happy to make revisions. I will also add this patch to
>>> the next CommitFest.
>>>
>>> Thank you,
>>>
>>> Siddharth Kothari
>>>
>>