Horton

Re: Speed up Clog Access by increasing CLOG buffers

Simon Riggs <simon@2ndquadrant.com>

View thread View on postgr.es
From: Simon Riggs <simon@2ndQuadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Simon Riggs <simon@2ndquadrant.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2015-11-17T13:00:54Z
Lists: pgsql-hackers 
On 17 November 2015 at 11:48, Amit Kapila <amit.kapila16@gmail.com> wrote:

> On Tue, Nov 17, 2015 at 5:04 PM, Simon Riggs <simon@2ndquadrant.com>
> wrote:
>
>> On 17 November 2015 at 11:27, Amit Kapila <amit.kapila16@gmail.com>
>> wrote:
>>
>> We are trying to speed up real cases, not just benchmarks.
>>>>
>>>> So +1 for the concept, patch is going in right direction though lets do
>>>> the full press-up.
>>>>
>>>>
>>> I have mentioned above the reason for not doing it for sub transactions,
>>> if
>>> you think it is viable to reserve space in shared memory for this
>>> purpose, then
>>> I can include the optimization for subtransactions as well.
>>>
>>
>> The number of subxids is unbounded, so as you say, reserving shmem isn't
>> viable.
>>
>> I'm interested in real world cases, so allocating 65 xids per process
>> isn't needed, but we can say is that the optimization shouldn't break down
>> abruptly in the presence of a small/reasonable number of subtransactions.
>>
>>
> I think in that case what we can do is if the total number of
> sub transactions is lesser than equal to 64 (we can find that by
> overflowed flag in PGXact) , then apply this optimisation, else use
> the existing flow to update the transaction status.  I think for that we
> don't even need to reserve any additional memory. Does that sound
> sensible to you?
>

I understand you to mean that the leader should look backwards through the
queue collecting xids while !(PGXACT->overflowed)

No additional shmem is required

-- 
Simon Riggs                http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Use group updates when setting transaction status in clog.

  2. Improve 64bit atomics support.

  3. Add ProcArrayGroupUpdate wait event.

  4. Make the different Unix-y semaphore implementations ABI-compatible.

  5. Fix broken ALTER INDEX documentation

  6. Code and docs review for commit 3187d6de0e5a9e805b27c48437897e8c39071d45.

  7. Partition the freelist for shared dynahash tables.

  8. Correct StartupSUBTRANS for page wraparound

  9. Make idle backends exit if the postmaster dies.

  10. contrib/sslinfo: add ssl_extension_info SRF

  11. Reduce ProcArrayLock contention by removing backends in batches.

  12. Fix `make installcheck` for serializable transactions.

  13. Lockless StrategyGetBuffer clock sweep hot path.

  14. Reduce sinval synchronization overhead.