Horton

Re: Speed up Clog Access by increasing CLOG buffers

Simon Riggs <simon@2ndquadrant.com>

View thread View on postgr.es
From: Simon Riggs <simon@2ndQuadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Simon Riggs <simon@2ndquadrant.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2015-11-17T11:34:17Z
Lists: pgsql-hackers 
On 17 November 2015 at 11:27, Amit Kapila <amit.kapila16@gmail.com> wrote:


> Attached patch group_update_clog_v1.patch
>>> implements this idea.
>>>
>>
>> I don't think we should be doing this only for transactions that don't
>> have subtransactions.
>>
>
> The reason for not doing this optimization for subtransactions is that we
> need to advertise the information that Group leader needs for updating
> the transaction status and if we want to do it for sub transactions, then
> all the subtransaction id's needs to be advertised.  Now here the tricky
> part is that number of subtransactions for which the status needs to
> be updated is dynamic, so reserving memory for it would be difficult.
> However, we can reserve some space in Proc like we do for XidCache
> (cache of sub transaction ids) and then use that to advertise that many
> Xid's at-a-time or just allow this optimization if number of
> subtransactions
> is lesser than or equal to the size of this new XidCache.  I am not sure
> if it is good idea to use the existing XidCache for this purpose in which
> case we need to have a separate space in PGProc for this purpose.  I
> don't see allocating space for 64 or so subxid's as a problem, however
> doing it for bigger number could be cause of concern.
>
>
>> We are trying to speed up real cases, not just benchmarks.
>>
>> So +1 for the concept, patch is going in right direction though lets do
>> the full press-up.
>>
>>
> I have mentioned above the reason for not doing it for sub transactions, if
> you think it is viable to reserve space in shared memory for this purpose,
> then
> I can include the optimization for subtransactions as well.
>

The number of subxids is unbounded, so as you say, reserving shmem isn't
viable.

I'm interested in real world cases, so allocating 65 xids per process isn't
needed, but we can say is that the optimization shouldn't break down
abruptly in the presence of a small/reasonable number of subtransactions.

-- 
Simon Riggs                http://www.2ndQuadrant.com/
<http://www.2ndquadrant.com/>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Use group updates when setting transaction status in clog.

  2. Improve 64bit atomics support.

  3. Add ProcArrayGroupUpdate wait event.

  4. Make the different Unix-y semaphore implementations ABI-compatible.

  5. Fix broken ALTER INDEX documentation

  6. Code and docs review for commit 3187d6de0e5a9e805b27c48437897e8c39071d45.

  7. Partition the freelist for shared dynahash tables.

  8. Correct StartupSUBTRANS for page wraparound

  9. Make idle backends exit if the postmaster dies.

  10. contrib/sslinfo: add ssl_extension_info SRF

  11. Reduce ProcArrayLock contention by removing backends in batches.

  12. Fix `make installcheck` for serializable transactions.

  13. Lockless StrategyGetBuffer clock sweep hot path.

  14. Reduce sinval synchronization overhead.