Re: Improve eviction algorithm in ReorderBuffer

Jeff Davis <pgsql@j-davis.com>

From: Jeff Davis <pgsql@j-davis.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, vignesh C <vignesh21@gmail.com>, Peter Smith <smithpb2250@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, Shubham Khanna <khannashubham1197@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-04-04T18:55:51Z
Lists: pgsql-hackers
On Thu, 2024-04-04 at 10:55 -0700, Jeff Davis wrote:
>   * Make a proper indexed binary heap API that accepts a hash
> function
> and provides both heap methods and HT methods that operate based on
> values (transaction size and transaction ID, respectively).
>   * Get rid of ReorderBuffer->by_txn and use the indexed binary heap
> instead.

An alternative idea:

* remove the hash table from binaryheap.c

* supply a new callback to the binary heap with type like:

  typedef void (*binaryheap_update_index)(
    bh_node_type node,
    int new_element_index);

* make the remove, update_up, and update_down methods take the element
index rather than the pointer

reorderbuffer.c would then do something like:

  void
  txn_update_heap_index(ReorderBufferTXN *txn, int new_element_index)
  {
     txn->heap_element_index = new_element_index;
  }

  ...

  txn_heap = binaryheap_allocate(..., txn_update_heap_index, ...);

and then binaryheap.c would effectively maintain txn-
>heap_element_index, so reorderbuffer.c can pass that to the APIs that
require the element index.


Another alternative is to keep the hash table in binaryheap.c, and
supply a hash function that hashes the xid. That leaves us with two
hash tables still, but it would be cleaner than hashing the pointer.
That might be best for right now, and we can consider these other ideas
later.

Regards,
	Jeff Davis




Commits

  1. Revert indexed and enlargable binary heap implementation.

  2. Replace binaryheap + index with pairingheap in reorderbuffer.c

  3. Improve eviction algorithm in ReorderBuffer using max-heap for many subtransactions.

  4. Add functions to binaryheap for efficient key removal and update.

  5. Make binaryheap enlargeable.