Horton

Re: tableam vs. TOAST

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Prabhat Sahu <prabhat.sahu@enterprisedb.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-09-05T19:36:22Z
Lists: pgsql-hackers 
Andres Freund <andres@anarazel.de> writes:
> Well, I still dislike making the toast chunk size configurable in a
> halfhearted manner.

It's hard to make it fully configurable without breaking our on-disk
storage, because of the lack of any explicit representation of the chunk
size in TOAST data.  You have to "just know" how big the chunks are
supposed to be.

However, it's reasonable to ask why we should treat it as an AM property,
especially a fixed AM property as this has it.  If somebody does
reimplement toast logic in some other AM, they might well decide it's
worth the storage cost to be more flexible about the chunk size ... but
too bad, this design won't let them do it.

I don't entirely understand why relation_toast_am is a callback
while toast_max_chunk_size isn't, either.  Why would they not both
be callbacks?  That would at least let an AM set a per-relation
max chunk size, if it wanted.

It seems like this design throws away most of the benefit of a fixed
chunk size (mostly, being able to do relevant modulo arithmetic with
shifts and masks rather than full-fledged integer division) without
getting much of anything in return.

			regards, tom lane

Commits

  1. tableam: New callback relation_fetch_toast_slice.

  2. tableam: Allow choice of toast AM.

  3. Move heap-specific detoasting logic into a separate function.

  4. Code cleanup for toast_fetch_datum and toast_fetch_datum_slice.

  5. Rename some toasting functions based on whether they are heap-specific.

  6. Create an API for inserting and deleting rows in TOAST tables.

  7. Split tuptoaster.c into three separate files.

  8. tableam: Move heap-specific logic from needs_toast_table below tableam.