Horton

Re: Inval reliability, especially for inplace updates

Noah Misch <noah@leadboat.com>

View thread
From: Noah Misch <noah@leadboat.com>
To: Alexander Lakhin <exclusion@gmail.com>, Nitin Motiani <nitinmotiani@google.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers@postgresql.org
Date: 2024-11-03T18:29:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Update .abi-compliance-history for PrepareToInvalidateCacheTuple().

  2. Assert lack of hazardous buffer locks before possible catalog read.

  3. For inplace update, send nontransactional invalidations.

  4. Update .abi-compliance-history for CacheInvalidateHeapTupleInplace().

  5. Revisit cosmetics of "For inplace update, send nontransactional invalidations."

  6. Correct comments of "Fix data loss at inplace update after heap_update()".

  7. Move I/O before the index_update_stats() buffer lock region.

  8. Revert "For inplace update, send nontransactional invalidations."

  9. Revert "WAL-log inplace update before revealing it to other sessions."

  10. Fix inplace update buffer self-deadlock.

  11. Remove duplicate words in comments

  12. At end of recovery, reset all sinval-managed caches.

  13. Fix data loss at inplace update after heap_update().

  14. Remove comment about xl_heap_inplace "AT END OF STRUCT".

  15. Reduce memory consumption for pending invalidation messages.

  16. Add support for streaming to built-in logical replication.

  17. WAL Log invalidations at command end with wal_level=logical.

  18. Introduce logical decoding.

  19. Rename and document some invalidation routines to make it clearer that

Attachments

On Fri, Nov 01, 2024 at 04:38:29PM -0700, Noah Misch wrote:
> This was a near miss to having a worst-in-years regression in a minor release,
> so I'm proposing this sequence:
> 
> - Revert from non-master branches commits 8e7e672 (inplace180, "WAL-log
>   inplace update before revealing it to other sessions.") and 243e9b4
>   (inplace160, "For inplace update, send nontransactional invalidations.").
> 
> - Back-patch inplace230-index_update_stats-io-before-buflock to harden commit
>   a07e03f (inplace110, "Fix data loss at inplace update after heap_update()").
> 
> - Push attached inplace240 to master.
> 
> - Make the commitfest entry a request for review of v17 inplace160+inplace240.
>   After some amount of additional review and master bake time, the reverted
>   patches would return to non-master branches.
> 
> If someone agrees or if nobody objects by 2024-11-02T15:00+0000, I'll make it
> so.  That's not much time, but I want to minimize buildfarm members hanging
> and maximize inplace230 bake time before the release wrap.

Pushed as 0bada39.

Buildfarm member hornet REL_15_STABLE was in the same hang.  Other buildfarm
runs 2024-10-25T13:51:02Z - 2024-11-02T16:04:56Z may hang the same way.  It's
early to make a comprehensive list of hung buildfarm members, since many
reported infrequently even before this period.  I'll wait a week or two and
then contact the likely-hung member owners.  I regret the damage.

To make explicit something I didn't call out above, v12-v17 commit a4668c9 "At
end of recovery, reset all sinval-managed caches." is still in the tree.
That's despite it originating to serve the now-reverted back branch versions
of 243e9b4.  It could prevent post-recovery corruption, and its risks are
orthogonal to the risks of the reverted code.

I'm attaching the v17 patch that now becomes the commitfest submission
associated with this thread.