Re: Use read streams in pg_visibility
Nazir Bilal Yavuz <byavuz81@gmail.com>
From: Nazir Bilal Yavuz <byavuz81@gmail.com>
To: Noah Misch <noah@leadboat.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-09-02T12:20:12Z
Lists: pgsql-hackers
Attachments
- v4-0001-Add-general-use-struct-and-callback-to-read-strea.patch (application/x-patch) patch v4-0001
- v4-0002-pg_prewarm-Use-generic-use-read-stream-struct-and.patch (application/x-patch) patch v4-0002
- v4-0003-RelationCopyStorageUsingBuffer-Use-generic-use-re.patch (application/x-patch) patch v4-0003
- v4-0004-Use-read-stream-in-pg_visibility-in-collect_visib.patch (application/x-patch) patch v4-0004
- v4-0005-Use-read-stream-in-pg_visibility-in-collect_corru.patch (application/x-patch) patch v4-0005
Hi,
On Sat, 31 Aug 2024 at 02:51, Noah Misch <noah@leadboat.com> wrote:
>
> To read blocks 10 and 11, I would expect to initialize the struct with one of:
>
> { .first=10, .nblocks=2 }
> { .first=10, .last_inclusive=11 }
> { .first=10, .last_exclusive=12 }
>
> With the patch's API, I would need {.first=10,.nblocks=12}. The struct field
> named "nblocks" behaves like a last_block_exclusive. Please either make the
> behavior an "nblocks" behavior or change the field name to replace the term
> "nblocks" with something matching the behavior. (I used longer field names in
> my examples here, to disambiguate those examples. It's okay if the final
> field names aren't those, as long as the field names and the behavior align.)
I decided to use 'current_blocknum' and 'last_exclusive'. I think
these are easier to understand and use.
> > Thanks for the information, I will check these. What I still do not
> > understand is how to make sure that only the second block is processed
> > and the first one is skipped. pg_check_visible() and pg_check_frozen()
> > returns TIDs that cause corruption in the visibility map, there is no
> > information about block numbers.
>
> I see what you're saying. collect_corrupt_items() needs a corrupt table to
> report anything; all corruption-free tables get the same output. Testing this
> would need extra C code or techniques like corrupt_page_checksum() to create
> the corrupt state. That wouldn't be a bad thing to have, but it's big enough
> that I'll consider it out of scope for $SUBJECT. With the callback change
> above, I'll be ready to push all this.
Thanks, updated patches are attached.
--
Regards,
Nazir Bilal Yavuz
Microsoft
Commits
-
Optimize pg_visibility with read streams.
- 65c310b310a6 18.0 landed
- ed1b1ee59fb3 18.0 landed
-
Revert "Optimize pg_visibility with read streams."
- ddfc556a6444 18.0 landed
-
Fix stack variable scope from previous commit.
- 1c61fd8b5279 18.0 landed
-
Add block_range_read_stream_cb(), to deduplicate code.
- c582b75851c2 18.0 landed