Horton

Re: PG 13 release notes, first draft

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: bruce@momjian.us
Cc: noah@leadboat.com, pgsql-hackers@postgresql.org
Date: 2020-05-14T00:51:41Z
Lists: pgsql-hackers 
At Wed, 13 May 2020 11:15:18 -0400, Bruce Momjian <bruce@momjian.us> wrote in 
> On Wed, May 13, 2020 at 11:56:33AM +0900, Kyotaro Horiguchi wrote:
> > > 
> > > 	Allow skipping of WAL for new tables and indexes if wal_level is
> > > 	'minimal' (Kyotaro Horiguchi)
> > > 
> > > 	Relations larger than wal_skip_threshold will have their files
> > > 	fsync'ed rather than writing their WAL records.  Previously this
> > > 	was done only for COPY operations, but the implementation had a
> > > 	bug that could cause data loss during crash recovery.
> > 
> > I see it. It is giving weight on improvement. Looks good the overall
> > structure of the description above.  However, wal-skipping is always
> > done regardless of table size. wal_skip_threshold is an optimization
> > to choose which to use fsync or FPI records (that is, not WAL records
> > in the common sense) at commit for speed.
> 
> Well, as far as users are concerned, everything wrtiten to  WAL is a WAL
> record.

I think that the significant point here is not that persistence is
ensured by which of fsync or WAL , but whether WAL records are written
for every insertion.  The commit-time WA is just an alternative of
fsync, which is faster than fsync'ing separate files for smaller
files.

> > So how about the following?
> > 
> > All kinds of bulk-insertion are not WAL-logged then fsync'ed at
> > commit.  Using FPI WAL records instead of fsync for relations smaller
> > than wal_skip_threshold. Previously this was done only for COPY
> > operations and always using fsync, but the implementation had a bug
> > that could cause data loss during crash recovery.
> 
> That is too much detail for the release notes.  We already will link to
> the docs.  Why put it here?

It is just an more accurate (not an detailed) version of the
previously proposed description.  If we simplify that, I choose to
remove explanation on wal_skip_threshold.

How about this?

WAL-logging is now skipped while all kinds of bulk-insertion, then
relations are sync'ed to disk at commit.  Previously this was done
only for COPY operations, but the implementation had a bug that could
cause data loss during crash recovery.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Doc: improve release notes' info about FROM UNPACKAGED feature removal.

  2. Doc: fix misstatement in v13 release notes.

  3. Doc: some more v13 release note tweaking.

  4. Doc: update v13 release notes through today, do a copy-editing pass.

  5. Doc: fill in "major enhancements" list in v13 release notes.

  6. doc: PG 13 relnotes: fix typos

  7. doc: PG 13 relnotes, update TOAST item to mention decompression

  8. pgbench: document that the default data loading is client-side

  9. psql \d: Display table where trigger is defined, if inherited

  10. Skip WAL for new relfilenodes, under wal_level=minimal.

  11. Introduce vacuum errcontext to display additional information.

  12. Fix lquery's behavior for consecutive '*' items.

  13. Improve the internal implementation of ereport().

  14. Add object names to partition integrity violations.

  15. Allow page lock to conflict among parallel group members.

  16. Allow relation extension lock to conflict among parallel group members.

  17. Introduce "anycompatible" family of polymorphic types.

  18. Improve checking of child pages in contrib/amcheck.

  19. doc: Add information about new SQL part SQL/MDA

  20. Added relation name in error messages for constraint checks.

  21. Avoid full scan of GIN indexes when possible

  22. Use carriage returns for data insertion logs in pgbench on terminal

  23. Use memcpy instead of a byte loop in pglz_decompress

  24. Add backtrace support for error reporting

  25. Improve pruning of a default partition

  26. Improve psql's \d output for partitioned indexes.

  27. initdb: Change authentication defaults