Re: refactoring basebackup.c (zstd)
Justin Pryzby <pryzby@telsasoft.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Document BaseBackupSync and BaseBackupWrite wait events.
- 749320cdc3fd 15.3 landed
- 4b1ad19a4e22 16.0 landed
-
Support long distance matching for zstd compression
- 2820adf7755d 16.0 landed
-
Fix possible NULL-pointer-deference in backup_compression.c.
- 8e053dc6dfbe 15.0 landed
-
Allow parallel zstd compression when taking a base backup.
- 51c0d186d99a 15.0 landed
-
Make PostgreSQL::Test::Cluster::run_log() return a useful value.
- ad4f2c47de44 15.0 landed
-
Fix a few goofs in new backup compression code.
- 61762426e6ed 15.0 landed
-
Replace BASE_BACKUP COMPRESSION_LEVEL option with COMPRESSION_DETAIL.
- ffd53659c46a 15.0 landed
-
Add 'basebackup_to_shell' contrib module.
- c6306db24bd9 15.0 landed
-
Allow extensions to add new backup targets.
- e4ba69f3f4a1 15.0 landed
-
Change HAVE_LIBLZ4 and HAVE_LIBZSTD tests to USE_LZ4 and USE_ZSTD.
- 75eae090876f 15.0 landed
-
pg_basebackup: Clean up some bogus file extension tests.
- d6f1cdeb9a9e 15.0 landed
-
pg_basebackup: Avoid unclean failure with server-compression and -D -.
- b2de45f9200d 15.0 landed
-
Fix LZ4 tests for remaining buffer space.
- 1d4be6be65ab 15.0 landed
-
Add support for zstd base backup compression.
- 7cf085f077df 15.0 landed
-
pg_basebackup: Allow client-side LZ4 (de)compression.
- 751b8d23b788 15.0 landed
-
Add suport for server-side LZ4 base backup compression.
- dab298471ff2 15.0 landed
-
Add min() and max() aggregates for xid8.
- 400fc6b6487d 15.0 cited
-
Remove superfluous variable.
- 82331ed4dd60 15.0 landed
-
pg_basebackup: Cleaner handling when compression is multiply specified.
- 51891d5a9560 15.0 landed
-
Allow server-side compression to be used with -Fp.
- d45099425eb1 15.0 landed
-
pg_basebackup: Fix a couple of recently-introduced bugs.
- dabf63bc9a5b 15.0 landed
-
Tidy up a few cosmetic issues related to pg_basebackup.
- e1f860f13459 15.0 landed
-
Server-side gzip compression.
- 0ad8032910d5 15.0 landed
-
Unbreak pg_basebackup/t/010_pg_basebackup.pl on msys
- 4f0bcc735038 15.0 cited
-
Suppress variable-set-but-not-used warning from clang 13.
- dc43fc9b3aa3 15.0 cited
-
Extend the options of pg_basebackup to control compression
- 5c649fe15336 15.0 cited
-
Support base backup targets.
- 3500ccc39b0d 15.0 landed
-
Modify pg_basebackup to use a new COPY subprotocol for base backups.
- cc333f32336f 15.0 landed
-
Document that tar archives are now properly terminated.
- 81fca310b38e 15.0 landed
-
Fix thinko in bbsink_throttle_manifest_contents.
- 1b098da20093 15.0 landed
-
Have the server properly terminate tar archives.
- 5a1007a5088c 15.0 landed
-
Minimal fix for unterminated tar archive problem.
- 57b5a9646d97 15.0 landed
-
Introduce 'bbstreamer' abstraction to modularize pg_basebackup.
- 23a1c6578c87 15.0 landed
-
Introduce 'bbsink' abstraction to modularize base backup code.
- bef47ff85df1 15.0 landed
-
Refactor basebackup.c's _tarWriteDir() function.
- 967a17fe2fa7 15.0 landed
-
Flexible options for CREATE_REPLICATION_SLOT.
- 0266e98c6b86 15.0 landed
-
Flexible options for BASE_BACKUP.
- 0ba281cb4bf9 15.0 landed
+++ b/configure
@@ -801,6 +805,7 @@ infodir
docdir
oldincludedir
includedir
+runstatedir
There's superfluous changes to ./configure unrelated to the changes in
configure.ac. Probably because you're using a different version of autotools,
or a vendor's patched copy. You can remove the changes with git checkout -p or
similar.
+++ b/src/backend/replication/basebackup_zstd.c
+bbsink *
+bbsink_zstd_new(bbsink *next, int compresslevel)
+{
+#ifndef HAVE_LIBZSTD
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("zstd compression is not supported by this build")));
+#else
This should have an return; like what's added by 71cbbbbe8 and 302612a6c.
Also, the parens() around errcode aren't needed since last year.
+ bbsink_zstd *sink;
+
+ Assert(next != NULL);
+ Assert(compresslevel >= 0 && compresslevel <= 22);
+
+ if (compresslevel < 0 || compresslevel > 22)
+ ereport(ERROR,
This looks like dead code in assert builds.
If it's unreachable, it can be elog().
+ * Compress the input data to the output buffer until we run out of input
+ * data. Each time the output buffer falls below the compression bound for
+ * the input buffer, invoke the archive_contents() method for then next sink.
*the next sink ?
Does anyone plan to include this for pg15 ? If so, I think at least the WAL
compression should have support added too. I'd plan to rebase Michael's patch.
https://www.postgresql.org/message-id/YNqWd2GSMrnqWIfx@paquier.xyz
--
Justin