Re: libpq compression (part 3)
Jacob Burroughs <jburroughs@instructure.com>
From: Jacob Burroughs <jburroughs@instructure.com>
To: Jelte Fennema-Nio <postgres@jeltef.nl>
Cc: Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-12-31T07:32:19Z
Lists: pgsql-hackers
Attachments
- v3-0001-Add-IO-stream-abstraction.patch (application/octet-stream) patch v3-0001
- v3-0002-Add-protocol-layer-compression-to-libpq.patch (application/octet-stream) patch v3-0002
- v3-0003-Add-network-traffic-stats.patch (application/octet-stream) patch v3-0003
- v3-0004-Add-basic-test-for-compression-functionality.patch (application/octet-stream) patch v3-0004
- v3-0005-DO-NOT-MERGE-enable-compression-for-CI.patch (application/octet-stream) patch v3-0005
> One thing I'm wondering: should it be possible for the client to change the compression it wants mid-connection? I can think of some scenarios where that would be useful to connection poolers: if a pooler does plain forwarding of the compressed messages, then it would need to be able to disable/enable compression if it wants to multiplex client connections with different compression settings over the same server connection. I have reworked this patch series to make it easier to extend to restart compression mid-connection once something in the vein of the discussion in "Add new protocol message to change GUCs for usage with future protocol-only GUCs" [1] happens. In particular, I have changed the `CompressedMessage` protocol message to signal the current compression algorithm any time the client should restart its streaming decompressor and otherwise implicitly use whatever compression algorithm and decompressor was used for previous `CompressedMessage` , which future work can leverage to trigger such a restart on update of the client-supported compression algorithms. > Option 2: > This one is even cleaner in the common case but a bit worse in the > uncommon case: just use one parameter and have > compression/decompression enabling be part of the compression detail > (e.g. "libpq_compression='gzip:no_de > compress;lz4:level=2,no_compress;zstd'" > or something like that, in which case the "none,gzip" case would > become "'libpq_compression=gzip:no_compress'"). See > https://www.postgresql.org/docs/current/app-pgbasebackup.html , > specifically the `--compress` flag, for how specifying compression > algorithms and details works. I ended up reworking this to use a version of this option in place of the `none` hackery, but naming the parameters `compress` and `decompress, so to disable compression but allow decompression you would specify `libpq_compression=gzip:compress=off`. Also my windows SSL test failures seem to have resolved themselves with either these changes or a rebase, so I think things are truly in a reviewable state now.