Horton

Re: [PING] [PATCH v2] parallel pg_restore: avoid disk seeks when jumping short distance forward

Dimitrios Apostolou <jimis@gmx.net>

View thread View on postgr.es
From: Dimitrios Apostolou <jimis@gmx.net>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Chao Li <li.evan.chao@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, pgsql-hackers@lists.postgresql.org
Date: 2025-10-20T20:50:44Z
Lists: pgsql-hackers 

  WriteDataToArchiveNone(ArchiveHandle *AH, CompressorState *cs,
                                            const void *data, size_t dLen)
  {
-       cs->writeF(AH, data, dLen);
+       NoneCompressorState *nonecs = (NoneCompressorState *) cs->private_data;
+       size_t          remaining = dLen;
+
+       while (remaining > 0)
+       {
+               size_t          chunk;
+
+               /* Dump buffer if full */
+               if (nonecs->bufdata >= nonecs->buflen)

Shouldn't this be equality check instead:
     if (nonecs->bufdata == nonecs->buflen)

And possibly also assert(nonecs->bufdata <= nonecs->buflen) ?

+               {
+                       cs->writeF(AH, nonecs->buffer, nonecs->bufdata);
+                       nonecs->bufdata = 0;
+               }
+               /* And fill it */
+               chunk = nonecs->buflen - nonecs->bufdata;
+               if (chunk > remaining)
+                       chunk = remaining;
+               memcpy(nonecs->buffer + nonecs->bufdata, data, chunk);
+               nonecs->bufdata += chunk;
+               data = ((const char *) data) + chunk;
+               remaining -= chunk;
+       }
  }

Commits

  1. Avoid short seeks in pg_restore.

  2. Don't rely on zlib's gzgetc() macro.

  3. Add more TAP test coverage for pg_dump.

  4. Split 002_pg_dump.pl into two test files.

  5. Align the data block sizes of pg_dump's various compression modes.

  6. Fix serious performance problems in LZ4Stream_read_internal.

  7. Fix poor buffering logic in pg_dump's lz4 and zstd compression code.

  8. Fix issue with reading zero bytes in Gzip_read.

  9. Restore test coverage of LZ4Stream_gets().