Horton
Menu

Re: cleanup patches for incremental backup

Nathan Bossart <nathandbossart@gmail.com>

View thread
From: Nathan Bossart <nathandbossart@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-14T21:00:10Z
Lists: pgsql-hackers

Attachments

On Wed, Jan 24, 2024 at 12:05:15PM -0600, Nathan Bossart wrote:
> There might be an overflow risk in the cutoff time calculation, but I doubt
> that's the root cause of these failures:
> 
> 	/*
> 	 * Files should only be removed if the last modification time precedes the
> 	 * cutoff time we compute here.
> 	 */
> 	cutoff_time = time(NULL) - 60 * wal_summary_keep_time;

I've attached a short patch for fixing this overflow risk.  Specifically,
it limits wal_summary_keep_time to INT_MAX / SECS_PER_MINUTE, just like
log_rotation_age.

I considering checking for overflow when we subtract the keep-time from the
result of time(2), but AFAICT there's only a problem if time_t is unsigned,
which Wikipedia leads me to believe is unusual [0], so I figured we might
be able to just wait this one out until 2038.

> Otherwise, I think we'll probably need to add some additional logging to
> figure out what is happening...

Separately, I suppose it's probably time to revert the temporary debugging
code adding by commit 5ddf997.  I can craft a patch for that, too.

[0] https://en.wikipedia.org/wiki/Unix_time#Representing_the_number

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Avoid overflow in MaybeRemoveOldWalSummaries().

  2. Revert "Temporary patch to help debug pg_walsummary test failures."

  3. Revise pg_walsummary's 002_blocks test to avoid spurious failures.

  4. Temporary patch to help debug pg_walsummary test failures.

  5. More documentation updates for incremental backup.

  6. Fix typos.

  7. Fix typo.

  8. Try to fix pg_walsummary buildfarm failures.

  9. Repair various defects in dc212340058b4e7ecfc5a7a81ec50e7a207bf288.

  10. Add new pg_walsummary tool.

  11. Add new function pg_get_wal_summarizer_state().

  12. Document WAL summarization information functions.