Horton
Menu

Thread

  1. PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

    loganathan P <plogandba@gmail.com> — 2026-03-03T14:21:31Z

    Dear Team,

We have PostgreSQL 15 and PostgreSQL 17 databases running in separate
environments on different servers. Each database is approximately 1.5 TB in
size and highly active, generating around 500 GB of archive logs per day.
We have VM SRM replication configured.

The archive log partition reaches 95–100% utilization before backups are
taken. After the backups are completed, we must manually remove the
archived log files to free up space.

Could you please advise whether PostgreSQL has any built-in parameters or
mechanisms to automatically delete archived log files once they have been
successfully backed up?



Thanks.

Regards,
Loganathan P

  2. Re: PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

    Laurenz Albe <laurenz.albe@cybertec.at> — 2026-03-03T14:48:01Z

    On Tue, 2026-03-03 at 19:51 +0530, loganathan P wrote:
> We have PostgreSQL 15 and PostgreSQL 17 databases running in separate environments on different servers.
> Each database is approximately 1.5 TB in size and highly active, generating around 500 GB of archive
> logs per day. We have VM SRM replication configured.
> 
> The archive log partition reaches 95–100% utilization before backups are taken. After the backups are
> completed, we must manually remove the archived log files to free up space.
> 
> Could you please advise whether PostgreSQL has any built-in parameters or mechanisms to automatically
> delete archived log files once they have been successfully backed up?

Since PostgreSQL doesn't know about your archives, there is no way for it to clean
up automatically.  You have to do that yourself.

There is the "pg_archivecleanup" utility that you can use with the backup file
generated in the WAL archive:

  pg_archivecleanup /mnt/server/archivedir 000000010000000000000010.00000020.backup

The easiest way would be to use a backup software like pgBackRest that does these
things for you.

Yours,
Laurenz Albe

  3. Re: PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

    Ron Johnson <ronljohnsonjr@gmail.com> — 2026-03-04T06:57:55Z

    On Tue, Mar 3, 2026 at 9:19 AM loganathan P <plogandba@gmail.com> wrote:

> Dear Team,
>
> We have PostgreSQL 15 and PostgreSQL 17 databases running in separate
> environments on different servers. Each database is approximately 1.5 TB in
> size and highly active, generating around 500 GB of archive logs per day.
>

How many days (or weeks) of PITR backups do you need to retain.


> We have VM SRM replication configured.
>

That's probably not wise, given the size and volume.  Physical replication
via pg_basebackup is quite easy to set up.
This is the command I use:
pg_basebackup \
            --pgdata=$PGDATA \
            --dbname=service=basebackup \
            --verbose --progress \
            --checkpoint=fast \
            --write-recovery-conf \
            --wal-method=stream \
            --create-slot --slot=$SlotName \
            --compress=server-lz4


> The archive log partition reaches 95–100% utilization before backups are
> taken. After the backups are completed, we must manually remove the
> archived log files to free up space.
>
> Could you please advise whether PostgreSQL has any built-in parameters or
> mechanisms to automatically delete archived log files
>

Lauren Albe is right: pgbackrest is probably the tool for you.  Besides
doing regular database backups, it manages WAL archives, encryption,
compression and can be configured to automatically purge the oldest saveset
(a full backup and its associated incremental backups plus archived WAL
files) after a certain number of days.

For example, I've got multiple 3-4TB databases and use pgbackrest to retain
28-35 days of PITR backups.  After the full backup on that 35th day,
pgbackrest automatically "expires" the oldest saveset and then we're back
down to 28 days of PITR backups.

It really is a wonderful do-everything tool.


> once they have been successfully backed up?
>

Where do you back them up to?

-- 
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

  4. R: PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

    Gianfranco Cocco <gianfranco.cocco@vargroup.com> — 2026-03-04T08:32:39Z

    Hi,
usually you could use a similar script that will delete archived WALs after a pg_basebackup

#!/bin/bash
# I'm looking for the latest backup so I can clean up the previous archive logs.
DATE=$(date +"%Y-%m-%d-%I-%M-%p");
ARCHIVEDIR='/pg_archive'
cd $ARCHIVEDIR
CHKPOINT=$(ls *.backup -Art | tail -n 1)
/usr/pgsql-15/bin/pg_archivecleanup -d $ARCHIVEDIR $CHKPOINT > /var/log/postgresql/RotateWAL-$DATE.log 2>&1


Regards.

Gianfranco



________________________________
Da: Ron Johnson <ronljohnsonjr@gmail.com>
Inviato: Mercoledì, 04 Marzo, 2026 07:57
A: pgsql-general <pgsql-general@postgresql.org>
Oggetto: Re: PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

On Tue, Mar 3, 2026 at 9:19 AM loganathan P <plogandba@gmail.com<mailto:plogandba@gmail.com>> wrote:
Dear Team,

We have PostgreSQL 15 and PostgreSQL 17 databases running in separate environments on different servers. Each database is approximately 1.5 TB in size and highly active, generating around 500 GB of archive logs per day.

How many days (or weeks) of PITR backups do you need to retain.

We have VM SRM replication configured.

That's probably not wise, given the size and volume.  Physical replication via pg_basebackup is quite easy to set up.
This is the command I use:
pg_basebackup \
            --pgdata=$PGDATA \
            --dbname=service=basebackup \
            --verbose --progress \
            --checkpoint=fast \
            --write-recovery-conf \
            --wal-method=stream \
            --create-slot --slot=$SlotName \
            --compress=server-lz4

The archive log partition reaches 95–100% utilization before backups are taken. After the backups are completed, we must manually remove the archived log files to free up space.

Could you please advise whether PostgreSQL has any built-in parameters or mechanisms to automatically delete archived log files

Lauren Albe is right: pgbackrest is probably the tool for you.  Besides doing regular database backups, it manages WAL archives, encryption, compression and can be configured to automatically purge the oldest saveset (a full backup and its associated incremental backups plus archived WAL files) after a certain number of days.

For example, I've got multiple 3-4TB databases and use pgbackrest to retain 28-35 days of PITR backups.  After the full backup on that 35th day, pgbackrest automatically "expires" the oldest saveset and then we're back down to 28 days of PITR backups.

It really is a wonderful do-everything tool.

once they have been successfully backed up?

Where do you back them up to?

--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!

  5. Re: PostgreSQL Archive Log Partition Reaching 95% – Need Automated Cleanup

    loganathan P <plogandba@gmail.com> — 2026-03-05T13:18:04Z

      Dear All,

Thank you for your response to my query. The backup retention period has
already been set to 30 days using the Commvault backup tool.

Thanks.

Regards,
Loganathan P



On Wed, Mar 4, 2026 at 2:08 PM Gianfranco Cocco <
Gianfranco.Cocco@vargroup.com> wrote:

> [image: Boxbe] <https://www.boxbe.com/overview> Gianfranco Cocco (
> Gianfranco.Cocco@vargroup.com) is not on your Guest List
> <https://www.boxbe.com/approved-list?tc_serial=61622565815&tc_rand=518829428&utm_source=stf&utm_medium=email&utm_campaign=ANNO_MWTP&utm_content=001&&key=wbiIqSkqIF%2Boz9rHX0y%2Bnhi7lp7fFkLa%2BhCf3axFg3A%3D&token=LuRq3xfTRgANaBqbyFNvYcp53AKL%2F308ZWhHVON5LPr4FuFO85uxUituqLCR6iqc>
> | Approve sender
> <https://www.boxbe.com/anno?tc_serial=61622565815&tc_rand=518829428&utm_source=stf&utm_medium=email&utm_campaign=ANNO_MWTP&utm_content=001&&key=wbiIqSkqIF%2Boz9rHX0y%2Bnhi7lp7fFkLa%2BhCf3axFg3A%3D&token=LuRq3xfTRgANaBqbyFNvYcp53AKL%2F308ZWhHVON5LPr4FuFO85uxUituqLCR6iqc>
> | Approve domain
> <https://www.boxbe.com/anno?tc_serial=61622565815&tc_rand=518829428&utm_source=stf&utm_medium=email&utm_campaign=ANNO_MWTP&utm_content=001&&dom&key=wbiIqSkqIF%2Boz9rHX0y%2Bnhi7lp7fFkLa%2BhCf3axFg3A%3D&token=LuRq3xfTRgANaBqbyFNvYcp53AKL%2F308ZWhHVON5LPr4FuFO85uxUituqLCR6iqc>
> Hi,
> usually you could use a similar script that will delete archived WALs
> after a pg_basebackup
>
> #!/bin/bash
> # I'm looking for the latest backup so I can clean up the previous archive
> logs.
> DATE=$(date +"%Y-%m-%d-%I-%M-%p");
> ARCHIVEDIR='/pg_archive'
> cd $ARCHIVEDIR
> CHKPOINT=$(ls *.backup -Art | tail -n 1)
> /usr/pgsql-15/bin/pg_archivecleanup -d $ARCHIVEDIR $CHKPOINT >
> /var/log/postgresql/RotateWAL-$DATE.log 2>&1
>
>
> Regards.
>
> Gianfranco
>
>
>
> ------------------------------
> *Da:* Ron Johnson <ronljohnsonjr@gmail.com>
> *Inviato:* Mercoledì, 04 Marzo, 2026 07:57
> *A:* pgsql-general <pgsql-general@postgresql.org>
> *Oggetto:* Re: PostgreSQL Archive Log Partition Reaching 95% – Need
> Automated Cleanup
>
> On Tue, Mar 3, 2026 at 9:19 AM loganathan P <plogandba@gmail.com> wrote:
>
> Dear Team,
>
> We have PostgreSQL 15 and PostgreSQL 17 databases running in separate
> environments on different servers. Each database is approximately 1.5 TB in
> size and highly active, generating around 500 GB of archive logs per day.
>
>
> How many days (or weeks) of PITR backups do you need to retain.
>
>
> We have VM SRM replication configured.
>
>
> That's probably not wise, given the size and volume.  Physical replication
> via pg_basebackup is quite easy to set up.
> This is the command I use:
> pg_basebackup \
>             --pgdata=$PGDATA \
>             --dbname=service=basebackup \
>             --verbose --progress \
>             --checkpoint=fast \
>             --write-recovery-conf \
>             --wal-method=stream \
>             --create-slot --slot=$SlotName \
>             --compress=server-lz4
>
>
> The archive log partition reaches 95–100% utilization before backups are
> taken. After the backups are completed, we must manually remove the
> archived log files to free up space.
>
> Could you please advise whether PostgreSQL has any built-in parameters or
> mechanisms to automatically delete archived log files
>
>
> Lauren Albe is right: pgbackrest is probably the tool for you.  Besides
> doing regular database backups, it manages WAL archives, encryption,
> compression and can be configured to automatically purge the oldest saveset
> (a full backup and its associated incremental backups plus archived WAL
> files) after a certain number of days.
>
> For example, I've got multiple 3-4TB databases and use pgbackrest to
> retain 28-35 days of PITR backups.  After the full backup on that 35th day,
> pgbackrest automatically "expires" the oldest saveset and then we're back
> down to 28 days of PITR backups.
>
> It really is a wonderful do-everything tool.
>
>
> once they have been successfully backed up?
>
>
> Where do you back them up to?
>
> --
> Death to <Redacted>, and butter sauce.
> Don't boil me, I'm still alive.
> <Redacted> lobster!
>