Thread

1. Allow pg_archivecleanup to ignore extensions

   Greg Smith <greg@2ndquadrant.com> — 2011-02-08T07:57:42Z

   One bit of feedback I keep getting from people who archive their WAL 
   files is that the fairly new pg_archivecleanup utility doesn't handle 
   the case where those archives are compressed.  As the sort of users who 
   are concerned about compression are also often ones with giant archives 
   they struggle to cleanup, they would certainly appreciate having a 
   bundled utility to take care of that. 
   
   The attached patch provides an additional option to the utility to 
   provide this capability.  It just strips a provided extension off any 
   matching file it considers before running the test for whether it should 
   be deleted or not.  It includes updates to the usage message and some 
   docs about how this might be used.  Code by Jaime Casanova and myself.
   
   Here's an example of it working:
   
   $ psql -c "show archive_command"
                     archive_command                  
   ----------------------------------------------------
    cp -i %p archive/%f < /dev/null && gzip archive/%f
   [Yes, I know that can be written more cleanly.  I call external scripts 
   with more serious error handling than you can put into a single command 
   line for this sort of thing in production.]
   
   $ psql -c "select pg_start_backup('test',true)"
   $ psql -c "select pg_stop_backup()"
   $ psql -c "checkpoint"
   $ psql -c "select pg_switch_xlog()"
   
   $ cd $PGDATA/archive
   $ ls
   000000010000000000000025.gz
   000000010000000000000026.gz
   000000010000000000000027.gz
   000000010000000000000028.00000020.backup.gz
   000000010000000000000028.gz
   000000010000000000000029.gz
   
   $ pg_archivecleanup -d -x .gz `pwd`  
   000000010000000000000028.00000020.backup
   pg_archivecleanup: keep WAL file 
   "/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000028" 
   and later
   pg_archivecleanup: removing file 
   "/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000025.gz"
   pg_archivecleanup: removing file 
   "/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000027.gz"
   pg_archivecleanup: removing file 
   "/home/gsmith/pgwork/data/archivecleanup/archive/000000010000000000000026.gz"
   $ ls
   000000010000000000000028.00000020.backup.gz
   000000010000000000000028.gz
   000000010000000000000029.gz
   
   We recenty got some on-list griping that pg_standby doesn't handle 
   archive files that are compressed, either.  Given how the job I'm 
   working on this week is going, I'll probably have to add that feature 
   next.  That's actually an easier source code hack than this one, because 
   of how the pg_standby code modularizes the concept of a restore command.
   
   -- 
   Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
   PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
   
   
   

2. Re: Allow pg_archivecleanup to ignore extensions

   Euler Taveira de Oliveira <euler@timbira.com> — 2011-02-08T17:28:42Z

   Em 08-02-2011 04:57, Greg Smith escreveu:
   > We recenty got some on-list griping that pg_standby doesn't handle
   > archive files that are compressed, either. Given how the job I'm working
   > on this week is going, I'll probably have to add that feature next.
   > That's actually an easier source code hack than this one, because of how
   > the pg_standby code modularizes the concept of a restore command.
   >
   This was already proposed a few years ago [1]. I have used a modified 
   pg_standby with this feature for a year or so.
   
   
   [1] 
   http://archives.postgresql.org/message-id/e4ccc24e0810222010p12bae2f4xa3a11cb2bc51bd89%40mail.gmail.com
   
   
   -- 
      Euler Taveira de Oliveira
      http://www.timbira.com/
   
   

3. Re: Allow pg_archivecleanup to ignore extensions

   Robert Haas <robertmhaas@gmail.com> — 2011-02-25T00:08:54Z

   On Tue, Feb 8, 2011 at 2:57 AM, Greg Smith <greg@2ndquadrant.com> wrote:
   > One bit of feedback I keep getting from people who archive their WAL files
   > is that the fairly new pg_archivecleanup utility doesn't handle the case
   > where those archives are compressed.  As the sort of users who are concerned
   > about compression are also often ones with giant archives they struggle to
   > cleanup, they would certainly appreciate having a bundled utility to take
   > care of that.
   
   Please add this patch to the currently open CommitFest at
   
   https://commitfest.postgresql.org/action/commitfest_view/open
   
   -- 
   Robert Haas
   EnterpriseDB: http://www.enterprisedb.com
   The Enterprise PostgreSQL Company