Thread
-
-f <output file> option for pg_dumpall
Dave Page <dpage@postgresql.org> — 2007-01-05T17:26:31Z
In pgAdmin we use pg_dump's -f option to write backup files. The IO streams are redirected to display status and errors etc. in the GUI. In order to enhance the interface to allow backup of entire clusters as well as role and tablespace definitions, we need to be able to get pg_dumpall to write it's output directly to a file in the same way, because we cannot redirect the child pg_dump IO streams (which also means we may miss errors, but I need to think about that some more). As far as I can see, adding a -f option to pg_dumpall should be straight forward, the only issue being that we'd need to pass pg_dump an additional (undocumented?) option to tell it to append to the output file instead of writing it as normal. Any thoughts or better ideas? Regards, Dave.
-
Re: -f <output file> option for pg_dumpall
Joshua D. Drake <jd@commandprompt.com> — 2007-01-05T17:35:03Z
> As far as I can see, adding a -f option to pg_dumpall should be straight > forward, the only issue being that we'd need to pass pg_dump an > additional (undocumented?) option to tell it to append to the output > file instead of writing it as normal. > > Any thoughts or better ideas? Yeah :) Getting rid of pg_dumpall entirely and merging it into pg_dump. It is kind of silly that we have two different versions. Although that may be more work than you were considering. Joshua D. Drake > > Regards, Dave. > > ---------------------------(end of broadcast)--------------------------- > TIP 1: if posting/reading through Usenet, please send an appropriate > subscribe-nomail command to majordomo@postgresql.org so that your > message can get through to the mailing list cleanly > -- === The PostgreSQL Company: Command Prompt, Inc. === Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240 Providing the most comprehensive PostgreSQL solutions since 1997 http://www.commandprompt.com/ Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate -
Re: -f <output file> option for pg_dumpall
Andrew Dunstan <andrew@dunslane.net> — 2007-01-05T17:40:20Z
Dave Page wrote: > In pgAdmin we use pg_dump's -f option to write backup files. The IO > streams are redirected to display status and errors etc. in the GUI. > > In order to enhance the interface to allow backup of entire clusters as > well as role and tablespace definitions, we need to be able to get > pg_dumpall to write it's output directly to a file in the same way, > because we cannot redirect the child pg_dump IO streams (which also > means we may miss errors, but I need to think about that some more). > > As far as I can see, adding a -f option to pg_dumpall should be straight > forward, the only issue being that we'd need to pass pg_dump an > additional (undocumented?) option to tell it to append to the output > file instead of writing it as normal. > > Any thoughts or better ideas? > > This seems a bit like piecemeal reform. Here are some things I'd like to see that affect this area: . merge pg_dump and pg_dumpall (e.g. add a flag to pg_dump that says do the lot) . multi-db non-text dumps And while we're about it, can we teach pg_restore to handle text dumps? Even if it just transparently called psql it would be nice. cheers andrew
-
Re: -f <output file> option for pg_dumpall
Darcy Buskermolen <darcy@ok-connect.com> — 2007-01-05T17:47:19Z
On Friday 05 January 2007 09:40, Andrew Dunstan wrote: > Dave Page wrote: > > In pgAdmin we use pg_dump's -f option to write backup files. The IO > > streams are redirected to display status and errors etc. in the GUI. > > > > In order to enhance the interface to allow backup of entire clusters as > > well as role and tablespace definitions, we need to be able to get > > pg_dumpall to write it's output directly to a file in the same way, > > because we cannot redirect the child pg_dump IO streams (which also > > means we may miss errors, but I need to think about that some more). > > > > As far as I can see, adding a -f option to pg_dumpall should be straight > > forward, the only issue being that we'd need to pass pg_dump an > > additional (undocumented?) option to tell it to append to the output > > file instead of writing it as normal. > > > > Any thoughts or better ideas? > > This seems a bit like piecemeal reform. Here are some things I'd like to > see that affect this area: > > . merge pg_dump and pg_dumpall (e.g. add a flag to pg_dump that says do > the lot) > . multi-db non-text dumps > > And while we're about it, can we teach pg_restore to handle text dumps? > Even if it just transparently called psql it would be nice. +1 on on having pg_restore work with text dumps. > > cheers > > andrew > > > > > ---------------------------(end of broadcast)--------------------------- > TIP 4: Have you searched our list archives? > > http://archives.postgresql.org
-
Re: -f <output file> option for pg_dumpall
Andreas Pflug <pgadmin@pse-consulting.de> — 2007-01-05T17:50:49Z
Dave Page wrote: > In pgAdmin we use pg_dump's -f option to write backup files. The IO > streams are redirected to display status and errors etc. in the GUI. > > In order to enhance the interface to allow backup of entire clusters as > well as role and tablespace definitions, we need to be able to get > pg_dumpall to write it's output directly to a file in the same way, > because we cannot redirect the child pg_dump IO streams (which also > means we may miss errors, but I need to think about that some more). > > As far as I can see, adding a -f option to pg_dumpall should be straight > forward, the only issue being that we'd need to pass pg_dump an > additional (undocumented?) option to tell it to append to the output > file instead of writing it as normal. > > Any thoughts or better ideas? > Use pgAdmin's "create script" funcion on the server. Regards, Andreas
-
Re: -f <output file> option for pg_dumpall
David Fetter <david@fetter.org> — 2007-01-05T19:02:13Z
On Fri, Jan 05, 2007 at 12:40:20PM -0500, Andrew Dunstan wrote: > Dave Page wrote: > >In pgAdmin we use pg_dump's -f option to write backup files. The IO > >streams are redirected to display status and errors etc. in the GUI. > > > >In order to enhance the interface to allow backup of entire clusters as > >well as role and tablespace definitions, we need to be able to get > >pg_dumpall to write it's output directly to a file in the same way, > >because we cannot redirect the child pg_dump IO streams (which also > >means we may miss errors, but I need to think about that some more). > > > >As far as I can see, adding a -f option to pg_dumpall should be straight > >forward, the only issue being that we'd need to pass pg_dump an > >additional (undocumented?) option to tell it to append to the output > >file instead of writing it as normal. > > > >Any thoughts or better ideas? > > > This seems a bit like piecemeal reform. Here are some things I'd like to > see that affect this area: > > . merge pg_dump and pg_dumpall (e.g. add a flag to pg_dump that says do > the lot) > . multi-db non-text dumps > > And while we're about it, can we teach pg_restore to handle text dumps? > Even if it just transparently called psql it would be nice. +1 on all of this :) Cheers, D (who can contribute code once we've agreed on what it should do) -- David Fetter <david@fetter.org> http://fetter.org/ phone: +1 415 235 3778 AIM: dfetter666 Skype: davidfetter Remember to vote! -
Re: -f <output file> option for pg_dumpall
Dave Page <dpage@postgresql.org> — 2007-01-05T20:03:33Z
David Fetter wrote: >> This seems a bit like piecemeal reform. Here are some things I'd like to >> see that affect this area: >> >> . merge pg_dump and pg_dumpall (e.g. add a flag to pg_dump that says do >> the lot) >> . multi-db non-text dumps >> >> And while we're about it, can we teach pg_restore to handle text dumps? >> Even if it just transparently called psql it would be nice. > > +1 on all of this :) Hmm, well I have no interest in the latter at present, but assuming the powers that be will allow me some time to do so, I will look at merging pg_dump and pg_dumpall as that seems to be the way people want to go. I'd also like to tweak the options for global objects to allow roles and tablespaces to be dumped seperately from each other if no-one objects. > Cheers, > D (who can contribute code once we've agreed on what it should do) Cool - fancy picking up the multi-db non text and pg_restore bits if I do the former? Regards, Dave.
-
Re: -f <output file> option for pg_dumpall
Dave Page <dpage@postgresql.org> — 2007-01-05T20:05:26Z
Andreas Pflug wrote: > Dave Page wrote: >> In pgAdmin we use pg_dump's -f option to write backup files. The IO >> streams are redirected to display status and errors etc. in the GUI. >> >> In order to enhance the interface to allow backup of entire clusters as >> well as role and tablespace definitions, we need to be able to get >> pg_dumpall to write it's output directly to a file in the same way, >> because we cannot redirect the child pg_dump IO streams (which also >> means we may miss errors, but I need to think about that some more). >> >> As far as I can see, adding a -f option to pg_dumpall should be straight >> forward, the only issue being that we'd need to pass pg_dump an >> additional (undocumented?) option to tell it to append to the output >> file instead of writing it as normal. >> >> Any thoughts or better ideas? >> > Use pgAdmin's "create script" funcion on the server. We said long ago we weren't going to replicate pg_dump functionality in pgAdmin - a brief discussion on the pgadmin-hackers list earlier indicates that people still feel the same way, and that time would be better spend fixing pg_dump/pg_dumpall. Regards, Dave.
-
Re: -f <output file> option for pg_dumpall
Tom Lane <tgl@sss.pgh.pa.us> — 2007-01-05T22:48:17Z
Dave Page <dpage@postgresql.org> writes: > As far as I can see, adding a -f option to pg_dumpall should be straight > forward, the only issue being that we'd need to pass pg_dump an > additional (undocumented?) option to tell it to append to the output > file instead of writing it as normal. Wouldn't it be easier/better to re-point stdout at the -f file, and not touch pg_dump at all? regards, tom lane
-
Re: -f <output file> option for pg_dumpall
Tom Lane <tgl@sss.pgh.pa.us> — 2007-01-05T22:52:37Z
Dave Page <dpage@postgresql.org> writes: > Hmm, well I have no interest in the latter at present, but assuming the > powers that be will allow me some time to do so, I will look at merging > pg_dump and pg_dumpall as that seems to be the way people want to go. I think this will be an exercise in time-wasting, and very possibly destabilize *both* tools. pg_dump has never been designed to reconnect to a different database; for instance there isn't any code for resetting all the internal state that it gathers. I think forking a separate pg_dump for each database is a perfectly fine arrangement, and should be left alone. regards, tom lane
-
Re: -f <output file> option for pg_dumpall
Tino Wildenhain <tino@wildenhain.de> — 2007-01-05T22:58:18Z
Tom Lane schrieb: > Dave Page <dpage@postgresql.org> writes: >> As far as I can see, adding a -f option to pg_dumpall should be straight >> forward, the only issue being that we'd need to pass pg_dump an >> additional (undocumented?) option to tell it to append to the output >> file instead of writing it as normal. > > Wouldn't it be easier/better to re-point stdout at the -f file, and not > touch pg_dump at all? Yeah, and maybe have a modifier like %n or something which would instead write different files and replaces %n with the name of the database... Regards Tino
-
Re: -f <output file> option for pg_dumpall
Peter Eisentraut <peter_e@gmx.net> — 2007-01-06T11:42:51Z
Dave Page wrote: > In pgAdmin we use pg_dump's -f option to write backup files. The IO > streams are redirected to display status and errors etc. in the GUI. > > In order to enhance the interface to allow backup of entire clusters > as well as role and tablespace definitions, we need to be able to get > pg_dumpall to write it's output directly to a file in the same way, > because we cannot redirect the child pg_dump IO streams (which also > means we may miss errors, but I need to think about that some more). I don't understand this. Errors are sent to stderr, and the real output is sent to stdout, where you can capture and save it. What is the problem? -- Peter Eisentraut http://developer.postgresql.org/~petere/
-
Re: -f <output file> option for pg_dumpall
Dave Page <dpage@postgresql.org> — 2007-01-06T20:28:39Z
Peter Eisentraut wrote: > Dave Page wrote: >> In pgAdmin we use pg_dump's -f option to write backup files. The IO >> streams are redirected to display status and errors etc. in the GUI. >> >> In order to enhance the interface to allow backup of entire clusters >> as well as role and tablespace definitions, we need to be able to get >> pg_dumpall to write it's output directly to a file in the same way, >> because we cannot redirect the child pg_dump IO streams (which also >> means we may miss errors, but I need to think about that some more). > > I don't understand this. Errors are sent to stderr, and the real output > is sent to stdout, where you can capture and save it. What is the > problem? > On Windows at least, I cannot redirect stdout/stderr of child processes, only the parent (ie. pg_dumpall) that is executed by pgAdmin (the pg_dump processes just seem to hang, and the debugger reports a deadlock with no other useful info). I'm sure there is a way round this but a) it'll probably mean hacking wxWidgets or pg_dump about in ways I'm sure Vadim or Tom would dislike, and b) it would seem reasonable to offer similar functionality in pg_dumpall to that already in pg_dump anyway. Regards, Dave.
-
Re: -f <output file> option for pg_dumpall
Jim Nasby <jim@nasby.net> — 2007-01-08T21:33:42Z
On Fri, Jan 05, 2007 at 08:05:26PM +0000, Dave Page wrote: > Andreas Pflug wrote: > >Dave Page wrote: > >>In pgAdmin we use pg_dump's -f option to write backup files. The IO > >>streams are redirected to display status and errors etc. in the GUI. > >> > >>In order to enhance the interface to allow backup of entire clusters as > >>well as role and tablespace definitions, we need to be able to get > >>pg_dumpall to write it's output directly to a file in the same way, > >>because we cannot redirect the child pg_dump IO streams (which also > >>means we may miss errors, but I need to think about that some more). > >> > >>As far as I can see, adding a -f option to pg_dumpall should be straight > >>forward, the only issue being that we'd need to pass pg_dump an > >>additional (undocumented?) option to tell it to append to the output > >>file instead of writing it as normal. > >> > >>Any thoughts or better ideas? > >> > >Use pgAdmin's "create script" funcion on the server. > > We said long ago we weren't going to replicate pg_dump functionality in > pgAdmin - a brief discussion on the pgadmin-hackers list earlier > indicates that people still feel the same way, and that time would be > better spend fixing pg_dump/pg_dumpall. It might make sense to provide a programmatic interface to pg_dump to provide tools like pgAdmin more flexibility. But it certainly doesn't make sense to re-create the dumping logic. In terms of integrating pg_dumpall and pg_dump; I don't really care if that happens, but I would like to see pg_dumpall supporting all the options that pg_dump does (a quick diff shows that pg_dumpall is currently missing support for -f, -F, -v, -Z, -C (which may or may not make sense), -E, -n & -t (might not make sense), -U and -W (probably OK). -- Jim Nasby jim@nasby.net EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
-
Re: -f <output file> option for pg_dumpall
Andreas Pflug <pgadmin@pse-consulting.de> — 2007-01-08T23:09:27Z
Jim C. Nasby wrote: > > > It might make sense to provide a programmatic interface to pg_dump to > provide tools like pgAdmin more flexibility. Are you talking about "pg_dump in a lib"? Certainly a good idea, because it allows better integration (e.g. progress bar). > But it certainly doesn't make sense to re-create the dumping logic. > > In terms of integrating pg_dumpall and pg_dump; I don't really care if > that happens, I can't make too much sense of integrating pg_dumpall anywhere. Dumping a whole cluster is certainly much of a planned job, not an interactive online one, because its output usually won't be usable except for disaster recovery. Not much function to "re-create" here, single exception is extracting cluster wide data, the -g option, that's why I mentioned scripting. But apparently this didn't get into pgadmin svn any more, so I need to retract this proposal. Regards, Andreas
-
Re: -f <output file> option for pg_dumpall
Dave Page <dpage@postgresql.org> — 2007-01-09T08:39:31Z
Andreas Pflug wrote: > Not much function to "re-create" here, single > exception is extracting cluster wide data, the -g option, that's why I > mentioned scripting. But apparently this didn't get into pgadmin svn any > more, so I need to retract this proposal. Eh? Your SCRIPT code is still there - or do you mean something else? Regards, Dave.
-
Re: -f <output file> option for pg_dumpall
Andreas Pflug <pgadmin@pse-consulting.de> — 2007-01-09T10:50:05Z
Dave Page wrote: > Andreas Pflug wrote: > >> Not much function to "re-create" here, single >> exception is extracting cluster wide data, the -g option, that's why I >> mentioned scripting. But apparently this didn't get into pgadmin svn any >> more, so I need to retract this proposal. >> > > Eh? Your SCRIPT code is still there - or do you mean something else? > No GetScript implementation for pgServer. Regards, Andreas
-
Re: -f <output file> option for pg_dumpall
Neil Conway <neilc@samurai.com> — 2007-01-11T19:36:42Z
On Fri, 2007-01-05 at 17:52 -0500, Tom Lane wrote: > I think this will be an exercise in time-wasting, and very possibly > destabilize *both* tools. pg_dump has never been designed to reconnect > to a different database; for instance there isn't any code for resetting > all the internal state that it gathers. That is merely an implementation issue. The question of whether pg_dump and pg_dumpall should be separate programs is a question of design, IMHO. I don't think they need to be integrated any time soon, but if we were to design pg_dump and pg_dumpall from scratch, it seems more logical to use a single program, and therefore that is the long-term direction I think we should head in. -Neil
-
Re: -f <output file> option for pg_dumpall
Neil Conway <neilc@samurai.com> — 2007-01-14T21:25:51Z
On Thu, 2007-01-11 at 14:36 -0500, Neil Conway wrote: > I don't think they need to be integrated any time soon, but if we were > to design pg_dump and pg_dumpall from scratch, it seems more logical to > use a single program On thinking about this some more, it might be useful to factor much of pg_dump's logic for reconstructing the state of a database into a shared library. This would make it relatively easy for developers to plug new archive formats into the library (in addition to the present 3 archive formats), or to make use of this functionality in other applications that want to reconstruct the logical state of a database from the content of the system catalogs. We could then provide a client app implemented on top of the library that would provide similar functionality to pg_dump. Moving pg_dump's functionality into the backend has been suggested in the past (and rejected for good reason), but I think this might be a more practical method for making the pg_dump logic more easily reusable. (While we're on the subject, this would also present a good opportunity to make pg_dump's command-line interface a little more sane. I wonder if pg_dump's syntax has gotten sufficiently complicated that some users might prefer to specify behavior via a configuration file, rather than a long string of command-line options.) -Neil
-
Re: -f <output file> option for pg_dumpall
Andrew Dunstan <andrew@dunslane.net> — 2007-01-15T15:13:16Z
Neil Conway wrote: > On Thu, 2007-01-11 at 14:36 -0500, Neil Conway wrote: > >> I don't think they need to be integrated any time soon, but if we were >> to design pg_dump and pg_dumpall from scratch, it seems more logical to >> use a single program >> > > On thinking about this some more, it might be useful to factor much of > pg_dump's logic for reconstructing the state of a database into a shared > library. This would make it relatively easy for developers to plug new > archive formats into the library (in addition to the present 3 archive > formats), or to make use of this functionality in other applications > that want to reconstruct the logical state of a database from the > content of the system catalogs. We could then provide a client app > implemented on top of the library that would provide similar > functionality to pg_dump. > > Moving pg_dump's functionality into the backend has been suggested in > the past (and rejected for good reason), but I think this might be a > more practical method for making the pg_dump logic more easily reusable. > > > I like this idea. For example, we might usefully map some of this to psql \ commands, without having to replicate the underlying logic. cheers andrew
-
Re: -f <output file> option for pg_dumpall
elein <elein@varlena.com> — 2007-01-23T00:05:49Z
On Mon, Jan 15, 2007 at 10:13:16AM -0500, Andrew Dunstan wrote: > > > Neil Conway wrote: > >On Thu, 2007-01-11 at 14:36 -0500, Neil Conway wrote: > > > >>I don't think they need to be integrated any time soon, but if we were > >>to design pg_dump and pg_dumpall from scratch, it seems more logical to > >>use a single program > >> > > > >On thinking about this some more, it might be useful to factor much of > >pg_dump's logic for reconstructing the state of a database into a shared > >library. This would make it relatively easy for developers to plug new > >archive formats into the library (in addition to the present 3 archive > >formats), or to make use of this functionality in other applications > >that want to reconstruct the logical state of a database from the > >content of the system catalogs. We could then provide a client app > >implemented on top of the library that would provide similar > >functionality to pg_dump. > > > >Moving pg_dump's functionality into the backend has been suggested in > >the past (and rejected for good reason), but I think this might be a > >more practical method for making the pg_dump logic more easily reusable. > > > > > > > > I like this idea. For example, we might usefully map some of this to > psql \ commands, without having to replicate the underlying logic. Don't we already do this with the .psqlrc file? --elein
-
Re: -f <output file> option for pg_dumpall
Andrew Dunstan <andrew@dunslane.net> — 2007-01-23T00:36:14Z
elein wrote: > On Mon, Jan 15, 2007 at 10:13:16AM -0500, Andrew Dunstan wrote: >> >> >> Neil Conway wrote: >> >On Thu, 2007-01-11 at 14:36 -0500, Neil Conway wrote: >> > >> >>I don't think they need to be integrated any time soon, but if we were >> >>to design pg_dump and pg_dumpall from scratch, it seems more logical >> to >> >>use a single program >> >> >> > >> >On thinking about this some more, it might be useful to factor much of >> >pg_dump's logic for reconstructing the state of a database into a >> shared >> >library. This would make it relatively easy for developers to plug new >> >archive formats into the library (in addition to the present 3 archive >> >formats), or to make use of this functionality in other applications >> >that want to reconstruct the logical state of a database from the >> >content of the system catalogs. We could then provide a client app >> >implemented on top of the library that would provide similar >> >functionality to pg_dump. >> > >> >Moving pg_dump's functionality into the backend has been suggested in >> >the past (and rejected for good reason), but I think this might be a >> >more practical method for making the pg_dump logic more easily >> reusable. >> > >> > >> > >> >> I like this idea. For example, we might usefully map some of this to >> psql \ commands, without having to replicate the underlying logic. > > Don't we already do this with the .psqlrc file? > No. \ commands are implemented in C code. cheers andrew