Thread

location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-11T18:44:15Z

The debate on the configuration file sparked a memory of an old patch I 
submitted in 7.1 days.

One of the things I do not like about PostgreSQL is, IMHO, is a 
backwards configuration process. Rather than specify a data directory, 
the administrator should specify a database configuration file. Within 
the configuration file is the location and names of the data directory 
and other information. Most admins want a central location for this 
information.

One of the things that is frustrating to me, is to have to hunt down 
where the data directory is so that I can administrate a DB. It can be 
anywhere, in any directory on any volume. If you had, say, a 
/usr/local/pgsql/admin, then you could have mydb.conf which could then 
be checked in to CVS. A standard location for configuration files is a 
more normal process as the location of the data directory is less so. I 
just don't think the PG data directory should not contain configuration 
information.

The original patch allowed a user to specify the location of the 
postgresql.conf file, rather than assuming it lived in $PGDATA
Also included in that patch, was the ability to specify the location of 
the PGDATA directory as well as the names of the pg_hba.conf and other 
configuration files.

It also allowed the user to use PG as it has always worked, The patch 
was not applied because a better solution was supposedly coming, but 
that was two major revisions ago. I would still like to see this 
functionality. Would anyone else?

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-11T18:55:08Z

On Tue, 2003-02-11 at 13:44, mlw wrote:
> The debate on the configuration file sparked a memory of an old patch I 
> submitted in 7.1 days.
> 
> One of the things I do not like about PostgreSQL is, IMHO, is a 
> backwards configuration process. Rather than specify a data directory, 
> the administrator should specify a database configuration file. Within 
> the configuration file is the location and names of the data directory 
> and other information. Most admins want a central location for this 
> information.
> 
> One of the things that is frustrating to me, is to have to hunt down 
> where the data directory is so that I can administrate a DB. It can be 
> anywhere, in any directory on any volume. If you had, say, a 
> /usr/local/pgsql/admin, then you could have mydb.conf which could then 
> be checked in to CVS. A standard location for configuration files is a 
> more normal process as the location of the data directory is less so. I 
> just don't think the PG data directory should not contain configuration 
> information.
> 
> The original patch allowed a user to specify the location of the 
> postgresql.conf file, rather than assuming it lived in $PGDATA
> Also included in that patch, was the ability to specify the location of 
> the PGDATA directory as well as the names of the pg_hba.conf and other 
> configuration files.
> 
> It also allowed the user to use PG as it has always worked, The patch 
> was not applied because a better solution was supposedly coming, but 
> that was two major revisions ago. I would still like to see this 
> functionality. Would anyone else?
> 

I'm going to be lazy and ask if you can post what the better solution
that was coming was (or a link to the thread). While I'll grant you that
the "it's coming" argument is pretty weak after two releases, that fact
that it may have been a better solution could still hold up.

Robert Treat

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-11T19:10:34Z

Robert Treat wrote:

>I'm going to be lazy and ask if you can post what the better solution
>that was coming was (or a link to the thread). While I'll grant you that
>the "it's coming" argument is pretty weak after two releases, that fact
>that it may have been a better solution could still hold up.
>
>Robert Treat
>  
>
AFAIK it wasn't actually done. It was more of a, "we should do something 
different" argument. At one point it was talked about rewriting the 
configuration system to allow "include" and other things.

Re: location of the configuration files

Curt Sampson <cjs@cynic.net> — 2003-02-12T04:53:17Z

I, personally, also think it makes more sense to pass to the postmaster
a configuration file that contains all the rest of the information about
the database system, including the disk locations of the various data
directories and whatnot.

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-12T06:43:00Z

mlw wrote:
> AFAIK it wasn't actually done. It was more of a, "we should do something 
> different" argument. At one point it was talked about rewriting the 
> configuration system to allow "include" and other things.

That seems like extreme overkill.  The PostgreSQL configuration
mechanism doesn't seem to me to be anywhere near complicated enough to
justify an "include" mechanism.

I agree with you: you should be able to specify all of the base
configuration information (including the location of the data
directories) in one file, and it makes perfect sense to me for the
location of the data directory to be a GUC variable.

I'd say the only thing the postmaster needs to know prior to startup
is the directory containing the postgresql.conf file.  An
administrator who wishes to set up multiple independent databases can
easily do so by using multiple config file directories.  When
consistency is required, he can easily use symlinks to point to master
config files where appropriate.

I assume $PGDATA was around long before GUC?

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-12T07:22:50Z

Kevin Brown <kevin@sysexperts.com> writes:
> I assume $PGDATA was around long before GUC?

Yes, it was.  But I have not yet seen an argument here that justifies
why $SOMECONFIGDIRECTORY/postgresql.conf is better than
$PGDATA/postgresql.conf.  The latter keeps all the related files
together.  The former seems only to introduce unnecessary complexity.
You can only justify it as simpler if you propose hardwiring a value for
$SOMECONFIGDIRECTORY ... which is a proposal that will not fly with any
of the core developers, because we all run multiple versions of Postgres
on our machines so that we can deal with back-version bug reports,
test installations, etc.  It is unlikely to fly with any of the RPM
packagers either, due to the wildly varying ideas out there about the
One True Place where applications should put their config files.

(This point was pretty much why mlw's previous proposal was rejected,
IIRC.)

			regards, tom lane

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-12T13:24:30Z

Tom Lane wrote:
> Kevin Brown <kevin@sysexperts.com> writes:
> > I assume $PGDATA was around long before GUC?
> 
> Yes, it was.  But I have not yet seen an argument here that justifies
> why $SOMECONFIGDIRECTORY/postgresql.conf is better than
> $PGDATA/postgresql.conf.  

Okay, here's one: most Unix systems store all of the configuration
files in a well known directory: /etc.  These days it's a hierarchy of
directories with /etc as the root of the hierarchy.  When an
administrator is looking for configuration files, the first place he's
going to look is in /etc and its subdirectories.  After that, he's
forced to look through the startup scripts to figure out where things
are located.  And if those aren't revealing, then he has to read
manpages and hope they're actually useful.  :-)  And if that doesn't
work, then he has to resort to tricks like doing "strings" on the
binaries (he doesn't necessarily have access to the sources that the
binaries were compiled from, which is all that matters here).

> The latter keeps all the related files together.  The former seems
> only to introduce unnecessary complexity.

Well, I'd say it's "unnecessary" only when you already know where the
data files are located -- which is true when you're a developer or
someone who is already familiar with the installation you're working
with.  But if you're just getting started and installed it from a
package like an RPM file, then you have to look in the package to see
where it created the data file areas, or look at the startup scripts,
etc.

> You can only justify it as simpler if you propose hardwiring a value
> for $SOMECONFIGDIRECTORY ...

Making things simpler from the standpoint of the code isn't the point.
Making things simpler for the DBA and/or Unix sysadmin is.

I'd say $SOMECONFIGDIRECTORY should be a hardwired default with a
command line override.

I doubt you'll get a whole lot of argument from the general user
community if you say that the hard wired default should be
/etc/postgresql.

> which is a proposal that will not fly with any of the core
> developers, because we all run multiple versions of Postgres on our
> machines so that we can deal with back-version bug reports, test
> installations, etc.

I absolutely agree that the config directory to use should be
something that can be controlled with a command line option.

> It is unlikely to fly with any of the RPM packagers either, due to
> the wildly varying ideas out there about the One True Place where
> applications should put their config files.

There seems to be substantial agreement among the distribution
maintainers that config files belong somewhere in /etc.  At least,
I've seen very little disagreement with that idea except from people
who believe that each package should have its own, separate directory
hierarchy.  And the fact that the vast majority of packages put their
config files somewhere in /etc supports this.

Debian, for instance, actually *does* put the PostgreSQL config files
in /etc/postgresql and creates symlinks in the data directory that
point to them.  This works, but it's a kludge.

There are highly practical reasons for putting all the config files
under /etc, not the least of which is that it makes backing up files
that are *very* likely to change from the default, and which are also
highly critical to the operation of the system, very easy.

You'll get A LOT more disagreement about where to put data files than
config files, as standards go.  And in the case of PostgreSQL, where
you put your data files is especially important for performance
reasons, so it therefore makes even less sense to put the config files
in the same location: it means that the config files could literally
be anywhere, and any administrator who is unfamiliar with the system
will have to dig through startup scripts (or worse!) to figure it out.

Oh, here's another reason $SOMECONFIGDIRECTORY is better than $PGDATA:
it allows much more appropriate separation of concern by default.

Most installations of PostgreSQL start the database from a startup
script that's run at boot time.  With $PGDATA, changing the target
data directory requires changing the startup script, which requires
root access to the system -- if it didn't require root access then the
entire system is open to the possibility of a world of hurt because
the DBA isn't necessarily the same guy as the Unix sysadmin and
therefore doesn't necessarily know his way around shell scripts in
general, and rc scripts in particular, the way the Unix admin will.
The possibility of hurt comes from the fact that the rc script runs at
root, at a time that the system is hardest to work with in the event
of a failure (many systems haven't even put up any console login
prompts and may not have even started any remote login facilities
before the PostgreSQL startup script runs).  A sufficiently bad
screwup on the part of the DBA with that kind of setup will require
the Unix sysadmin to go to single user mode or worse to fix it.  So
unless the Unix sysadmin really trusts the DBA, he's not going to
allow the DBA that kind of access.  Instead he'll kludge something
together so that the DBA can make the appropriate kinds of changes
without compromising the system.  But every shop will do this a
different way.  Even Debian, which usually is pretty good about
dealing with issues like these, doesn't address this.

But it shouldn't even be necessary for a shop to kludge around the
problem: with $SOMECONFIGDIRECTORY, the Unix sysadmin can safely give
write permissions to the DBA on the config files (and even on the
entire directory they reside in), the DBA can point the database at
whatever directory he wants the data to reside in, and the Unix admin
only has to set up the storage areas, set the permissions, and let the
DBA loose on it -- he doesn't have to touch the startup scripts at
all, the DBA doesn't have to be much of a shell script wizard, and
everyone is (relatively) happy.  And, even better, the DBA will be
able to do this at most shops he works for because his knowledge will
be based on the standard PostgreSQL install.

If we want to get wider acceptance amongst the heavy database users
(which often *do* separate DBAs from Unix sysadmins), we may want to
think about things like this from time to time...

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-12T15:39:24Z

On Wed, 2003-02-12 at 08:24, Kevin Brown wrote:
> Tom Lane wrote:
> > You can only justify it as simpler if you propose hardwiring a value
> > for $SOMECONFIGDIRECTORY ...
> 
> Making things simpler from the standpoint of the code isn't the point.
> Making things simpler for the DBA and/or Unix sysadmin is.
> 
> I'd say $SOMECONFIGDIRECTORY should be a hardwired default with a
> command line override.
> 
> I doubt you'll get a whole lot of argument from the general user
> community if you say that the hard wired default should be
> /etc/postgresql.
> 
> > which is a proposal that will not fly with any of the core
> > developers, because we all run multiple versions of Postgres on our
> > machines so that we can deal with back-version bug reports, test
> > installations, etc.
> 
> I absolutely agree that the config directory to use should be
> something that can be controlled with a command line option.
> 
> > It is unlikely to fly with any of the RPM packagers either, due to
> > the wildly varying ideas out there about the One True Place where
> > applications should put their config files.
> 
> There seems to be substantial agreement among the distribution
> maintainers that config files belong somewhere in /etc.  At least,
> I've seen very little disagreement with that idea except from people
> who believe that each package should have its own, separate directory
> hierarchy.  And the fact that the vast majority of packages put their
> config files somewhere in /etc supports this.
> 
> Debian, for instance, actually *does* put the PostgreSQL config files
> in /etc/postgresql and creates symlinks in the data directory that
> point to them.  This works, but it's a kludge.
> 

Seems like a good compromise would be to make the hard wired default
$SOMECONFIGDIRECTORY be $PGDATA; this makes each version of the software
more self contained/ less likely to interfere with another installation.
(This becomes really handy when doing major upgrades). If you really
have a strong desire to change this, you can.

As I see it, this change would (should?) need to be something that could
be changed in the configure script when building postgresql, as well
changeable via a command line option, any other places?

Robert Treat

Re: location of the configuration files

Christopher Kings-Lynne <chriskl@familyhealth.com.au> — 2003-02-13T01:37:16Z

> Okay, here's one: most Unix systems store all of the configuration
> files in a well known directory: /etc.  These days it's a hierarchy of
> directories with /etc as the root of the hierarchy.  When an
> administrator is looking for configuration files, the first place he's
> going to look is in /etc and its subdirectories.  After that, he's
> forced to look through the startup scripts to figure out where things
> are located.  And if those aren't revealing, then he has to read
> manpages and hope they're actually useful.  :-)  And if that doesn't
> work, then he has to resort to tricks like doing "strings" on the
> binaries (he doesn't necessarily have access to the sources that the
> binaries were compiled from, which is all that matters here).

No goddammit - /usr/local/etc.  Why can't the Linux community respect
history!!!!

It is the ONE TRUE PLACE dammit!!!

Chris

(btw, there is humour + seriousness in above post...)

Re: location of the configuration files

Rod Taylor <rbt@rbt.ca> — 2003-02-13T02:17:01Z

> > binaries (he doesn't necessarily have access to the sources that the
> > binaries were compiled from, which is all that matters here).
> 
> No goddammit - /usr/local/etc.  Why can't the Linux community respect
> history!!!!

History?  It's the only way to make a read-only / (enforced by
secure-level) and still be able to change the user applications.

I don't mind /usr/X11R6/etc either, but it's not exactly appropriate for
PostgreSQL ;)

-- 
Rod Taylor <rbt@rbt.ca>

PGP Key: http://www.rbt.ca/rbtpub.asc

Re: location of the configuration files

J. M. Brenner <doom@kzsu.stanford.edu> — 2003-02-13T03:03:33Z

"Christopher Kings-Lynne" <chriskl@familyhealth.com.au> wrote: 

> > Okay, here's one: most Unix systems store all of the configuration
> > files in a well known directory: /etc.  These days it's a hierarchy of
> > directories with /etc as the root of the hierarchy.  When an
> > administrator is looking for configuration files, the first place he's
> > going to look is in /etc and its subdirectories.  

> No goddammit - /usr/local/etc.  Why can't the Linux community respect
> history!!!!
> 
> It is the ONE TRUE PLACE dammit!!!

Well, to the extent that you're serious, you understand that 
a lot of people feel that /usr/local should be reserved for 
stuff that's installed by the local sysadmin, and your
vendor/distro isn't supposed to be messing with it. 

Which means if the the vendor installed Postgresql (say, the
Red Hat Database) you'd expect config files to be in /etc.
If the postgresql is compiled from source by local admin, 
you might look somewhere in /usr/local.

I've got the vauge feeling that this is all more than a
little silly... directory locations floating about depending
on who did what, as thought it were such a radical thing 
to do a ./configure, make & make install.  But this is a 
pretty common feeling among the unix world (more wide spread
than just in the Linux world).

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-13T03:08:23Z

On Wednesday 12 February 2003 20:37, Christopher Kings-Lynne wrote:
> > Okay, here's one: most Unix systems store all of the configuration
> > files in a well known directory: /etc.  These days it's a hierarchy of

> No [snip] - /usr/local/etc.  Why can't the Linux community respect
> history!!!!

> It is the ONE TRUE PLACE [snip]

If PostgreSQL is supported as a part of the base operating system in a Linux 
distribution, and that distribution wishes to be Linux Standards Base 
compliant (most do), then PostgreSQL cannot go in /usr/local -- period.

IDIC at work.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T05:31:01Z

Tom Lane wrote:

>Kevin Brown <kevin@sysexperts.com> writes:
>  
>
>>I assume $PGDATA was around long before GUC?
>>    
>>
>
>Yes, it was.  But I have not yet seen an argument here that justifies
>why $SOMECONFIGDIRECTORY/postgresql.conf is better than
>$PGDATA/postgresql.conf.  The latter keeps all the related files
>together.  The former seems only to introduce unnecessary complexity.
>You can only justify it as simpler if you propose hardwiring a value for
>$SOMECONFIGDIRECTORY ... which is a proposal that will not fly with any
>of the core developers, because we all run multiple versions of Postgres
>on our machines so that we can deal with back-version bug reports,
>test installations, etc.  It is unlikely to fly with any of the RPM
>packagers either, due to the wildly varying ideas out there about the
>One True Place where applications should put their config files.
>
>(This point was pretty much why mlw's previous proposal was rejected,
>IIRC.)
>  
>
I wasn't talking about a "default directory" I was talking about 
configuring a database in a configuration file.

While I accept that the PostgreSQL group can not be playing catch-up 
with other databases, this does not preclude the notion accepting common 
practices and adopting them.

Understand, I really like PostgreSQL. I like it better than Oracle, and 
it is my DB of choice.  That being said, I see what other DBs do right. 
Putting the configuration in the data directory is "wrong," no other 
database or service under UNIX or Windows does this, Period.

Does the PostgreSQL team know better than the rest of the world?

The idea that a, more or less, arbitrary data location determines the 
database configuration is wrong. It should be obvious to any 
administrator that a configuration file location which controls the 
server is the "right" way to do it.  Regardless of where ever you choose 
to put the default configuration file, it is EASIER to configure a 
database by using a file in a standard configuration directory (/etc, 
/usr/etc, /usr/local/etc, /usr/local/pgsql/conf or what ever). The data 
directory should not contain configuration data as it is typically 
dependent on where the admin chooses to mount storage.

I am astounded that this point of view is missed by the core group.

Mark.

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-13T05:51:07Z

mlw <pgsql@mohawksoft.com> writes:
> The idea that a, more or less, arbitrary data location determines the 
> database configuration is wrong. It should be obvious to any 
> administrator that a configuration file location which controls the 
> server is the "right" way to do it.

I guess I'm just dense, but I entirely fail to see why this is the One
True Way To Do It.  What you seem to be proposing (ignoring
syntactic-sugar issues) is that we replace "postmaster -D
/some/data/dir" by "postmaster -config /some/config/file".  I am not
seeing the nature of the improvement.  It looks to me like the sysadmin
must now grant the Postgres DBA write access on *two* directories, viz
/some/config/ and /wherever/the/data/directory/is.  How is that better
than granting write access on one directory?  Given that we can't manage
to standardize the data directory location across multiple Unixen, how
is it that we will be more successful at standardizing a config file
location?

All I see here is an arbitrary break with our past practice.  I do not
see any net improvement.

			regards, tom lane

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T06:10:41Z


Tom Lane wrote:

>mlw <pgsql@mohawksoft.com> writes:
>  
>
>>The idea that a, more or less, arbitrary data location determines the 
>>database configuration is wrong. It should be obvious to any 
>>administrator that a configuration file location which controls the 
>>server is the "right" way to do it.
>>    
>>
>
>I guess I'm just dense, but I entirely fail to see why this is the One
>True Way To Do It.  What you seem to be proposing (ignoring
>syntactic-sugar issues) is that we replace "postmaster -D
>/some/data/dir" by "postmaster -config /some/config/file".  I am not
>seeing the nature of the improvement.  It looks to me like the sysadmin
>must now grant the Postgres DBA write access on *two* directories, viz
>/some/config/ and /wherever/the/data/directory/is.  How is that better
>than granting write access on one directory?  Given that we can't manage
>to standardize the data directory location across multiple Unixen, how
>is it that we will be more successful at standardizing a config file
>location?
>
>All I see here is an arbitrary break with our past practice.  I do not
>see any net improvement.
>
>  
>
There is a pretty well understood convention that a configuration file 
will be located in some standard location depending on your distro. 
Would you disagree with that?

There is also a convention that most servers are configured by a 
configuration file, located in a central location. Look at sendmail, 
named,, et al.

Here is the test, configure a server, with sendmail, named, apache, and 
PostgreSQL. Tell me which of these systems doesn't configure right.

Re: location of the configuration files

Peter Bierman <bierman@apple.com> — 2003-02-13T06:14:50Z

At 12:31 AM -0500 2/13/03, mlw wrote:
>The idea that a, more or less, arbitrary data location determines 
>the database configuration is wrong. It should be obvious to any 
>administrator that a configuration file location which controls the 
>server is the "right" way to do it.

Isn't the database data itself a rather significant portion of the 
'configuration' of the database?

What do you gain by having the postmaster config and the database 
data live in different locations?

-pmb

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T06:31:47Z

Peter Bierman wrote:

> At 12:31 AM -0500 2/13/03, mlw wrote:
>
>> The idea that a, more or less, arbitrary data location determines the 
>> database configuration is wrong. It should be obvious to any 
>> administrator that a configuration file location which controls the 
>> server is the "right" way to do it.
>
>
>
> Isn't the database data itself a rather significant portion of the 
> 'configuration' of the database?
>
> What do you gain by having the postmaster config and the database data 
> live in different locations? 

While I don't like to use another product as an example, I think amongst 
the number of things Oracle does right is that it has a fairly standard 
way for an admin to find everything. All one needs to do is find the 
"ORACLE_HOME" directory, and everything can be found from there.

If, assume, PostgreSQL worked like every other system. It would have 
either an entry in /etc or some other directory specified by configure.

Somene please tell me how what I'm proposing differs from things like 
sendmail, named, or anyother standards based UNIX server?

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-13T07:36:45Z

Before I get started, I should note that it may be a good compromise
to have the data directory be the same as the config file directory,
when neither the config file nor the command line specify something
different.  So the changes I think may make the most sense are:

1.  We add a new GUC variable which specifies where the data is.
    The data is assumed to reside in the same place the config files
    reside unless the GUC variable is defined (either in
    postgresql.conf or on the command line, as usual for a GUC
    variable).  Both -D and $PGDATA therefore retain their current
    semantics unless overridden by the GUC variable, in which case
    they fall back to the new semantics of specifying only where the
    config files can be found.

2.  We add a configure option that specifies what the hardcoded
    fallback directory should be when neither -D nor $PGDATA are
    specified: /etc/postgresql when the option isn't specified to
    configure.

3.  We supply a different default startup script and a different
    default configuration file (but can make the older versions
    available in the distribution as well if we wish).  The former
    uses neither $PGDATA nor -D (or uses /etc/postgresql for them),
    and the latter uses the new GUC variable to specify a data
    directory location (/var/lib/postgres by default?)

This combination should work nicely for transitioning and for package
builders.  It accomplishes all of the goals mentioned in this thread
and will cause minimal pain for developers, since they can use their
current methods.  Sounds like it'll make Tom happy, at least.  :-)

Tom Lane wrote:
> mlw <pgsql@mohawksoft.com> writes:
> > The idea that a, more or less, arbitrary data location determines the 
> > database configuration is wrong. It should be obvious to any 
> > administrator that a configuration file location which controls the 
> > server is the "right" way to do it.
> 
> I guess I'm just dense, but I entirely fail to see why this is the One
> True Way To Do It.  

But we're not saying it's the One True Way, just saying that it's a
way that has very obvious benefits over the way we're using now, if
your job is to manage a system that someone else set up.

> What you seem to be proposing (ignoring syntactic-sugar issues) is
> that we replace "postmaster -D /some/data/dir" by "postmaster
> -config /some/config/file".  I am not seeing the nature of the
> improvement.

The nature of the improvement is that the configuration of a
PostgreSQL install will becomes obvious to anyone who looks in the
obvious places.  Remember, the '-D ...' is optional!  The PGDATA
environment variable can be used instead, and *is* used in what few
installations I've seen.  That's not something that shows up on the
command line when looking at the process list, which forces the
administrator to hunt down the data directory through other means.

> It looks to me like the sysadmin must now grant the Postgres DBA
> write access on *two* directories, viz /some/config/ and
> /wherever/the/data/directory/is.  How is that better than granting
> write access on one directory?

The difference in where you grant write access isn't a benefit to be
gained here.  The fact that you no longer have to give root privileges
to the DBA so that he can change the data directory as needed is the
benefit (well, one of them, at least).  A standard packaged install
can easily set the /etc/postgresql directory up with write permissions
for the postgres user by default, so the sysadmin won't even have to
touch it if he doesn't want to.

A big production database box is usually managed by one or more system
administrators and one or more DBAs.  Their roles are largely
orthogonal.  The sysadmins have the responsibility of keeping the
boxes up and making sure they don't fall over or crawl to a
standstill.  The DBAs have the responsibility of maximizing the
performance and availability of the database and *that's all*.  Giving
the DBAs root privileges means giving them the power to screw up the
system in ways that they can't recover from and might not even know
about.  The ways you can take down a system by misconfiguring the
database are bad enough.  No sane sysadmin is going to give the DBA
the power to run an arbitrary script as root at a time during the boot
cycle that the system is the most difficult to manage unless he thinks
the DBA is *really* good at system administration tasks, too.  And
that's assuming the sysadmin even *has* the authority to grant the DBA
that kind of access.  Many organizations keep a tight rein on who can
do what in an effort to minimize the damage from screwups.

The point is that the DBA isn't likely to have root access to the box.
When the DBA lacks that ability, the way we currently do things places
greater demand on the sysadmin than is necessary, because root access
is required to change the startup scripts, as it should be, and the
location of the data, as it should *not* be.

> Given that we can't manage to standardize the data directory
> location across multiple Unixen, how is it that we will be more
> successful at standardizing a config file location?

A couple of ways.

Firstly, as we mentioned before, just about every other daemon that
runs on a Unix system has its configuration file somewhere in the /etc
hierarchy.  By putting our config files in that same hierarchy we'll
be *adhering* to a standard.  We don't have to worry about
"standardizing" that config file location because it's *already* a
standard that we're currently ignoring.

Secondly, standards arise as a result of being declared standards and
by most people using them.  So simply by making /etc/postgresql the
default configuration directory, *that* will become the standard
place.  Most people won't mess with the default install if they don't
have to.

Right now they almost *have to* mess with the default install, because
there is no standard place on a Unix system for high speed, highly
reliable disk access.  And that means that, right now, there *is* no
standard place for our config files -- it's wherever the person who
configured the database decided the data should be, and he made that
decision based on performance and reliability considerations, not on
any standards.

> All I see here is an arbitrary break with our past practice.  I do not
> see any net improvement.

That's probably because you're looking at this from the point of view
of a developer.  From that standpoint there really isn't any net
improvement, because *you* still have to specify something on the
command line to get your test databases going.  As a developer you
*always* install and manage your own database installations, so *of
course* you'll always know where the config files are.  But that's not
how it works in the production world.

The break we'd be making is *not* arbitrary, and that's much of the
point: it's a break towards existing standards, and there are good
reasons for doing it, benefits to be had by adhering to those
standards.

The way we currently handle configuration files is fine for research
and development use -- the environment from which PostgreSQL sprang.
But now we're talking about getting it used in production
environments, and their requirements are very different.

Since it is *we* who are not currently adhering to the standard,
shouldn't the burden of proof (so to speak) be on those who wish to
keep things as they are?

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-13T11:20:46Z

> Well, to the extent that you're serious, you understand that 
> a lot of people feel that /usr/local should be reserved for 
> stuff that's installed by the local sysadmin, and your
> vendor/distro isn't supposed to be messing with it. 
> 
> Which means if the the vendor installed Postgresql (say, the
> Red Hat Database) you'd expect config files to be in /etc.
> If the postgresql is compiled from source by local admin, 
> you might look somewhere in /usr/local.

Indeed. For better or worse, there is a Filesystem Hierarcy Standard,
and most of the important Linux distros, BSDs and some legacy Unixen
stick to it, so so should we.

Configuration files should be in /etc/postgresql/, or at the very least
symlinked from there.

Martin

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-13T12:00:11Z

On Wed, 12 Feb 2003, J. M. Brenner wrote:

>
> "Christopher Kings-Lynne" <chriskl@familyhealth.com.au> wrote:
>
> > > Okay, here's one: most Unix systems store all of the configuration
> > > files in a well known directory: /etc.  These days it's a hierarchy of
> > > directories with /etc as the root of the hierarchy.  When an
> > > administrator is looking for configuration files, the first place he's
> > > going to look is in /etc and its subdirectories.
>
> > No goddammit - /usr/local/etc.  Why can't the Linux community respect
> > history!!!!
> >
> > It is the ONE TRUE PLACE dammit!!!
>
> Well, to the extent that you're serious, you understand that
> a lot of people feel that /usr/local should be reserved for
> stuff that's installed by the local sysadmin, and your
> vendor/distro isn't supposed to be messing with it.
>
> Which means if the the vendor installed Postgresql (say, the
> Red Hat Database) you'd expect config files to be in /etc.
> If the postgresql is compiled from source by local admin,
> you might look somewhere in /usr/local.

Then why not ~postgres/etc ??  Or substitute ~postgres with the
db admin user you (or the distro) decided on at installation time.
Gives a common location no matter who installed it or where it was
installed.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Curt Sampson <cjs@cynic.net> — 2003-02-13T13:09:41Z

On Wed, 12 Feb 2003, Peter Bierman wrote:

> What do you gain by having the postmaster config and the database
> data live in different locations?

You can then standardize a location for the configuration files.

Everybody has room in /etc for another 10K of data. Where you have
room for something that might potentially be a half terrabyte of
data, and is not infrequently several gigabytes or more, is pretty
system-depenendent.

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC

Re: location of the configuration files

Chris Browne <cbbrowne@acm.org> — 2003-02-13T13:32:51Z

In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote:
> On Wed, 12 Feb 2003, Peter Bierman wrote:
>
>> What do you gain by having the postmaster config and the database
>> data live in different locations?
>
> You can then standardize a location for the configuration files.
>
> Everybody has room in /etc for another 10K of data. Where you have
> room for something that might potentially be a half terrabyte of
> data, and is not infrequently several gigabytes or more, is pretty
> system-depenendent.

Ah, but this has two notable problems:

1.  It assumes that there is "a location" for "the configuration files
    for /the single database instance./"

    If I have a second database instance, that may conflict.

2.  It assumes I have write access to /etc

    If I'm a Plain Old User, as opposed to root, I may only have
    read-only access to /etc.

These conditions have both been known to occur...
-- 
If this was helpful, <http://svcs.affero.net/rm.php?r=cbbrowne> rate me
http://www.ntlug.org/~cbbrowne/rdbms.html
"Some people, when confronted with a Unix problem, think "I know, I'll
use sed." Now they have two problems."  -- Jamie Zawinski

Re: location of the configuration files

Curt Sampson <cjs@cynic.net> — 2003-02-13T14:18:00Z

On Thu, 13 Feb 2003, Christopher Browne wrote:

> 1.  It assumes that there is "a location" for "the configuration files
>     for /the single database instance./"

No; it assumes that there's a location for "the default instance." If
you have more than one, you could have one default and one elsewhere, or
just do what I often do, which is put in an empty config file except for
a comment saying "we have several instances of <xxx> on this machine; look
in <yyy> for them."

> 2.  It assumes I have write access to /etc
>
>     If I'm a Plain Old User, as opposed to root, I may only have
>     read-only access to /etc.

Right. It's dependent on the sysadmin to create /etc/postgres/ and make
it writeable, or set up proper symlinks, or whatever.

Fortunately, the files in /etc are only the defaults, to be used if
they're not overridden on the command line. If you're in a situation
like #2, you're basically stuck where we are now all the time: you have
to just put it somewhere and hope that, if someone else needs to find
it, they can.

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T14:23:20Z

Christopher Browne wrote:

>In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote:
>  
>
>>On Wed, 12 Feb 2003, Peter Bierman wrote:
>>
>>    
>>
>>>What do you gain by having the postmaster config and the database
>>>data live in different locations?
>>>      
>>>
>>You can then standardize a location for the configuration files.
>>
>>Everybody has room in /etc for another 10K of data. Where you have
>>room for something that might potentially be a half terrabyte of
>>data, and is not infrequently several gigabytes or more, is pretty
>>system-depenendent.
>>    
>>
>
>Ah, but this has two notable problems:
>
>1.  It assumes that there is "a location" for "the configuration files
>    for /the single database instance./"
>
>    If I have a second database instance, that may conflict.
>
>2.  It assumes I have write access to /etc
>
>    If I'm a Plain Old User, as opposed to root, I may only have
>    read-only access to /etc.
>
>These conditions have both been known to occur...
>  
>
These are not issues at all. You could put the configuration file 
anywhere, just as you can for any UNIX service.

postmaster --config=/home/myhome/mydb.conf

I deal with a number of PG databases on a number of sites, and it is a 
real pain in the ass to get to a PG box and hunt around for data 
directory so as to be able to administer the system. What's really 
annoying is when you have to find the data directory when someone else 
set up the system.

Configuring postgresql via a configuration file which specifies all the 
data, i.e. data directory, name of other configuration files, etc. is 
the right way to do it. Even if you have reasons against it, even if you 
think it is a bad idea, a bad standard is almost always a better 
solution than an arcane work of perfection.

Personally, however, I think the configuration issue is a no-brainer and 
I am amazed that people are balking. EVERY other service on a UNIX box 
is configured in this way, why not do it this way in PostgreSQL? The 
patch I submitted allowed the configuration to work as it currently 
does, but allowed for the more standard configuration file methodology.

I just don't understand what the resistance is, it makes no sense.

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-13T14:50:22Z

On Thursday 13 February 2003 08:32, Christopher Browne wrote:
> In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote:
> > Everybody has room in /etc for another 10K of data. Where you have
> > room for something that might potentially be a half terrabyte of
> > data, and is not infrequently several gigabytes or more, is pretty
> > system-depenendent.

> 1.  It assumes that there is "a location" for "the configuration files
>     for /the single database instance./"

>     If I have a second database instance, that may conflict.

If you run multiple servers of any kind, the second and subsequent servers 
must have a command line switch specifying the location of the config file.  
This is the way named, sendmail, et al do it.  I have run multiple nameds on 
a single box, by using alternate config file locations.

> 2.  It assumes I have write access to /etc

>     If I'm a Plain Old User, as opposed to root, I may only have
>     read-only access to /etc.

So you start postmaster with a config file switch pointing to your config file 
in your tree.  Or you specify the default location with a configure switch at 
compile time.  Or you do it the same way you would run any other typical 
daemon as a regular user.  How does Apache, AOLserver (my favorite), 
sendmail, jabberd, named, or any other typical daemon do it?  

For example, AOLserver can easily be installed and run as a plain user (just 
not on port 80).  The command line switch '-t' specifies the tcl 
configuration script's location.  There is no default.  The configuration 
script then specifies pageroot and the like -- and a webserver is very much 
like running PostgreSQL -- you have configuration, you have logs, and you 
have the spool (database or pageroot).  All can be in different locations at 
the discretion of the admin.  And hardcoding dependencies like this stifles 
the discretion of the admin.

> These conditions have both been known to occur...

Just because the situation is known to occur doesn't mean the whole direction 
of a project should hinge on those corner cases.  They should be allowed but 
not forced.

For better or for worse, thanks to Karl DeBisschop, the latest RPMs have the 
ability to start multiple postmasters with their data trees and 
configurations in different places.  This is all done in the startup script, 
and required no new functionality from postmaster.   I personally think it is 
for the better; YMMV.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Nigel J. Andrews <nandrews@investsystems.co.uk> — 2003-02-13T15:03:09Z

On Thu, 13 Feb 2003, Curt Sampson wrote:

> On Thu, 13 Feb 2003, Christopher Browne wrote:
> 
> > 1.  It assumes that there is "a location" for "the configuration files
> >     for /the single database instance./"
> 
> No; it assumes that there's a location for "the default instance." If
> you have more than one, you could have one default and one elsewhere, or
> just do what I often do, which is put in an empty config file except for
> a comment saying "we have several instances of <xxx> on this machine; look
> in <yyy> for them."
> 
> > 2.  It assumes I have write access to /etc
> >
> >     If I'm a Plain Old User, as opposed to root, I may only have
> >     read-only access to /etc.
> 
> Right. It's dependent on the sysadmin to create /etc/postgres/ and make
> it writeable, or set up proper symlinks, or whatever.
> 
> Fortunately, the files in /etc are only the defaults, to be used if
> they're not overridden on the command line. If you're in a situation
> like #2, you're basically stuck where we are now all the time: you have
> to just put it somewhere and hope that, if someone else needs to find
> it, they can.

It doesn't follow this line of argument directly but it's to do with this
thread...

Is everyone forgetting that wherever the configuration file is stored and
whether or not it needs a command line argument to specify it the database is
not going to start up automatically unless at least part of the installation is
done as root anyway?

As I like to install software as a non root user normally anyway I am happy
that the config file lives somewhere not requiring write access by the
installer. However, I think having it in an etc directory is a good thing
(tm). So, colour me an uncommited, fence sitter :)

I'm not talking distribution/package installation here but just plain system
administration. Being an untrusting soul I do _not_ want to type make install
as root and find things installed outside of where I say I want things placed.
That includes configuration files. Doing this as a normal user protects the
system from bad software which assumes things about the host system. It also
simplifies switching between versions of software, try doing that if your
config is /etc/postgresql/postgres.conf.

-- 
Nigel J. Andrews

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-13T15:03:42Z

mlw <pgsql@mohawksoft.com> writes:
> Here is the test, configure a server, with sendmail, named, apache, and 
> PostgreSQL. Tell me which of these systems doesn't configure right.

AFAIK, only one of those four is designed to support multiple instances
running on a single machine.  This is not unrelated.

			regards, tom lane

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T15:19:11Z

Tom Lane wrote:

>mlw <pgsql@mohawksoft.com> writes:
>  
>
>>Here is the test, configure a server, with sendmail, named, apache, and 
>>PostgreSQL. Tell me which of these systems doesn't configure right.
>>    
>>
>
>AFAIK, only one of those four is designed to support multiple instances
>running on a single machine.  This is not unrelated.
>
>  
>
While I will agree with you on sendmail and named, Apache is often run 
more than once with different options.
Furthermore, I hate to keep bringing it up, Oracle does use the 
configuration file methodology.

Tom, I just don't understand why this is being resisted so vigorously. 
What is wrong with starting PostgreSQL as:

postmaster -C /etc/postgresql.conf

UNIX admins would love to have this as a methodology, I don't think you 
can deny this, can you? I, as a long term PG user, really really want 
this, because in the long run, it makes PostgreSQL easier to administer.

If a patch allows PG to function as it does, but also allows a 
configuration file methodology, why not?

Re: location of the configuration files

Nigel J. Andrews <nandrews@investsystems.co.uk> — 2003-02-13T15:22:39Z

On Thu, 13 Feb 2003, mlw wrote:

> Tom, I just don't understand why this is being resisted so vigorously. 
> What is wrong with starting PostgreSQL as:
> 
> postmaster -C /etc/postgresql.conf
> 
> UNIX admins would love to have this as a methodology, I don't think you 
> can deny this, can you? I, as a long term PG user, really really want 
> this, because in the long run, it makes PostgreSQL easier to administer.
> 
> If a patch allows PG to function as it does, but also allows a 
> configuration file methodology, why not?

I forgot to say that I don't see why this facility can't be included in
addition to the existing scheme.


-- 
Nigel J. Andrews

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-13T15:32:19Z

On Thursday 13 February 2003 10:03, Tom Lane wrote:
> mlw <pgsql@mohawksoft.com> writes:
> > Here is the test, configure a server, with sendmail, named, apache, and
> > PostgreSQL. Tell me which of these systems doesn't configure right.

> AFAIK, only one of those four is designed to support multiple instances
> running on a single machine.  This is not unrelated.

One can run many nameds on a single machine by specifying '-c 
alternate_named.conf' , which then points to a different set of zone files, 
listens to either a different port or address, etc.  I have personally done 
this, and it worked as if it were designed to do so.

Apache can easily have multiple instances by passing the location of 
httpd.conf on the command line.  Everything else comes from that. Although 
Apache's virtual hosting is typically use instead, it may be necessary for 
large sites to run multiple instances with degrees of separation at the 
config file level.

I use AOLserver, which in version 3 is designed from the ground up for many 
(even thousands) of instances on a single box.  You access this capability 
with the '-t' switch (it stands for 'tcl config script' -- previous versions 
of AOLserver had an 'ini' file accessed with '-i', and version 3 added the 
tcl config script and deprecated the ini file).  In fact, since there is no 
default, you MUST use -t.  The tcl config script specifies all the parameters 
that instance needs (with the exception of the user and group ID the server 
should run as, if started as root (for port 80 access) -- but that doesn't 
effect PostgreSQL since our port is above 1024).  Two instances can even 
share a tcl config script, as long as the virtual server name is specified on 
the command line, and the tcl config has multiple virtual server sections.  

I personally only lightly use this feature, running a mere half dozen 
AOLserver's on one of my production servers.  All of which share a single 
PostgreSQL instance; but that's a different story.  

AOLserver is an excellent example here, as everything that has a location is 
configurable.  During ./configure, you can specify the prefix and the other 
standard autoconf type options.  This includes the location of the 
--enable-thread built Tcl 8.4 that you have to have first.  I have two 
versions of AOLserver on that machine, and they coexist very well, because I 
_can_ be so specific in where everything lies.  I run OpenACS on two of those 
instances, and, due to the size of that install I have those two pageroots on 
a separate filesystem from the binaries and config script.  It was just a 
single tcl config entry.  No biggie.

Even sendmail has a -c switch to specify the location of sendmail.cf: so, yes, 
you can run multiple instances, although it could be argued that it wasn't 
designed in.

Next?
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T15:59:11Z


Tom Lane wrote:

>mlw <pgsql@mohawksoft.com> writes:
>  
>
>>Here is the test, configure a server, with sendmail, named, apache, and 
>>PostgreSQL. Tell me which of these systems doesn't configure right.
>>    
>>
>
>AFAIK, only one of those four is designed to support multiple instances
>running on a single machine.  This is not unrelated.
>  
>
Also, using an explicit configuration file will also help you run 
multiple postgresql's on the same machien in a consistent manner, for 
instance:

postmaster -C /etc/postgres/common.conf -D /RAID0/postgres -p 5432
postmaster -C /etc/postgres/common.conf -D /RAID1/postgres -p 5433

Please, Tom, tell me why this is such a bad idea?

I will make the patch, I will submit it, will you guys put it in?

If not, why?

>  
>

Re: location of the configuration files

Bruno Wolff III <bruno@wolff.to> — 2003-02-13T16:28:52Z

On Thu, Feb 13, 2003 at 15:03:09 +0000,
  "Nigel J. Andrews" <nandrews@investsystems.co.uk> wrote:
> 
> Is everyone forgetting that wherever the configuration file is stored and
> whether or not it needs a command line argument to specify it the database is
> not going to start up automatically unless at least part of the installation is
> done as root anyway?

Users can use cron to start there own instance. Your cron script can check
if the server is running every (say) 15 minutes and start the server
if it isn't.

Re: location of the configuration files

Robert Treat <rtreat@webmd.net> — 2003-02-13T16:32:24Z

On Thu, 2003-02-13 at 09:23, mlw wrote: 
> I deal with a number of PG databases on a number of sites, and it is a
> real pain in the ass to get to a PG box and hunt around for data
> directory so as to be able to administer the system. What's really
> annoying is when you have to find the data directory when someone else
> set up the system.
> 

find / -name postgresql.conf -print

you now know where all of your configuration files are and where the
data for each of those servers is as well. 

(Not I'm not against the idea...)

Robert Treat

Re: location of the configuration files

Bruno Wolff III <bruno@wolff.to> — 2003-02-13T16:36:51Z

On Thu, Feb 13, 2003 at 09:23:20 -0500,
  mlw <pgsql@mohawksoft.com> wrote:
> 
> Personally, however, I think the configuration issue is a no-brainer and 
> I am amazed that people are balking. EVERY other service on a UNIX box 
> is configured in this way, why not do it this way in PostgreSQL? The 
> patch I submitted allowed the configuration to work as it currently 
> does, but allowed for the more standard configuration file methodology.

If you are interested in reading a contrary position, you can read
Berstein's arguments for his recommended way to install services at:
http://cr.yp.to/unix.html

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T16:47:22Z

Robert Treat wrote:

>On Thu, 2003-02-13 at 09:23, mlw wrote: 
>  
>
>>I deal with a number of PG databases on a number of sites, and it is a
>>real pain in the ass to get to a PG box and hunt around for data
>>directory so as to be able to administer the system. What's really
>>annoying is when you have to find the data directory when someone else
>>set up the system.
>>
>>    
>>
>
>find / -name postgresql.conf -print
>

LOL, That is NOT an option.  It can take hours on some systems. 
Specifically, one of the systems is  freedb server, it has over 300,000 
files in a directory tree.

Re: location of the configuration files

Stephan Szabo <sszabo@megazone23.bigpanda.com> — 2003-02-13T16:49:21Z

On Thu, 13 Feb 2003, mlw wrote:

>
>
> Robert Treat wrote:
>
> >On Thu, 2003-02-13 at 09:23, mlw wrote:
> >
> >
> >>I deal with a number of PG databases on a number of sites, and it is a
> >>real pain in the ass to get to a PG box and hunt around for data
> >>directory so as to be able to administer the system. What's really
> >>annoying is when you have to find the data directory when someone else
> >>set up the system.

You realize that the actual code feature doesn't necessarily help this
case, right? Putting configuration in /etc and having a configuration file
option on the command line are separate concepts.

I think the feature is worthwhile, but I have some initial condition
functionality questions that may have been answered in the previous patch,
but I don't remember at this point.

Mostly these have to deal with initial creation.  Does the user specify an
output location to initdb, do they just specify a data dir as now where
the configuration goes but then they need to move it somewhere, does
initdb now do nothing relating to configuration file and the user should
make one on his own.  Related, is the admin expected to have already made
(say) /etc/postgresql to stick the config in and set the permissions
correctly (since initdb doesn't run as root)?

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T16:53:26Z


Bruno Wolff III wrote:

>On Thu, Feb 13, 2003 at 09:23:20 -0500,
>  mlw <pgsql@mohawksoft.com> wrote:
>  
>
>>Personally, however, I think the configuration issue is a no-brainer and 
>>I am amazed that people are balking. EVERY other service on a UNIX box 
>>is configured in this way, why not do it this way in PostgreSQL? The 
>>patch I submitted allowed the configuration to work as it currently 
>>does, but allowed for the more standard configuration file methodology.
>>    
>>
>
>If you are interested in reading a contrary position, you can read
>Berstein's arguments for his recommended way to install services at:
>http://cr.yp.to/unix.html
>
>  
>
Where, specificaly are his arguements against a configuration file 
methodology?

>  
>

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T17:13:00Z

Stephan Szabo wrote:

>On Thu, 13 Feb 2003, mlw wrote:
>
>  
>
>>Robert Treat wrote:
>>
>>    
>>
>>>On Thu, 2003-02-13 at 09:23, mlw wrote:
>>>
>>>
>>>      
>>>
>>>>I deal with a number of PG databases on a number of sites, and it is a
>>>>real pain in the ass to get to a PG box and hunt around for data
>>>>directory so as to be able to administer the system. What's really
>>>>annoying is when you have to find the data directory when someone else
>>>>set up the system.
>>>>        
>>>>
>
>You realize that the actual code feature doesn't necessarily help this
>case, right? Putting configuration in /etc and having a configuration file
>option on the command line are separate concepts.
>
>I think the feature is worthwhile, but I have some initial condition
>functionality questions that may have been answered in the previous patch,
>but I don't remember at this point.
>
>Mostly these have to deal with initial creation.  Does the user specify an
>output location to initdb, do they just specify a data dir as now where
>the configuration goes but then they need to move it somewhere, does
>initdb now do nothing relating to configuration file and the user should
>make one on his own.  Related, is the admin expected to have already made
>(say) /etc/postgresql to stick the config in and set the permissions
>correctly (since initdb doesn't run as root)?
>
My patch only works on the PostgreSQL server code. No changes have been 
made to the initialization scripts.

The patch declares three extra configuration file parameters:
hbafile= '/etc/postgres/pg_hba.conf'
identfile='/etc/postgres/pg_ident.conf'
datadir='/RAID0/postgres'

The command line option is a capital 'C,' as in:
postmaster -C /etc/postgresql.conf

I have no problem leaving the default configuration files remaining in 
the data directory as sort of a maintenance / boot strap sort of thing, 
so I don't see any reason to alter the installation.

As for this feature helping or not, I think it will. I think it 
accomplishes two things:
(1) Separates configuration from data.
(2) Allows an administrator to create a convention across multiple 
systems regardless of the location and mount points of the database storage.
(3) Lastly, it is a familiar methodology to DBAs not familiar with 
PostgreSQL.

Again, I don't see a valid reason for not including the patch. Yes, if 
you don't want to configure PostgreSQL that way, then so be it, but why 
not add the functionality for those who do?

I can envision the configuration file methodology of managing a database 
becoming the "preferred" approach over time as it is a more familiar and 
standard way of configuring servers on UNIX.

>  
>

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T17:28:14Z

On Thu, 2003-02-13 at 13:32, Christopher Browne wrote:
> > Everybody has room in /etc for another 10K of data. Where you have
> > room for something that might potentially be a half terrabyte of
> > data, and is not infrequently several gigabytes or more, is pretty
> > system-depenendent.
> 
> Ah, but this has two notable problems:
> 
> 1.  It assumes that there is "a location" for "the configuration files
>     for /the single database instance./"
> 
>     If I have a second database instance, that may conflict.

I think that moving configuration to [/usr/local]/etc/postgresql implies
the need for sub-directories by port, possibly with a default config to
be used if there is no port-specific config file.

> 2.  It assumes I have write access to /etc
> 
>     If I'm a Plain Old User, as opposed to root, I may only have
>     read-only access to /etc.

The location should be configurable; I hope we're talking about the
default here.  For distributions it should be /etc/postgresql; for local
builds it should be /usr/local/etc/postgresql, assuming you have root
access.  If you don't, the -c configfile switch suggested elsewhere in
this debate would be needed.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T17:28:35Z

On Thu, 2003-02-13 at 12:00, Vince Vielhaber wrote:
> > Which means if the the vendor installed Postgresql (say, the
> > Red Hat Database) you'd expect config files to be in /etc.
> > If the postgresql is compiled from source by local admin,
> > you might look somewhere in /usr/local.
> 
> Then why not ~postgres/etc ??  Or substitute ~postgres with the
> db admin user you (or the distro) decided on at installation time.
> Gives a common location no matter who installed it or where it was
> installed.

Because it doesn't comply with FHS.  All projects should remember that
they coexist with many others and should do their best to stick to
common standards.

The default config file location should be set as a parameter to
./configure, which should default to /usr/local/etc/postgresql.  Those
of us who build for distributions will change it to /etc/postgresql.

I suppose if we want to run different postmasters simultaneously, we
could have /etc/postgresql/5432/ and so on for each port number being
used.  Perhaps have a default set in /etc/postgresql/ which can be used
if there is no port-specific directory, but a postmaster using those
defaults would have to have PGDATA specified on the command line.

On the same lines, I have just had a request (as Debian maintainer) to
move the location of postmaster.pid to the /var/run hierarchy; firstly,
so as to make it easier for the administrator to find, and secondly so
as to make it easier to configure SE Linux policy for file access.  (SE
Linux is the highly secure version produced by the NSA.)

I'm not entirely sure why SE Linux has a problem, seeing that postgres
needs read-write access to all the files in $PGDATA, but assuming the
need is verified, I could do this by moving the pid file from
$PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for
other ports.  This would also have the benefit of being more FHS
compliant  What do people think about that?

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-13T17:52:23Z

On 13 Feb 2003, Oliver Elphick wrote:

> On Thu, 2003-02-13 at 12:00, Vince Vielhaber wrote:
> > > Which means if the the vendor installed Postgresql (say, the
> > > Red Hat Database) you'd expect config files to be in /etc.
> > > If the postgresql is compiled from source by local admin,
> > > you might look somewhere in /usr/local.
> >
> > Then why not ~postgres/etc ??  Or substitute ~postgres with the
> > db admin user you (or the distro) decided on at installation time.
> > Gives a common location no matter who installed it or where it was
> > installed.
>
> Because it doesn't comply with FHS.  All projects should remember that
> they coexist with many others and should do their best to stick to
> common standards.
>
> The default config file location should be set as a parameter to
> ./configure, which should default to /usr/local/etc/postgresql.  Those
> of us who build for distributions will change it to /etc/postgresql.

Seems to me that if FHS allows such a mess, it's reason enough to avoid
compliance.  Either that or those of you who build for distributions are
making an ill advised change.  Simply because the distribution makes the
decision to add PostgreSQL, or some other package, to it's distribution
doesn't make it a requirement to change the location of the config files.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T18:18:31Z

On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote:
> Seems to me that if FHS allows such a mess, it's reason enough to avoid
> compliance.  Either that or those of you who build for distributions are
> making an ill advised change.  Simply because the distribution makes the
> decision to add PostgreSQL, or some other package, to it's distribution
> doesn't make it a requirement to change the location of the config files.

Debian (and FHS) specifically requires that.  All configuration files
MUST be under /etc; the reason is to make the system administrator's job
easier.  Part of the raison d'etre of a distribution is to rationalise
the idiosyncrasies of individual projects.  The locations used by
locally-built packages are up to the local administrator, but they
really should not be in /etc and are recommended to be under /usr/local.

I really don't see why there is such a not-invented-here mentality about
this issue.  I say again, standards-compliance is the best way.  It
makes life easier for everyone if standards are followed.  Don't we
pride ourselves on being closer to the SQL spec than other databases? 
Any way, if PostgreSQL stays as it is, I will continue to have to ensure
that initdb creates symlinks to /etc/postgresql/, as happens now.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T18:45:27Z

Oliver Elphick wrote:
> On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote:
> > Seems to me that if FHS allows such a mess, it's reason enough to avoid
> > compliance.  Either that or those of you who build for distributions are
> > making an ill advised change.  Simply because the distribution makes the
> > decision to add PostgreSQL, or some other package, to it's distribution
> > doesn't make it a requirement to change the location of the config files.
> 
> Debian (and FHS) specifically requires that.  All configuration files
> MUST be under /etc; the reason is to make the system administrator's job
> easier.  Part of the raison d'etre of a distribution is to rationalise
> the idiosyncrasies of individual projects.  The locations used by
> locally-built packages are up to the local administrator, but they
> really should not be in /etc and are recommended to be under /usr/local.
> 
> I really don't see why there is such a not-invented-here mentality about
> this issue.  I say again, standards-compliance is the best way.  It
> makes life easier for everyone if standards are followed.  Don't we
> pride ourselves on being closer to the SQL spec than other databases? 
> Any way, if PostgreSQL stays as it is, I will continue to have to ensure
> that initdb creates symlinks to /etc/postgresql/, as happens now.

It doesn't have anything to do with "not-invented-here", which is a
common refrain by people who don't like our decisions, like "Why don't
you use mmap()?  Oh, it's because I thought of it and you didn't".  Does
anyone seriously believe that is the motiviation of anyone in this
project!  I certainly don't.

Now, on to this configuration discussion.  Seems moving the config file
out of $PGDATA requies either:
	
	1) we specifiy both the config directory and the data directory on
	postmaster start
	
	2) we specify the pgdata directory inside postgresql.conf or
	other config file

Is this accurate?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T19:06:23Z


Bruce Momjian wrote:

>Oliver Elphick wrote:
>  
>
>>On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote:
>>    
>>
>>>Seems to me that if FHS allows such a mess, it's reason enough to avoid
>>>compliance.  Either that or those of you who build for distributions are
>>>making an ill advised change.  Simply because the distribution makes the
>>>decision to add PostgreSQL, or some other package, to it's distribution
>>>doesn't make it a requirement to change the location of the config files.
>>>      
>>>
>>Debian (and FHS) specifically requires that.  All configuration files
>>MUST be under /etc; the reason is to make the system administrator's job
>>easier.  Part of the raison d'etre of a distribution is to rationalise
>>the idiosyncrasies of individual projects.  The locations used by
>>locally-built packages are up to the local administrator, but they
>>really should not be in /etc and are recommended to be under /usr/local.
>>
>>I really don't see why there is such a not-invented-here mentality about
>>this issue.  I say again, standards-compliance is the best way.  It
>>makes life easier for everyone if standards are followed.  Don't we
>>pride ourselves on being closer to the SQL spec than other databases? 
>>Any way, if PostgreSQL stays as it is, I will continue to have to ensure
>>that initdb creates symlinks to /etc/postgresql/, as happens now.
>>    
>>
>
>It doesn't have anything to do with "not-invented-here", which is a
>common refrain by people who don't like our decisions, like "Why don't
>you use mmap()?  Oh, it's because I thought of it and you didn't".  Does
>anyone seriously believe that is the motiviation of anyone in this
>project!  I certainly don't.
>
>Now, on to this configuration discussion.  Seems moving the config file
>out of $PGDATA requies either:
>	
>	1) we specifiy both the config directory and the data directory on
>	postmaster start
>	
>	2) we specify the pgdata directory inside postgresql.conf or
>	other config file
>
>Is this accurate?
>  
>
The patch that I have adds three settings to postgresql.conf and one 
command line parameter.

hba_conf = 'filename'
ident_conf='filename'
data_dir='path'

The command linae parameter is  -C, used as:

postmaster -C /usr/local/etc/postgresql.conf

I think this will help administrators.

Bruce, can you shed some light as to why this is being so strongly 
rejected. I just don't see any downside. I just don't get it.

I will be resubmitting my patch for the 7.3.2 tree.


>  
>

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T19:13:14Z

On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote:
> Now, on to this configuration discussion.  Seems moving the config file
> out of $PGDATA requies either:
> 	
> 	1) we specifiy both the config directory and the data directory on
> 	postmaster start
> 	
> 	2) we specify the pgdata directory inside postgresql.conf or
> 	other config file
> 
> Is this accurate?

The default start would read the config file from its predefined
location, set by ./configure.  No command line options would be
necessary for the postmaster to run, though they could be provided.

The config file should contain the pgdata location; this and any other
parameter should be overridden if a different location is specified by a
command-line option. I think the config should be able to contain all
information that can be specified on the command line (except, of
course, the location of the configuration file.)

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T19:15:36Z

On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote:
> Oliver Elphick wrote:
> > On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote:
> > > Seems to me that if FHS allows such a mess, it's reason enough to avoid
> > > compliance.  Either that or those of you who build for distributions are
> > > making an ill advised change.  Simply because the distribution makes the
> > > decision to add PostgreSQL, or some other package, to it's distribution
> > > doesn't make it a requirement to change the location of the config files.
> > ...
> > I really don't see why there is such a not-invented-here mentality about
> > this issue.  I say again, standards-compliance is the best way.  It
> > makes life easier for everyone if standards are followed.  Don't we
> > pride ourselves on being closer to the SQL spec than other databases? 
> > Any way, if PostgreSQL stays as it is, I will continue to have to ensure
> > that initdb creates symlinks to /etc/postgresql/, as happens now.
> 
> It doesn't have anything to do with "not-invented-here", which is a
> common refrain by people who don't like our decisions, like "Why don't
> you use mmap()?  Oh, it's because I thought of it and you didn't".  Does
> anyone seriously believe that is the motiviation of anyone in this
> project!  I certainly don't.

My apologies.  I withdraw the comment, which was provoked mostly by
Vince's response, quoted above.  I agree that it is not characteristic
of the project.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T19:17:51Z

mlw wrote:
> >It doesn't have anything to do with "not-invented-here", which is a
> >common refrain by people who don't like our decisions, like "Why don't
> >you use mmap()?  Oh, it's because I thought of it and you didn't".  Does
> >anyone seriously believe that is the motiviation of anyone in this
> >project!  I certainly don't.
> >
> >Now, on to this configuration discussion.  Seems moving the config file
> >out of $PGDATA requies either:
> >	
> >	1) we specifiy both the config directory and the data directory on
> >	postmaster start
> >	
> >	2) we specify the pgdata directory inside postgresql.conf or
> >	other config file
> >
> >Is this accurate?
> >  
> >
> The patch that I have adds three settings to postgresql.conf and one 
> command line parameter.
> 
> hba_conf = 'filename'
> ident_conf='filename'
> data_dir='path'
> 
> The command linae parameter is  -C, used as:
> 
> postmaster -C /usr/local/etc/postgresql.conf
> 
> I think this will help administrators.
> 
> Bruce, can you shed some light as to why this is being so strongly 
> rejected. I just don't see any downside. I just don't get it.
> 
> I will be resubmitting my patch for the 7.3.2 tree.

Well, in a sense, it trades passing one parameter, PGDATA, for another. 
I see your point that we should specify configuration first, and let
everything pass from there.  However, it does add extra configuration
parameters, and because you still need to specify/create pgdata, it adds
an extra level of abstraction to setting up the server.

Also, there is nothing preventing someone from symlinking the
configuration files from pgdata to somewhere else.

I don't think separate params for each config file is good.  At the
most, I think we will specify the configuration _directory_ for all the
config files, perhaps pgsql/etc, and have pgdata default to ../data, or
honor $PGDATA.  That might be the cleanest.

Of course, that now gives us $PGCONFIG and $PGDATA, and possible
intraction if postgresql.conf specifies a different pgdata from $PGDATA.
As you can see, it could get messy.

And, if you specify pgdata in postgresql.conf, it prevents you from
using that file by different postmasters.

My best guess would be to not specify pgdata in postgresql.conf, and
have a new $PGCONFIG param to specify the configuration directory, but
if we do that, $PGDATA/postgresql.conf becomes meaningless, which could
also be confusing.  Maybe we don't allow those files to exist in $PGDATA
if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA.  See,
I am getting myself confused.  :-)

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T19:23:38Z

On Thu, 2003-02-13 at 14:06, mlw wrote:
> 
> I will be resubmitting my patch for the 7.3.2 tree.
> 

I'm no core developer, but surely this wont be included in the 7.3.x
branch. Any change needs to be made against CVS head.

Robert Treat

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T19:28:48Z

Robert Treat wrote:
> On Thu, 2003-02-13 at 14:06, mlw wrote:
> > 
> > I will be resubmitting my patch for the 7.3.2 tree.
> > 
> 
> I'm no core developer, but surely this wont be included in the 7.3.x
> branch. Any change needs to be made against CVS head.

I assume he meant he will repost his 7.3.2-based patch and we will merge
it into CVS HEAD if it is accepted.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T19:30:43Z

On Thu, 2003-02-13 at 12:13, mlw wrote:
> 
> My patch only works on the PostgreSQL server code. No changes have been
> made to the initialization scripts.
> 
> The patch declares three extra configuration file parameters:
> hbafile= '/etc/postgres/pg_hba.conf'
> identfile='/etc/postgres/pg_ident.conf'
> datadir='/RAID0/postgres'
> 

If we're going to do this, I think we need to account for all of the
files in the directory including PG_VERSION, postmaster.opts,
postmaster.pid. In the end if we can't build so that we are either fully
FHS compliant and/or LSB compliant, we've not done enough work on it.

Robert Treat

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T19:43:27Z

Robert Treat wrote:
> On Thu, 2003-02-13 at 12:13, mlw wrote:
> > 
> > My patch only works on the PostgreSQL server code. No changes have been
> > made to the initialization scripts.
> > 
> > The patch declares three extra configuration file parameters:
> > hbafile= '/etc/postgres/pg_hba.conf'
> > identfile='/etc/postgres/pg_ident.conf'
> > datadir='/RAID0/postgres'
> > 
> 
> If we're going to do this, I think we need to account for all of the
> files in the directory including PG_VERSION, postmaster.opts,
> postmaster.pid. In the end if we can't build so that we are either fully
> FHS compliant and/or LSB compliant, we've not done enough work on it.

Woh, how do we move some of those files into /etc or /var/run if we
aren't running as root?  We certainly don't want to require that.  I
guess /etc/postgresql will work if that directory is owned by the
PostgreSQL superuser, but /var/run will be a problem.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T19:47:12Z

On Thu, 2003-02-13 at 19:30, Robert Treat wrote:
> If we're going to do this, I think we need to account for all of the
> files in the directory including PG_VERSION, postmaster.opts,

Not PG_VERSION; that is intimately associated with the data itself and
ought to stay in the data directory.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T19:48:28Z


Bruce Momjian wrote:

>Well, in a sense, it trades passing one parameter, PGDATA, for another. 
>I see your point that we should specify configuration first, and let
>everything pass from there.  However, it does add extra configuration
>parameters, and because you still need to specify/create pgdata, it adds
>an extra level of abstraction to setting up the server.
>
While this is true, it is not uncommon, and it is more or less expected 
by most UNIX admins.

>
>Also, there is nothing preventing someone from symlinking the
>configuration files from pgdata to somewhere else.
>
Stop!!!  symlinks are not sufficient. When happens when a native Win32 
version comes out? there are no symlinks. Also, most of the admins I 
know don't like to use simlinks as they are not self documenting. 
Symlinks are "bad."

>
>I don't think separate params for each config file is good.  At the
>most, I think we will specify the configuration _directory_ for all the
>config files, perhaps pgsql/etc, and have pgdata default to ../data, or
>honor $PGDATA.  That might be the cleanest.
>
The problem with that is that you are back to symlinking shared files. 
Symlinks are a kludge.

>
>Of course, that now gives us $PGCONFIG and $PGDATA, and possible
>intraction if postgresql.conf specifies a different pgdata from $PGDATA.
>As you can see, it could get messy.
>
I don't see it as very messy, for instance:

postmaster -C /etc/postgres/postgresql.conf -D /RAID0/postgres -p 5432
postmaster -C /etc/postgres/postgresql.conf -D /RAID1/postgres -p 5433

That looks like a real clean way to run multiple PostgreSQL servers on 
the same box using the same configuration files.

>And, if you specify pgdata in postgresql.conf, it prevents you from
>using that file by different postmasters.
>
Not true, command line parameters, as a rule, override configuration 
file defaults.

>
>My best guess would be to not specify pgdata in postgresql.conf, and
>have a new $PGCONFIG param to specify the configuration directory, but
>if we do that, $PGDATA/postgresql.conf becomes meaningless, which could
>also be confusing.  Maybe we don't allow those files to exist in $PGDATA
>if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA.  See,
>I am getting myself confused.  :-)
>
I think you are making it too complicated.

I wouldn't remove the default configration set, it would be useful as a 
failsafe or maintainence feature.

Re: location of the configuration files

Stephan Szabo <sszabo@megazone23.bigpanda.com> — 2003-02-13T19:50:59Z

On Thu, 13 Feb 2003, mlw wrote:

> Stephan Szabo wrote:
> >>>On Thu, 2003-02-13 at 09:23, mlw wrote:
> >>>>I deal with a number of PG databases on a number of sites, and it is a
> >>>>real pain in the ass to get to a PG box and hunt around for data
> >>>>directory so as to be able to administer the system. What's really
> >>>>annoying is when you have to find the data directory when someone else
> >>>>set up the system.
> >>>>
> >>>>
> >
> >You realize that the actual code feature doesn't necessarily help this
> >case, right? Putting configuration in /etc and having a configuration file
> >option on the command line are separate concepts.

Re-read my statement and yours about the case you were mentioning. ;)
Sure, putting the files in /etc lets you find them easily.  However, if
you're doing things like finding configuration made by someone else and
said configuration isn't in /etc (which if they wanted to they could do
now with symlinks I believe - yes symlinks aren't a complete solution, but
I think they're reasonable on most of our current ports) then you still
have to search the system for the configuration file, except now it might
not even be postgresql.conf. That's why I said the two issues aren't the
same.

> >I think the feature is worthwhile, but I have some initial condition
> >functionality questions that may have been answered in the previous patch,
> >but I don't remember at this point.
> >
> >Mostly these have to deal with initial creation.  Does the user specify an
> >output location to initdb, do they just specify a data dir as now where
> >the configuration goes but then they need to move it somewhere, does
> >initdb now do nothing relating to configuration file and the user should
> >make one on his own.  Related, is the admin expected to have already made
> >(say) /etc/postgresql to stick the config in and set the permissions
> >correctly (since initdb doesn't run as root)?
> >
> My patch only works on the PostgreSQL server code. No changes have been
> made to the initialization scripts.
>
> The patch declares three extra configuration file parameters:
> hbafile= '/etc/postgres/pg_hba.conf'
> identfile='/etc/postgres/pg_ident.conf'
> datadir='/RAID0/postgres'
>
> The command line option is a capital 'C,' as in:
> postmaster -C /etc/postgresql.conf
>
> I have no problem leaving the default configuration files remaining in
> the data directory as sort of a maintenance / boot strap sort of thing,
> so I don't see any reason to alter the installation.
>
>
> As for this feature helping or not, I think it will. I think it
> accomplishes two things:
> (1) Separates configuration from data.
> (2) Allows an administrator to create a convention across multiple
> systems regardless of the location and mount points of the database storage.
> (3) Lastly, it is a familiar methodology to DBAs not familiar with
> PostgreSQL.

I agree on all these points ("I think the feature is worthwhile, but...").
I just wonder if we were going to do this, we might as well look at all of
the various things people want and decide what we want to do, for example,
people commenting on default configuration locations through configure,
how does this interact with what we have now, etc.  I'd rather have a
month spent arguing out a behavior rather than just adding a new behavior
that we'll need to possibly revisit again in the future. :)

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T19:51:27Z


Robert Treat wrote:

>On Thu, 2003-02-13 at 12:13, mlw wrote:
>  
>
>>My patch only works on the PostgreSQL server code. No changes have been
>>made to the initialization scripts.
>>
>>The patch declares three extra configuration file parameters:
>>hbafile= '/etc/postgres/pg_hba.conf'
>>identfile='/etc/postgres/pg_ident.conf'
>>datadir='/RAID0/postgres'
>>
>>    
>>
>
>If we're going to do this, I think we need to account for all of the
>files in the directory including PG_VERSION, postmaster.opts,
>postmaster.pid. In the end if we can't build so that we are either fully
>FHS compliant and/or LSB compliant, we've not done enough work on it.
>
>Robert Treat
>
>
>  
>
postmaster.opts, PG_VERSION, and postmaster.pid are not configuration 
parameters.

PG_VERSION is VERY important, it is how you know the version of the 
database.
Postmaster.pid is a postgres writable value
AFAIK, postmaster.opts is also a postgres writable value.

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T19:58:26Z


Bruce Momjian wrote:

>Robert Treat wrote:
>  
>
>>On Thu, 2003-02-13 at 12:13, mlw wrote:
>>    
>>
>>>My patch only works on the PostgreSQL server code. No changes have been
>>>made to the initialization scripts.
>>>
>>>The patch declares three extra configuration file parameters:
>>>hbafile= '/etc/postgres/pg_hba.conf'
>>>identfile='/etc/postgres/pg_ident.conf'
>>>datadir='/RAID0/postgres'
>>>
>>>      
>>>
>>If we're going to do this, I think we need to account for all of the
>>files in the directory including PG_VERSION, postmaster.opts,
>>postmaster.pid. In the end if we can't build so that we are either fully
>>FHS compliant and/or LSB compliant, we've not done enough work on it.
>>    
>>
>
>Woh, how do we move some of those files into /etc or /var/run if we
>aren't running as root?  We certainly don't want to require that.  I
>guess /etc/postgresql will work if that directory is owned by the
>PostgreSQL superuser, but /var/run will be a problem.
>
>  
>
I don't think those files need to move. As I said in another post, they 
are postgres writable and should in the PostgreSQL data directory. 
However, I suppose, that those also could be configuration parameters? No?

PG_VERSION obviously should not move.
postmaster.opts gets created when postmaster is run, correct?

The only issue would be the PID file, and I don't have strong feelings 
about it, except that using a /var/run system will make running multiple 
postmasters a pain.

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T20:08:39Z


Stephan Szabo wrote:

>On Thu, 13 Feb 2003, mlw wrote:
>
>  
>
>>Stephan Szabo wrote:
>>    
>>
>>>>>On Thu, 2003-02-13 at 09:23, mlw wrote:
>>>>>          
>>>>>
>>>>>>I deal with a number of PG databases on a number of sites, and it is a
>>>>>>real pain in the ass to get to a PG box and hunt around for data
>>>>>>directory so as to be able to administer the system. What's really
>>>>>>annoying is when you have to find the data directory when someone else
>>>>>>set up the system.
>>>>>>
>>>>>>
>>>>>>            
>>>>>>
>>>You realize that the actual code feature doesn't necessarily help this
>>>case, right? Putting configuration in /etc and having a configuration file
>>>option on the command line are separate concepts.
>>>      
>>>
>
>Re-read my statement and yours about the case you were mentioning. ;)
>Sure, putting the files in /etc lets you find them easily.  However, if
>you're doing things like finding configuration made by someone else and
>said configuration isn't in /etc (which if they wanted to they could do
>now with symlinks I believe - yes symlinks aren't a complete solution, but
>I think they're reasonable on most of our current ports) then you still
>have to search the system for the configuration file, except now it might
>not even be postgresql.conf. That's why I said the two issues aren't the
>same.
>
>  
>
>>>I think the feature is worthwhile, but I have some initial condition
>>>functionality questions that may have been answered in the previous patch,
>>>but I don't remember at this point.
>>>
>>>Mostly these have to deal with initial creation.  Does the user specify an
>>>output location to initdb, do they just specify a data dir as now where
>>>the configuration goes but then they need to move it somewhere, does
>>>initdb now do nothing relating to configuration file and the user should
>>>make one on his own.  Related, is the admin expected to have already made
>>>(say) /etc/postgresql to stick the config in and set the permissions
>>>correctly (since initdb doesn't run as root)?
>>>
>>>      
>>>
>>My patch only works on the PostgreSQL server code. No changes have been
>>made to the initialization scripts.
>>
>>The patch declares three extra configuration file parameters:
>>hbafile= '/etc/postgres/pg_hba.conf'
>>identfile='/etc/postgres/pg_ident.conf'
>>datadir='/RAID0/postgres'
>>
>>The command line option is a capital 'C,' as in:
>>postmaster -C /etc/postgresql.conf
>>
>>I have no problem leaving the default configuration files remaining in
>>the data directory as sort of a maintenance / boot strap sort of thing,
>>so I don't see any reason to alter the installation.
>>
>>
>>As for this feature helping or not, I think it will. I think it
>>accomplishes two things:
>>(1) Separates configuration from data.
>>(2) Allows an administrator to create a convention across multiple
>>systems regardless of the location and mount points of the database storage.
>>(3) Lastly, it is a familiar methodology to DBAs not familiar with
>>PostgreSQL.
>>    
>>
>
>I agree on all these points ("I think the feature is worthwhile, but...").
>I just wonder if we were going to do this, we might as well look at all of
>the various things people want and decide what we want to do, for example,
>people commenting on default configuration locations through configure,
>how does this interact with what we have now, etc.  I'd rather have a
>month spent arguing out a behavior rather than just adding a new behavior
>that we'll need to possibly revisit again in the future. :)
>

I have absolutely no problem debating and augmenting the feature. None 
what so ever, I am more pushing to get momentum to actually do it. In 
7.1 I proposed this, and was told that it wasn't needed because (a) 
symlinks provide all the functionality you need and (b) that they were 
going to redesign the configuration system. That was well over a year 
ago (two?). I am willing to do the work, but what's the point if the 
core group isn't even going to use it?

Most of the admins I know don't use symlinks as they can not carry 
comments. Without knowing,  you can change or delete a file that does 
not appear to be in use but which kills a working server. Symlinks are 
dangerous in production systems, it is easy to screw them up with scp 
when administering a cluster of computers.


>  
>

Re: location of the configuration files

scott.marlowe <scott.marlowe@ihs.com> — 2003-02-13T20:11:13Z

On 13 Feb 2003, Martin Coxall wrote:

> 
> > Well, to the extent that you're serious, you understand that 
> > a lot of people feel that /usr/local should be reserved for 
> > stuff that's installed by the local sysadmin, and your
> > vendor/distro isn't supposed to be messing with it. 
> > 
> > Which means if the the vendor installed Postgresql (say, the
> > Red Hat Database) you'd expect config files to be in /etc.
> > If the postgresql is compiled from source by local admin, 
> > you might look somewhere in /usr/local.
> 
> Indeed. For better or worse, there is a Filesystem Hierarcy Standard,
> and most of the important Linux distros, BSDs and some legacy Unixen
> stick to it, so so should we.
> 
> Configuration files should be in /etc/postgresql/, or at the very least
> symlinked from there.

So, how do we handle things like installing three or four versions at the 
same time.  This isn't the same thing as /etc/fstab.  While we only would 
likely need to have one fstab or whatever, with postgresql, it's not 
unreasonable to want to intall more than one copy or version for various 
reason.

Generally things that live in /etc are owned and operated by the OS.  
Postgresql, by it's definition is a userspace program, not an OS owned 
one.

I've found having a $PGDATA var where EVERYTHING lives to be a huge 
advantage when you need to run a half dozen instances of pgsql under 
different accounts or for different versions on the same box.

Now, if we could do it like X, where the base stuff is all in the 
/etc/X11R6 directory, but your own personal config lives in your home 
directory, then we're right as rain.  but what parts of postgresql would 
always be common to all flavors that might need to be run at the same 
time?  Not much.

Re: location of the configuration files

Steve Crawford <scrawford@pinpointresearch.com> — 2003-02-13T20:28:07Z

I don't see why we can't keep everyone happy and let the users choose the 
setup they want. To wit, make the following, probably simple, changes:

1) Have postgresql default to using /etc/postgresql.conf
2) Add a setting in postgresql.conf specifying the data directory
3) Change the meaning of -D to mean "use this config file"
4) In the absence of a specified data directory in postgresql.conf, use the 
location of the postgresql.conf file as the data directory

I see several advantages:

1) Anyone who doesn't want to change doesn't have to - leaving the data 
directory spec out of postgresql.conf and starting with -D will be 
essentially identical to how things are now (except it would be -D 
/foo/bar/postgresql.conf instead of -D /foo/bar/ - even this could be 
overcome with a bit of bailing wire saying if -D specifies a directory, look 
for postgresql.conf in that directory).

2) Postgresql will be more "familiar" to those who expect or desire configs 
to be in /etc.

3) Adding a postgresql.conf line for data location sets the stage for being 
able to specify directories for all sorts of files (WAL, index, etc.) without 
the need for symlinks.

4) Multiple config files could be more easily managed for 
testing/benchmarking/etc.

Cheers,
Steve

On Wednesday 12 February 2003 10:14 pm, Peter Bierman wrote:
> At 12:31 AM -0500 2/13/03, mlw wrote:
> >The idea that a, more or less, arbitrary data location determines
> >the database configuration is wrong. It should be obvious to any
> >administrator that a configuration file location which controls the
> >server is the "right" way to do it.
>
> Isn't the database data itself a rather significant portion of the
> 'configuration' of the database?
>
> What do you gain by having the postmaster config and the database
> data live in different locations?
>
> -pmb
>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)

Re: location of the configuration files

scott.marlowe <scott.marlowe@ihs.com> — 2003-02-13T21:07:50Z

On Thu, 13 Feb 2003, mlw wrote:

> 
> 
> Christopher Browne wrote:
> 
> >In the last exciting episode, cjs@cynic.net (Curt Sampson) wrote:
> >
> >>On Wed, 12 Feb 2003, Peter Bierman wrote:
> >>
> >>>What do you gain by having the postmaster config and the database
> >>>data live in different locations?
> >>>
> >>You can then standardize a location for the configuration files.
> >>
> >>Everybody has room in /etc for another 10K of data. Where you have
> >>room for something that might potentially be a half terrabyte of
> >>data, and is not infrequently several gigabytes or more, is pretty
> >>system-depenendent.
> >
> >Ah, but this has two notable problems:
> >
> >1.  It assumes that there is "a location" for "the configuration files
> >    for /the single database instance./"
> >
> >    If I have a second database instance, that may conflict.
> >
> >2.  It assumes I have write access to /etc
> >
> >    If I'm a Plain Old User, as opposed to root, I may only have
> >    read-only access to /etc.
> >
> >These conditions have both been known to occur...
> >  
> >
> These are not issues at all. You could put the configuration file 
> anywhere, just as you can for any UNIX service.
> 
> postmaster --config=/home/myhome/mydb.conf
> 
> I deal with a number of PG databases on a number of sites, and it is a 
> real pain in the ass to get to a PG box and hunt around for data 
> directory so as to be able to administer the system. What's really 
> annoying is when you have to find the data directory when someone else 
> set up the system.

Really?  I would think it's easier to do this:

su - pgsuper
cd $PGDATA
pwd

Than to try to figure out what someone entered when they ran ./configure 
--config=...

> Configuring postgresql via a configuration file which specifies all the 
> data, i.e. data directory, name of other configuration files, etc. is 
> the right way to do it. Even if you have reasons against it, even if you 
> think it is a bad idea, a bad standard is almost always a better 
> solution than an arcane work of perfection.

Wrong, I strongly disagree with this sentament.  Conformity to standards 
for simple conformity's sake is as wrong as sticking to the old way 
because it's what we're all comfy with.  

> Personally, however, I think the configuration issue is a no-brainer and 
> I am amazed that people are balking. EVERY other service on a UNIX box 
> is configured in this way, why not do it this way in PostgreSQL? The 
> patch I submitted allowed the configuration to work as it currently 
> does, but allowed for the more standard configuration file methodology.

If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which 
is not the standard way you're listing.  If I install openldap from 
.tar.gz, I get a /usr/local/etc/openldap directory, close, but still not 
the same.  The fact is, it's the packagers that put things into /etc and 
whatnot, and I can see the postgresql RPMs or debs or whatever having that 
as the default, but for custom built software, NOTHING that I know of 
builds from source and uses /etc without a switch to tell it to, just like 
postgresql can do now.

> I just don't understand what the resistance is, it makes no sense.

I agree, but from the other side of the fence.

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-13T21:21:48Z

On 13 Feb 2003, Oliver Elphick wrote:

> On Thu, 2003-02-13 at 18:45, Bruce Momjian wrote:
> > Oliver Elphick wrote:
> > > On Thu, 2003-02-13 at 17:52, Vince Vielhaber wrote:
> > > > Seems to me that if FHS allows such a mess, it's reason enough to avoid
> > > > compliance.  Either that or those of you who build for distributions are
> > > > making an ill advised change.  Simply because the distribution makes the
> > > > decision to add PostgreSQL, or some other package, to it's distribution
> > > > doesn't make it a requirement to change the location of the config files.
> > > ...
> > > I really don't see why there is such a not-invented-here mentality about
> > > this issue.  I say again, standards-compliance is the best way.  It
> > > makes life easier for everyone if standards are followed.  Don't we
> > > pride ourselves on being closer to the SQL spec than other databases?
> > > Any way, if PostgreSQL stays as it is, I will continue to have to ensure
> > > that initdb creates symlinks to /etc/postgresql/, as happens now.
> >
> > It doesn't have anything to do with "not-invented-here", which is a
> > common refrain by people who don't like our decisions, like "Why don't
> > you use mmap()?  Oh, it's because I thought of it and you didn't".  Does
> > anyone seriously believe that is the motiviation of anyone in this
> > project!  I certainly don't.
>
> My apologies.  I withdraw the comment, which was provoked mostly by
> Vince's response, quoted above.  I agree that it is not characteristic
> of the project.

I certainly wasn't trying to provoke anything.  It just seems odd to me
that when the distribution installs a package and places it's config files
in /etc and later the admin happens to upgrade by the instructions with
the package, it's acceptable for the config files to now be in two places
and you don't find it confusing.  What happens when a new admin comes on
and tries to figure out which config file is which?   Ever try to figure
out where the hell Pine's config really is?

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Peter Eisentraut <peter_e@gmx.net> — 2003-02-13T21:25:49Z

mlw writes:

> AFAIK it wasn't actually done. It was more of a, "we should do something
> different" argument. At one point it was talked about rewriting the
> configuration system to allow "include" and other things.

The core of the problem was, and continues to be, this:  If you move
postgresql.conf somewhere else, then someone else will also want to move
pg_hba.conf and all the rest.  And that opens up a number of security and
cumbersome-to-install problems.

Just having an option that says, the configuration file is "there", is a
first step but not a complete solution.

-- 
Peter Eisentraut   peter_e@gmx.net

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T21:26:37Z


Peter Eisentraut wrote:

>mlw writes:
>
>  
>
>>AFAIK it wasn't actually done. It was more of a, "we should do something
>>different" argument. At one point it was talked about rewriting the
>>configuration system to allow "include" and other things.
>>    
>>
>
>The core of the problem was, and continues to be, this:  If you move
>postgresql.conf somewhere else, then someone else will also want to move
>pg_hba.conf and all the rest.  And that opens up a number of security and
>cumbersome-to-install problems.
>
>Just having an option that says, the configuration file is "there", is a
>first step but not a complete solution.
>
The location of pg_hba.conf and pg_ident.conf can be specified within 
the postgresql.conf file if desired.

I don't understand the security concerns, what security issues can there be?

>  
>

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T21:44:05Z


scott.marlowe wrote:

>>These are not issues at all. You could put the configuration file 
>>anywhere, just as you can for any UNIX service.
>>
>>postmaster --config=/home/myhome/mydb.conf
>>
>>I deal with a number of PG databases on a number of sites, and it is a 
>>real pain in the ass to get to a PG box and hunt around for data 
>>directory so as to be able to administer the system. What's really 
>>annoying is when you have to find the data directory when someone else 
>>set up the system.
>>    
>>
>
>Really?  I would think it's easier to do this:
>
>su - pgsuper
>cd $PGDATA
>pwd
>
>Than to try to figure out what someone entered when they ran ./configure 
>--config=...
>  
>
Why do you think PGDATA would be set for root?

>  
>
>>Configuring postgresql via a configuration file which specifies all the 
>>data, i.e. data directory, name of other configuration files, etc. is 
>>the right way to do it. Even if you have reasons against it, even if you 
>>think it is a bad idea, a bad standard is almost always a better 
>>solution than an arcane work of perfection.
>>    
>>
>
>Wrong, I strongly disagree with this sentament.  Conformity to standards 
>for simple conformity's sake is as wrong as sticking to the old way 
>because it's what we're all comfy with.  
>
It isn't conformity for conformitys sake. It is following an established 
practice, like driving on the same side of the road or stopping at red 
lights.

>
>  
>
>>Personally, however, I think the configuration issue is a no-brainer and 
>>I am amazed that people are balking. EVERY other service on a UNIX box 
>>is configured in this way, why not do it this way in PostgreSQL? The 
>>patch I submitted allowed the configuration to work as it currently 
>>does, but allowed for the more standard configuration file methodology.
>>    
>>
>
>If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which 
>is not the standard way you're listing.  If I install openldap from 
>.tar.gz, I get a /usr/local/etc/openldap directory, close, but still not 
>the same.  The fact is, it's the packagers that put things into /etc and 
>whatnot, and I can see the postgresql RPMs or debs or whatever having that 
>as the default, but for custom built software, NOTHING that I know of 
>builds from source and uses /etc without a switch to tell it to, just like 
>postgresql can do now.
>
You are confusing the default location of a file with the ability to use 
the file. The default I have proposed all along was to use the existing 
practice of keeping everything in the $PGDATA directory.

The change I wish to make to the code allows this to be changed. Most 
admins want configuration and data separate. Most admins do not want to 
use symlinks because they are dangerous in a production environment.

I would rather have a simpler solution sooner than a perfect solution never.

Re: location of the configuration files

Sean Chittenden <sean@chittenden.org> — 2003-02-13T22:06:25Z

> All I see here is an arbitrary break with our past practice.  I do
> not see any net improvement.

<FreeBSD Port Maintainer>
Well, given that there's a trend to make PostgreSQL more usable, I can
say with absolute certainty, that an FAQ that I get about once a week
is (and granted only from new users) "where is the postgresql.conf?  I
don't see it in ${LOCALBASE}/etc/."  PostgreSQL is one of a few ports
in an extreme minority that uses a local configuration directive and
it violates the policy of least surprise for sysadmins.

PS LOCALBASE/PREFIX is /usr/local 99.999% of the time
</FreeBSD Port Maintainer>

With my DBA hat on, however, here are a few reasons that I'd like to
see the conf moved out of the data directory:

1) pg_dumpall > foo && rm -rf $PGDATA && initdb

   As a DBA I don't have to worry about backing up my config file when
   doing upgrades since the config file is located in an external
   directory.

2) Backing up config files in ${LOCALBASE}/etc is a pretty common
   practice.  Having to make a special case for postgresql's kind of a
   PITA.


Suggestions:

1) gmake install installs a default configuration file in
   ${LOCALBASE}/etc/postgresql.conf.default.  Promote that DBAs should
   diff postgresql.conf.default with postgresql.conf and make
   adjustments as they see fit, but gmake install will _not_, under
   any circumstances, touch postgresql.conf (by default, it should cp
   postgresql.conf.default to postgresql.conf that way things "just
   work" out of the box).

2) Leave the current functionality in place.  Being able to have
   multiple databases on the same machine is a _really_ nice feature
   of PostgreSQL.  If you want multiple databases, having the config
   file in $PGDATA makes some sense because with multiple
   installations, you want to keep everything together... though it
   doesn't make much sense if you have only one installation per
   server... and really, the only reason to have multiple
   installations is to handle username collisions (hint hint).

3) In the absence of a PGDATA environment variable (don't want to
   break backward compatible installations) being set, the future
   behavior allow for a default location of a config file (if no CLI
   switch is specified for an explicit location) that points to a
   config file.  The path would be ${PREFIX}/etc and would provide
   most admins with a standard launching off point for running/tuning
   their databases.  The config file would have to specify the data
   directory as well as the path to the hba.conf, which should be
   outside of the datadir as well (speaking of the hba.conf, am I the
   only one who things that hba.conf should be converted into a system
   catalog?  ::shrug::)

Just some random thoughts from someone who's had to deal with this on
all of the mentioned levels (new users, single installations, multiple
installations, and multiple copies running via daemontools). -sc


PS If there is no huge press for this, I should have the time do do
this in a few weeks if someone doesn't beat me to it.

-- 
Sean Chittenden

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-13T22:22:56Z

On Thu, 2003-02-13 at 21:21, Vince Vielhaber wrote:
> I certainly wasn't trying to provoke anything.  It just seems odd to me
> that when the distribution installs a package and places it's config files
> in /etc and later the admin happens to upgrade by the instructions with
> the package, it's acceptable for the config files to now be in two places
> and you don't find it confusing.  What happens when a new admin comes on
> and tries to figure out which config file is which?   Ever try to figure
> out where the hell Pine's config really is?

I've not used pine, and there doesn't seem to be an official Debian
package, (it doesn't allow any changes to its source, I believe, which
makes it ineligible).  But if it were an official package, I know I
should look in /etc/pine.

If the admin installs a local build of something he has installed as a
package, he will presumably take care to separate the two.  If his local
build is to replace the package, he should purge the installed package,
so that there are no traces of it left.  Since he is administering a
distribution installation, it is certainly his responsibility to
understand the difference between local and distributed packages, as
well as the different places that each should put their configuration
files.  (Incidentally, Debian's changes from the upstream configuration
are documented in the package.)  In the end, though, when we package for
a distribution, we expect people to use the packages.  If they want to
build from source, the packages system lets them do it.  Anyone who is
building from the upstream source must be presumed to know what he is
doing and take responsibility for it.

What your comments strongly suggest to me is that projects like
PostgreSQL and pine, along with everything else, should comply with FHS;
then there will be no confusion because everyone will be following the
smae standards.  Messes arise when people ignore standards; we have all
seen the dreadful examples of MySQL and the Beast, haven't we?

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "The earth is the LORD'S, and the fullness thereof; the
      world, and they that dwell therein."            
                                   Psalms 24:1

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T22:25:13Z

On Thu, 2003-02-13 at 14:28, Bruce Momjian wrote:
> Robert Treat wrote:
> > On Thu, 2003-02-13 at 14:06, mlw wrote:
> > > 
> > > I will be resubmitting my patch for the 7.3.2 tree.
> > > 
> > 
> > I'm no core developer, but surely this wont be included in the 7.3.x
> > branch. Any change needs to be made against CVS head.
> 
> I assume he meant he will repost his 7.3.2-based patch and we will merge
> it into CVS HEAD if it is accepted.
> 

IIRC he originally wrote the patch for a pre 7.3 version, so it seems
like he'd be reworking it for 7.3.x with the above statement. I'm only
suggesting he rework it against CVS head if he doesn't have plans to do
so already. Course if yall are willing to merge it in for him, none of
this really matters does it? :-)

Robert Treat

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T22:29:24Z

On Thu, 2003-02-13 at 14:43, Bruce Momjian wrote:
> Robert Treat wrote:
> > On Thu, 2003-02-13 at 12:13, mlw wrote:
> > > 
> > > My patch only works on the PostgreSQL server code. No changes have been
> > > made to the initialization scripts.
> > > 
> > > The patch declares three extra configuration file parameters:
> > > hbafile= '/etc/postgres/pg_hba.conf'
> > > identfile='/etc/postgres/pg_ident.conf'
> > > datadir='/RAID0/postgres'
> > > 
> > 
> > If we're going to do this, I think we need to account for all of the
> > files in the directory including PG_VERSION, postmaster.opts,
> > postmaster.pid. In the end if we can't build so that we are either fully
> > FHS compliant and/or LSB compliant, we've not done enough work on it.
> 
> Woh, how do we move some of those files into /etc or /var/run if we
> aren't running as root?  We certainly don't want to require that.  I
> guess /etc/postgresql will work if that directory is owned by the
> PostgreSQL superuser, but /var/run will be a problem.
> 

Seems like some are saying one of the problems with the current system
is it doesn't follow FHS or LSB. If those are valid reasons to change
the system, it seems like a change which doesn't actually address those
concerns would not be acceptable. (Unless those really aren't valid
concerns...)

Robert Treat

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T22:44:29Z

On Thu, 2003-02-13 at 14:51, mlw wrote:
> 
> 
> Robert Treat wrote:
> 
> 
> On Thu, 2003-02-13 at 12:13, mlw wrote:
> 
>   
> 
> My patch only works on the PostgreSQL server code. No changes have been
> 
> made to the initialization scripts.
> 
> 
> 
> The patch declares three extra configuration file parameters:
> 
> hbafile= '/etc/postgres/pg_hba.conf'
> 
> identfile='/etc/postgres/pg_ident.conf'
> 
> datadir='/RAID0/postgres'
> 
> 
> If we're going to do this, I think we need to account for all of the
> 
> files in the directory including PG_VERSION, postmaster.opts,
> 
> postmaster.pid. In the end if we can't build so that we are either fully
> 
> FHS compliant and/or LSB compliant, we've not done enough work on it.
> 
> 
> postmaster.opts, PG_VERSION, and postmaster.pid are not configuration
> parameters. 
>

So? I'm not saying they all have to be moved, just they all need to be
accounted for. 
 
> PG_VERSION is VERY important, it is how you know the version of the
> database.
> Postmaster.pid is a postgres writable value 
> AFAIK, postmaster.opts is also a postgres writable value.
> 

IIRC the postmaster.pid file should be in /var/run according to FHS, I'm
not sure about postmaster.opts though...

Again, if we're going to make a change, let's make sure we think it
through.

Robert Treat

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T22:53:53Z

Oliver Elphick wrote:
> What your comments strongly suggest to me is that projects like
> PostgreSQL and pine, along with everything else, should comply with FHS;
> then there will be no confusion because everyone will be following the
> smae standards.  Messes arise when people ignore standards; we have all
> seen the dreadful examples of MySQL and the Beast, haven't we?

Can the FHS handle installing PostgreSQL as non-root?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-13T22:55:16Z

Robert Treat wrote:
> IIRC the postmaster.pid file should be in /var/run according to FHS, I'm
> not sure about postmaster.opts though...
> 
> Again, if we're going to make a change, let's make sure we think it
> through.

Can non-root write to /var/run?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-13T22:59:17Z

On Thu, 2003-02-13 at 15:08, mlw wrote:
> Stephan Szabo wrote:
> 
> Re-read my statement and yours about the case you were mentioning. ;)
> 
> Sure, putting the files in /etc lets you find them easily.  However, if
> 
> you're doing things like finding configuration made by someone else and
> 
> said configuration isn't in /etc (which if they wanted to they could do
> 
> now with symlinks I believe - yes symlinks aren't a complete solution,
> but
> 
> I think they're reasonable on most of our current ports) then you still
> 
> have to search the system for the configuration file, except now it
> might
> 
> not even be postgresql.conf. That's why I said the two issues aren't the
> 
> same.
> 

Actually, I'd almost go so far as to say it will make it worse. In the
current system, if you can figure out where $PGDATA is, you've found
everything you need for that installation. In the new system, there's no
telling where people will put things, and it certainly won't be any
easier to find it. THinking on the above Stephan, you'd almost have to
require that the config file be called postgresql.conf in order to run,
anything else leads to real scary scenario's.

> On Thu, 13 Feb 2003, mlw wrote:
> 
> I have absolutely no problem debating and augmenting the feature. None
> what so ever, I am more pushing to get momentum to actually do it. 

Stick with it, I think most of us here can see the value in the option,
but there are valid concerns that it be implemented correctly.
Personally I think a postgresql installation is much more like an apache
installation, which generally contains all of the files (data and
config) under /usr/local/apache. Maybe someone can dig more to see if
that system is more appropriate a comparison than something like bind.

Robert Treat

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T23:06:15Z


Bruce Momjian wrote:

>Robert Treat wrote:
>  
>
>>IIRC the postmaster.pid file should be in /var/run according to FHS, I'm
>>not sure about postmaster.opts though...
>>
>>Again, if we're going to make a change, let's make sure we think it
>>through.
>>    
>>
>
>Can non-root write to /var/run?
>
>  
>
Shouldn't be able too

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-13T23:07:46Z

On 13 Feb 2003, Oliver Elphick wrote:

> What your comments strongly suggest to me is that projects like
> PostgreSQL and pine, along with everything else, should comply with FHS;
> then there will be no confusion because everyone will be following the
> smae standards.  Messes arise when people ignore standards; we have all
> seen the dreadful examples of MySQL and the Beast, haven't we?

Actually FHS says the opposite.  If the distribution installs PostgreSQL
then the config files belong in /etc/postgresql.  If the admin does then
they belong in /usr/local/etc/postgresql.  FHS is out of their tree.  If
PostgreSQL or any other package is not critical to the basic operation of
the operating system, it's config files shouldn't be polluting /etc.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-13T23:25:20Z


Robert Treat wrote:

>On Thu, 2003-02-13 at 14:51, mlw wrote:
>  
>
>>Robert Treat wrote:
>>
>>
>>On Thu, 2003-02-13 at 12:13, mlw wrote:
>>
>>  
>>
>>My patch only works on the PostgreSQL server code. No changes have been
>>
>>made to the initialization scripts.
>>
>>
>>
>>The patch declares three extra configuration file parameters:
>>
>>hbafile= '/etc/postgres/pg_hba.conf'
>>
>>identfile='/etc/postgres/pg_ident.conf'
>>
>>datadir='/RAID0/postgres'
>>
>>
>>If we're going to do this, I think we need to account for all of the
>>
>>files in the directory including PG_VERSION, postmaster.opts,
>>
>>postmaster.pid. In the end if we can't build so that we are either fully
>>
>>FHS compliant and/or LSB compliant, we've not done enough work on it.
>>
>>
>>postmaster.opts, PG_VERSION, and postmaster.pid are not configuration
>>parameters. 
>>
>>    
>>
>
>So? I'm not saying they all have to be moved, just they all need to be
>accounted for. 
>
OK, what was the point?

>>PG_VERSION is VERY important, it is how you know the version of the
>>database.
>>Postmaster.pid is a postgres writable value 
>>AFAIK, postmaster.opts is also a postgres writable value.
>>
>>    
>>
>
>IIRC the postmaster.pid file should be in /var/run according to FHS, I'm
>not sure about postmaster.opts though...
>
>Again, if we're going to make a change, let's make sure we think it
>through.
>
I'm not a big fan of the "/var/run" directory convention, especially 
when we expect multiple instances of the server to be able to run 
concurrently. I suppose it can be a parameter in both the configuration 
file and command line.

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-13T23:26:00Z

On Thursday 13 February 2003 17:53, Bruce Momjian wrote:
> Oliver Elphick wrote:
> > What your comments strongly suggest to me is that projects like
> > PostgreSQL and pine, along with everything else, should comply with FHS;
> > then there will be no confusion because everyone will be following the
> > smae standards.  Messes arise when people ignore standards; we have all
> > seen the dreadful examples of MySQL and the Beast, haven't we?

> Can the FHS handle installing PostgreSQL as non-root?

Once again, no one is trying to make an FHS install the default 'let's force 
everyone to think our way or no way' coercion.

We just want the option.

For those who wish to do non-root installs, nothing would need to change.  You 
can still put it into /usr/local/pgsql (assuming you have permissions to put 
it there) or your home directory, or wherever.

I deal with RPMs; Oliver deals with .deb's.  Neither can be installed as 
non-root.  The daemon can of course run as non-root (and it does, which is 
exactly correct); but the installation of the files is done as root _always_ 
in an RPM or deb environment.  So I really don't care about non-root 
installs; sorry.  I wonder what percentage of our users are not the 
administrator of the machine on which they are running PostgreSQL?

I dispute the statement made earlier in the thread (not by Bruce) that 
PostgreSQL is by definition not an OS service.  This is false, and needs to 
be realized by this community.  PostgreSQL is becoming an essential OS core 
service in many cases: virtually all Linux distributions (the lion's share of 
our current distribution) include PostgreSQL as a core service.  Many of our 
new users see PostgreSQL as 'SQL server' in the Red Hat installation menu.

Now, on a Win32 server, what is PostgreSQL going to be considered?  It is 
probably going to run as a service, right? So you need to be Administrator 
there to perform the install, right?

This isn't the same environment, Bruce, that you got into back when it was 
still Postgres95.  We are in the big leagues OS-wise, and we need to act like 
it.  Assuming that we are a 'userspace' program (which is a misnomer anyway, 
as _anything_ non-kernel is 'userspace') is not going to cut it anymore.  

So we need to fit in to an OS environment, whether it is FreeBSD, OS/X, Win32, 
Solaris, or Linux.  In FreeBSD, as the ports maintainer excellently posted, 
PostgreSQL should live in LOCALBASE.  We should make that easy.  In Win32, 
configuration might be better stored in the system registry (Argh! Did I 
actually say THAT! Yuck!) -- we should make even that easy.  In OS/X we 
should use the OS/X paradigm (whatever that is).  And we should make it easy 
to make PostgreSQL LSB-compliant for our very large Linux user community.  We 
should be adaptable to the accepted administration paradigm on whatever 
system we are running -- this should be a minimum.

These concerns vastly outweigh the occasional non-root install from source, in 
my mind at least.  I am not opposed to that way even being the default; after 
all, leaving the default the same as now agrees with the principle of least 
surprise (although we really don't ascribe to that; witness the 7.2-7.3 
migration fiasco -- 7.3 should have been 8.0 to warn people of the major 
changes going on in client connections).  But I do advocate _allowing_ the 
configuration options Mark has enumerated -- although I really wish we could 
use the lowercase c instead, for consistency with other OS services.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-13T23:30:45Z

On Thursday 13 February 2003 18:07, Vince Vielhaber wrote:
> Actually FHS says the opposite.  If the distribution installs PostgreSQL
> then the config files belong in /etc/postgresql.  If the admin does then
> they belong in /usr/local/etc/postgresql.  FHS is out of their tree.  If
> PostgreSQL or any other package is not critical to the basic operation of
> the operating system, it's config files shouldn't be polluting /etc.

PostgreSQL is as critical as PHP, Apache, or whatever other package is being 
backended by PostgreSQL.  If the package is provided by the distributor, 
consider it part of the OS.  If it isn't, well, it isn't.

This is so that local admin installed (from source -- not from binary package) 
files don't get clobbered by the next operating system binary upgrade.  In 
that context the FHS (LSB) mandate makes lots of sense.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-13T23:41:06Z

On Thu, 13 Feb 2003, Lamar Owen wrote:

> On Thursday 13 February 2003 18:07, Vince Vielhaber wrote:
> > Actually FHS says the opposite.  If the distribution installs PostgreSQL
> > then the config files belong in /etc/postgresql.  If the admin does then
> > they belong in /usr/local/etc/postgresql.  FHS is out of their tree.  If
> > PostgreSQL or any other package is not critical to the basic operation of
> > the operating system, it's config files shouldn't be polluting /etc.
>
> PostgreSQL is as critical as PHP, Apache, or whatever other package is being
> backended by PostgreSQL.  If the package is provided by the distributor,
> consider it part of the OS.  If it isn't, well, it isn't.

You completely miss my point, but lately you've been real good at that.

Can the system boot without PHP, Apache, PostgreSQL, Mysql and/or Pine?
Can the root user log in without PHP, Apache, PostgreSQL, Mysql and/or Pine?
Can any user log in without PHP, Apache, PostgreSQL, Mysql and/or Pine?

Note, I'm not even including an MTA here.  I said BASIC OPERATION.

If a package is not critical as I just outlined, it shouldn't matter who
installed it.

After the last go around with you Lamar, this will be my last response
to you on this.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-14T00:10:04Z

On Thursday 13 February 2003 18:41, Vince Vielhaber wrote:
> On Thu, 13 Feb 2003, Lamar Owen wrote:
> > PostgreSQL is as critical as PHP, Apache, or whatever other package is
> > being backended by PostgreSQL.  If the package is provided by the
> > distributor, consider it part of the OS.  If it isn't, well, it isn't.

> You completely miss my point, but lately you've been real good at that.

No, Vince, I understand your point.  But understand mine: it does matter who 
installed it.

> Note, I'm not even including an MTA here.  I said BASIC OPERATION.

> If a package is not critical as I just outlined, it shouldn't matter who
> installed it.

'Critical' is in the eye of the admin of the system in question.  For my 
servers, if, for instance, sshd doesn't come up, then there's a major 
problem, as they are all headless.  If the webserver doesn't come up, I have 
other problems, as OpenACS is mission-critical here.  So what's critical is a 
question for the individual sysadmin.

So, to continue your point, what is 'critical' to the 'basic operation' of the 
system shouldn't pollute /etc.  So, let's eliminate the /etc/mail, 
/etc/samba, /etc/xinetd.d, /etc/X11, /etc/httpd, and the other subtrees foung 
in at least Red Hat 8.  While we're at it, many other files in /etc need to 
go: named.conf for one.  It depends on what you consider 'critical'.  
PostgreSQL is at least as critical on my systems as some of the other things 
that already 'pollute' /etc.

> After the last go around with you Lamar, this will be my last response
> to you on this.

Aw Vince, I don't know what your problem is with conflicting opinions.  But 
that's your choice.  And Open Source is about _choice_.  You are free to 
admin your systems your way, and I'm free to do so my way.  And all's well.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Adam Haberlach <adam@newsnipple.com> — 2003-02-14T00:22:40Z

On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote:
> On Thu, 2003-02-13 at 15:08, mlw wrote:
> > Stephan Szabo wrote:
> > 
> > On Thu, 13 Feb 2003, mlw wrote:
> > 
> > I have absolutely no problem debating and augmenting the feature. None
> > what so ever, I am more pushing to get momentum to actually do it. 
> 
> Stick with it, I think most of us here can see the value in the option,
> but there are valid concerns that it be implemented correctly.
> Personally I think a postgresql installation is much more like an apache
> installation, which generally contains all of the files (data and
> config) under /usr/local/apache. Maybe someone can dig more to see if
> that system is more appropriate a comparison than something like bind.

	I think you are making a pretty uninformed, if not just plain wrong 
generalization.  I've run exactly one system with apache configuration 
files in /usr/local/apache, and even then, the data was not there.

	A quick straw poll of the people I know who actually do run real systems
also mentioned that they use packaging systems like encap or rpm to manage
upgrades, and would almost never put datafiles into /usr/local.

RedHat (7.3 at least)'s default httpd datafiles go in /var/www/html and
config goes in /etc/httpd

One OpenBSD user I talked to puts his in /home/www and config files in
/etc/httpd.  The defaults are /var/www and /var/www/conf

Another user reports:
On systems that I set up I have /web/{apache|httpd}/ and put all 
the config info there.
And /web/sites/name/ holds site data.

What does this mean?

People will put things in different places, and there are typically
very good reasons for this.  This is ESPECIALLY true when one wants to
have configuration files, at least the base ones in a common place such
as /etc or /usr/local/etc in order to make backup of configuration easy
and clean, while leaving data somewhere else for performance or magnitude
of partition reasons.  It just makes sense to ME to have postgresql.conf
reside in /etc, yet put my data in /var/data/postgresql, yet retain the
option to put my data in /raid/data/postgresql at a later date, when the
new hardware comes in.

Yes, symlinks are an option on most systems.  No, they are not a good
one on most systems.

What _I_ would like to see:

o. a default postgresql.conf location of $PREFIX/data/postgresql.conf
o. a default PGDATA location of whatever directory postgresql.conf is in
(this should maintain backward compatibility)
o. a ./configure - time option to override the location of the postgresql.conf
o. a run-time option to override the location of the postgresql.conf
o. options in postgresql.conf to specify the location of PGDATA and PID files.

($PREFIX is already settable at ./configure - time)

This would allow:
	o. Config files in /usr/local/pgsql/data, /etc, /usr/local/etc, ~postgresql, 
	or /dev/.hidden-node, whichever you prefer, so long as you either know
	the compile-time default, or are willing to specify it at startup.

	o. Datafiles to be in /usr/local/pgsql/data, /var/data, /raid0, /nfs/bigmount
	or whichever you prefer, so long as you either know the compile-time default,
	or are willing to specify it in a config file that you specify at startup.

Does it add complexity to the system?  Sure -- a very little bit, IMHO, especially
compared to the BTREE-folding that I see being bantered about.

Is it some work?  Sure -- a very little bit, and it seems that it has already
been done.

However, this seems, to me, to be a very small addition that has some real-world
(and yes, we need to start paying attention to the real world) advantages.

And finally, don't go telling me that I'm wrong to put my data and config files
where I am.  You can offer advice, but I'm probably going to ignore it because
I like where they are and don't need to explain why.

-- 
Adam Haberlach         | "Because manholes are round."
adam@mediariffic.com   |
http://mediariffic.com |

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T01:09:34Z

Lamar Owen wrote:
> This isn't the same environment, Bruce, that you got into back when it was 
> still Postgres95.  We are in the big leagues OS-wise, and we need to act like 
> it.  Assuming that we are a 'userspace' program (which is a misnomer anyway, 
> as _anything_ non-kernel is 'userspace') is not going to cut it anymore.  

So you are saying this isn't my grandma's database anymore.  :-)

Anyway, I think I have _a_ proposal that we can use to work toward a
goal.

First, a few conclusions:

	We can't use /var/run because we need the postmaster to create
	those, and it isn't root.

	Right now, the fact that we mix the config stuff with the
	data isn't ideal.  Someone mentioned:

		pg_dumpall > foo && rm -rf $PGDATA && initdb

	discards all the config files.

So, I propose we change a few things.  The good news is that this is
something only administrators deal with;  client apps don't deal with
it.

OK, first, we keep postmaster.pid and postmaster.opts in /data.  We
can't put them in /var/run, and /data seems like the best spot for them.

That leaves postgresql.conf, pg_hba.conf, and pg_ident.conf.  I
recommend moving them all, by default, into pgsql/etc.  I recommend we
add these to postgresql.conf:

	data_dir = ../data
	pg_hba_dir = ./
	pg_ident_dir = ./

Those paths are relative to postgresql.conf.

We then add a PGCONFIG variable and postmaster -C flag to point to the
config _directory_.  That way, if folks want to move all of this into
/etc, then easily do that.  This also pulls those files out of /data so
they are easier to back up.

We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps
removing the other duplicate postmaster flags that have postgresql.conf
entries.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-14T01:39:13Z

On Thursday 13 February 2003 20:09, Bruce Momjian wrote:
> Lamar Owen wrote:
> > This isn't the same environment, Bruce, that you got into back when it
> > was still Postgres95.

> So you are saying this isn't my grandma's database anymore.  :-)

I actually thought of saying it that way, too. :-)

> Anyway, I think I have _a_ proposal that we can use to work toward a
> goal.

> First, a few conclusions:

> 	We can't use /var/run because we need the postmaster to create
> 	those, and it isn't root.

It isn't without precedent to have a directory under /var/run.  Maybe 
/var/run/postgresql.  Under this one could have a uniquely named pid file.  I 
say uniquely named so that multiple postmasters could run.  Naming those 
files could be fun. /var/run/postgresql would be owned by the postmaster run 
user.  This of course requires root to install -- but would be completely 
optional.

> 		pg_dumpall > foo && rm -rf $PGDATA && initdb

> 	discards all the config files.

Yes, this is a big deal.  It makes it more difficult to properly restore.  
While it's not impossible to do so now, of course, it just could be a little 
easier.

> So, I propose we change a few things.

> OK, first, we keep postmaster.pid and postmaster.opts in /data.  We
> can't put them in /var/run, and /data seems like the best spot for them.

Can we make that configurable?  The default in pgdata is fine; just having the 
option is good.

> That leaves postgresql.conf, pg_hba.conf, and pg_ident.conf.  I
> recommend moving them all, by default, into pgsql/etc.  I recommend we
> add these to postgresql.conf:

> 	data_dir = ../data
> 	pg_hba_dir = ./
> 	pg_ident_dir = ./

> Those paths are relative to postgresql.conf.

And these are all just defaults, easily changed.  Good.

> We then add a PGCONFIG variable and postmaster -C flag to point to the
> config _directory_.  That way, if folks want to move all of this into
> /etc, then easily do that.  This also pulls those files out of /data so
> they are easier to back up.

Yes.  I'm thinking along the lines of this sort of structure:
/etc
|---postgresql
                |----- name of postmaster one (unique ID of some kind)
                |----- name of postmaster two
                    .
                    .

Not difficult.

> We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps
> removing the other duplicate postmaster flags that have postgresql.conf
> entries.

Now I really _like_ this idea.  By removing it to 7.5, and therefore 
deprecating it in 7.4, this brings best practice into effect.

However, at the same time, I wouldn't be opposed to leaving them in place, 
either, for backwards compatibility.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T01:47:11Z

Bruce Momjian wrote:
> I don't think separate params for each config file is good.  At the
> most, I think we will specify the configuration _directory_ for all the
> config files, perhaps pgsql/etc, and have pgdata default to ../data, or
> honor $PGDATA.  That might be the cleanest.
> 
> Of course, that now gives us $PGCONFIG and $PGDATA, and possible
> intraction if postgresql.conf specifies a different pgdata from $PGDATA.
> As you can see, it could get messy.

Uh...why are we having to mess with environment variables at all?
It's one thing for shell scripts to make use of them, but another
thing entirely for an executable like the postmaster to do the same.

Seems logical to me to eliminate the use of $PGDATA in the postmaster
entirely.  It usually gets started from a shell script, so let the
shell script pass the appropriate parameter telling the postmaster
where to find the data, or the config files, or whatever.

> And, if you specify pgdata in postgresql.conf, it prevents you from
> using that file by different postmasters.

Not at all.  Don't GUC variables that are specified on the command
line override the ones in the configuration file?

> My best guess would be to not specify pgdata in postgresql.conf, and
> have a new $PGCONFIG param to specify the configuration directory, but
> if we do that, $PGDATA/postgresql.conf becomes meaningless, which could
> also be confusing.  Maybe we don't allow those files to exist in $PGDATA
> if $PGCONFIG is used, _and_ $PGCONFIG is not the same as $PGDATA.  See,
> I am getting myself confused.  :-)

I think the solution is real simple:

1.  Eliminate the use of $PGDATA in the postmaster.  It causes far
    more headaches than it's worth.  Instead, require that -D be
    passed on the command line.  It's fine if the postmaster *sets*
    $PGDATA in order to minimize any changes that need to be made
    elsewhere, but the postmaster should not use it until it sets it.
    The postmaster right now reads all the config files (including
    postgresql.conf) from the directory specified by the -D option.
    Keep it that way.

2.  Add a GUC variable that specifies where the data is.  If this
    variable is not defined either on the command line or in the
    config file, then assume that the data is in the same place as the
    config file.  Obviously files like PG_VERSION are associated with
    the data and not with the config, so they get treated
    appropriately.

The above addresses *everyone's* concerns that I've seen thus far, I
think.  Thoughts?


-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T02:13:29Z

Lamar Owen wrote:
> > First, a few conclusions:
> 
> > 	We can't use /var/run because we need the postmaster to create
> > 	those, and it isn't root.
> 
> It isn't without precedent to have a directory under /var/run.  Maybe 
> /var/run/postgresql.  Under this one could have a uniquely named pid file.  I 
> say uniquely named so that multiple postmasters could run.  Naming those 
> files could be fun. /var/run/postgresql would be owned by the postmaster run 
> user.  This of course requires root to install -- but would be completely 
> optional.

But how do you handle the default then, where you have postmaster.pid in
/data?  Do we rename it to postmaster.pid.5432 so it can sit in
/var/run/postgresql alone with other backends?

Another issue is that pg_ctl looks at that file, so moving it around is
going to be tricky.  Also, this brings up a new issue that pg_ctl all of
a sudden can't just look at $PGDATA but must instead grope through
postgresql.conf to find the data directory location.  That could be
interesting.  Of course, it can still supply the /data path on the
command line, but if we use only $PGCONFIG, we would need to have it
find /data automatically from postgresql.conf.

> > OK, first, we keep postmaster.pid and postmaster.opts in /data.  We
> > can't put them in /var/run, and /data seems like the best spot for them.
> 
> Can we make that configurable?  The default in pgdata is fine; just having the 
> option is good.

Basically, I am saying that unless someone wants to use this
configurability, it is going to cause code confusion so it is best
avoided.
`
> Yes.  I'm thinking along the lines of this sort of structure:
> /etc
> |---postgresql
>                 |----- name of postmaster one (unique ID of some kind)
>                 |----- name of postmaster two
>                     .
>                     .
> 
> Not difficult.

Yes, that would work easily.

> > We can also firm up stuff in 7.5 by removing PGDATA and -D, and perhaps
> > removing the other duplicate postmaster flags that have postgresql.conf
> > entries.
> 
> Now I really _like_ this idea.  By removing it to 7.5, and therefore 
> deprecating it in 7.4, this brings best practice into effect.
> 
> However, at the same time, I wouldn't be opposed to leaving them in place, 
> either, for backwards compatibility.

The problem is that we would be having too many ways to specify the
/data directory.

I am now wondering if we even want pg_hba_dir and pg_ident_dir.  Seems
we can assume they are in the same directory as postgresql.conf.  That
leaves only data_dir as new for postgresql.conf.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T02:20:05Z

Vince Vielhaber wrote:
> On Thu, 13 Feb 2003, Lamar Owen wrote:
> 
> > On Thursday 13 February 2003 18:07, Vince Vielhaber wrote:
> > > Actually FHS says the opposite.  If the distribution installs PostgreSQL
> > > then the config files belong in /etc/postgresql.  If the admin does then
> > > they belong in /usr/local/etc/postgresql.  FHS is out of their tree.  If
> > > PostgreSQL or any other package is not critical to the basic operation of
> > > the operating system, it's config files shouldn't be polluting /etc.
> >
> > PostgreSQL is as critical as PHP, Apache, or whatever other package is being
> > backended by PostgreSQL.  If the package is provided by the distributor,
> > consider it part of the OS.  If it isn't, well, it isn't.
> 
> You completely miss my point, but lately you've been real good at that.
> 
> Can the system boot without PHP, Apache, PostgreSQL, Mysql and/or
> Pine?

Yep.

> Can the root user log in without PHP, Apache, PostgreSQL, Mysql
> and/or Pine?

Hopefully.

> Can any user log in without PHP, Apache, PostgreSQL, Mysql and/or
> Pine?

That depends, doesn't it?  There exist PAM modules that allow
authentication against a database, for instance.  If you're using them
and the database doesn't come up, the users can't log in.  So suddenly
the database config files belong in /etc?

The mission of the box is what counts.  If the mission of the box is
to be a web server then I'm probably not going to care whether
non-root users can log into it: that simply doesn't factor into the
mission profile.  The web server process is going to be as critical to
the mission of the box as almost anything else on it, as will anything
the web server process depends on -- which may well include a
database.

> Note, I'm not even including an MTA here.  I said BASIC OPERATION.

So by your reasoning sendmail.cf doesn't belong in /etc??  I dare say
that's news to most of us.  Where, then, *does* it belong?

> If a package is not critical as I just outlined, it shouldn't matter
> who installed it.

Oh, it matters a great deal, because people upgrade their OS installs
from time to time.  Many OS distributions come with a lot of packages
that aren't "critical" as you define them but which nevertheless will
cause much pain and suffering for the sysadmin if they install
themselves over what the sysadmin has previously built by hand.

The purpose for differentiating between a package that was compiled
and installed from the source by the sysadmin and a prebuilt package
that was provided to the sysadmin by the vendor is to keep them from
stepping on each other -- if the sysadmin went to the trouble of
compiling and installing a package from the source instead of using a
prebuilt version from the vendor, then he probably did so for a very
good reason, and is going to be *really* annoyed if an OS upgrade
blows away his work.

There are some good reasons for putting all the config files in /etc,
one of them being that it gives you *one* directory full of config
files to worry about backing up instead of many.  If you've got other
ideas I'm certainly interested in hearing the reasoning behind them.
But from the point of view of maintaining a widely deployed package
like PostgreSQL, the conventions the distributions and sysadmins use
matter a great deal, whether or not you happen to agree with those
conventions.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Your Name <cbbrowne@cbbrowne.com> — 2003-02-14T02:45:47Z

> On 13 Feb 2003, Oliver Elphick wrote:
> 
> > What your comments strongly suggest to me is that projects like
> > PostgreSQL and pine, along with everything else, should comply with FHS;
> > then there will be no confusion because everyone will be following the
> > smae standards.  Messes arise when people ignore standards; we have all
> > seen the dreadful examples of MySQL and the Beast, haven't we?
> 
> Actually FHS says the opposite.  If the distribution installs PostgreSQL
> then the config files belong in /etc/postgresql.  If the admin does then
> they belong in /usr/local/etc/postgresql.  FHS is out of their tree.  If
> PostgreSQL or any other package is not critical to the basic operation of
> the operating system, it's config files shouldn't be polluting /etc.

I suspect you may be conflating BSD usage with Linux usage here...

The point isn't of being "critical to basic operation of the operating 
system;" it is of whether or not the software is being "package-managed" or 
not.

One of the operating principles in FHS is that "/usr/local" is an area that 
the distribution should never "pollute."  And so, a "package-managed" 
PostgreSQL installation should never touch that area.

Looking at FHS, for a moment: http://www.pathname.com/fhs/2.2/

3.7.1 Purpose
/etc contains configuration files and directories that are specific to the 
current system.

3.7.4  Indicates that 

"Host-specific configuration files for add-on application software packages 
must be installed within the directory /etc/opt/<package>, where <package> is 
the name of the subtree in /opt where the static data from that package is 
stored."

3.12 indicates: /opt is reserved for the installation of add-on application 
software packages.

A package to be installed in /opt must locate its static files in a separate 
/opt/<package> directory tree, where <package> is a name that describes the 
software package.

Then comes 5.1, on /var

/var contains variable data files. This includes spool directories and files, 
administrative and logging data, and transient and temporary files.

It would make most sense, based on FHS, for PostgreSQL information to 
assortedly reside in:

- /etc/opt/postgresql or /etc/postgresql, for static config information;
- Binaries could assortedly live in /usr/bin or /opt/postgresql;
- Logs should live in /var/log or /var/log/postgresql;
- Data could assortedly live in /var/lib/postgresql, /var/opt/postgresql;
- PIDs should live in /var/lock or /var/lock/postgresql.

None of these choices should come as any spectacular shock to anyone; there 
are an assortment of sets of bigotry out there surrounding the Proper Purposes 
of /opt and /usr/local, and there's probably enough wriggle room there to 
avoid overly enraging anyone that (for instance) felt calling a directory 
"/opt" would make someone deserving of carpet bombing by B-52s.

Interestingly, the Debian install of PostgreSQL somewhat resembles this, with, 
assortedly:

/etc/postgresql
/etc/postgresql/postgresql.conf
/etc/postgresql/postmaster.conf
/etc/postgresql/pg_hba.conf
/etc/postgresql/pg_ident.conf
/etc/init.d/postgresql
/usr/share/doc/postgresql
/usr/share/man/man1/pg_ctl.1.gz
/usr/lib/postgresql
/usr/lib/postgresql/bin/postgres
/usr/lib/postgresql/bin/enable_lang
/usr/lib/postgresql/bin/initdb
/usr/lib/postgresql/bin/initlocation
/usr/lib/postgresql/bin/ipcclean
/usr/lib/postgresql/bin/pg_ctl
/usr/lib/postgresql/bin/pg_dumpall
/var/run/postgresql

(This is obviously incomplete; this just gives the flavor that there are files 
in a reasonably rational but diverse assortment of places.)

Note that the server software hides in /usr/lib/postgresql/bin; it's not stuff 
you should normally run from the command line, so, quel surprise, it is 
stashed somewhere that's unlikely to be in your $PATH.

Stashing _everything_ in /var/lib/postgres would seem a tad surprising.

Mind you, if I need to manage 4 instances on one box, I might very well 
install several instances some place custom, say /opt/postgres, or similar, 
and in that case, it's probably preferable for everything to reside clearly 
underneath that, and for my custom backup scripts to work in that area.

But if I'm managing 4 instances on one box, it should be quite evident that 
I'm going well beyond what any packaging system is likely to be prepared to 
handle.  Again, quel surprise.
--
(reverse (concatenate 'string "gro.gultn@" "enworbbc"))
http://www3.sympatico.ca/cbbrowne/linuxxian.html
"Of  _course_ it's the murder weapon.   Who would frame someone with a
fake?"

Re: location of the configuration files

Curt Sampson <cjs@cynic.net> — 2003-02-14T02:46:39Z

On Thu, 13 Feb 2003, scott.marlowe wrote:

> If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which
> is not the standard way you're listing.

I'm going to stay out of this argument from now on, but this struck a sore
point.

/usr is designed to be a filesystem that can be shared. Is the stuff in
/usr/local/apache/conf really supposed to be shared amongst all machines
of that architecture on your site that run apache?

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-14T02:47:19Z

On Thursday 13 February 2003 21:13, Bruce Momjian wrote:
> Lamar Owen wrote:
> > It isn't without precedent to have a directory under /var/run.  Maybe
> > /var/run/postgresql.  Under this one could have a uniquely named pid
> > file.

> But how do you handle the default then, where you have postmaster.pid in
> /data?  Do we rename it to postmaster.pid.5432 so it can sit in
> /var/run/postgresql alone with other backends?

Well, you can have the default as 'postmaster.pid' if it wasn't named.  But 
more thought is needed. I'll have to admit; the wisdom of AOLserver having a 
full-fledged tcl config script is beginning to look better and better.

> Another issue is that pg_ctl looks at that file, so moving it around is
> going to be tricky.

pg_ctl could be interesting.

> I am now wondering if we even want pg_hba_dir and pg_ident_dir.  Seems
> we can assume they are in the same directory as postgresql.conf.  That
> leaves only data_dir as new for postgresql.conf.

Ok, if we're going this far already, tell me exactly why we have three config 
files.  Why not really Unify things and fulfil the full promise of Grand 
Unified Configuration by rolling hba and ident into postgresql.conf.  Is 
there a compelling reason not to do so?  The structure of that configuration 
data would have to change, for sure.  Although I seem to remember this being 
suggested once before, but my mind draws a blank trying to recall it.  Just a 
suggestion; maybe not even a good one, but something that crossed my mind.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T02:49:16Z

Oliver Elphick <olly@lfix.co.uk> writes:
> I'm not entirely sure why SE Linux has a problem, seeing that postgres
> needs read-write access to all the files in $PGDATA, but assuming the
> need is verified, I could do this by moving the pid file from
> $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for
> other ports.  This would also have the benefit of being more FHS
> compliant  What do people think about that?

No chance at all.  Breaking the connection between the data directory
and the postmaster.pid file means we don't have an interlock against
starting two postmasters in the same data directory.

I do not see the argument for moving the pid file anyway.  Surely no
one's going to tell us that the postmaster shouldn't have write access
to the data directory?

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-14T02:59:25Z

On Thursday 13 February 2003 21:49, Tom Lane wrote:
> Oliver Elphick <olly@lfix.co.uk> writes:
> > need is verified, I could do this by moving the pid file from
> > $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for
> > other ports.  This would also have the benefit of being more FHS
> > compliant  What do people think about that?

> No chance at all.  Breaking the connection between the data directory
> and the postmaster.pid file means we don't have an interlock against
> starting two postmasters in the same data directory.

It's not a pid file in the /var/run sense, really.  It's an interlock for 
PGDATA.  So it might be argued that postmaster.pid is misnamed.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T03:11:05Z

Lamar Owen wrote:
> > I am now wondering if we even want pg_hba_dir and pg_ident_dir.  Seems
> > we can assume they are in the same directory as postgresql.conf.  That
> > leaves only data_dir as new for postgresql.conf.
> 
> Ok, if we're going this far already, tell me exactly why we have three config 
> files.  Why not really Unify things and fulfil the full promise of Grand 
> Unified Configuration by rolling hba and ident into postgresql.conf.  Is 
> there a compelling reason not to do so?  The structure of that configuration 
> data would have to change, for sure.  Although I seem to remember this being 
> suggested once before, but my mind draws a blank trying to recall it.  Just a 
> suggestion; maybe not even a good one, but something that crossed my mind.

postgresql.conf is var=val, while the others are column-based.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T04:00:02Z

Lamar Owen <lamar.owen@wgcr.org> writes:
> Ok, if we're going this far already, tell me exactly why we have three config
> files.  Why not really Unify things and fulfil the full promise of Grand 
> Unified Configuration by rolling hba and ident into postgresql.conf.  Is 
> there a compelling reason not to do so?

Lack of backwards compatibility; unnecessary complexity.  Unifying those
files would create a big headache in terms of having to unify their
syntax.  And there are some basic semantic differences too.  For
instance, order matters in pg_hba.conf, but not in postgresql.conf.

Another reason not to do it is that there are differences in the
security requirements of these files.  postgresql.conf probably doesn't
contain anything that needs to be hidden from prying eyes, but I'd be
inclined to want to keep the other two mode 600.

---

Okay, I've been laying low all day, but here are my thoughts on the
discussion:

I do see the value in being able to (as opposed to being forced to,
please) keep hand-edited config files in a separate location from
the machine-processed data files.  We have already gone some distance
in that direction over the past few releases --- there's much less in
the top $PGDATA directory than there once was.  It makes sense to let
people keep hand-edited files away from what initdb will overwrite.

I would favor a setup that allows a -C *directory* (not file) to be
specified as a postmaster parameter separately from the -D directory;
then the hand-editable config files would be sought in -C not -D.  In
the absence of -C the config files should be sought in -D, same as they
ever were (thus simplifying life for people like me who run many
postmasters and don't give a darn about FHS ;-)).

I don't see any great value in a separate postgresql.conf parameter for
each secondary config file; that just means clutter to me, especially
if we add more such files in future.  I am also distinctly not in favor
of eliminating the PGDATA environment variable; that reads to me as
"we are going to force you to do it our way rather than the way you've
always done it, even if you like the old way".

To make the RPM packagers happy, I guess that the default -C directory
has to be settable via configure.  We do not currently have a default
-D directory, and I didn't hear anyone arguing in favor of adding one.
So that leaves the following possible combinations that the postmaster
might see at startup, for which I propose the following behaviors:

1. No -C switch, no -D switch, no PGDATA found in environment: seek
postgresql.conf in the default -C directory established at configure
time.  Use the 'datadir' specified therein as -D.  Fail if postgresql.conf
doesn't define a datadir value.

2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
as both -C and -D.  (Minor detail: if the postgresql.conf in the $PGDATA
directory specifies a different directory as datadir, do we follow that
or raise an error?  I'd be inclined to say "follow it" but maybe there
is an argument for erroring out.)

(In all the following cases, any environment PGDATA value is ignored.)

3. No -C switch, -D switch on command line: use -D value as both -C and -D,
proceed as in case 2.

4. -C switch, no -D switch on command line: seek postgresql.conf in
-C directory, use the datadir it specifies.

5. -C and -D on command line: seek postgresql.conf in -C directory,
use -D as datadir overriding what is in postgresql.conf (this is just
the usual rule that command line switches override postgresql.conf).

Cases 2 and 3 are backwards-compatible with our historical behavior,
so that anyone who likes the historical behavior will not be unhappy.
Cases 1 and 4 I think will make mlw and our packagers happy.  Case 5
is just the logical conclusion for that combination.

In all cases, pg_hba.conf and pg_ident.conf would be sought in the
same directory as postgresql.conf.  The other stuff in the toplevel
$PGDATA directory should stay where it is, IMHO.

I would venture that the configure-time-default for -C should be
${prefixdir}/etc if configure is not told differently, while the
packagers would probably set it to /etc/postgresql/ (ie, the
config files should live in a subdirectory that can be owned by
postgres user).  I'm not wedded to that though.

Another interesting question is whether the installed-by-default
postgresql.conf should specify a datadir value, and if so what.
If initdb installs it, it can and probably should insert the actual
datadir location the user gave to initdb into the file.  But should
initdb install any config files at all anymore?  I'm leaning to the
thought that initdb should store default config files into $PGDATA
same as it ever did, and then it's up to the user (or package install
scripts) to move them to the desired -C directory if appropriate.
Or I suppose we could add a -C parameter to initdb to tell it where to
put 'em.

Comments?

			regards, tom lane

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T05:59:42Z

Tom Lane wrote:
> I don't see any great value in a separate postgresql.conf parameter for
> each secondary config file; that just means clutter to me, especially
> if we add more such files in future.  I am also distinctly not in favor
> of eliminating the PGDATA environment variable; that reads to me as
> "we are going to force you to do it our way rather than the way you've
> always done it, even if you like the old way".

The scripts needn't ignore PGDATA at all.  Only postmaster.  Since the
vast majority of people start the postmaster from a script, this winds
up being a minor issue, except for the fact that without PGDATA
administrators will be able to count on looking at the output of 'ps'
to determine where the postmaster is looking for either the config
file or the data directory.  In other words, they'll have somewhere to
start from without having to poke through scripts that might not even
have been used (what happens when a user defines PGDATA and starts a
postmaster?  The administrator will have to go to more extreme
lengths, like using lsof, to figure out where the data directory is.
Not all systems have such tools).

> Comments?

I agree with your assessment for the most part, except for PGDATA.
There's no good reason I can think of for the postmaster to look at
it.  It's fine if it sets it for processes it forks to inherit, but it
shouldn't pay attention to it on startup.  Some people might complain,
but there's little difference in doing a "postmaster -D $PGDATA" and
just "postmaster", and people who are starting things by hand
hopefully aren't so inflexible as to demand that PGDATA remain treated
as-is.  People who really care can create a simple little 'pm.sh'
script that simply does a "postmaster -D $PGDATA", which will save
them typing even over just doing a "postmaster" from the command line.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T06:17:43Z

On Thu, 2003-02-13 at 23:06, mlw wrote:
> 
> Bruce Momjian wrote:

> > Can non-root write to /var/run?
> > 
> >   
> Shouldn't be able too

But it should be able to write under /var/run/postgresql, which the
distribution will set up with the correct permissions.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T06:18:22Z

On Thu, 2003-02-13 at 22:53, Bruce Momjian wrote:
> Oliver Elphick wrote:
> > What your comments strongly suggest to me is that projects like
> > PostgreSQL and pine, along with everything else, should comply with FHS;
> > then there will be no confusion because everyone will be following the
> > smae standards.  Messes arise when people ignore standards; we have all
> > seen the dreadful examples of MySQL and the Beast, haven't we?
> 
> Can the FHS handle installing PostgreSQL as non-root?

Certainly.  It is only necessary to set permissions correctly in
/etc/postgresql and /var/run/postgresql.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T06:26:12Z

On Fri, 2003-02-14 at 02:49, Tom Lane wrote:
> Oliver Elphick <olly@lfix.co.uk> writes:
> > I'm not entirely sure why SE Linux has a problem, seeing that postgres
> > needs read-write access to all the files in $PGDATA, but assuming the
> > need is verified, I could do this by moving the pid file from
> > $PGDATA/postmaster.pid to /var/run/postgresql/5432.pid and similarly for
> > other ports.  This would also have the benefit of being more FHS
> > compliant  What do people think about that?
> 
> No chance at all.  Breaking the connection between the data directory
> and the postmaster.pid file means we don't have an interlock against
> starting two postmasters in the same data directory.

Yes; that would take a lot of effort to get round. Not worth it, I
think.

> I do not see the argument for moving the pid file anyway.  Surely no
> one's going to tell us that the postmaster shouldn't have write access
> to the data directory?

I'm waiting for a response on that one; I don't understand it either.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T06:30:21Z

Kevin Brown <kevin@sysexperts.com> writes:
> I agree with your assessment for the most part, except for PGDATA.
> There's no good reason I can think of for the postmaster to look at
> it.

The other side of that coin is, what's the good reason to remove it?
There's a long way between "I don't want my setup to depend on PGDATA"
and "I don't think your setup should be allowed to depend on PGDATA".
If you don't want to use it, then don't use it.  Why do you need to
tell me how I'm allowed to run my installation?

> ... people who are starting things by hand hopefully aren't so
> inflexible as to demand that PGDATA remain treated as-is.

Yes, I could reconfigure my scripts to not depend on this.  You have
not given me an adequate argument why I should have to.

			regards, tom lane

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T06:37:30Z

On Fri, 2003-02-14 at 02:45, cbbrowne@cbbrowne.com wrote:
> 3.7.1 Purpose
> /etc contains configuration files and directories that are specific to the 
> current system.
> 
> 3.7.4  Indicates that 
> 
> "Host-specific configuration files for add-on application software packages 
> must be installed within the directory /etc/opt/<package>, where <package> is 
> the name of the subtree in /opt where the static data from that package is 
> stored."
> 
> 3.12 indicates: /opt is reserved for the installation of add-on application 
> software packages.
> 
> A package to be installed in /opt must locate its static files in a separate 
> /opt/<package> directory tree, where <package> is a name that describes the 
> software package.
...
> It would make most sense, based on FHS, for PostgreSQL information to 
> assortedly reside in:
> 
> - /etc/opt/postgresql or /etc/postgresql, for static config information;

I feel that /opt (and therefore /etc/opt) are intended for the use of
vendors; so commercial packages designed to fit in with FHS should use
those.  I don't think they are for locally built stuff.

No matter; it illustrates the main point, which is that these things
should be easily configurable.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

Andrew Dunstan <andrew@dunslane.net> — 2003-02-14T07:35:03Z

Apache explicitly supports a number of different layouts for files out of
the box (and provides support for you to roll your own very easily). From
the manual:

The second, and more flexible way to configure the install path locations
for Apache is using the config.layout file. Using this method, it is
possible to separately specify the location for each type of file within the
Apache installation. The config.layout file contains several example
configurations, and you can also create your own custom configuration
following the examples. The different layouts in this file are grouped into
<Layout FOO>...</Layout> sections and referred to by name as in FOO.
  --enable-layout=LAYOUT
  Use the named layout in the config.layout file to specify the installation
paths.
Maybe pg could benefit from something similar?

cheers

andrew

----- Original Message -----
From: "scott.marlowe" <scott.marlowe@ihs.com>
Sent: Thursday, February 13, 2003 4:07 PM
[snip]
> If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which
> is not the standard way you're listing.  If I install openldap from
> .tar.gz, I get a /usr/local/etc/openldap directory, close, but still not
> the same.  The fact is, it's the packagers that put things into /etc and
> whatnot, and I can see the postgresql RPMs or debs or whatever having that
> as the default, but for custom built software, NOTHING that I know of
> builds from source and uses /etc without a switch to tell it to, just like
> postgresql can do now.

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T10:58:49Z

Tom Lane wrote:
> Kevin Brown <kevin@sysexperts.com> writes:
> > I agree with your assessment for the most part, except for PGDATA.
> > There's no good reason I can think of for the postmaster to look at
> > it.
> 
> The other side of that coin is, what's the good reason to remove it?
> There's a long way between "I don't want my setup to depend on PGDATA"
> and "I don't think your setup should be allowed to depend on PGDATA".
> If you don't want to use it, then don't use it.  Why do you need to
> tell me how I'm allowed to run my installation?

I'm not talking about getting rid of ALL dependency on PGDATA in our
entire distribution, only postmaster's.

Recall that the main purpose of making any of these changes at all is
to make life easier for the guys who have to manage the systems that
will be running PostgreSQL.  Agreed?

So: imagine you're the newly-hired DBA and your boss points you to the
system and says "administrate the database on that".  You go over to
the computer and start looking around.

You do a "ps" and see a postmaster process running.  You know that
it's the process that is listening for connections.  The "ps" listing
only says "/usr/bin/postmaster".  No arguments to clue you in,
nothing.  Where do you look to figure out where the data is?  How do
you figure out what port it's listening on?

Well, we're already agreed on how to deal with that question: you look
in /etc/postgresql, and because this is a relatively new install (and
the PostgreSQL maintainers, who are very wise and benevolent, made
that the default location for configs :-), it has a postgresql.conf
file with a line that says "data_directory = /var/lib/pgsql".  It
doesn't mention a port to listen to so you know that it's listening on
port 5432.  As a DBA, you're all set.

Now let's repeat that scenario, except that instead of seeing one
postmaster process, you see five.  And they all say
"/usr/bin/postmaster" in the "ps" listing.  No arguments to clue you
in or anything, as before.  You might be able to figure out where one
of them is going by looking at /etc/postgresql, but what about the
rest?  Now you're stuck unless you want to do a "find" (time consuming
and I/O intensive -- a good way to slow the production database down a
bit), or you're knowledgeable enough to use 'lsof' or black magic like
digging into kernel memory to figure out where the config files and
data directories are, or you have enough knowledge to pore through the
startup scripts and understand what they're doing.

Lest you think that this is an unlikely scenario, keep in mind that
most startup scripts, including pg_ctl, currently start the postmaster
without arguments and rely on PGDATA, so a shop that hasn't already
been bitten by this *will* be.  Right now shops that wish to avoid the
trap I described have to go to *extra* lengths: they have to make
exactly the same kinds of changes to the scripts that I'm talking
about us making (putting an explicit '-D "$PGDATA"' where none now
exists) or they have to resort to tricks like renaming the postmaster
executable and creating a shell script in its place that will invoke
the (renamed) postmaster with '-D "$PGDATA"'.

It's not so bad if only a few shops have to make those changes.  But
what if it's thousands?  Yeah, the distribution guys can patch the
scripts to do this, but why should they have to?  They, and the shops
that run PostgreSQL, are our customers.

All of that is made possible because the postmaster can use an
inherited PGDATA for the location of the config files and (if the
config files don't say differently in our new scheme) the data
directory, and pg_ctl takes advantage of that fact (as do most startup
scripts that I've seen, that don't just invoke pg_ctl).

I'm not arguing that we should remove the use of PGDATA *everywhere*,
only in postmaster (and then, only postmaster's use of an *inherited*
PGDATA.  It should still set PGDATA so its children can use it).  It
means changing pg_ctl and the startup scripts we ship.  The earlier we
make these changes, the less overall pain there will be in the long
run.

> > ... people who are starting things by hand hopefully aren't so
> > inflexible as to demand that PGDATA remain treated as-is.
> 
> Yes, I could reconfigure my scripts to not depend on this.  You have
> not given me an adequate argument why I should have to.

[By this I'm assuming you're referring to the scripts you use for
testing, and not the ones that ship with the distribution]

I'm not arguing that you should get rid of all the references to
PGDATA in your scripts or anything crazy like that.  The changes I'm
talking about are minor: where you see "postmaster" without any "-D"
arguments, you simply add '-D "$PGDATA"' to it, before any other
arguments that you might also be passing.  That's it.  Nothing else
should be needed.

The reason for removing postmaster's use of an inherited PGDATA is the
same as the reason for making the other changes we already agree
should be made: to make things easier for the guys in the field who
have to manage production systems.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T12:17:17Z

Kevin Brown wrote:
> Tom Lane wrote:
> > Kevin Brown <kevin@sysexperts.com> writes:
> > > I agree with your assessment for the most part, except for PGDATA.
> > > There's no good reason I can think of for the postmaster to look at
> > > it.
> > 
> > The other side of that coin is, what's the good reason to remove it?
> > There's a long way between "I don't want my setup to depend on PGDATA"
> > and "I don't think your setup should be allowed to depend on PGDATA".
> > If you don't want to use it, then don't use it.  Why do you need to
> > tell me how I'm allowed to run my installation?
> 
> I'm not talking about getting rid of ALL dependency on PGDATA in our
> entire distribution, only postmaster's.
> 
> Recall that the main purpose of making any of these changes at all is
> to make life easier for the guys who have to manage the systems that
> will be running PostgreSQL.  Agreed?
> 
> So: imagine you're the newly-hired DBA and your boss points you to the
> system and says "administrate the database on that".  You go over to
> the computer and start looking around.
> 
> You do a "ps" and see a postmaster process running.  You know that
> it's the process that is listening for connections.  The "ps" listing
> only says "/usr/bin/postmaster".  No arguments to clue you in,
> nothing.  Where do you look to figure out where the data is?  How do
> you figure out what port it's listening on?

If you want ps to display the data dir, you should use -D.  Remember, it
is mostly important for multiple postmaster, so if you are doing that,
just use -D, but don't prevent single-postmaster folks from using
PGDATA.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T12:32:31Z

OK, here is an updated proposal.  I think we have decided:

	Moving postmaster.pid and postmaster.opts isn't worth it.

	We don't want per-file GUC variables, but assume it is in
	the same config directory as postgresql.conf.  I don't
	see any valid reason they would want to put them somewhere
	different than postgresql.conf.

So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
postmaster.

Regarding Tom's idea of replacing data_dir with a full path during
initdb, I think we are better having it be relative to the config
directory, that way if they move pgsql/, the system still works.
However, if the config directory is in a different lead directory path,
we should replace it with the full path, e.g. /usr/local/pgsql/data and
/usr/local/pgsql/etc use relative paths, ../data, while /etc/postgresql
and /usr/local/pgsql/data get an absolute path.

My idea is to introduce the above capabilities in 7.4, and keep the
config files in /data.  This will allow package folks to move the config
files in 7.4.

I also think we should start telling people to use PGCONFIG rather than
PGDATA.  Then, in 7.5, we move the default config file location to
pgsql/etc, and tell folks to point there rather than /data.  I think
there is major value to getting those config files out of the initdb
creation area and for backups.

I am now wondering if we should add PGCONFIG and move them out of data
all in the same release.  Not sure if delaying the split is valuable.

---------------------------------------------------------------------------

Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > Ok, if we're going this far already, tell me exactly why we have three config
> > files.  Why not really Unify things and fulfil the full promise of Grand 
> > Unified Configuration by rolling hba and ident into postgresql.conf.  Is 
> > there a compelling reason not to do so?
> 
> Lack of backwards compatibility; unnecessary complexity.  Unifying those
> files would create a big headache in terms of having to unify their
> syntax.  And there are some basic semantic differences too.  For
> instance, order matters in pg_hba.conf, but not in postgresql.conf.
> 
> Another reason not to do it is that there are differences in the
> security requirements of these files.  postgresql.conf probably doesn't
> contain anything that needs to be hidden from prying eyes, but I'd be
> inclined to want to keep the other two mode 600.
> 
> ---
> 
> Okay, I've been laying low all day, but here are my thoughts on the
> discussion:
> 
> I do see the value in being able to (as opposed to being forced to,
> please) keep hand-edited config files in a separate location from
> the machine-processed data files.  We have already gone some distance
> in that direction over the past few releases --- there's much less in
> the top $PGDATA directory than there once was.  It makes sense to let
> people keep hand-edited files away from what initdb will overwrite.
> 
> I would favor a setup that allows a -C *directory* (not file) to be
> specified as a postmaster parameter separately from the -D directory;
> then the hand-editable config files would be sought in -C not -D.  In
> the absence of -C the config files should be sought in -D, same as they
> ever were (thus simplifying life for people like me who run many
> postmasters and don't give a darn about FHS ;-)).
> 
> I don't see any great value in a separate postgresql.conf parameter for
> each secondary config file; that just means clutter to me, especially
> if we add more such files in future.  I am also distinctly not in favor
> of eliminating the PGDATA environment variable; that reads to me as
> "we are going to force you to do it our way rather than the way you've
> always done it, even if you like the old way".
> 
> To make the RPM packagers happy, I guess that the default -C directory
> has to be settable via configure.  We do not currently have a default
> -D directory, and I didn't hear anyone arguing in favor of adding one.
> So that leaves the following possible combinations that the postmaster
> might see at startup, for which I propose the following behaviors:
> 
> 1. No -C switch, no -D switch, no PGDATA found in environment: seek
> postgresql.conf in the default -C directory established at configure
> time.  Use the 'datadir' specified therein as -D.  Fail if postgresql.conf
> doesn't define a datadir value.
> 
> 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
> as both -C and -D.  (Minor detail: if the postgresql.conf in the $PGDATA
> directory specifies a different directory as datadir, do we follow that
> or raise an error?  I'd be inclined to say "follow it" but maybe there
> is an argument for erroring out.)
> 
> (In all the following cases, any environment PGDATA value is ignored.)
> 
> 3. No -C switch, -D switch on command line: use -D value as both -C and -D,
> proceed as in case 2.
> 
> 4. -C switch, no -D switch on command line: seek postgresql.conf in
> -C directory, use the datadir it specifies.
> 
> 5. -C and -D on command line: seek postgresql.conf in -C directory,
> use -D as datadir overriding what is in postgresql.conf (this is just
> the usual rule that command line switches override postgresql.conf).
> 
> Cases 2 and 3 are backwards-compatible with our historical behavior,
> so that anyone who likes the historical behavior will not be unhappy.
> Cases 1 and 4 I think will make mlw and our packagers happy.  Case 5
> is just the logical conclusion for that combination.
> 
> In all cases, pg_hba.conf and pg_ident.conf would be sought in the
> same directory as postgresql.conf.  The other stuff in the toplevel
> $PGDATA directory should stay where it is, IMHO.
> 
> I would venture that the configure-time-default for -C should be
> ${prefixdir}/etc if configure is not told differently, while the
> packagers would probably set it to /etc/postgresql/ (ie, the
> config files should live in a subdirectory that can be owned by
> postgres user).  I'm not wedded to that though.
> 
> Another interesting question is whether the installed-by-default
> postgresql.conf should specify a datadir value, and if so what.
> If initdb installs it, it can and probably should insert the actual
> datadir location the user gave to initdb into the file.  But should
> initdb install any config files at all anymore?  I'm leaning to the
> thought that initdb should store default config files into $PGDATA
> same as it ever did, and then it's up to the user (or package install
> scripts) to move them to the desired -C directory if appropriate.
> Or I suppose we could add a -C parameter to initdb to tell it where to
> put 'em.
> 
> Comments?
> 
> 			regards, tom lane
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: [MLIST] Re: location of the configuration files

David Walker <pgsql@grax.com> — 2003-02-14T12:32:37Z

In reference to determining what port postgres or any program is listening on
On my Redhat Linux machines
netstat --inet -nlp
when run as root
produces a nice list of all programs listening on the network with IP and port 
number the process is listening on, the name of the process and the pid.

The environment used to start each of these postmasters can be found at
cat /proc/${POSTMASTER-PID}/environ | tr "\000" "\n"

I'm not arguing one way or the other on your issue, just hope these tips make 
the "black magic" a little easier to use.

On Friday 14 February 2003 04:58 am, Kevin Brown wrote:
> Now let's repeat that scenario, except that instead of seeing one
> postmaster process, you see five.  And they all say
> "/usr/bin/postmaster" in the "ps" listing.  No arguments to clue you
> in or anything, as before.  You might be able to figure out where one
> of them is going by looking at /etc/postgresql, but what about the
> rest?  Now you're stuck unless you want to do a "find" (time consuming
> and I/O intensive -- a good way to slow the production database down a
> bit), or you're knowledgeable enough to use 'lsof' or black magic like
> digging into kernel memory to figure out where the config files and
> data directories are, or you have enough knowledge to pore through the
> startup scripts and understand what they're doing.

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T13:11:58Z

On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote:
> If you want ps to display the data dir, you should use -D.  Remember, it
> is mostly important for multiple postmaster, so if you are doing that,
> just use -D, but don't prevent single-postmaster folks from using
> PGDATA.

Could not the ps line be rewritten to show this, as the backend's ps
lines are rewritten?

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T13:29:29Z

I am not sure if it is a good idea to be mucking with it.  For backend,
we do the entire thing, so it is clear we modified something.

---------------------------------------------------------------------------

Oliver Elphick wrote:
> On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote:
> > If you want ps to display the data dir, you should use -D.  Remember, it
> > is mostly important for multiple postmaster, so if you are doing that,
> > just use -D, but don't prevent single-postmaster folks from using
> > PGDATA.
> 
> Could not the ps line be rewritten to show this, as the backend's ps
> lines are rewritten?
> 
> -- 
> Oliver Elphick                                Oliver.Elphick@lfix.co.uk
> Isle of Wight, UK                             http://www.lfix.co.uk/oliver
> GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
>                  ========================================
>      "God be merciful unto us, and bless us; and cause his 
>       face to shine upon us."          Psalms 67:1 
> 
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T13:41:48Z

Bruce Momjian wrote:
> If you want ps to display the data dir, you should use -D.  Remember, it
> is mostly important for multiple postmaster, so if you are doing that,
> just use -D, but don't prevent single-postmaster folks from using
> PGDATA.

Perhaps the best compromise would be to change pg_ctl so that it uses
-D explicitly when invoking postmaster.  That's an easy change.

Could you describe how you and other developers use PGDATA?  I'm quite
interested in knowing why there seems to be so much resistance to
removing the "potential_DataDir = getenv("PGDATA");" line from
postmaster.c.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T13:47:37Z

Bruce Momjian wrote:
> If you want ps to display the data dir, you should use -D.  Remember, it
> is mostly important for multiple postmaster, so if you are doing that,
> just use -D, but don't prevent single-postmaster folks from using
> PGDATA.

Perhaps another reasonable approach would be to put an #ifdef/#endif
around the "potential_DataDir = getenv("PGDATA");" line in postmater.c
and create a configure option to enable it.  That way you guys get the
behavior you want for testing but production builds could disable it
if that's viewed as desirable.  You'd want to make the error message
that's produced when no data directory is specified depend on the same
#ifdef variable, of course.

Then the group would get to fight it out over whether the configure
default should be "enable" or "disable".  :-)

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T14:00:44Z

Kevin Brown wrote:
> Bruce Momjian wrote:
> > If you want ps to display the data dir, you should use -D.  Remember, it
> > is mostly important for multiple postmaster, so if you are doing that,
> > just use -D, but don't prevent single-postmaster folks from using
> > PGDATA.
> 
> Perhaps the best compromise would be to change pg_ctl so that it uses
> -D explicitly when invoking postmaster.  That's an easy change.
> 
> Could you describe how you and other developers use PGDATA?  I'm quite
> interested in knowing why there seems to be so much resistance to
> removing the "potential_DataDir = getenv("PGDATA");" line from
> postmaster.c.

I just set PGDATA in my login and I don't have to deal with it again.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-14T14:07:09Z

> > If you are interested in reading a contrary position, you can read
> > Berstein's arguments for his recommended way to install services at:
> > http://cr.yp.to/unix.html

But since DJB is a class-A monomaniac, he may not be the best person to
listen to. /var/qmail/control for qmail configuration files? Yeah, good
one, DJB.

-- 
Martin Coxall <coxall@cream.org>

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-14T14:07:11Z

> Generally things that live in /etc are owned and operated by the OS.  
> Postgresql, by it's definition is a userspace program, not an OS owned 
> one.

Partially true. The FHS specifies that the /etc top layer is for system-own3d 
stuff, but the subdirectories off it are explicitly used for user space programs
and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3, /etc/tomcat4...)

Martin Coxall

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-14T14:07:17Z

On Thu, 2003-02-13 at 20:28, Steve Crawford wrote:
> I don't see why we can't keep everyone happy and let the users choose the 
> setup they want. To wit, make the following, probably simple, changes:
> 
> 1) Have postgresql default to using /etc/postgresql.conf

/etc/postgres/postgresql.conf, if we want to be proper FHS-bitches.

> 2) Add a setting in postgresql.conf specifying the data directory
> 3) Change the meaning of -D to mean "use this config file"
> 4) In the absence of a specified data directory in postgresql.conf, use the 
> location of the postgresql.conf file as the data directory

Shouldn't it in that case default to, say /var/lib/postgres?

-- 
Martin Coxall <coxall@cream.org>

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T14:12:38Z

Bruce Momjian wrote:
> I just set PGDATA in my login and I don't have to deal with it
> again.

Hmm...you don't use pg_ctl to start/stop/whatever the database?  You
invoke the postmaster directly (I can easily see that you would, just
asking if you do)?

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T14:15:58Z

Kevin Brown wrote:
> Bruce Momjian wrote:
> > I just set PGDATA in my login and I don't have to deal with it
> > again.
> 
> Hmm...you don't use pg_ctl to start/stop/whatever the database?  You
> invoke the postmaster directly (I can easily see that you would, just
> asking if you do)?

I can use either to start/stop it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T14:19:59Z

Bruce Momjian wrote:
> I just set PGDATA in my login and I don't have to deal with it
> again.

Duh....I just realized a reason you guys might care about this so
much.

It's because you want to be able to start the postmaster from within a
debugger (or profiler, or whatever), and you don't want to have to
mess with command line options from there, right?

Sounds like fixing pg_ctl to use -D explicitly when invoking the
postmaster is the right change to make here, since that's probably how
the majority of the production shops are going to be starting the
database anyway.  Takes care of the majority of the "visibility"
problem and leaves PGDATA intact.  Thoughts?

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-14T14:21:16Z

On 14 Feb 2003, Martin Coxall wrote:

>
> > > If you are interested in reading a contrary position, you can read
> > > Berstein's arguments for his recommended way to install services at:
> > > http://cr.yp.to/unix.html
>
> But since DJB is a class-A monomaniac, he may not be the best person to
> listen to. /var/qmail/control for qmail configuration files? Yeah, good
> one, DJB.

I'm guessing that rather than reading it the above mentioned link you
chose to waste our time with this instead.  Good one, MC.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T14:22:26Z

Give it up.  As long as we have -D, we will allow PGDATA. If you don't
want to use it, don't use it.

---------------------------------------------------------------------------

Kevin Brown wrote:
> Bruce Momjian wrote:
> > I just set PGDATA in my login and I don't have to deal with it
> > again.
> 
> Duh....I just realized a reason you guys might care about this so
> much.
> 
> It's because you want to be able to start the postmaster from within a
> debugger (or profiler, or whatever), and you don't want to have to
> mess with command line options from there, right?
> 
> 
> Sounds like fixing pg_ctl to use -D explicitly when invoking the
> postmaster is the right change to make here, since that's probably how
> the majority of the production shops are going to be starting the
> database anyway.  Takes care of the majority of the "visibility"
> problem and leaves PGDATA intact.  Thoughts?
> 
> 
> 
> -- 
> Kevin Brown					      kevin@sysexperts.com
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T14:29:34Z

My point is that folks with multiple postmasters may not want to use
PGDATA, but for folks who have single postmasters, it makes things
easier and less error-prone.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> 
> Give it up.  As long as we have -D, we will allow PGDATA. If you don't
> want to use it, don't use it.
> 
> ---------------------------------------------------------------------------
> 
> Kevin Brown wrote:
> > Bruce Momjian wrote:
> > > I just set PGDATA in my login and I don't have to deal with it
> > > again.
> > 
> > Duh....I just realized a reason you guys might care about this so
> > much.
> > 
> > It's because you want to be able to start the postmaster from within a
> > debugger (or profiler, or whatever), and you don't want to have to
> > mess with command line options from there, right?
> > 
> > 
> > Sounds like fixing pg_ctl to use -D explicitly when invoking the
> > postmaster is the right change to make here, since that's probably how
> > the majority of the production shops are going to be starting the
> > database anyway.  Takes care of the majority of the "visibility"
> > problem and leaves PGDATA intact.  Thoughts?
> > 
> > 
> > 
> > -- 
> > Kevin Brown					      kevin@sysexperts.com
> > 
> > ---------------------------(end of broadcast)---------------------------
> > TIP 2: you can get off all lists at once with the unregister command
> >     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> > 
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 5: Have you checked our extensive FAQ?
> 
> http://www.postgresql.org/users-lounge/docs/faq.html
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-14T14:33:53Z

On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote:
> On 14 Feb 2003, Martin Coxall wrote:
> 
> >
> > > > If you are interested in reading a contrary position, you can read
> > > > Berstein's arguments for his recommended way to install services at:
> > > > http://cr.yp.to/unix.html
> >
> > But since DJB is a class-A monomaniac, he may not be the best person to
> > listen to. /var/qmail/control for qmail configuration files? Yeah, good
> > one, DJB.
> 
> I'm guessing that rather than reading it the above mentioned link you
> chose to waste our time with this instead.  Good one, MC.

Yeah, I've read it several times, and have often linked to it as an
example of why one should be wary of DJB's software. It seems to me that
since DJB doesn't follow his own advice regarding the filesystem
hierarchy (see both qmail and djbdns), it'd be odd for him to expect
anyone else to. *Especially* seing as he's a bit mental. ("I'm not going
to take this any more. I demand cross-platform compatibility!")

-- 
Martin Coxall <coxall@cream.org>

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T14:35:54Z

Bruce Momjian wrote:
> OK, here is an updated proposal.  I think we have decided:
> 
> 	Moving postmaster.pid and postmaster.opts isn't worth it.
> 
> 	We don't want per-file GUC variables, but assume it is in
> 	the same config directory as postgresql.conf.  I don't
> 	see any valid reason they would want to put them somewhere
> 	different than postgresql.conf.
> 
> So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
> postmaster.

Agreed.  One additional thing: when pg_ctl invokes the postmaster, it
should explicitly specify -C on the postmaster command line, and if it
doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should
explicitly specify a -D as well.  Pg_ctl is going to have to be
modified to take a -C argument anyway, so we may as well go all the
way to do the right thing here.

This way, people who start the database using the standard tools we
supply will know exactly what's going on when they get a "ps" listing.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T14:40:11Z

Bruce Momjian wrote:
> 
> Give it up.  As long as we have -D, we will allow PGDATA. If you don't
> want to use it, don't use it.

Agreed.

I'm not sure I see how this diminishes the argument for fixing pg_ctl
so that it passes an explicit -D option to the postmaster when
invoking it...

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T14:40:27Z

Kevin Brown wrote:
> Bruce Momjian wrote:
> > OK, here is an updated proposal.  I think we have decided:
> > 
> > 	Moving postmaster.pid and postmaster.opts isn't worth it.
> > 
> > 	We don't want per-file GUC variables, but assume it is in
> > 	the same config directory as postgresql.conf.  I don't
> > 	see any valid reason they would want to put them somewhere
> > 	different than postgresql.conf.
> > 
> > So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
> > postmaster.
> 
> Agreed.  One additional thing: when pg_ctl invokes the postmaster, it
> should explicitly specify -C on the postmaster command line, and if it
> doesn't find a data_dir in $PGCONFIG/postgresql.conf then it should
> explicitly specify a -D as well.  Pg_ctl is going to have to be
> modified to take a -C argument anyway, so we may as well go all the
> way to do the right thing here.
> 
> This way, people who start the database using the standard tools we
> supply will know exactly what's going on when they get a "ps" listing.

No.  If you want ps to display, don't use environment variables. Many
don't care --- especially those with only one postmaster.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T14:58:49Z

Bruce Momjian wrote:
> > This way, people who start the database using the standard tools we
> > supply will know exactly what's going on when they get a "ps" listing.
> 
> No.  If you want ps to display, don't use environment variables. Many
> don't care --- especially those with only one postmaster.

You know that the code in pg_ctl doesn't send an explicit -D to the
postmaster even if pg_ctl itself is invoked with a -D argument, right?
The only way to make pg_ctl do that is by using the "-o" option.

A typical vendor-supplied install is going to invoke pg_ctl to do the
dirty work.  That's why I'm focusing on pg_ctl.

I completely understand your need for keeping PGDATA in postmaster.  I
don't understand why pg_ctl *shouldn't* be changed to invoke
postmaster with an explicit -D option.  It might be desirable for ps
to not show any arguments to postmaster in some circumstances (I have
no idea what those would be), but why in the world would you want that
to be the *default*?  Why would we want the default behavior to make
things harder on administrators and not easier?

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:02:02Z

Kevin Brown <kevin@sysexperts.com> writes:
> Tom Lane wrote:
>> The other side of that coin is, what's the good reason to remove it?
>> There's a long way between "I don't want my setup to depend on PGDATA"
>> and "I don't think your setup should be allowed to depend on PGDATA".
>> If you don't want to use it, then don't use it.  Why do you need to
>> tell me how I'm allowed to run my installation?

> I'm not talking about getting rid of ALL dependency on PGDATA in our
> entire distribution, only postmaster's.

We're obviously talking past each other.  You are arguing that under
circumstances X, Y, or Z, depending on a PGDATA setting is a bad idea.
You are then drawing the conclusion that I should not be allowed to
depend on PGDATA, whether or not I care about X, Y, or Z.

I am happy to design an arrangement that allows you not to depend on
PGDATA if you don't want to.  But I don't see why you need to break
my configuration procedures in order to fix yours.  As I outlined last
night, it's possible to do what you want without breaking backwards
compatibility for those that like PGDATA.

			regards, tom lane

Re: location of the configuration files

Vince Vielhaber <vev@michvhf.com> — 2003-02-14T15:07:32Z

On 14 Feb 2003, Martin Coxall wrote:

> On Fri, 2003-02-14 at 14:21, Vince Vielhaber wrote:
> > On 14 Feb 2003, Martin Coxall wrote:
> >
> > >
> > > > > If you are interested in reading a contrary position, you can read
> > > > > Berstein's arguments for his recommended way to install services at:
> > > > > http://cr.yp.to/unix.html
> > >
> > > But since DJB is a class-A monomaniac, he may not be the best person to
> > > listen to. /var/qmail/control for qmail configuration files? Yeah, good
> > > one, DJB.
> >
> > I'm guessing that rather than reading it the above mentioned link you
> > chose to waste our time with this instead.  Good one, MC.
>
> Yeah, I've read it several times, and have often linked to it as an
> example of why one should be wary of DJB's software. It seems to me that
> since DJB doesn't follow his own advice regarding the filesystem
> hierarchy (see both qmail and djbdns), it'd be odd for him to expect
> anyone else to. *Especially* seing as he's a bit mental. ("I'm not going
> to take this any more. I demand cross-platform compatibility!")

I seriously doubt your ability to judge anyone's mental stability.
I can also see that you prefer cross-platform INcompatibility.  Your
position and mindset are now crystal clear.

Vince.
-- 
 Fast, inexpensive internet service 56k and beyond!  http://www.pop4.net/
   http://www.meanstreamradio.com       http://www.unknown-artists.com
         Internet radio: It's not file sharing, it's just radio.

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T15:10:03Z

Tom Lane wrote:
> I am happy to design an arrangement that allows you not to depend on
> PGDATA if you don't want to.  But I don't see why you need to break
> my configuration procedures in order to fix yours.  As I outlined last
> night, it's possible to do what you want without breaking backwards
> compatibility for those that like PGDATA.

Yes, I agree.  I hadn't really thought of all the possible benefits of
PGDATA.  Sorry.  :-(

Would you agree that it would be a beneficial change to have pg_ctl
pass explicit arguments to postmaster?  It would go a long way towards
eliminating most of the situations I described.

A warning in the documentation about the consequences of using PGDATA
might not be a bad idea, either...

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:20:56Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
> postmaster.

Wait one second.  You are blithely throwing in a PGCONFIG variable
without any detailed proposal of exactly how it will work.  Does
that override a PGDATA environment variable?  How do they interact?

Also, please note Kevin Brown's nearby arguments against using PGDATA
at all, which surely apply with equal force to a PGCONFIG variable.
Now, I don't buy that Kevin's arguments are enough reason to break
backwards compatibility by removing PGDATA --- but I think they are
enough reason not to introduce a new environment variable.  PGCONFIG
wouldn't offer any backwards-compatibility value, and that tilts the
scales against it.

> Regarding Tom's idea of replacing data_dir with a full path during
> initdb, I think we are better having it be relative to the config
> directory, that way if they move pgsql/, the system still works.

Good thought, but you're assuming that initdb knows where the config
files will eventually live.  If we do that, then moving the config
files breaks the installation.  I think it will be fairly common to
let initdb drop its proposed config files into $PGDATA, and then
manually place them where they should go (or even more likely,
manually merge them with a prior version).  Probably better to force
datadir to be an absolute path in the config file.  (In fact, on safety
grounds I'd argue in favor of rejecting a datadir value taken from the
config file that wasn't absolute.)

> I also think we should start telling people to use PGCONFIG rather than
> PGDATA.  Then, in 7.5, we move the default config file location to
> pgsql/etc, and tell folks to point there rather than /data.

I agree with none of this.  This is not improvement, this is only change
for the sake of change.  The packagers will do what they want to do
(and are already doing, mostly) regardless.

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:27:23Z

Oliver Elphick <olly@lfix.co.uk> writes:
> On Fri, 2003-02-14 at 12:17, Bruce Momjian wrote:
>> If you want ps to display the data dir, you should use -D.  Remember, it
>> is mostly important for multiple postmaster, so if you are doing that,
>> just use -D, but don't prevent single-postmaster folks from using
>> PGDATA.

> Could not the ps line be rewritten to show this, as the backend's ps
> lines are rewritten?

I for one would rather it didn't do that.  I already set my postmaster
command lines the way I want 'em, and I don't want the code overriding
that.  (I prefer to use explicit -p arguments to distinguish the various
postmasters I have running --- shorter and easier to read than explicit
-D would be.  At least for me.)

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:30:07Z

Kevin Brown <kevin@sysexperts.com> writes:
> I'm quite interested in knowing why there seems to be so much resistance to
> removing the "potential_DataDir = getenv("PGDATA");" line from
> postmaster.c.

Backwards compatibility.  Also, you still haven't explained why
"I don't want to use PGDATA" should translate to "no one should
be allowed to use PGDATA".  If you don't set PGDATA, what problem
is there for you in that line being there?

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:35:41Z

Martin Coxall <coxall@cream.org> writes:
> Partially true. The FHS specifies that the /etc top layer is for system-own3d
> stuff, but the subdirectories off it are explicitly used for user space programs
> and, well, everything. (/etc/apache, /etc/postgres, /etc/tomcat3,
> /etc/tomcat4...)

FHS or no FHS, I would think that the preferred arrangement would be to
keep Postgres' config files in a postgres-owned subdirectory, not
directly in /etc.  That way you need not be root to edit them.  (My idea
of an editor, Emacs, always wants to write a backup file, so I dislike
having to edit files that live in directories I can't write.)

Here's a pretty topic for a flamewar: should it be /etc/postgres/ or
/etc/postgresql/ ?

			regards, tom lane

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-14T15:38:27Z

On Thu, 2003-02-13 at 19:22, Adam Haberlach wrote:
> On Thu, Feb 13, 2003 at 05:59:17PM -0500, Robert Treat wrote:
> > On Thu, 2003-02-13 at 15:08, mlw wrote:
> > > Stephan Szabo wrote:
> > > 
> > > On Thu, 13 Feb 2003, mlw wrote:
> > Personally I think a postgresql installation is much more like an apache
> > installation, which generally contains all of the files (data and
> > config) under /usr/local/apache. Maybe someone can dig more to see if
> > that system is more appropriate a comparison than something like bind.
> 
> 	I think you are making a pretty uninformed, if not just plain wrong 
> generalization.  I've run exactly one system with apache configuration 
> files in /usr/local/apache, and even then, the data was not there.

Uh... the last time I built apache from source, it stuck everything
under /usr/local/apache. It uses a conf directory for the config files,
and htdocs for the "data" files...  That is it's default configuration.

<snip stories of all the different ways people run apache>

You know, this is why I actually suggested looking closer at apache. By
default, everything is crammed in one directory, but if you want to, you
can configure it "six different ways to sunday". That seems to be a big
plus IMO

> 
> What does this mean?
> 
> People will put things in different places, and there are typically
> very good reasons for this.  This is ESPECIALLY true when one wants to
> have configuration files, at least the base ones in a common place such
> as /etc or /usr/local/etc in order to make backup of configuration easy
> and clean, while leaving data somewhere else for performance or magnitude
> of partition reasons.  It just makes sense to ME to have postgresql.conf
> reside in /etc, yet put my data in /var/data/postgresql, yet retain the
> option to put my data in /raid/data/postgresql at a later date, when the
> new hardware comes in.

Is anyone arguing against this? I'm certainly not. But maybe my needs
are more varied than yours. On my local development box, I run multiple
versions of apache, compiled with different versions of php. It really
helps to keep all of apache's stuff centralized, and using things like
rpms actually overly complicates this. Now sure, that's a development
machine, but on the phppgadmin demo server, which is essentially a
production system, I run three different versions of postgresql. In
fact, I need to upgrade two of those (to 7.2.4 and 7.3.2), I shudder to
think about doing that if postgresql forced me to use the /etc/
directory for all of my config files. Now sure, this probably isn't
typical use, but I would say that when it comes time to upgrade major
versions, unless you running an operation where you can have large
amounts of downtime, postgresql needs to have the ability to have
multiple versions install that don't conflict with each other, and it
needs to do this easily. The upgrade process is hard enough already.

<snip> 
> However, this seems, to me, to be a very small addition that has some real-world
> (and yes, we need to start paying attention to the real world) advantages.
> 
> And finally, don't go telling me that I'm wrong to put my data and config files
> where I am.  You can offer advice, but I'm probably going to ignore it because
> I like where they are and don't need to explain why.
> 

Have I wronged you in some former life? I've very little concern for
where you put your data files, and have no idea why you'd think I'd
criticize your setup. 

Robert Treat

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T15:51:16Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> My point is that folks with multiple postmasters may not want to use
> PGDATA, but for folks who have single postmasters, it makes things
> easier and less error-prone.

Actually, for multi postmasters too.  I have little shell-environment
config files that switch my entire world view between different source
trees and installation trees, for example this one sets me up to mess
with the 7.2 branch:

STDPATH=${STDPATH:-$PATH}
STDMANPATH=${STDMANPATH:-$MANPATH}

PGSRCROOT=$HOME/REL7_2/pgsql
PGINSTROOT=$HOME/version72
PATH=$PGINSTROOT/bin:/opt/perl5.6.1/bin:$STDPATH
MANPATH=$PGINSTROOT/man:$STDMANPATH
PGLIB=$PGINSTROOT/lib
PGDATA=$PGINSTROOT/data
PMOPTIONS="-p 5472 -i -F"
PMLOGFILE=server72.log

export PGSRCROOT PGINSTROOT PATH MANPATH PGLIB PGDATA STDPATH STDMANPATH
export PMOPTIONS PMLOGFILE

After sourcing one of these, I can use pg_ctl as well as a half dozen
other convenient little scripts that do things like remake and reinstall
the backend:

#!/bin/sh

pg_ctl -w stop

cd $PGSRCROOT/src/backend

make install-bin

startpg

or this one that fires up gdb on a crashed backend:

#!/bin/sh

# Usage: gdbcore

cd $HOME

CORES=`find $PGDATA/base -name core -type f -print`

if [ x"$CORES" != x"" ]
then
    ls -l $CORES
fi

if [ `echo "$CORES" | wc -w` -eq 1 ]
then
    exec gdb $PGINSTROOT/bin/postgres "$CORES"
else
    exec gdb $PGINSTROOT/bin/postgres
fi

This is vastly less error-prone than keeping track of the various
related elements in my head.

Now, it's certainly true that I could still make this work if I had
to explicitly say -D $PGDATA to the postmaster.  But that would clutter
my ps display.  I am happy with -p as the ps indicator of which
postmaster is which; I don't want more stuff in there.

			regards, tom lane

Re: location of the configuration files

Shridhar Daithankar <shridhar_daithankar@persistent.co.in> — 2003-02-14T16:02:48Z

On Friday 14 Feb 2003 9:05 pm, you wrote:
> Martin Coxall <coxall@cream.org> writes:
> Here's a pretty topic for a flamewar: should it be /etc/postgres/ or
> /etc/postgresql/ ?

I vote for /etc/pgsql. Keeping in line of unix philosophy of cryptic and short 
names. Who wants a descriptive names anyway..:-)

Seriously, the traffic on last three days ahd very high noise ratio. 
Especially the whole discussion of PGDATA stuff fails to register as 
significant IMO. Right now, I can do things the way I want to do and I guess 
it is pretty much same with everyone else. Is it last topic left to improve?

Keep it simple and on tpoic guys. This is hackers. Keep it low volume 
otherwise, two years down the lines, archives will be unsearchable..

 Shridhar

Re: location of the configuration files

scott.marlowe <scott.marlowe@ihs.com> — 2003-02-14T16:14:52Z

On Thu, 13 Feb 2003, mlw wrote:

> 
> 
> scott.marlowe wrote:
> 
> >>These are not issues at all. You could put the configuration file 
> >>anywhere, just as you can for any UNIX service.
> >>
> >>postmaster --config=/home/myhome/mydb.conf
> >>
> >>I deal with a number of PG databases on a number of sites, and it is a 
> >>real pain in the ass to get to a PG box and hunt around for data 
> >>directory so as to be able to administer the system. What's really 
> >>annoying is when you have to find the data directory when someone else 
> >>set up the system.
> >>    
> >>
> >
> >Really?  I would think it's easier to do this:
> >
> >su - pgsuper
> >cd $PGDATA
> >pwd
> >
> >Than to try to figure out what someone entered when they ran ./configure 
> >--config=...
> >  
> >
> Why do you think PGDATA would be set for root?

Did you not notice the "su - pgsuper" line above?  You know, the one where 
you become the account that runs that instance of the database.  Again, I 
ask you, isn't that easier than trying to find out what someone typed when 
they typed ./configure --config=?

> >>Configuring postgresql via a configuration file which specifies all the 
> >>data, i.e. data directory, name of other configuration files, etc. is 
> >>the right way to do it. Even if you have reasons against it, even if you 
> >>think it is a bad idea, a bad standard is almost always a better 
> >>solution than an arcane work of perfection.
> >>    
> >>
> >
> >Wrong, I strongly disagree with this sentament.  Conformity to standards 
> >for simple conformity's sake is as wrong as sticking to the old way 
> >because it's what we're all comfy with.  
> >
> It isn't conformity for conformitys sake. It is following an established 
> practice, like driving on the same side of the road or stopping at red 
> lights.

But this isn't the same thing at all.  Apache, when built from a tar ball, 
goes into /usr/local/apache/ and ALL it's configuration files are there.  
When installed as a package, my OS manufacturer decides where that goes.  
Those are the two "standard" ways of doing things.  I like that postgresql 
installs into the /usr/local/pgsql directory from a tar ball.  I like the 
fact that it uses $PGDATA to tell it where the cluster is, so that all my 
scripts, like pg_ctl, just know where it is without a -D switch each time.

Re: location of the configuration files

Oliver Elphick <olly@lfix.co.uk> — 2003-02-14T16:44:49Z

On Fri, 2003-02-14 at 15:35, Tom Lane wrote:
> Here's a pretty topic for a flamewar: should it be /etc/postgres/ or
> /etc/postgresql/ ?

It should be configurable!

Debian uses /etc/postgresql, if you want to stick to what quite a lot of
people are familiar with.

-- 
Oliver Elphick                                Oliver.Elphick@lfix.co.uk
Isle of Wight, UK                             http://www.lfix.co.uk/oliver
GPG: 1024D/3E1D0C1C: CA12 09E0 E8D5 8870 5839  932A 614D 4C34 3E1D 0C1C
                 ========================================
     "God be merciful unto us, and bless us; and cause his 
      face to shine upon us."          Psalms 67:1

Re: location of the configuration files

scott.marlowe <scott.marlowe@ihs.com> — 2003-02-14T16:46:35Z

On Fri, 14 Feb 2003, Curt Sampson wrote:

> On Thu, 13 Feb 2003, scott.marlowe wrote:
> 
> > If I do a .tar.gz install of apache, I get /usr/local/apache/conf, which
> > is not the standard way you're listing.
> 
> I'm going to stay out of this argument from now on, but this struck a sore
> point.
> 
> /usr is designed to be a filesystem that can be shared. Is the stuff in
> /usr/local/apache/conf really supposed to be shared amongst all machines
> of that architecture on your site that run apache?

Interesting.  I've always viewed usr EXCEPT for local this way.  In 
fact, on most of my boxes I create a seperate mount point for /usr/local 
so it's easier to backup and maintain, and it doesn't fill up the /usr 
directory.

Asking for everything in a directory with the name local in it to be 
shared is kind of counter intuitive to me.

Re: location of the configuration files

Martin Coxall <coxall@cream.org> — 2003-02-14T16:57:58Z

> I seriously doubt your ability to judge anyone's mental stability.
> I can also see that you prefer cross-platform INcompatibility.  Your
> position and mindset are now crystal clear.

Come now- don't take it personally. All I said is, as someone who
wrestles daily with QMail, we should prefer the FHS over DJB's way of
doing things and that DJB is a little, ahem, egocentric at times.
Neither of these things was meant as a mortal insult to you personally,
and if I offended you I apologise.

Anyway, it looks like it's all been agreed over there, anyway.

-- 
Martin Coxall <coxall@cream.org>

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T17:33:42Z

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > So, we add data_dir to postgresql.conf, and add -C/PGCONFIG to
> > postmaster.
> 
> Wait one second.  You are blithely throwing in a PGCONFIG variable
> without any detailed proposal of exactly how it will work.  Does
> that override a PGDATA environment variable?  How do they interact?

I am just throwing out ideas. I don't think we are near interaction
issues yet.

I think the big question is whether we want the default to install the
configs in a separate directory, pgsql/etc, or just allow the
specification of a separate location.  Advantages of pgsql/etc are
initdb-safe, and easier backups.

I do think PGCONFIG would be helpful for the same reason that PGDATA is.
However, there is clearly a problem of how does data_dir interact with
PGDATA.

The big question is whether PGDATA is still our driving config variable,
and PGCONFIG/-C is just an additional option, or whether we are moving
in a direction where PGCONFIG/-C is going to be the driving value, and
data_dir is going to be read as part of that.

> Also, please note Kevin Brown's nearby arguments against using PGDATA
> at all, which surely apply with equal force to a PGCONFIG variable.
> Now, I don't buy that Kevin's arguments are enough reason to break
> backwards compatibility by removing PGDATA --- but I think they are
> enough reason not to introduce a new environment variable.  PGCONFIG
> wouldn't offer any backwards-compatibility value, and that tilts the
> scales against it.

Weren't you just showing how you set environment variables to easily
configure stuff.  If you use a separate configure dir, isn't PGCONFIG
part of that?

> > Regarding Tom's idea of replacing data_dir with a full path during
> > initdb, I think we are better having it be relative to the config
> > directory, that way if they move pgsql/, the system still works.
> 
> Good thought, but you're assuming that initdb knows where the config
> files will eventually live.  If we do that, then moving the config
> files breaks the installation.  I think it will be fairly common to
> let initdb drop its proposed config files into $PGDATA, and then
> manually place them where they should go (or even more likely,
> manually merge them with a prior version).  Probably better to force
> datadir to be an absolute path in the config file.  (In fact, on safety
> grounds I'd argue in favor of rejecting a datadir value taken from the
> config file that wasn't absolute.)

Maybe.  Not sure.

> > I also think we should start telling people to use PGCONFIG rather than
> > PGDATA.  Then, in 7.5, we move the default config file location to
> > pgsql/etc, and tell folks to point there rather than /data.
> 
> I agree with none of this.  This is not improvement, this is only change
> for the sake of change.  The packagers will do what they want to do
> (and are already doing, mostly) regardless.

Well, it is a step forward in terms of initdb-safe and easier backups.
Several people said they liked that.  I thought you were one of them.

This is back to the big question, who drives things in the default
install, config file or pgdata.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T18:09:37Z

Bruce Momjian wrote:
> The big question is whether PGDATA is still our driving config variable,
> and PGCONFIG/-C is just an additional option, or whether we are moving
> in a direction where PGCONFIG/-C is going to be the driving value, and
> data_dir is going to be read as part of that.

I'm actually leaning towards PGCONFIG + PGDATA.

Yeah, it may be a surprise given my previous arguments, but I can't
help but think that the advantages you get with PGDATA will also exist
for PGCONFIG.

My previous arguments for removing PGDATA from postmaster can be dealt
with by fixing pg_ctl to use explicit command line directives when
invoking postmaster -- no changes to postmaster needed.  PGCONFIG
would be no different in that regard.

Sorry if I seem a big gung-ho on the administrator point of view, but
as a system administrator myself I understand and feel their pain.
:-)

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Greg Stark <gsstark@mit.edu> — 2003-02-14T18:16:26Z

"scott.marlowe" <scott.marlowe@ihs.com> writes:

> But this isn't the same thing at all.  Apache, when built from a tar ball, 
> goes into /usr/local/apache/ and ALL it's configuration files are there.  

Two comments:

1) Even in that case the config files go into /usr/local/apache/conf and the
   other kinds of files like data logs and cache files, all go in other
   subdirectories.

2) What you describe is only true if you configure with the default
   "--with-layout=Apache". The naming should perhaps be a clue that this isn't
   a conventional layout. If you configure with --with-layout=GNU you get the
   conventional Unix layout in /usr/local, If you use --with-layout=RedHat you
   get the conventional layout in /usr directly which is mainly useful for
   distribution packagers.

Putting stuff in a subdirectory like /usr/local/apache or /usr/local/pgsql is
unfortunately a widespread practice. It does have some advantages over the
conventional layout in /usr/local/{etc,bin,...} directly. But the major
disadvantage is that users can't run programs without adding dozens of entries
to their paths, can't compile programs without dozens of -L and -I lines, etc.

GNU autoconf script makes it pretty easy to configure packages to work either
though, and /usr/local is the purview of the local admin. As long as it's easy
to configure postgres to install "properly" with --prefix=/usr/local it won't
be any more of an offender than lots of other packages like apache, kde, etc.

Though I'll mention, please make it $prefix/etc not $prefix/conf. No need to
be gratuitously non-standard on an arbitrary name, and no need to pollute
/usr/local with multiple redundant directories.

-- 
greg

Re: location of the configuration files

Bruno Wolff III <bruno@wolff.to> — 2003-02-14T18:27:49Z

On Thu, Feb 13, 2003 at 11:53:26 -0500,
  mlw <pgsql@mohawksoft.com> wrote:
> 
> Where, specificaly are his arguements against a configuration file 
> methodology?

I don't think he is argueing against a configuration methodology, but
rather against the methodology being used in Unix distributions.
In particular he doesn't file the Linux File Standard because it
puts the same software in different places depending on whether the
vendor or using installed it.

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T18:37:22Z

Kevin Brown wrote:
> Bruce Momjian wrote:
> > The big question is whether PGDATA is still our driving config variable,
> > and PGCONFIG/-C is just an additional option, or whether we are moving
> > in a direction where PGCONFIG/-C is going to be the driving value, and
> > data_dir is going to be read as part of that.
> 
> I'm actually leaning towards PGCONFIG + PGDATA.
> 
> Yeah, it may be a surprise given my previous arguments, but I can't
> help but think that the advantages you get with PGDATA will also exist
> for PGCONFIG.
> 
> My previous arguments for removing PGDATA from postmaster can be dealt
> with by fixing pg_ctl to use explicit command line directives when
> invoking postmaster -- no changes to postmaster needed.  PGCONFIG
> would be no different in that regard.

I see your point --- pg_ctl does a PGDATA trick when passed -D:

        -D)
            shift
            # pass environment into new postmaster
            PGDATA="$1"
            export PGDATA

It should pass -D just like it was given.

> Sorry if I seem a big gung-ho on the administrator point of view, but
> as a system administrator myself I understand and feel their pain.

Making things easy for sysadmins is an important feature of PostgreSQL.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruno Wolff III <bruno@wolff.to> — 2003-02-14T18:50:17Z

On Fri, Feb 14, 2003 at 02:58:49 -0800,
  Kevin Brown <kevin@sysexperts.com> wrote:
> 
> Lest you think that this is an unlikely scenario, keep in mind that
> most startup scripts, including pg_ctl, currently start the postmaster
> without arguments and rely on PGDATA, so a shop that hasn't already
> been bitten by this *will* be.  Right now shops that wish to avoid the
> trap I described have to go to *extra* lengths: they have to make
> exactly the same kinds of changes to the scripts that I'm talking
> about us making (putting an explicit '-D "$PGDATA"' where none now
> exists) or they have to resort to tricks like renaming the postmaster
> executable and creating a shell script in its place that will invoke
> the (renamed) postmaster with '-D "$PGDATA"'.

On at least some systems ps will dump process' environment and could be
easily used to check PGDATA.

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-14T18:56:28Z

Bruce Momjian wrote:
> I see your point --- pg_ctl does a PGDATA trick when passed -D:
> 
>         -D)
>             shift
>             # pass environment into new postmaster
>             PGDATA="$1"
>             export PGDATA
> 
> It should pass -D just like it was given.

Yes, exactly.

Now, the more interesting question in my mind is: if pg_ctl isn't
passed -D but inherits PGDATA, should it nonetheless pass -D
explicitly to the postmaster?  We can make it do that, and it would
have the benefit of making transparent what would otherwise be opaque.

I'm inclined to answer "yes" to that question, but only because
someone who *really* doesn't want the postmaster to show up with a -D
argument in "ps" can start the postmaster directly without using
pg_ctl at all.  Tom made a good argument for sometimes wanting to keep
the ps output clean, but it's not clear to me that it should
necessarily apply to pg_ctl.

But you guys might have a different perspective on that.  :-)

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Steve Crawford <scrawford@pinpointresearch.com> — 2003-02-14T19:24:19Z

On Friday 14 February 2003 6:07 am, Martin Coxall wrote:
> On Thu, 2003-02-13 at 20:28, Steve Crawford wrote:
> > I don't see why we can't keep everyone happy and let the users choose the
> > setup they want. To wit, make the following, probably simple, changes:
> >
> > 1) Have postgresql default to using /etc/postgresql.conf
>
> /etc/postgres/postgresql.conf, if we want to be proper FHS-bitches.
>
> > 2) Add a setting in postgresql.conf specifying the data directory
> > 3) Change the meaning of -D to mean "use this config file"
> > 4) In the absence of a specified data directory in postgresql.conf, use
> > the location of the postgresql.conf file as the data directory
>
> Shouldn't it in that case default to, say /var/lib/postgres?

Idea 4 was just a way to preserve current behaviour for those who desire. 
Moving postgresql.conf requires adding the data directory info into 
postgresql.conf or specifying it in some other way. If, in the absence of any 
specification in postgresql.conf, postgres just looks in the same directory 
as postgresql.conf then it will be almost identical to the current setup.

Cheers,
Steve

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T20:10:13Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think the big question is whether we want the default to install the
> configs in a separate directory, pgsql/etc, or just allow the
> specification of a separate location.  Advantages of pgsql/etc are
> initdb-safe, and easier backups.

I don't see why we don't just let initdb install suggested config files
into the new $PGDATA directory, same as it ever did.  Then (as long as
we don't use relative paths in the config files) people can move them
somewhere else if they like, or not if they prefer not to.  Adding more
mechanism than that just adds complexity without buying much (except the
possibility of initdb overwriting your old config files, which is
exactly what I thought we wanted to avoid).

> The big question is whether PGDATA is still our driving config variable,
> and PGCONFIG/-C is just an additional option, or whether we are moving
> in a direction where PGCONFIG/-C is going to be the driving value, and
> data_dir is going to be read as part of that.

I thought the idea was to allow both approaches.  We are not moving in
the direction of one or the other, we are giving people a choice of how
they want to drive it.

> Weren't you just showing how you set environment variables to easily
> configure stuff.  If you use a separate configure dir, isn't PGCONFIG
> part of that?

I'm just pointing out that there's no backward-compatibility argument
for PGCONFIG.  It should only be put in if the people who want to use
the -C-is-driver approach want it.  Kevin clearly doesn't ;-).

			regards, tom lane

Re: location of the configuration files

Peter Eisentraut <peter_e@gmx.net> — 2003-02-14T20:14:30Z

Tom Lane writes:

> I would favor a setup that allows a -C *directory* (not file) to be
> specified as a postmaster parameter separately from the -D directory;

A directory is not going to satisfy people.

> I don't see any great value in a separate postgresql.conf parameter for
> each secondary config file; that just means clutter to me,

Not to other people.

> 1. No -C switch, no -D switch, no PGDATA found in environment: seek
> postgresql.conf in the default -C directory established at configure
> time.  Use the 'datadir' specified therein as -D.  Fail if postgresql.conf
> doesn't define a datadir value.

OK.

> 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
> as both -C and -D.

This behavior would be pretty inconsistent.  But maybe it's the best we
can do.

> 3. No -C switch, -D switch on command line: use -D value as both -C and -D,
> proceed as in case 2.

Same as above.

> 4. -C switch, no -D switch on command line: seek postgresql.conf in
> -C directory, use the datadir it specifies.

OK.

> 5. -C and -D on command line: seek postgresql.conf in -C directory,
> use -D as datadir overriding what is in postgresql.conf (this is just
> the usual rule that command line switches override postgresql.conf).

But that usual rule seems to be in conflict with cases 2 and 3 above.
(The usual rule is that a command-line option overrides a postgresql.conf
parameter.  The rule in 3, for example is, that a command-line option (the
same one!) overrides where postgresql.conf is in the first place.)

> I would venture that the configure-time-default for -C should be
> ${prefixdir}/etc if configure is not told differently,

Yeah, we already have that as --sysconfdir.

-- 
Peter Eisentraut   peter_e@gmx.net

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T20:19:36Z

Peter Eisentraut wrote:
> Tom Lane writes:
> 
> > I would favor a setup that allows a -C *directory* (not file) to be
> > specified as a postmaster parameter separately from the -D directory;
> 
> A directory is not going to satisfy people.

Who is asking to put postgresql.conf, pg_hba.conf, and pg_ident.conf in
different directories?  I haven't heard anyone ask for that.

> > I don't see any great value in a separate postgresql.conf parameter for
> > each secondary config file; that just means clutter to me,
> 
> Not to other people.
> 
> > 1. No -C switch, no -D switch, no PGDATA found in environment: seek
> > postgresql.conf in the default -C directory established at configure
> > time.  Use the 'datadir' specified therein as -D.  Fail if postgresql.conf
> > doesn't define a datadir value.
> 
> OK.
> 
> > 2. No -C switch, no -D switch, PGDATA found in environment: use $PGDATA
> > as both -C and -D.
> 
> This behavior would be pretty inconsistent.  But maybe it's the best we
> can do.

What happens if postgresql.conf then defines data_dir?  Seems we ignore it.

This brings up the same issue of whether -C/PGCONFIG is a inferior
option to -D/PGDATA, and whether we keep the config files in /data by
default.

> > 3. No -C switch, -D switch on command line: use -D value as both -C and -D,
> > proceed as in case 2.
> 
> Same as above.

> 
> > 4. -C switch, no -D switch on command line: seek postgresql.conf in
> > -C directory, use the datadir it specifies.
> 
> OK.

Here we are saying the -C doesn't override postgresql.conf as the proper
PGDATA value.  Is that what we want?  We had the question above over how
a data_dir in postgresql.conf is handled.

> 
> > 5. -C and -D on command line: seek postgresql.conf in -C directory,
> > use -D as datadir overriding what is in postgresql.conf (this is just
> > the usual rule that command line switches override postgresql.conf).
> 
> But that usual rule seems to be in conflict with cases 2 and 3 above.
> (The usual rule is that a command-line option overrides a postgresql.conf
> parameter.  The rule in 3, for example is, that a command-line option (the
> same one!) overrides where postgresql.conf is in the first place.)

Yes, the big question seems to be if we are defaulting -C to be the same
as -D, whether that is an actual specification of -D that should
override postgresql.conf.

This is part of the reason I don't like the -D assumes -C and stuff like
that.

I think we need to move the config files to pgsql/etc, for backup and
initdb safety, and move toward having PGCONFIG/-C as the driving
parameter.  I think having both function equally and defaulting if the
other is not specified is going to breed confusion.

I am willing to make thing a little difficult for backward compatibility
to do this, and I think because it is only administrators, they will
welcome the improvement.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-14T20:27:53Z

Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> I would favor a setup that allows a -C *directory* (not file) to be
>> specified as a postmaster parameter separately from the -D directory;

> A directory is not going to satisfy people.

Why not?  Who won't it satisfy, and what's their objection?

AFAICS, you can either set -C to /etc if you want your PG config files
loose in /etc, or you can set it to /etc/postgresql/ if you want them
in a privately-owned directory.  Which other arrangements are needed?

			regards, tom lane

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T20:53:45Z

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I think the big question is whether we want the default to install the
> > configs in a separate directory, pgsql/etc, or just allow the
> > specification of a separate location.  Advantages of pgsql/etc are
> > initdb-safe, and easier backups.
> 
> I don't see why we don't just let initdb install suggested config files
> into the new $PGDATA directory, same as it ever did.  Then (as long as
> we don't use relative paths in the config files) people can move them
> somewhere else if they like, or not if they prefer not to.  Adding more
> mechanism than that just adds complexity without buying much (except the
> possibility of initdb overwriting your old config files, which is
> exactly what I thought we wanted to avoid).
> 
> > The big question is whether PGDATA is still our driving config variable,
> > and PGCONFIG/-C is just an additional option, or whether we are moving
> > in a direction where PGCONFIG/-C is going to be the driving value, and
> > data_dir is going to be read as part of that.
> 
> I thought the idea was to allow both approaches.  We are not moving in
> the direction of one or the other, we are giving people a choice of how
> they want to drive it.

That's where I am unsure.  Is the initdb-safe and backup advantages
enough to start to migrate those out to data/?  I need to hear
comments on that.

One new idea is to move the config files into data/etc.  That makes it
clear which are config files, and makes backup a little easier.  It
would make -C more logical because you are not moving a clear directory.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T22:17:22Z

Kevin Brown wrote:
> Bruce Momjian wrote:
> > The big question is whether PGDATA is still our driving config variable,
> > and PGCONFIG/-C is just an additional option, or whether we are moving
> > in a direction where PGCONFIG/-C is going to be the driving value, and
> > data_dir is going to be read as part of that.
> 
> I'm actually leaning towards PGCONFIG + PGDATA.
> 
> Yeah, it may be a surprise given my previous arguments, but I can't
> help but think that the advantages you get with PGDATA will also exist
> for PGCONFIG.
> 
> My previous arguments for removing PGDATA from postmaster can be dealt
> with by fixing pg_ctl to use explicit command line directives when
> invoking postmaster -- no changes to postmaster needed.  PGCONFIG
> would be no different in that regard.

The following patch propogates pg_ctl -D to the postmaster as a -D flag.
I see no other pg_ctl flags that make sense to propogate.

Applied.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-14T22:41:26Z

I don't want to over-engineer this.  Propogating -D into postmaster
makes sense, but grabbing PGDATA doesn't to me.

---------------------------------------------------------------------------

Kevin Brown wrote:
> Bruce Momjian wrote:
> > I see your point --- pg_ctl does a PGDATA trick when passed -D:
> > 
> >         -D)
> >             shift
> >             # pass environment into new postmaster
> >             PGDATA="$1"
> >             export PGDATA
> > 
> > It should pass -D just like it was given.
> 
> Yes, exactly.
> 
> Now, the more interesting question in my mind is: if pg_ctl isn't
> passed -D but inherits PGDATA, should it nonetheless pass -D
> explicitly to the postmaster?  We can make it do that, and it would
> have the benefit of making transparent what would otherwise be opaque.
> 
> I'm inclined to answer "yes" to that question, but only because
> someone who *really* doesn't want the postmaster to show up with a -D
> argument in "ps" can start the postmaster directly without using
> pg_ctl at all.  Tom made a good argument for sometimes wanting to keep
> the ps output clean, but it's not clear to me that it should
> necessarily apply to pg_ctl.
> 
> But you guys might have a different perspective on that.  :-)
> 
> 
> 
> -- 
> Kevin Brown					      kevin@sysexperts.com
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Curt Sampson <cjs@cynic.net> — 2003-02-15T04:44:00Z

On Fri, 14 Feb 2003, scott.marlowe wrote:

> Asking for everything in a directory with the name local in it to be
> shared is kind of counter intuitive to me.

Not really. If you install a particular program that doesn't come with
the OS on one machine on your site, why would you not want to install it
separately on all of the others?

Typically, I want my favourite non-OS utilities on all machines, not
just one. (Even if I don't use them on all machines.) Thus /usr/local is
for site-local stuff.

cjs
-- 
Curt Sampson  <cjs@cynic.net>   +81 90 7737 2974   http://www.netbsd.org
    Don't you know, in this new Dark Age, we're all light.  --XTC

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-15T14:48:57Z

Robert Treat wrote:

>Seems like some are saying one of the problems with the current system
>is it doesn't follow FHS or LSB. If those are valid reasons to change
>the system, it seems like a change which doesn't actually address those
>concerns would not be acceptable. (Unless those really aren't valid
>concerns...)
>
>  
>
I did not start this thread to make PostgreSQL FHS compatible, someone 
else brought that up.

As I said somewhere else, I'm an old fashioned UNIX guy, capability 
without policy. The patch that I submitted for 7.3.2 will allow the user 
to configure PostgreSQL with a configuration file outside the $PGDATA 
directory. That's all I care about. If someone wants to get on the FHS 
bandwagon, that's fine. PostgreSQL should allow that ability but should 
not require it.

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-15T14:53:23Z


Martin Coxall wrote:

>On Thu, 2003-02-13 at 20:28, Steve Crawford wrote:
>  
>
>>I don't see why we can't keep everyone happy and let the users choose the 
>>setup they want. To wit, make the following, probably simple, changes:
>>
>>1) Have postgresql default to using /etc/postgresql.conf
>>    
>>
>
>/etc/postgres/postgresql.conf, if we want to be proper FHS-bitches.
>
>  
>
>>2) Add a setting in postgresql.conf specifying the data directory
>>3) Change the meaning of -D to mean "use this config file"
>>4) In the absence of a specified data directory in postgresql.conf, use the 
>>location of the postgresql.conf file as the data directory
>>    
>>
>
>Shouldn't it in that case default to, say /var/lib/postgres?
>
I would really like to push back this whole discussion to adding the 
ability the flexibility to configure PostgreSQ as opposed to determining 
a specific configuration strategy.

Adding the ability is easy. Let the distros determine their strategy. 
Trying to enforce one way over another will make this continue on 
forever and will never be solved.

>  
>

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-15T16:00:11Z

mlw <pgsql@mohawksoft.com> writes:
> I would really like to push back this whole discussion to adding the 
> ability the flexibility to configure PostgreSQ as opposed to determining 
> a specific configuration strategy.

> Adding the ability is easy. Let the distros determine their strategy. 
> Trying to enforce one way over another will make this continue on 
> forever and will never be solved.

I agree that we shouldn't be in the business of dictating choices.

But it is important to examine what the plausible choices are, so that
we can be sure the solution we provide will accommodate all of them.
So I don't think this part of the thread has been useless.

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T00:36:51Z

On Friday 14 February 2003 15:10, Tom Lane wrote:
> I don't see why we don't just let initdb install suggested config files
> into the new $PGDATA directory, same as it ever did.

Ok, let me take another tack.

Just exactly why does initdb need to drop any config files anywhere?  We 
provide templates; initdb can initialize the data structure.  If we can by 
default (as part of make install) put the config file templates in 
$SYSCONFDIR (as set by ./configure), then why does initdb need to retouch 
them?  I say that having configured PostgreSQL like this: (this is for 7.2.4, 
not 7.3.x)
--enable-locale --with-CXX --prefix=/usr --disable-rpath --with-perl 
--enable-multibyte --with-tcl --with-odbc --enable-syslog --with-python 
--with-openssl --with-pam --with-krb5=/usr/kerberos --enable-nls 
--sysconfdir=/etc/pgsql --mandir=/usr/share/man --docdir=/usr/share/doc 
--includedir=/usr/include --datadir=/usr/share/pgsql

So, in my case, it would be preferable to me for initdb to not make a default 
postgresql.conf, pg_hba.conf, and pg_ident.conf.  The make install process 
should populate sysconfdir (/etc/pgsql here) with those files.

Why does initdb even need to be involved now (I know the historical reason)?

Comments?
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T00:38:23Z

On Friday 14 February 2003 11:02, "Shridhar Daithankar wrote:
> Especially the whole discussion of PGDATA stuff fails to register as
> significant IMO. Right now, I can do things the way I want to do and I
> guess it is pretty much same with everyone else. Is it last topic left to
> improve?

If it weren't significant to a few, then there wouldn't be the traffic.  If 
there's too much traffic, well, there are alternatives.

> Keep it simple and on tpoic guys. This is hackers. Keep it low volume
> otherwise, two years down the lines, archives will be unsearchable..

The system configuration of PostgreSQL is on topic for -hackers.  IMNSHO.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T01:19:59Z

Lamar Owen <lamar.owen@wgcr.org> writes:
> Just exactly why does initdb need to drop any config files anywhere?

Because we'd like it to edit the correct datadir into the config file,
to take just the most obvious example.  There has also been a great deal
of discussion recently about other things initdb might automatically put
into the config file after looking at the system environment.  That's
not happened yet, but we'd really be restricting ourselves to say that
initdb can never customize the config files.

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T01:24:57Z

On Saturday 15 February 2003 20:19, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > Just exactly why does initdb need to drop any config files anywhere?

> Because we'd like it to edit the correct datadir into the config file,
> to take just the most obvious example.

Shouldn't we be consistent and have initdb use the datadir set in the config 
file, which could be supplied by a ./configure switch?

>  There has also been a great deal
> of discussion recently about other things initdb might automatically put
> into the config file after looking at the system environment.  That's
> not happened yet, but we'd really be restricting ourselves to say that
> initdb can never customize the config files.

Customize != writing the original.

I'm looking at a packager point of view here.  The initdb is done well after 
the package is made, and installed.  It would be ideal from this point of 
view to have things fully configured pre-initdb (and thus pre-packaging).

But I understand that this might not be ideal from a multipostmaster point of 
view.  Surely these two can be reconciled.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Robert Treat <xzilla@users.sourceforge.net> — 2003-02-16T01:34:16Z

On Saturday 15 February 2003 09:48 am, mlw wrote:
> Robert Treat wrote:
> >Seems like some are saying one of the problems with the current system
> >is it doesn't follow FHS or LSB. If those are valid reasons to change
> >the system, it seems like a change which doesn't actually address those
> >concerns would not be acceptable. (Unless those really aren't valid
> >concerns...)
>
> I did not start this thread to make PostgreSQL FHS compatible, someone
> else brought that up.
>
> As I said somewhere else, I'm an old fashioned UNIX guy, capability
> without policy. The patch that I submitted for 7.3.2 will allow the user
> to configure PostgreSQL with a configuration file outside the $PGDATA
> directory. That's all I care about. If someone wants to get on the FHS
> bandwagon, that's fine. PostgreSQL should allow that ability but should
> not require it.

If we're going to go through the trouble to change the way things work, we 
might as well try to get something that will allow instalation to match real 
desired configurations out there, like FHS and LSB, or how Oliver and Lamar 
want for packaging without symlinks.  If the goal is just to get something 
that you like, apply the patch locally and be done with it. 

Robert Treat

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T02:06:20Z

Lamar Owen <lamar.owen@wgcr.org> writes:
> On Saturday 15 February 2003 20:19, Tom Lane wrote:
>> Lamar Owen <lamar.owen@wgcr.org> writes:
> Just exactly why does initdb need to drop any config files anywhere?

>> Because we'd like it to edit the correct datadir into the config file,
>> to take just the most obvious example.

> Shouldn't we be consistent and have initdb use the datadir set in the config 
> file, which could be supplied by a ./configure switch?

That'd mean there is no way to perform an initdb into a nonstandard
location without first hand-preparing a config file.  I don't much care
for that.

> I'm looking at a packager point of view here.  The initdb is done well after 
> the package is made, and installed.  It would be ideal from this point of 
> view to have things fully configured pre-initdb (and thus pre-packaging).

This point of view means that no post-configure knowledge can be
applied.  We might as well forget the separate initdb step altogether
and have "make install" do it.

I realize that from a packager's point of view, the separate initdb step
is not very useful.  But it is from my point of view.

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T03:03:56Z

On Saturday 15 February 2003 21:06, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > Shouldn't we be consistent and have initdb use the datadir set in the
> > config file, which could be supplied by a ./configure switch?

> That'd mean there is no way to perform an initdb into a nonstandard
> location without first hand-preparing a config file.  I don't much care
> for that.

Six of one and half-dozen of another.  And that's my real point.  We do things 
quite differently from many other standard services, even those which are 
built from the ground up for multiple instances.  Making things more 
consistent for admins, even if it's not what we are used to or what we might 
like (because it's familiar) should at least be thought about.  I'm not 
advocating changing just for the sake of change; but getting a new fresh look 
at our current setup can't hurt.

> > I'm looking at a packager point of view here.  The initdb is done well
> > after the package is made, and installed.  It would be ideal from this
> > point of view to have things fully configured pre-initdb (and thus
> > pre-packaging).

> This point of view means that no post-configure knowledge can be
> applied.  We might as well forget the separate initdb step altogether
> and have "make install" do it.

I wouldn't complain.  Although that isn't conducive to the multiple instance 
case.  The necessary post-configure knowledge would be in the instance 
postgresql.conf file.  One place for it.  But this is hypothetical; fishing 
around the waters here at this point. Realize that my own packages apply an 
initdb automatically if an install isn't found the first time the system 
initscript is started.  It is virtually automatic.  With the multiple 
postmaster support, creating a couple of files and a symlink (for the 
initscript), and starting the new initscript symlink does all the dirty work.  
But it could be easier.

> I realize that from a packager's point of view, the separate initdb step
> is not very useful.  But it is from my point of view.

Would you mind elucidating which point of view is yours?  General idea of who 
else might have the same point of view, and why you find the initdb in its 
current form to be more useful than alternatives.  Again, I'm fishing for 
knowledge -- if nothing else it gives me an answer to those users who send me 
nastygrams about the way things are right now.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T05:16:44Z

Lamar Owen <lamar.owen@wgcr.org> writes:
>> I realize that from a packager's point of view, the separate initdb step
>> is not very useful.  But it is from my point of view.

> Would you mind elucidating which point of view is yours?

Primarily, one that wants to have multiple postmasters running, of the
same or different versions; including test and temporary installations
that mustn't conflict with the existing primary installation on a machine.

Currently, I don't need to do anything more than set PGDATA or say -D
to initdb in order to set up the data directory wherever I like.  I also
don't need to worry about whether I'm selecting the wrong config file.

You're talking about making manual installations significantly more
difficult (and error-prone, I think) in order to simplify automated
installs.  Now you've acknowledged that your script can do what
you want it to, and in fact already does.  Why is it good to make my
life more difficult to make a script's easier?  Cycles are cheap.
I like to think that my time is worth something.

Nor will I buy an argument that only a few developers have need for test
installations.  Ordinary users will want to do that anytime they are
doing preliminary tests on a new PG version before migrating their
production database to it.  To the extent that you make manual selection
of a nonstandard datadir location more difficult and error-prone, you
are hurting them too.

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T05:31:03Z

On Sunday 16 February 2003 00:16, Tom Lane wrote:
> Lamar Owen <lamar.owen@wgcr.org> writes:
> > Would you mind elucidating which point of view is yours?

> Primarily, one that wants to have multiple postmasters running, of the
> same or different versions; including test and temporary installations
> that mustn't conflict with the existing primary installation on a machine.

Well, due to our upgrading difficulty, having multiple versions running has 
its advantages.

> You're talking about making manual installations significantly more
> difficult (and error-prone, I think) in order to simplify automated
> installs.  Now you've acknowledged that your script can do what
> you want it to, and in fact already does.  Why is it good to make my
> life more difficult to make a script's easier?  Cycles are cheap.
> I like to think that my time is worth something.

The script's been out there for awhile.  It does some things well, and some 
things not so well.  The config files are still coresident with the database, 
and backup is more difficult than it can be.  Meeting all these needs (with 
configure switches, configuration file directives, etc) would be a good 
thing.  And that's what I'm after; maximum usability for the maximum 
audience.  I believe pretty strongly that the usage to which you or I would 
put PostgreSQL is probably quite different from the average user's way of 
using PostgreSQL.  Most probably the typical user has a single installation 
with multiple databases with little need to run isolated postmasters.

> Nor will I buy an argument that only a few developers have need for test
> installations.  Ordinary users will want to do that anytime they are
> doing preliminary tests on a new PG version before migrating their
> production database to it.  To the extent that you make manual selection
> of a nonstandard datadir location more difficult and error-prone, you
> are hurting them too.

While I'm not going to speak for all users, I know that I don't put a 
development database system on my production servers.  The production machine 
only runs production servers, period.  Hardware is cheap.  I have development 
machines for development databases.  One also has the error-prone PATH issues 
with multiple versions, which, if you are running a typical RPM installation 
becomes quite difficult to manage, since the RPM version's executables are in 
/usr/bin.  This could be changed; I've thought about changing it.  But I'm 
not sure of the best way to make multiple versions peacefully and seamlessly 
coexist -- particularly when older versions may not even build on the newer 
OS: but we've been over that discussion.

Care for a poll?
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-16T06:37:07Z

Tom Lane wrote:
> Currently, I don't need to do anything more than set PGDATA or say -D
> to initdb in order to set up the data directory wherever I like.  I also
> don't need to worry about whether I'm selecting the wrong config file.

So in your case, what's the advantage of having initdb write anything
to a config file, when you're probably also relying on PGDATA or -D to
start the database (if you're not, then fair enough.  But see below)?

I'd expect initdb to initialize a database.  If I were running initdb
without a lot of foreknowledge of its side effects, I think I'd
probably be a bit surprised to find that it had touched my config
file.  Initdb doesn't have prior knowledge of how you intend to
start the database so that it refers to the database initdb just
created, so it can't really know whether it's desirable to touch the
config file.

If it's desirable for initdb to be able to write to the config file,
wouldn't it be more appropriate for that to an option that has to be
explicitly enabled on initdb's command line?  I don't know how often
you'd want it to write into the config file for your purposes, but
having it do so automatically seems to violate the principle of least
surprise.

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Peter Eisentraut <peter_e@gmx.net> — 2003-02-16T14:10:05Z

Tom Lane writes:

> AFAICS, you can either set -C to /etc if you want your PG config files
> loose in /etc, or you can set it to /etc/postgresql/ if you want them
> in a privately-owned directory.  Which other arrangements are needed?

People might want to share them between servers, or allow a user to select
from a few pre-configured ones that which reside in the same directory.

-- 
Peter Eisentraut   peter_e@gmx.net

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T17:54:23Z

Peter Eisentraut <peter_e@gmx.net> writes:
> Tom Lane writes:
>> AFAICS, you can either set -C to /etc if you want your PG config files
>> loose in /etc, or you can set it to /etc/postgresql/ if you want them
>> in a privately-owned directory.  Which other arrangements are needed?

> People might want to share them between servers, or allow a user to select
> from a few pre-configured ones that which reside in the same directory.

You can accomplish that without the need to customize the .conf file
names; you just make, eg,

	/etc/postgres/myconfig/postgresql.conf
	/etc/postgres/yourconfig/postgresql.conf
	/etc/postgres/herconfig/postgresql.conf

(plus additional config files as needed in each of these directories)
and then the postmaster start command is

	postmaster -C /etc/postgres/myconfig

I see no real gain in flexibility in allowing people to choose random
names for the individual config files.  Also, it'd defeat the
ultimate-fallback approach of doing "find / -name postgresql.conf"
to figure out where the config files are hiding in an unfamiliar
installation.

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T18:15:45Z

Lamar Owen <lamar.owen@wgcr.org> writes:
> The script's been out there for awhile.  It does some things well, and some 
> things not so well.  The config files are still coresident with the database,
> and backup is more difficult than it can be.  Meeting all these needs (with 
> configure switches, configuration file directives, etc) would be a good 
> thing.

Sure.  I'm happy to change the software in a way that *allows* moving the
config files elsewhere.  But it's not apparent to me why you insist on
forcing people who are perfectly happy with their existing configuration
arrangements to change them.  I have not seen any reason in this
discussion why we can't support both a separate-config-location approach
and the traditional single-location one.

Please remember that the existing approach has been evolved over quite
a few releases.  It may not satisfy the dictates of the FHS religion,
but it does meet some people's needs perfectly well.  Let's look for a
solution that permits coexistence, rather than one that forces change
on people who don't need or want change.

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-16T19:20:38Z

Kevin Brown <kevin@sysexperts.com> writes:
> So in your case, what's the advantage of having initdb write anything
> to a config file, when you're probably also relying on PGDATA or -D to
> start the database (if you're not, then fair enough.  But see below)?

Keep in mind that initdb doesn't currently *need* to put the datadir
location into the config file.  It *will* need to do so if we separate
config and data dirs.  Or at least, *somebody* will need to do so.
It's not apparent to me how it simplifies life not to have initdb do it.
Especially when there are other configuration items that initdb should
or already does record: locale settings, database encoding.  And we
have already been talking about improving PG's self-tuning capability.
initdb would be the natural place to look around for information like
available RAM and adjust the config-file settings like sort_mem
accordingly.

Basically, the notion that initdb shouldn't write a config file seems
like a complete dead end to me.  It cannot possibly be more convenient
than the alternatives.  We'd be giving up a lot of current and future
functionality --- and for what?

> I'd expect initdb to initialize a database.  If I were running initdb
> without a lot of foreknowledge of its side effects, I think I'd
> probably be a bit surprised to find that it had touched my config
> file.

If we do it the way I suggested (dump into the datadir, which is
initially empty, same as always) then it cannot overwrite your existing
config files.  Think of it as providing a suggested config file to
compare against what you have.

			regards, tom lane

Re: location of the configuration files

Lamar Owen <lamar.owen@wgcr.org> — 2003-02-16T23:56:57Z

On Sunday 16 February 2003 13:15, Tom Lane wrote:
> Sure.  I'm happy to change the software in a way that *allows* moving the
> config files elsewhere.

So we agree.  Perfect.

>  But it's not apparent to me why you insist on
> forcing people who are perfectly happy with their existing configuration
> arrangements to change them.

Me? Trying to force things to change?  You misunderstand me.  No, I'm trying 
to understand the rationale for a (relative to the way other 
designed-multiple daemons do things) different, non-standard configuration 
process.  I understand better now; the exercise was a success.  Many thanks.
-- 
Lamar Owen
WGCR Internet Radio
1 Peter 4:11

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-17T02:40:08Z

Tom Lane wrote:

>Peter Eisentraut <peter_e@gmx.net> writes:
>  
>
>>Tom Lane writes:
>>    
>>
>>>I would favor a setup that allows a -C *directory* (not file) to be
>>>specified as a postmaster parameter separately from the -D directory;
>>>      
>>>
>
>  
>
>>A directory is not going to satisfy people.
>>    
>>
>
>Why not?  Who won't it satisfy, and what's their objection?
>
>AFAICS, you can either set -C to /etc if you want your PG config files
>loose in /etc, or you can set it to /etc/postgresql/ if you want them
>in a privately-owned directory.  Which other arrangements are needed?
>
>  
>
The idea of using a "directory" puts us back to using symlinks to share 
files.

While I know the core development teams thinks that symlinks are a 
viable configuration option, most admins, myself included, do not like 
to use symlinks because they do not have the ability to carry 
documentation, i.e. comments in a configuration file, and are DANGEROUS 
in a production environment.

Any configuration strategy that depends on symlinks is inadequate and 
poorly designed.

>  
>

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-17T02:48:47Z

mlw <pgsql@mohawksoft.com> writes:
> The idea of using a "directory" puts us back to using symlinks to share 
> files.

So?  If you want to share files, you're probably sharing all three
config files and don't need a separate directory at all.  This is
not a sufficient argument to make me buy into the mess of letting
people choose nonstandard configuration file names --- especially
when most of the opposite camp seems to be more interested in choosing
*standard* names for things.  Why does that policy stop short at the
directory name?

			regards, tom lane

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-17T03:33:24Z

Tom Lane wrote:
> Keep in mind that initdb doesn't currently *need* to put the datadir
> location into the config file.  It *will* need to do so if we separate
> config and data dirs.  Or at least, *somebody* will need to do so.
> It's not apparent to me how it simplifies life not to have initdb do it.
> Especially when there are other configuration items that initdb should
> or already does record: locale settings, database encoding.  

Is it possible for the database engine to properly deal with a
database when it is told to use a different database encoding than the
one the database was initdb'd with?

If it's not, then that suggests to me that the database encoding is
something that doesn't belong in the configuration file but rather in
some other place that is intimately tied with the database itself and
which is difficult/impossible to change, like perhaps a read-only
system table that gets created at initdb time.

> And we have already been talking about improving PG's self-tuning
> capability.  initdb would be the natural place to look around for
> information like available RAM and adjust the config-file settings
> like sort_mem accordingly.

I agree here, and since you're thinking of just putting the resulting
config file in the database data directory, then as a DBA I wouldn't
be terribly surprised by it ... especially if it came back with a
message that told me what it had done.

> If we do it the way I suggested (dump into the datadir, which is
> initially empty, same as always) then it cannot overwrite your existing
> config files.  Think of it as providing a suggested config file to
> compare against what you have.

There is one minor complication: what if there's an existing config
file in the target directory?

One use for initdb would be as a quick way to completely wipe the
database and start over (e.g., if the encoding were found to be
incorrect), but the config file that's already there could easily
contain a lot of customization that the administrator would want to
retain.  Which suggests that we should consider writing to a file
using a slightly different name (e.g., postgresql.conf.initdb), at
least in the event that a config file already exists in the target
directory.

Not sure what the overall right thing to do here is...

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Mark Woodward <pgsql@mohawksoft.com> — 2003-02-17T03:41:37Z

Tom Lane wrote:

>mlw <pgsql@mohawksoft.com> writes:
>  
>
>>The idea of using a "directory" puts us back to using symlinks to share 
>>files.
>>    
>>
>
>So?  If you want to share files, you're probably sharing all three
>config files and don't need a separate directory at all.  This is
>not a sufficient argument to make me buy into the mess of letting
>people choose nonstandard configuration file names --- especially
>when most of the opposite camp seems to be more interested in choosing
>*standard* names for things.  Why does that policy stop short at the
>directory name?
>  
>
symlinks suck. Sorry Tom, but they are *BAD* in a production server. You 
can not add comments to symlinks. Most of the admins I know, myself 
included, HATE symlinks and use them as a last resort. Requiring 
symlinks is just pointless, we are talking about a few lines of code hat 
has nothing to do with performance.

The patch that I submitted allows PostgreSQL to work as it always has, 
but adds the ability for a configuration file to do what is normally 
done with fixed names in $PGDATA.

I have said before, I do not like policy, I like flexibility, forcing a 
directory is similarly restricting as requiring the files in $PGDATA.

Why is this such a problem? MANY people want to configure PostgreSQL 
this way, but the patch I submitted allows it, but does not force 
anything. Any configuration solution that requires symlinks is flawed.

>  
>

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-17T03:55:49Z

mlw wrote:
> symlinks suck. Sorry Tom, but they are *BAD* in a production
> server. 

Well, at least they're better than hard links.  ;-)


-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-17T18:39:07Z

Kevin Brown <kevin@sysexperts.com> writes:
> Is it possible for the database engine to properly deal with a
> database when it is told to use a different database encoding than the
> one the database was initdb'd with?

It can't be "told to use a different database encoding".  However, the
default *client* encoding matches the database encoding, and that is
something that can be set in the config file.

>> If we do it the way I suggested (dump into the datadir, which is
>> initially empty, same as always) then it cannot overwrite your existing
>> config files.  Think of it as providing a suggested config file to
>> compare against what you have.

> There is one minor complication: what if there's an existing config
> file in the target directory?

If there's anything at all in the target directory, initdb refuses to
run.

			regards, tom lane

Re: location of the configuration files

Andrew Sullivan <andrew@libertyrms.info> — 2003-02-17T18:51:16Z

On Fri, Feb 14, 2003 at 10:35:41AM -0500, Tom Lane wrote:

> FHS or no FHS, I would think that the preferred arrangement would be to
> keep Postgres' config files in a postgres-owned subdirectory, not
> directly in /etc.  That way you need not be root to edit them.  (My idea

Besides, what are you going to do for people installing on a box
where they don't have root?  Are they going to need a whole mess of
extra directories in their private copy?

> of an editor, Emacs, always wants to write a backup file, so I dislike
> having to edit files that live in directories I can't write.)
> 
> Here's a pretty topic for a flamewar: should it be /etc/postgres/ or
> /etc/postgresql/ ?

Wow, two flamewar topics in one mail.  I'm impressed.

Andrew "ed is the one true editor" Sullivan

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

Re: location of the configuration files

Andrew Sullivan <andrew@libertyrms.info> — 2003-02-17T19:03:31Z

On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote:
> Nor will I buy an argument that only a few developers have need for test
> installations.  Ordinary users will want to do that anytime they are
> doing preliminary tests on a new PG version before migrating their
> production database to it.  To the extent that you make manual selection
> of a nonstandard datadir location more difficult and error-prone, you
> are hurting them too.

Not only that.  For safety's sake, you may need to run multiple
postmasters on one machine (so that database user X can't DoS
database user Y, for instance).  And making that sort of
production-grade work more difficult and error-prone would also be
bad.

A

-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110

Re: location of the configuration files

scott.marlowe <scott.marlowe@ihs.com> — 2003-02-18T17:00:38Z

On Sat, 15 Feb 2003, Curt Sampson wrote:

> On Fri, 14 Feb 2003, scott.marlowe wrote:
> 
> > Asking for everything in a directory with the name local in it to be
> > shared is kind of counter intuitive to me.
> 
> Not really. If you install a particular program that doesn't come with
> the OS on one machine on your site, why would you not want to install it
> separately on all of the others?
> 
> Typically, I want my favourite non-OS utilities on all machines, not
> just one. (Even if I don't use them on all machines.) Thus /usr/local is
> for site-local stuff.

Good point.  Of course, in apache, it's quite easy to use the -f switch to 
pick the file you're running on.  so, with a 

httpd -f /usr/local/apache/conf/`uname -a|cut -d " " -f 2`.conf

I can pick and choose the file to run.  So, yes, I would gladly use it in 
a cluster, and all the files would be in one place, easy to backup.

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-19T02:55:09Z

I have a new idea.  You know how we have search_path where you can
specify multiple schema names.  What if we allow the config_dirs/-C to
specify multiple directories to search for config files.  That way, we
can use only one variable, and we can allow people to place different
config files in different directories.

---------------------------------------------------------------------------

Andrew Sullivan wrote:
> On Sun, Feb 16, 2003 at 12:16:44AM -0500, Tom Lane wrote:
> > Nor will I buy an argument that only a few developers have need for test
> > installations.  Ordinary users will want to do that anytime they are
> > doing preliminary tests on a new PG version before migrating their
> > production database to it.  To the extent that you make manual selection
> > of a nonstandard data_dir location more difficult and error-prone, you
> > are hurting them too.
> 
> Not only that.  For safety's sake, you may need to run multiple
> postmasters on one machine (so that database user X can't DoS
> database user Y, for instance).  And making that sort of
> production-grade work more difficult and error-prone would also be
> bad.
> 
> A
> 
> -- 
> ----
> Andrew Sullivan                         204-4141 Yonge Street
> Liberty RMS                           Toronto, Ontario Canada
> <andrew@libertyrms.info>                              M2P 2A8
>                                          +1 416 646 3304 x110
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 2: you can get off all lists at once with the unregister command
>     (send "unregister YourEmailAddressHere" to majordomo@postgresql.org)
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Kevin Brown <kevin@sysexperts.com> — 2003-02-19T12:09:52Z

Bruce Momjian wrote:
> 
> I have a new idea.  You know how we have search_path where you can
> specify multiple schema names.  What if we allow the config_dirs/-C to
> specify multiple directories to search for config files.  That way, we
> can use only one variable, and we can allow people to place different
> config files in different directories.

That's an interesting idea.  Were you thinking, perhaps, that you
could put, say, a postgresql.conf file in multiple directories and
have the settings in the latest one override the settings in earlier
ones?  That would mean you could set up a single postgresql.conf that
has settings common to all of your instances (settings related to the
system such as shared buffers, and default settings that would apply
to any instance if not overridden), and a postgresql.conf file for
each instance that defines the instance-specific configuration
information.

I'm not sure that's quite what you had in mind, though.  :-)

-- 
Kevin Brown					      kevin@sysexperts.com

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-19T15:35:56Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I have a new idea.  You know how we have search_path where you can
> specify multiple schema names.  What if we allow the config_dirs/-C to
> specify multiple directories to search for config files.  That way, we
> can use only one variable, and we can allow people to place different
> config files in different directories.

Hm, a search path for config files?  I could support that if it
satisfies the folks who object to specifying config directories
rather than file names.

One thing that would have to be thought about is whether to re-search
the path on each config file reload --- if you first find pg_hba.conf
in, say, the third directory on the path, should you pay attention if
one materializes in the second directory later?  Or do you keep going
back to the same well?  I can see arguments either way.

			regards, tom lane

Re: location of the configuration files

Andrew Dunstan <andrew@dunslane.net> — 2003-02-19T15:58:20Z

The problem I have with Bruce's scheme is that you could put your config
file where you want it and someone else puts one somewhere higher in the
search path and you have no idea what went wrong. It sounds to me like a
recipe for an SA's nightmare. Other people have claimed to speak from the SA
perspective - do they see this too?

andrew

----- Original Message -----
From: "Tom Lane" <tgl@sss.pgh.pa.us>
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I have a new idea.  You know how we have search_path where you can
> > specify multiple schema names.  What if we allow the config_dirs/-C to
> > specify multiple directories to search for config files.  That way, we
> > can use only one variable, and we can allow people to place different
> > config files in different directories.
>
> Hm, a search path for config files?  I could support that if it
> satisfies the folks who object to specifying config directories
> rather than file names.
>
> One thing that would have to be thought about is whether to re-search
> the path on each config file reload --- if you first find pg_hba.conf
> in, say, the third directory on the path, should you pay attention if
> one materializes in the second directory later?  Or do you keep going
> back to the same well?  I can see arguments either way.

Re: location of the configuration files

Bruce Momjian <pgman@candle.pha.pa.us> — 2003-02-19T15:58:22Z

Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I have a new idea.  You know how we have search_path where you can
> > specify multiple schema names.  What if we allow the config_dirs/-C to
> > specify multiple directories to search for config files.  That way, we
> > can use only one variable, and we can allow people to place different
> > config files in different directories.
> 
> Hm, a search path for config files?  I could support that if it
> satisfies the folks who object to specifying config directories
> rather than file names.
> 
> One thing that would have to be thought about is whether to re-search
> the path on each config file reload --- if you first find pg_hba.conf
> in, say, the third directory on the path, should you pay attention if
> one materializes in the second directory later?  Or do you keep going
> back to the same well?  I can see arguments either way.

Oh, I hadn't thought of that.  I would vote for researching the path,
but I am not sure why.

The bigger question is whether you can modify config_dirs while the
postmaster is running.  I would think not.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-19T16:02:06Z

Bruce Momjian <pgman@candle.pha.pa.us> writes:
> The bigger question is whether you can modify config_dirs while the
> postmaster is running.  I would think not.

There would be no way to do that, because the only way to set it would
be from -C on the command line or a PGCONFIG environment variable.
But I can't see a real good reason why you'd need to.

			regards, tom lane

Re: location of the configuration files

Tom Lane <tgl@sss.pgh.pa.us> — 2003-02-19T16:40:20Z

"Andrew Dunstan" <andrew@dunslane.net> writes:
> The problem I have with Bruce's scheme is that you could put your config
> file where you want it and someone else puts one somewhere higher in the
> search path and you have no idea what went wrong. It sounds to me like a
> recipe for an SA's nightmare. Other people have claimed to speak from the SA
> perspective - do they see this too?

If you have "your" file you'd put it in the directory at the front of
the search path.  End of problem.  Any additional directories would be
for config files that you *want* to share.

Offhand I find it hard to visualize needing more than two directories in
this path (private and shared), unless we grow to having many more
config files than we do now.  But we may as well build the feature with
no artificial restriction about path length.

Search path management seems well understood for $PATH --- people do
get burnt by having the wrong $PATH, but it doesn't qualify as a
nightmare...

			regards, tom lane