Thread

1. Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Amol Inamdar <amol.aai@gmail.com> — 2025-07-14T05:49:55Z

   Dear PostgreSQL Community,
   
   I'm currently running PostgreSQL version 16.6 inside a Docker container
   (base image: UBI 9), using Docker Compose. The PostgreSQL data directory
   is mounted from an NFS volume hosted on a z/OS NFS server.
   
   The environment has a few constraints:
   
   - The NFS server runs on z/OS with AT-TLS enabled.
   - It’s a highly secure and access-controlled setup.
   - Due to platform restrictions on z/OS, the mounted NFS directory cannot
     be owned by the PostgreSQL user (e.g., `postgres`) inside the container.
   - As a result, PostgreSQL fails to start because of the directory
     ownership validation check.
   
   Given the secure nature of the NFS server, I’d like to ask:
   
   1. Is there a supported or recommended way to bypass the ownership
      check on the data directory?
   2. What are the potential risks or implications of doing so in a secure
      NFS environment?
   3. I'm considering building a custom PostgreSQL image by modifying the
      `miscinit.c` file—specifically, disabling the ownership check in the
      `checkDataDir()` function. Is this a reasonable approach, and are
      there any caveats or unintended side effects I should be aware of?
   
   **Disclaimer**: The z/OS NFS server is secured using AT-TLS and enforces
   strict access control policies. My intention is not to weaken
   PostgreSQL’s security model, but to adapt to platform-specific
   constraints while maintaining overall security integrity.
   
   Any insights, experiences, or alternative suggestions would be greatly
   appreciated.
   
   Best regards,
   Amol
   

2. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Laurenz Albe <laurenz.albe@cybertec.at> — 2025-07-14T12:20:14Z

   On Mon, 2025-07-14 at 11:19 +0530, Amol Inamdar wrote:
   > I'm currently running PostgreSQL version 16.6 inside a Docker container
   > (base image: UBI 9), using Docker Compose. The PostgreSQL data directory
   > is mounted from an NFS volume hosted on a z/OS NFS server.
   > 
   > The environment has a few constraints:
   > 
   > - It’s a highly secure and access-controlled setup.
   > - Due to platform restrictions on z/OS, the mounted NFS directory cannot
   >   be owned by the PostgreSQL user (e.g., `postgres`) inside the container.
   > - As a result, PostgreSQL fails to start because of the directory
   >   ownership validation check.
   
   It is not a good idea to have a mount point be the data directory.
   The proper solution is to create the data directory inside the
   mount point.  That way, the permissions of the data directory don't
   have to be the same as the permissions of the mount point.
   
   Yours,
   Laurenz Albe
   
   
   
   

3. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Amol Inamdar <amol.aai@gmail.com> — 2025-07-14T12:29:12Z

   Hi Laurenz
   
   Thanks for the reply,
   
   If I am not mistaken, below is my understanding of your suggestion.
   
   Suppose that My mount point on the NFS server is say /nfs-mount/postgres/
   and you are suggesting to have a data directory as say
   /nfs-mount/postgres/db or something like that ?
   and assign this value to the PGDATA ?
   
   If that is the case, then when and who should be creating the directory DB
   ?
   
   Please correct me if I am wrong about the understanding.
   
   Thanks,
   Amol
   
   
   
   On Mon, Jul 14, 2025 at 5:50 PM Laurenz Albe <laurenz.albe@cybertec.at>
   wrote:
   
   > On Mon, 2025-07-14 at 11:19 +0530, Amol Inamdar wrote:
   > > I'm currently running PostgreSQL version 16.6 inside a Docker container
   > > (base image: UBI 9), using Docker Compose. The PostgreSQL data directory
   > > is mounted from an NFS volume hosted on a z/OS NFS server.
   > >
   > > The environment has a few constraints:
   > >
   > > - It’s a highly secure and access-controlled setup.
   > > - Due to platform restrictions on z/OS, the mounted NFS directory cannot
   > >   be owned by the PostgreSQL user (e.g., `postgres`) inside the
   > container.
   > > - As a result, PostgreSQL fails to start because of the directory
   > >   ownership validation check.
   >
   > It is not a good idea to have a mount point be the data directory.
   > The proper solution is to create the data directory inside the
   > mount point.  That way, the permissions of the data directory don't
   > have to be the same as the permissions of the mount point.
   >
   > Yours,
   > Laurenz Albe
   >
   
   
   -- 
   -regards
   Amol
   

4. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Laurenz Albe <laurenz.albe@cybertec.at> — 2025-07-14T12:44:19Z

   On Mon, 2025-07-14 at 17:59 +0530, Amol Inamdar wrote:
   > If I am not mistaken, below is my understanding of your suggestion. 
   > 
   > Suppose that My mount point on the NFS server is say /nfs-mount/postgres/ 
   > and you are suggesting to have a data directory as say /nfs-mount/postgres/db or something like that ? 
   > and assign this value to the PGDATA ? 
   > 
   > If that is the case, then when and who should be creating the directory DB ? 
   > 
   > Please correct me if I am wrong about the understanding.
   
   You understood me perfectly well.
   
   The data directory can either be created by "initdb", in which case
   the mount point must allow the PostgreSQL user to create a directory.
   You could set the group of the mount point to the group of the
   PostgreSQL user and use permissions 1770, which should be perfectly safe.
   
   Alternatively, the root user could create the data directory with the
   correct ownership and permissions prior to running "initdb".
   
   Yours,
   Laurenz Albe
   
   
   
   

5. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Amol Inamdar <amol.aai@gmail.com> — 2025-07-14T13:02:23Z

   Thanks Laurenz,
   
   The data directory can either be created by "initdb", in which case
   the mount point must allow the PostgreSQL user to create a directory.
   You could set the group of the mount point to the group of the
   PostgreSQL user and use permissions 1770, which should be perfectly safe.
   
   This exactly is the problem we are facing, to give you a summary,
   our NFS server is enabled with AT-TLS authentication
   and we are accessing the server via a proxy server (Haproxy).
   This acts as our NFS client and it is configured with the
   required client certificates.
   
   The outcome of above configuration is that any directory created
   in the NFS mount is always owned by the user in the certificates
   and if that user isn't present in the proxy container it is marked
   as nobody:nogroup, we tried various things like
   created the user similar to postgres user so that the users ids match but
   always ended up giving error  “data directory “/var/lib” has wrong ownership
   
   
   Hence, we thought of skipping this check (Directory owner and postgres user
   validation) and
   wanted to understand the implication of the same.
   
   Thanks,
   Amol,
   
   
   
   On Mon, Jul 14, 2025 at 6:14 PM Laurenz Albe <laurenz.albe@cybertec.at>
   wrote:
   
   > On Mon, 2025-07-14 at 17:59 +0530, Amol Inamdar wrote:
   > > If I am not mistaken, below is my understanding of your suggestion.
   > >
   > > Suppose that My mount point on the NFS server is say
   > /nfs-mount/postgres/
   > > and you are suggesting to have a data directory as say
   > /nfs-mount/postgres/db or something like that ?
   > > and assign this value to the PGDATA ?
   > >
   > > If that is the case, then when and who should be creating the directory
   > DB ?
   > >
   > > Please correct me if I am wrong about the understanding.
   >
   > You understood me perfectly well.
   >
   > The data directory can either be created by "initdb", in which case
   > the mount point must allow the PostgreSQL user to create a directory.
   > You could set the group of the mount point to the group of the
   > PostgreSQL user and use permissions 1770, which should be perfectly safe.
   >
   > Alternatively, the root user could create the data directory with the
   > correct ownership and permissions prior to running "initdb".
   >
   > Yours,
   > Laurenz Albe
   >
   
   
   -- 
   -regards
   Amol
   

6. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-14T14:07:20Z

   Laurenz Albe <laurenz.albe@cybertec.at> writes:
   > It is not a good idea to have a mount point be the data directory.
   
   ^^^ This. ^^^
   
   That is primarily for safety reasons: if for some reason the
   filesystem gets dismounted, or hasn't come on-line yet during
   a reboot, you do not want Postgres to be able to write on the
   underlying mount-point directory.  There is a sobering tale
   in this old thread:
   
   https://www.postgresql.org/message-id/flat/41BFAB7C.5040108%40joeconway.com
   
   Now it didn't help any that they were using a start script that
   would automatically run initdb if it didn't see a data directory
   where expected.  But even without that, you are in for a world of
   hurt if the mount drops while the server is running and the server
   has any ability to write on the underlying storage; it will think
   whatever it was able to write is safely down on disk.  To prevent
   that, the server must not have write permissions on the mount
   point, which dictates making a separate data directory (with
   different ownership/permissions) just below the mount.
   
   Do not bypass that ownership/permissions check.  It is there
   for very good reasons.
   
   			regards, tom lane
   
   
   
   

7. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Laurenz Albe <laurenz.albe@cybertec.at> — 2025-07-14T14:41:26Z

   On Mon, 2025-07-14 at 18:32 +0530, Amol Inamdar wrote:
   > > The data directory can either be created by "initdb", in which case
   > > the mount point must allow the PostgreSQL user to create a directory.
   > > You could set the group of the mount point to the group of the
   > > PostgreSQL user and use permissions 1770, which should be perfectly safe.
   > 
   > This exactly is the problem we are facing, to give you a summary, 
   > our NFS server is enabled with AT-TLS authentication
   > and we are accessing the server via a proxy server (Haproxy). 
   > This acts as our NFS client and it is configured with the 
   > required client certificates.
   > 
   > The outcome of above configuration is that any directory created 
   > in the NFS mount is always owned by the user in the certificates 
   > and if that user isn't present in the proxy container it is marked 
   > as nobody:nogroup, we tried various things like
   > created the user similar to postgres user so that the users ids match but 
   > always ended up giving error  “data directory “/var/lib” has wrong ownership 
   > 
   > Hence, we thought of skipping this check (Directory owner and postgres user validation) and 
   > wanted to understand the implication of the same.
   
   No; don't.
   
   Simply mount the directory once, create a subdirectory with the
   appropriate ownership and permissions, and there you go.
   Problem solved.
   
   Yours,
   Laurenz Albe
   
   
   
   

8. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Peter J. Holzer <hjp-pgsql@hjp.at> — 2025-07-14T18:20:16Z

   On 2025-07-14 10:07:20 -0400, Tom Lane wrote:
   > Laurenz Albe <laurenz.albe@cybertec.at> writes:
   > > It is not a good idea to have a mount point be the data directory.
   > 
   > ^^^ This. ^^^
   > 
   > That is primarily for safety reasons: if for some reason the
   > filesystem gets dismounted, or hasn't come on-line yet during
   > a reboot, you do not want Postgres to be able to write on the
   > underlying mount-point directory.  There is a sobering tale
   > in this old thread:
   > 
   > https://www.postgresql.org/message-id/flat/41BFAB7C.5040108%40joeconway.com
   > 
   > Now it didn't help any that they were using a start script that
   > would automatically run initdb if it didn't see a data directory
   > where expected.  But even without that, you are in for a world of
   > hurt if the mount drops while the server is running and the server
   > has any ability to write on the underlying storage; it will think
   > whatever it was able to write is safely down on disk.  To prevent
   > that, the server must not have write permissions on the mount
   > point, which dictates making a separate data directory (with
   > different ownership/permissions) just below the mount.
   
   Be careful: There are two different directorys involved in a mount
   point. The one in the parent filesystem and the one in the mounted file
   system.
   
   E.g.:
   
   # mkdir /mnt/demo
   # stat /mnt/demo
     File: /mnt/demo
     Size: 4096      	Blocks: 8          IO Block: 4096   directory
   Device: 254,0	Inode: 317         Links: 2
   Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
   
   So here we have inode 317 on device 254,0 and unsurprisingly it belongs
   to root and is only writable by root.
   
   Now let's mount a filesystem:
   
   # mount /dev/vgroot/demo /mnt/demo
   # stat /mnt/demo
     File: /mnt/demo
     Size: 4096      	Blocks: 8          IO Block: 4096   directory
   Device: 254,13	Inode: 2           Links: 3
   Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
   
   This is now inode 2 on device 254,13 (i.e. the root directory on
   /dev/vgroot/demo). Since I just created that it also belongs to root.
   Let's change that:
   
   # chown postgres: /mnt/demo
   # stat /mnt/demo
     File: /mnt/demo
     Size: 4096      	Blocks: 8          IO Block: 4096   directory
   Device: 254,13	Inode: 2           Links: 3
   Access: (0755/drwxr-xr-x)  Uid: (  108/postgres)   Gid: (  114/postgres)
   
   Ok, now it belongs to postgres and postgres could create files,
   subdirectories, etc.
   
   But when I unmount it:
   
   # umount /mnt/demo
   # stat /mnt/demo
     File: /mnt/demo
     Size: 4096      	Blocks: 8          IO Block: 4096   directory
   Device: 254,0	Inode: 317         Links: 2
   Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
   
   Lo and behold: The original Inode is back and unchanged.
   
   The user postgres couldn't write to that directory if it tried to.
   
   That said, there are of course some remaining failure modes:
   
   1) An admin might notice the wrong permissions and "fix" them without
      investigating the cause
   2) Some automated procedure (running as root) might "fix" the
      permissions at startup (like the initdb in the example) or during an
      upgrade.
   3) use your imagination ;-).
   
           hjp
   
   -- 
      _  | Peter J. Holzer    | Story must make more sense than reality.
   |_|_) |                    |
   | |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
   __/   | http://www.hjp.at/ |       challenge!"
   

9. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

   Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-14T18:30:45Z

   "Peter J. Holzer" <hjp-pgsql@hjp.at> writes:
   > On 2025-07-14 10:07:20 -0400, Tom Lane wrote:
   >> That is primarily for safety reasons: if for some reason the
   >> filesystem gets dismounted, or hasn't come on-line yet during
   >> a reboot, you do not want Postgres to be able to write on the
   >> underlying mount-point directory.
   
   > Be careful: There are two different directorys involved in a mount
   > point. The one in the parent filesystem and the one in the mounted file
   > system.
   
   True, and the safety requirement really is only that the parent
   filesystem's mount-point directory not be writable by us.
   But normal practice is that both directories are root-owned,
   or at least owned by highly privileged users.
   
   (I have a vague idea that there are system-level security hazards,
   not specific to Postgres, if mount-point directories are publicly
   writable.  Don't feel like researching that though.)
   
   			regards, tom lane
   
   
   
   

10. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Benjamin Wang <benjamin.ahrtr@gmail.com> — 2025-07-14T19:02:27Z

    I am not sure whether PostgreSQL depends on system call `fsyncdata` to
     sync data to disk. If yes, then I don't think it's safe to use NFS.
    When `fsyncdata` returns success, it doesn't mean the data has
    really been synced to disk. But if PostgreSQL crashes right after
    it returns success to clients. Eventually it breaks the D (Durability) of
    ACID.
    
    Benjamin
    
    On Mon, Jul 14, 2025 at 7:31 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
    
    > "Peter J. Holzer" <hjp-pgsql@hjp.at> writes:
    > > On 2025-07-14 10:07:20 -0400, Tom Lane wrote:
    > >> That is primarily for safety reasons: if for some reason the
    > >> filesystem gets dismounted, or hasn't come on-line yet during
    > >> a reboot, you do not want Postgres to be able to write on the
    > >> underlying mount-point directory.
    >
    > > Be careful: There are two different directorys involved in a mount
    > > point. The one in the parent filesystem and the one in the mounted file
    > > system.
    >
    > True, and the safety requirement really is only that the parent
    > filesystem's mount-point directory not be writable by us.
    > But normal practice is that both directories are root-owned,
    > or at least owned by highly privileged users.
    >
    > (I have a vague idea that there are system-level security hazards,
    > not specific to Postgres, if mount-point directories are publicly
    > writable.  Don't feel like researching that though.)
    >
    >                         regards, tom lane
    >
    >
    >
    

11. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    David G. Johnston <david.g.johnston@gmail.com> — 2025-07-14T19:08:26Z

    On Mon, Jul 14, 2025 at 12:02 PM Benjamin Wang <benjamin.ahrtr@gmail.com>
    wrote:
    
    > I am not sure whether PostgreSQL depends on system call `fsyncdata` to
    >  sync data to disk.
    >
    
    https://www.postgresql.org/docs/current/runtime-config-wal.html#GUC-WAL-SYNC-METHOD
    
    David J.
    

12. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-14T19:10:45Z

    Benjamin Wang <benjamin.ahrtr@gmail.com> writes:
    > I am not sure whether PostgreSQL depends on system call `fsyncdata` to
    >  sync data to disk. If yes, then I don't think it's safe to use NFS.
    
    Well, that's a whole other discussion.  The point about mount
    directories applies to any sort of dismountable storage.
    
    (But I too would not use Postgres-over-NFS for any critical data.
    Too many moving parts.  It's tough enough to ensure crash safety
    with local storage.)
    
    			regards, tom lane
    
    
    
    

13. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Laurenz Albe <laurenz.albe@cybertec.at> — 2025-07-15T06:02:06Z

    On Mon, 2025-07-14 at 14:30 -0400, Tom Lane wrote:
    > (I have a vague idea that there are system-level security hazards,
    > not specific to Postgres, if mount-point directories are publicly
    > writable.  Don't feel like researching that though.)
    
    Well, if you are using an ext? file system, there is a lost+found
    directory where fsck places links to orphaned inodes.
    If the PostgreSQL user owns the mount point and wants to use
    "initdb" to create a data directory in it, the program will fail
    and complain that the directory is not empty.  The danger is great
    that the user removes the lost+found directory to solve the problem.
    
    True, one could re-create it with "mklost+found", but if a DBA
    is uneducated enough to remove the directory in the first place,
    the risk is high that he wouldn't think of creating it again,
    which is a problem if the file system ever becomes corrupted.
    
    All this doesn't apply to NFS, but it is yet another reason
    (apart from the safety of a subdirectory that doesn't exist
    on the file system underlying the mount point) why we should
    continue to recommend that the data directory be not a mount point.
    
    Yours,
    Laurenz Albe
    
    
    
    

14. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Amol Inamdar <amol.aai@gmail.com> — 2025-07-15T11:35:04Z

    Thanks Laurenz.
    
    On Mon, Jul 14, 2025 at 8:11 PM Laurenz Albe <laurenz.albe@cybertec.at>
    wrote:
    
    > On Mon, 2025-07-14 at 18:32 +0530, Amol Inamdar wrote:
    > > > The data directory can either be created by "initdb", in which case
    > > > the mount point must allow the PostgreSQL user to create a directory.
    > > > You could set the group of the mount point to the group of the
    > > > PostgreSQL user and use permissions 1770, which should be perfectly
    > safe.
    > >
    > > This exactly is the problem we are facing, to give you a summary,
    > > our NFS server is enabled with AT-TLS authentication
    > > and we are accessing the server via a proxy server (Haproxy).
    > > This acts as our NFS client and it is configured with the
    > > required client certificates.
    > >
    > > The outcome of above configuration is that any directory created
    > > in the NFS mount is always owned by the user in the certificates
    > > and if that user isn't present in the proxy container it is marked
    > > as nobody:nogroup, we tried various things like
    > > created the user similar to postgres user so that the users ids match
    > but
    > > always ended up giving error  “data directory “/var/lib” has wrong
    > ownership
    > >
    > > Hence, we thought of skipping this check (Directory owner and postgres
    > user validation) and
    > > wanted to understand the implication of the same.
    >
    > No; don't.
    >
    > Simply mount the directory once, create a subdirectory with the
    > appropriate ownership and permissions, and there you go.
    > Problem solved.
    >
    > Yours,
    > Laurenz Albe
    >
    
    
    -- 
    -regards
    Amol
    

15. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Amol Inamdar <amol.aai@gmail.com> — 2025-07-15T11:36:29Z

    Thanks Tom and Laurenz for the explanation.
    Let me try out a few things and get back to you if needed.
    
    Thanks,
    Amol
    
    On Mon, Jul 14, 2025 at 7:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
    
    > Laurenz Albe <laurenz.albe@cybertec.at> writes:
    > > It is not a good idea to have a mount point be the data directory.
    >
    > ^^^ This. ^^^
    >
    > That is primarily for safety reasons: if for some reason the
    > filesystem gets dismounted, or hasn't come on-line yet during
    > a reboot, you do not want Postgres to be able to write on the
    > underlying mount-point directory.  There is a sobering tale
    > in this old thread:
    >
    > https://www.postgresql.org/message-id/flat/41BFAB7C.5040108%40joeconway.com
    >
    > Now it didn't help any that they were using a start script that
    > would automatically run initdb if it didn't see a data directory
    > where expected.  But even without that, you are in for a world of
    > hurt if the mount drops while the server is running and the server
    > has any ability to write on the underlying storage; it will think
    > whatever it was able to write is safely down on disk.  To prevent
    > that, the server must not have write permissions on the mount
    > point, which dictates making a separate data directory (with
    > different ownership/permissions) just below the mount.
    >
    > Do not bypass that ownership/permissions check.  It is there
    > for very good reasons.
    >
    >                         regards, tom lane
    >
    
    
    -- 
    -regards
    Amol
    

16. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Amol Inamdar <amol.aai@gmail.com> — 2025-07-16T13:24:50Z

    Hi All,
    
    I would like to rephrase the question a little bit, below is how our setup
    going to be
    
       1. NFS mount point is for /nfs-mount/postgres (and permissions locked
       down so that Postgres cannot create directories in here)
       2. Postgres data directory is /nfs-mount/postgres/db
       3.
    
       With secured NFS + AT-TLS setup Postgres will be able to write to data
       directory but not parent dir, however the file ownership information
       Postgres sees from the stat() call will not match the Postgres user in the
       container (even though the AT-TLS strict access control will ensure only
       the Posgres user can read/write to this directory)
    
    Considering the above scenario/setup, what is the danger of removing the
    ownership check in miscinit.c checkDataDir() function ?
    
    
    On Tue, Jul 15, 2025 at 5:06 PM Amol Inamdar <amol.aai@gmail.com> wrote:
    
    > Thanks Tom and Laurenz for the explanation.
    > Let me try out a few things and get back to you if needed.
    >
    > Thanks,
    > Amol
    >
    > On Mon, Jul 14, 2025 at 7:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
    >
    >> Laurenz Albe <laurenz.albe@cybertec.at> writes:
    >> > It is not a good idea to have a mount point be the data directory.
    >>
    >> ^^^ This. ^^^
    >>
    >> That is primarily for safety reasons: if for some reason the
    >> filesystem gets dismounted, or hasn't come on-line yet during
    >> a reboot, you do not want Postgres to be able to write on the
    >> underlying mount-point directory.  There is a sobering tale
    >> in this old thread:
    >>
    >>
    >> https://www.postgresql.org/message-id/flat/41BFAB7C.5040108%40joeconway.com
    >>
    >> Now it didn't help any that they were using a start script that
    >> would automatically run initdb if it didn't see a data directory
    >> where expected.  But even without that, you are in for a world of
    >> hurt if the mount drops while the server is running and the server
    >> has any ability to write on the underlying storage; it will think
    >> whatever it was able to write is safely down on disk.  To prevent
    >> that, the server must not have write permissions on the mount
    >> point, which dictates making a separate data directory (with
    >> different ownership/permissions) just below the mount.
    >>
    >> Do not bypass that ownership/permissions check.  It is there
    >> for very good reasons.
    >>
    >>                         regards, tom lane
    >>
    >
    >
    > --
    > -regards
    > Amol
    >
    
    
    -- 
    -regards
    Amol
    

17. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Ron Johnson <ronljohnsonjr@gmail.com> — 2025-07-16T14:17:43Z

    Quoting Tom's earlier email:
    "(But I too *would not use Postgres-over-NFS for any critical data*.
    Too many moving parts.  It's tough enough to ensure crash safety
    with local storage.)"
    
    You're going through a lot of security effort to implement a Worst Practice.
    
    On Wed, Jul 16, 2025 at 9:25 AM Amol Inamdar <amol.aai@gmail.com> wrote:
    
    > Hi All,
    >
    > I would like to rephrase the question a little bit, below is how our setup
    > going to be
    >
    >    1. NFS mount point is for /nfs-mount/postgres (and permissions locked
    >    down so that Postgres cannot create directories in here)
    >    2. Postgres data directory is /nfs-mount/postgres/db
    >    3.
    >
    >    With secured NFS + AT-TLS setup Postgres will be able to write to data
    >    directory but not parent dir, however the file ownership information
    >    Postgres sees from the stat() call will not match the Postgres user in the
    >    container (even though the AT-TLS strict access control will ensure only
    >    the Posgres user can read/write to this directory)
    >
    > Considering the above scenario/setup, what is the danger of removing the
    > ownership check in miscinit.c checkDataDir() function ?
    >
    >
    > On Tue, Jul 15, 2025 at 5:06 PM Amol Inamdar <amol.aai@gmail.com> wrote:
    >
    >> Thanks Tom and Laurenz for the explanation.
    >> Let me try out a few things and get back to you if needed.
    >>
    >> Thanks,
    >> Amol
    >>
    >> On Mon, Jul 14, 2025 at 7:37 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
    >>
    >>> Laurenz Albe <laurenz.albe@cybertec.at> writes:
    >>> > It is not a good idea to have a mount point be the data directory.
    >>>
    >>> ^^^ This. ^^^
    >>>
    >>> That is primarily for safety reasons: if for some reason the
    >>> filesystem gets dismounted, or hasn't come on-line yet during
    >>> a reboot, you do not want Postgres to be able to write on the
    >>> underlying mount-point directory.  There is a sobering tale
    >>> in this old thread:
    >>>
    >>>
    >>> https://www.postgresql.org/message-id/flat/41BFAB7C.5040108%40joeconway.com
    >>>
    >>> Now it didn't help any that they were using a start script that
    >>> would automatically run initdb if it didn't see a data directory
    >>> where expected.  But even without that, you are in for a world of
    >>> hurt if the mount drops while the server is running and the server
    >>> has any ability to write on the underlying storage; it will think
    >>> whatever it was able to write is safely down on disk.  To prevent
    >>> that, the server must not have write permissions on the mount
    >>> point, which dictates making a separate data directory (with
    >>> different ownership/permissions) just below the mount.
    >>>
    >>> Do not bypass that ownership/permissions check.  It is there
    >>> for very good reasons.
    >>>
    >>>                         regards, tom lane
    >>>
    >>
    >>
    >> --
    >> -regards
    >> Amol
    >>
    >
    >
    > --
    > -regards
    > Amol
    >
    
    
    -- 
    Death to <Redacted>, and butter sauce.
    Don't boil me, I'm still alive.
    <Redacted> lobster!
    

18. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Laurenz Albe <laurenz.albe@cybertec.at> — 2025-07-16T15:48:10Z

    On Wed, 2025-07-16 at 18:54 +0530, Amol Inamdar wrote:
    > I would like to rephrase the question a little bit, below is how our setup going to be 
    >    1. NFS mount point is for /nfs-mount/postgres (and permissions locked down so
    >       that Postgres cannot create directories in here)
    >    2. Postgres data directory is /nfs-mount/postgres/db
    >    3. With secured NFS + AT-TLS setup Postgres will be able to write to data directory
    >       but not parent dir, however the file ownership information Postgres sees from the
    >       stat() call will not match the Postgres user in the container (even though the
    >       AT-TLS strict access control will ensure only the Posgres user can read/write to
    >       this directory)
    > Considering the above scenario/setup, what is the danger of removing the ownership check
    > in miscinit.c checkDataDir() function ? 
    
    The danger is that somebody else than the PostgreSQL user has permissions on
    the data directory.  You will argue that that somebody is root, and root has
    these permissions anyway.
    
    But there is another reason why PostgreSQL insists that the PostgreSQL user
    owns the data directory: at startup, the postmaster checks if the data
    directory belongs to the current user and fails if not.  This is a protection
    against starting the postmaster with the wrong user.
    
    There are certainly ways to do it differently, but I'd argue that they would
    be more complicated, and the current simple solution is robust.
    
    If you pre-create the data directory with the appropriate permissions,
    what keeps you from giving ownership to the correct user too?
    
    Yours,
    Laurenz Albe
    
    
    
    

19. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Greg Sabino Mullane <htamfids@gmail.com> — 2025-07-17T00:41:44Z

    On Wed, Jul 16, 2025 at 9:25 AM Amol Inamdar <amol.aai@gmail.com> wrote:
    
    >
    >    1. NFS mount point is for /nfs-mount/postgres (and permissions locked
    >    down so that Postgres cannot create directories in here)
    >    2. Postgres data directory is /nfs-mount/postgres/db
    >    3.
    >
    >    With secured NFS + AT-TLS setup Postgres will be able to write to data
    >    directory but not parent dir, however the file ownership information
    >    Postgres sees from the stat() call will not match the Postgres user in the
    >    container (even though the AT-TLS strict access control will ensure only
    >    the Posgres user can read/write to this directory)
    >
    > This thread is fascinating. It's like combining two of the most annoying
    technologies in the world, NFS and SELinux, into something worse than
    either of them.
    
    Many people use Docker, and NFS, and Postgres all the time. Stop trying to
    push on a string.  Conform your process to Postgres' fairly minimal and
    sane requirements, rather than the other way around.
    
    Cheers,
    Greg
    
    --
    Crunchy Data - https://www.crunchydata.com
    Enterprise Postgres Software Products & Tech Support
    

20. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Amol Inamdar <amol.aai@gmail.com> — 2025-07-17T04:52:37Z

    @Laurenz Albe <laurenz.albe@cybertec.at>
    If you pre-create the data directory with the appropriate permissions,
    what keeps you from giving ownership to the correct user too?
    
    Our NFS server is not a regular linux based server,
    it's on zOS (Mainframes) with AT-TLS security enabled,
    hence it doesn't allow changing of ownership.
    
    Basically, we have tried everything we could
    to change the directory ownership to match with the postgres user
    and that as of now looks impossible, unless we make changes in the
    environment.
    
    To summarize*, we are not able to change the ownership of the data
    directory *
    *due to the Mainframe NFS server limitations when enabled with AT-TLS
    security *
    *Hence we wanted to check if bypassing this check is ok if it could be
    assured *
    *that only the postgres user can write here (NFS-AT-TLS ensures that). *
    
    I wouldn't get into details of explaining why changing ownership is not
    possible,
    as that would take this discussion to another context, hence avoiding.
    
    Thanks in advance
    
    On Wed, Jul 16, 2025 at 9:18 PM Laurenz Albe <laurenz.albe@cybertec.at>
    wrote:
    
    > On Wed, 2025-07-16 at 18:54 +0530, Amol Inamdar wrote:
    > > I would like to rephrase the question a little bit, below is how our
    > setup going to be
    > >    1. NFS mount point is for /nfs-mount/postgres (and permissions locked
    > down so
    > >       that Postgres cannot create directories in here)
    > >    2. Postgres data directory is /nfs-mount/postgres/db
    > >    3. With secured NFS + AT-TLS setup Postgres will be able to write to
    > data directory
    > >       but not parent dir, however the file ownership information
    > Postgres sees from the
    > >       stat() call will not match the Postgres user in the container
    > (even though the
    > >       AT-TLS strict access control will ensure only the Posgres user can
    > read/write to
    > >       this directory)
    > > Considering the above scenario/setup, what is the danger of removing the
    > ownership check
    > > in miscinit.c checkDataDir() function ?
    >
    > The danger is that somebody else than the PostgreSQL user has permissions
    > on
    > the data directory.  You will argue that that somebody is root, and root
    > has
    > these permissions anyway.
    >
    > But there is another reason why PostgreSQL insists that the PostgreSQL user
    > owns the data directory: at startup, the postmaster checks if the data
    > directory belongs to the current user and fails if not.  This is a
    > protection
    > against starting the postmaster with the wrong user.
    >
    > There are certainly ways to do it differently, but I'd argue that they
    > would
    > be more complicated, and the current simple solution is robust.
    >
    > If you pre-create the data directory with the appropriate permissions,
    > what keeps you from giving ownership to the correct user too?
    >
    > Yours,
    > Laurenz Albe
    >
    
    
    -- 
    -regards
    Amol
    

21. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-17T05:14:33Z

    Amol Inamdar <amol.aai@gmail.com> writes:
    > @Laurenz Albe <laurenz.albe@cybertec.at>
    >> If you pre-create the data directory with the appropriate permissions,
    >> what keeps you from giving ownership to the correct user too?
    
    > Our NFS server is not a regular linux based server,
    > it's on zOS (Mainframes) with AT-TLS security enabled,
    > hence it doesn't allow changing of ownership.
    
    Not only is that not a fit storage substrate for Postgres,
    it's pretty hard to imagine that it's a fit substrate for
    anything.  "Every file on this filesystem must belong to the
    same owner" is a concept that should have gone out with
    floppy disks.
    
    You need some extremely fundamental re-examination of your
    design decisions.  At the moment I am content to say that
    Postgres does not support this storage mechanism and we
    do not intend to do so in the future.
    
    			regards, tom lane
    
    
    
    

22. Re: Bypassing Directory Ownership Check in PostgreSQL 16.6 with Secure z/OS NFS (AT-TLS)

    Ron Johnson <ronljohnsonjr@gmail.com> — 2025-07-17T13:13:15Z

    On Wed, Jul 16, 2025 at 8:42 PM Greg Sabino Mullane <htamfids@gmail.com>
    wrote:
    
    > On Wed, Jul 16, 2025 at 9:25 AM Amol Inamdar <amol.aai@gmail.com> wrote:
    >
    >>
    >>    1. NFS mount point is for /nfs-mount/postgres (and permissions locked
    >>    down so that Postgres cannot create directories in here)
    >>    2. Postgres data directory is /nfs-mount/postgres/db
    >>    3.
    >>
    >>    With secured NFS + AT-TLS setup Postgres will be able to write to
    >>    data directory but not parent dir, however the file ownership information
    >>    Postgres sees from the stat() call will not match the Postgres user in the
    >>    container (even though the AT-TLS strict access control will ensure only
    >>    the Posgres user can read/write to this directory)
    >>
    >> This thread is fascinating. It's like combining two of the most annoying
    > technologies in the world, NFS and SELinux, into something worse than
    > either of them.
    >
    > Many people use Docker, and NFS, and Postgres all the time. Stop trying to
    > push on a string.  Conform your process to Postgres' fairly minimal and
    > sane requirements, rather than the other way around.
    >
    
    Unless "all databases must be stored on the mainframe, Because Mainframes
    Are Secure" is dogma in that shop, and there's no way the CISO will make an
    exception for some random program off the Internet.  "Heck, it's probably
    got malware in it!!"
    
    -- 
    Death to <Redacted>, and butter sauce.
    Don't boil me, I'm still alive.
    <Redacted> lobster!