Horton

Thread

Commits

  1. config: Rename "Asynchronous Behavior" to "I/O"

  2. config: Split "Worker Processes" out of "Asynchronous Behavior"

  1. postgresql.conf.sample ordering for IO, worker related GUCs

    Andres Freund <andres@anarazel.de> — 2025-01-31T01:19:05Z

    Hi,

While working on polishing the AIO patchset, I was trying to figure out where
to fit the new GUCs. So far I had a new "top-level" #--- style section named
"WIP AIO GUC docs" which I suspect you all won't let me get away with.

There is an existing (sub-)section which already has a few related GUCs and
could fit AIO related ones.

#------------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#------------------------------------------------------------------------------
...
# - Asynchronous Behavior -


Except that the list of GUCs in it seems confusing:

#backend_flush_after = 0                # measured in pages, 0 disables
#effective_io_concurrency = 1           # 1-1000; 0 disables prefetching
#maintenance_io_concurrency = 10        # 1-1000; 0 disables prefetching
#io_combine_limit = 128kB               # usually 1-32 blocks (depends on OS)
#max_worker_processes = 8               # (change requires restart)
#max_parallel_workers_per_gather = 2    # limited by max_parallel_workers
#max_parallel_maintenance_workers = 2   # limited by max_parallel_workers
#max_parallel_workers = 8               # number of max_worker_processes that
                                        # can be used in parallel operations
#parallel_leader_participation = on


The first four GUCs make some sense under the heading, but why are the worker
GUCs in the same section?


Seems like it would make sense to introduce a new "Worker Processes"
sub-section?


I am wondering if it'd be better to get rid of "Asynchronous Behavior" and
instead have "# - IO -" or such?  E.g. io_combine_limit isn't really about
anything asynchronous, but still seems like it should belong in the same
section as the rest?


I also wonder if fsync, recovery_init_sync_method, data_sync_retry shouldn't
be in that new subsection. Or maybe it should even be a top-level section.



FWIW, AIO currently adds the following GUCs:

#io_method = worker                     # worker, io_uring, sync
                                        # (change requires restart)
#io_max_concurrency = -1		# Max number of IOs that may be in
					# flight at the same time in one backend
					# (change requires restart)
#io_workers = 3                         # 1-32; relevant only if io_method = worker


Greetings,

Andres Freund

  2. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-01-31T01:55:36Z

    Andres Freund <andres@anarazel.de> writes:
> While working on polishing the AIO patchset, I was trying to figure out where
> to fit the new GUCs. So far I had a new "top-level" #--- style section named
> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
> There is an existing (sub-)section which already has a few related GUCs and
> could fit AIO related ones.

I think the normal theory for postgresql.conf.sample is that it should
match the organization of config.sgml.  What are you planning there?

			regards, tom lane

  3. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Andres Freund <andres@anarazel.de> — 2025-01-31T02:24:05Z

    Hi

On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>Andres Freund <andres@anarazel.de> writes:
>> While working on polishing the AIO patchset, I was trying to figure out where
>> to fit the new GUCs. So far I had a new "top-level" #--- style section named
>> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
>> There is an existing (sub-)section which already has a few related GUCs and
>> could fit AIO related ones.
>
>I think the normal theory for postgresql.conf.sample is that it should
>match the organization of config.sgml.  What are you planning there?

Pretty much the same. I.e. I'm thinking that the worker stuff should be it's own subsection and that the existing IO parameters should be moved to either a new subsection or a new top level section.  But I'm wondering how others think it should be structured...

Andres
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

  4. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Andres Freund <andres@anarazel.de> — 2025-01-31T15:24:58Z

    Hi,

On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
> On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >Andres Freund <andres@anarazel.de> writes:
> >> While working on polishing the AIO patchset, I was trying to figure out where
> >> to fit the new GUCs. So far I had a new "top-level" #--- style section named
> >> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
> >> There is an existing (sub-)section which already has a few related GUCs and
> >> could fit AIO related ones.
> >
> >I think the normal theory for postgresql.conf.sample is that it should
> >match the organization of config.sgml.  What are you planning there?
> 
> Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
> own subsection and that the existing IO parameters should be moved to either
> a new subsection or a new top level section.  But I'm wondering how others
> think it should be structured...

Here are draft changes for the minimal thing I think we should do.

I don't really know what to do about the "IO" abbreviation. The other sections
un-abbreviate abbreviations, but I suspect that Input/Output will be less
informative than IO to most...

I still wonder if we instead ought to create a top-level "IO" section instead
of leaving it under "Resource Usage".  How many IOs we combine, how
aggressively we flush unflushed data, etc only kinda fits into the resource
usage category.

Greetings,

Andres Freund

  5. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-01-31T15:33:44Z

    Andres Freund <andres@anarazel.de> writes:
> I don't really know what to do about the "IO" abbreviation. The other sections
> un-abbreviate abbreviations, but I suspect that Input/Output will be less
> informative than IO to most...

I'd lean towards writing "I/O", but it's not a point I'd quibble over.

			regards, tom lane

  6. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Robert Treat <rob@xzilla.net> — 2025-01-31T22:22:43Z

    On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
> > On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > >Andres Freund <andres@anarazel.de> writes:
> > >> While working on polishing the AIO patchset, I was trying to figure out where
> > >> to fit the new GUCs. So far I had a new "top-level" #--- style section named
> > >> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
> > >> There is an existing (sub-)section which already has a few related GUCs and
> > >> could fit AIO related ones.
> > >
> > >I think the normal theory for postgresql.conf.sample is that it should
> > >match the organization of config.sgml.  What are you planning there?
> >
> > Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
> > own subsection and that the existing IO parameters should be moved to either
> > a new subsection or a new top level section.  But I'm wondering how others
> > think it should be structured...
>
> Here are draft changes for the minimal thing I think we should do.
>
> I don't really know what to do about the "IO" abbreviation. The other sections
> un-abbreviate abbreviations, but I suspect that Input/Output will be less
> informative than IO to most...
>
> I still wonder if we instead ought to create a top-level "IO" section instead
> of leaving it under "Resource Usage".  How many IOs we combine, how
> aggressively we flush unflushed data, etc only kinda fits into the resource
> usage category.
>

+1 from me, though I did pause on whether it should be called
"background workers" rather than "worker processes", but I think this
is the right direction.


Robert Treat
https://xzilla.net

  7. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Andres Freund <andres@anarazel.de> — 2025-01-31T23:21:54Z

    Hi, 

On January 31, 2025 5:22:43 PM EST, Robert Treat <rob@xzilla.net> wrote:
>On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
>>
>> Hi,
>>
>> On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
>> > On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> > >Andres Freund <andres@anarazel.de> writes:
>> > >> While working on polishing the AIO patchset, I was trying to figure out where
>> > >> to fit the new GUCs. So far I had a new "top-level" #--- style section named
>> > >> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
>> > >> There is an existing (sub-)section which already has a few related GUCs and
>> > >> could fit AIO related ones.
>> > >
>> > >I think the normal theory for postgresql.conf.sample is that it should
>> > >match the organization of config.sgml.  What are you planning there?
>> >
>> > Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
>> > own subsection and that the existing IO parameters should be moved to either
>> > a new subsection or a new top level section.  But I'm wondering how others
>> > think it should be structured...
>>
>> Here are draft changes for the minimal thing I think we should do.
>>
>> I don't really know what to do about the "IO" abbreviation. The other sections
>> un-abbreviate abbreviations, but I suspect that Input/Output will be less
>> informative than IO to most...
>>
>> I still wonder if we instead ought to create a top-level "IO" section instead
>> of leaving it under "Resource Usage".  How many IOs we combine, how
>> aggressively we flush unflushed data, etc only kinda fits into the resource
>> usage category.
>>
>
>+1 from me, though I did pause on whether it should be called
>"background workers" rather than "worker processes", but I think this
>is the right direction.

+1 for the patch as-is, or my suggestion to make it a top level section? 

Greetings, 

Andres 
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

  8. Re: postgresql.conf.sample ordering for IO, worker related GUCs

    Andres Freund <andres@anarazel.de> — 2025-02-11T17:57:46Z

    Hi,

On 2025-01-31 18:21:54 -0500, Andres Freund wrote:
> On January 31, 2025 5:22:43 PM EST, Robert Treat <rob@xzilla.net> wrote:
> >On Fri, Jan 31, 2025 at 10:25 AM Andres Freund <andres@anarazel.de> wrote:
> >>
> >> Hi,
> >>
> >> On 2025-01-30 21:24:05 -0500, Andres Freund wrote:
> >> > On January 30, 2025 8:55:36 PM EST, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> > >Andres Freund <andres@anarazel.de> writes:
> >> > >> While working on polishing the AIO patchset, I was trying to figure out where
> >> > >> to fit the new GUCs. So far I had a new "top-level" #--- style section named
> >> > >> "WIP AIO GUC docs" which I suspect you all won't let me get away with.
> >> > >> There is an existing (sub-)section which already has a few related GUCs and
> >> > >> could fit AIO related ones.
> >> > >
> >> > >I think the normal theory for postgresql.conf.sample is that it should
> >> > >match the organization of config.sgml.  What are you planning there?
> >> >
> >> > Pretty much the same. I.e. I'm thinking that the worker stuff should be it's
> >> > own subsection and that the existing IO parameters should be moved to either
> >> > a new subsection or a new top level section.  But I'm wondering how others
> >> > think it should be structured...
> >>
> >> Here are draft changes for the minimal thing I think we should do.
> >>
> >> I don't really know what to do about the "IO" abbreviation. The other sections
> >> un-abbreviate abbreviations, but I suspect that Input/Output will be less
> >> informative than IO to most...
> >>
> >> I still wonder if we instead ought to create a top-level "IO" section instead
> >> of leaving it under "Resource Usage".  How many IOs we combine, how
> >> aggressively we flush unflushed data, etc only kinda fits into the resource
> >> usage category.
> >>
> >
> >+1 from me, though I did pause on whether it should be called
> >"background workers" rather than "worker processes", but I think this
> >is the right direction.
>
> +1 for the patch as-is, or my suggestion to make it a top level section?

I pushed the patch as I had it, save for replacing IO with I/O where
appropriate. If we want to go for a larger restructuring in the future, this
won't make it harder, and perhaps even a little simpler.

Greetings,

Andres Freund