Thread

1. PostgreSQL 17 Beta 1 release announcement draft

   Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-16T01:45:35Z

   Hi,
   
   Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
   draft. This contains a user-facing summary of some of the features that 
   will be available in the Beta, as well as a call to test. I've made an 
   effort to group them logically around the different workflows they affect.
   
   A few notes:
   
   * The section with the features is not 80-char delimited. I will do that 
   before the final copy
   
   * There is an explicit callout that we've added in the SQL/JSON features 
   that were previously reverted in PG15. I want to ensure we're 
   transparent about that, but also use it as a hook to get people testing.
   
   When reviewing:
   
   * Please check for correctness of feature descriptions, keeping in mind 
   this is targeted for a general audience
   
   * Please indicate if you believe there's a notable omission, or if we 
   should omit any of these callouts
   
   * Please indicate if a description is confusing - I'm happy to rewrite 
   to ensure it's clearer.
   
   Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
   beta release takes some extra effort, I want to ensure all changes are 
   in with time to spare before release day.
   
   Thanks,
   
   Jonathan
   

2. Re: PostgreSQL 17 Beta 1 release announcement draft

   David Rowley <dgrowleyml@gmail.com> — 2024-05-16T02:42:37Z

   Thanks for writing that up. It's always exciting to see this each year.
   
   On Thu, 16 May 2024 at 13:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
   > * Please indicate if you believe there's a notable omission, or if we
   > should omit any of these callouts
   
   I'd say the streaming read stuff added in b5a9b18cd and subsequent
   commits like b7b0f3f27 and 041b96802 are worth a mention. I'd be happy
   to see this over the IS NOT NULL qual stuff that I worked on in there
   or even the AVX512 bit counting. Speeding up a backwater aggregate
   function is nice, but IMO, not compatible with reducing the number
   reads.
   
   There's some benchmarking in a youtube video:
   https://youtu.be/QAYzWAlxCYc?si=L0UT6Lrf067ZBv46&t=237
   
   > * Please indicate if a description is confusing - I'm happy to rewrite
   > to ensure it's clearer.
   >
   > Please provide feedback no later than Wed 2024-05-22 18:00 UTC.
   
   The only other thing I saw from a quick read was a stray "the" in "the
   copy proceed even if the there is an error inserting a row."
   
   David
   
   
   
   

3. Re: PostgreSQL 17 Beta 1 release announcement draft

   Kashif Zeeshan <kashi.zeeshan@gmail.com> — 2024-05-16T04:11:04Z

   Hi Jonathan
   
   Did the review and did not find any issues.
   
   Regards
   Kashif Zeeshan
   Bitnine Global
   
   On Thu, May 16, 2024 at 6:45 AM Jonathan S. Katz <jkatz@postgresql.org>
   wrote:
   
   > Hi,
   >
   > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
   > draft. This contains a user-facing summary of some of the features that
   > will be available in the Beta, as well as a call to test. I've made an
   > effort to group them logically around the different workflows they affect.
   >
   > A few notes:
   >
   > * The section with the features is not 80-char delimited. I will do that
   > before the final copy
   >
   > * There is an explicit callout that we've added in the SQL/JSON features
   > that were previously reverted in PG15. I want to ensure we're
   > transparent about that, but also use it as a hook to get people testing.
   >
   > When reviewing:
   >
   > * Please check for correctness of feature descriptions, keeping in mind
   > this is targeted for a general audience
   >
   > * Please indicate if you believe there's a notable omission, or if we
   > should omit any of these callouts
   >
   > * Please indicate if a description is confusing - I'm happy to rewrite
   > to ensure it's clearer.
   >
   > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
   > beta release takes some extra effort, I want to ensure all changes are
   > in with time to spare before release day.
   >
   > Thanks,
   >
   > Jonathan
   >
   

4. Re: PostgreSQL 17 Beta 1 release announcement draft

   Thom Brown <thom@linux.com> — 2024-05-16T05:10:57Z

   On Thu, May 16, 2024, 02:45 Jonathan S. Katz <jkatz@postgresql.org> wrote:
   
   > Hi,
   >
   > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
   > draft. This contains a user-facing summary of some of the features that
   > will be available in the Beta, as well as a call to test. I've made an
   > effort to group them logically around the different workflows they affect.
   >
   > A few notes:
   >
   > * The section with the features is not 80-char delimited. I will do that
   > before the final copy
   >
   > * There is an explicit callout that we've added in the SQL/JSON features
   > that were previously reverted in PG15. I want to ensure we're
   > transparent about that, but also use it as a hook to get people testing.
   >
   > When reviewing:
   >
   > * Please check for correctness of feature descriptions, keeping in mind
   > this is targeted for a general audience
   >
   > * Please indicate if you believe there's a notable omission, or if we
   > should omit any of these callouts
   >
   > * Please indicate if a description is confusing - I'm happy to rewrite
   > to ensure it's clearer.
   >
   > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
   > beta release takes some extra effort, I want to ensure all changes are
   > in with time to spare before release day.
   >
   
   "Now as of PostgreSQL 17, you can now use parallel index builds for [BRIN](
   https://www.postgresql.org/docs/17/brin.html) indexes."
   
   The 2nd "now" is redundant.
   
   
   "Finally, PostgreSQL 17 adds more explicitly SIMD instructions, including
   AVX-512 support for the [`bit_count](
   https://www.postgresql.org/docs/17/functions-bitstring.html) function."
   
   Would "SIMD-explicit instructions" be better? Also, I know you may not be
   using markdown for the final version, but the bit_count backtick isn't
   matched by a closing backtick.
   
   
   "[`COPY`](https://www.postgresql.org/docs/17/sql-copy.html), used to
   efficiently bulk load data into PostgreSQL"
   
   The "used to" makes me stumble into reading it as meaning "it previously
   could efficiently bulk load data".
   
   Perhaps just add a "which is" before "used"?
   
   
   "PostgreSQL 17 includes a built-in collation provider that provides similar
   semantics to the `C` collation provided by libc."
   
   "provider", "provides", and "provided" feels too repetitive.
   
   How about, "PostgreSQL 17 includes a built-in collation provider with
   semantics similar to the `C` collation offered by libc."?
   
   
   Regards
   
   Thom
   

5. Re: PostgreSQL 17 Beta 1 release announcement draft

   Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-05-16T05:15:58Z

   Hi,
   
   On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
   > Hi,
   > 
   > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
   
   Thanks for working on it!
   
   I've one comment:
   
   > PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into an operation.
   
   Instead of "to give more insight into an operation", what about "to give more
   insight about what a session is waiting for (should it be active)"?
   
   Regards,
   
   -- 
   Bertrand Drouvot
   PostgreSQL Contributors Team
   RDS Open Source Databases
   Amazon Web Services: https://aws.amazon.com
   
   
   
   

6. Re: PostgreSQL 17 Beta 1 release announcement draft

   zaidagilist <zaidagilist@gmail.com> — 2024-05-16T05:36:49Z

   Hello,
   
   I am trying to open the 17 docs but it looks removed. Getting
   following message "Page not found"
   
   https://www.postgresql.org/docs/17/
   
   
   Regards,
   Zaid Shabbir
   
   On Thu, May 16, 2024 at 10:16 AM Bertrand Drouvot
   <bertranddrouvot.pg@gmail.com> wrote:
   >
   > Hi,
   >
   > On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
   > > Hi,
   > >
   > > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
   >
   > Thanks for working on it!
   >
   > I've one comment:
   >
   > > PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into an operation.
   >
   > Instead of "to give more insight into an operation", what about "to give more
   > insight about what a session is waiting for (should it be active)"?
   >
   > Regards,
   >
   > --
   > Bertrand Drouvot
   > PostgreSQL Contributors Team
   > RDS Open Source Databases
   > Amazon Web Services: https://aws.amazon.com
   >
   >
   

7. Re: PostgreSQL 17 Beta 1 release announcement draft

   Thom Brown <thom@linux.com> — 2024-05-16T05:48:52Z

   On Thu, May 16, 2024, 06:37 zaidagilist <zaidagilist@gmail.com> wrote:
   
   > Hello,
   >
   > I am trying to open the 17 docs but it looks removed. Getting
   > following message "Page not found"
   >
   > https://www.postgresql.org/docs/17/
   >
   >
   > Regards,
   > Zaid Shabbir
   >
   
   That link isn't set up yet, but will be (or should be) when the
   announcement goes out.
   
   Regards
   
   Thom
   
   >
   

8. Re: PostgreSQL 17 Beta 1 release announcement draft

   David Rowley <dgrowleyml@gmail.com> — 2024-05-16T06:19:42Z

   On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
   > I am trying to open the 17 docs but it looks removed. Getting
   > following message "Page not found"
   >
   > https://www.postgresql.org/docs/17/
   
   It's called "devel" for "development" until we branch sometime before July:
   
   https://www.postgresql.org/docs/devel/
   
   David
   
   
   
   

9. Re: PostgreSQL 17 Beta 1 release announcement draft

   Jelte Fennema <postgres@jeltef.nl> — 2024-05-16T10:41:50Z

   On Thu, 16 May 2024 at 03:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
   > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
   > draft.
   
   I think we can quickly mention c4ab7da6061 in the COPY paragraph, in
   some benchmarks it improved perf by close to 2x. Something like this:
   "has improved performance in PostgreSQL 17 when the source encoding
   matches the destination encoding *and when sending large rows from
   server to client*"
   
   Also, I think it's a bit weird to put the current COPY paragraph under
   Developer Experience. I think if you want to keep it there instead of
   move it to the per section, we should put the line about IGNORE_ERROR
   first instead of the perf improvements. Now the IGNORE_ERROR addition
   seems more of an afterthought.
   
   s/IGNORE_ERROR/ON_ERROR
   
   I think it would be good to clarify if the following applies when
   upgrading from or to PostgreSQL 17:
   "Starting with PostgreSQL 17, you no longer need to drop logical
   replication slots when using pg_upgrade"
   
   Finally, I personally would have included a lot more links for the new
   items in this document. Some that would benefit from being a link
   imho:
   - pg_createsubscriber
   - JSON_TABLE
   - SQL/JSON constructor
   - SQL/JSON query functions
   - ON_ERROR
   - sslnegotiation
   - PQchangePassword
   - pg_maintain
   
   
   
   

10. Re: PostgreSQL 17 Beta 1 release announcement draft

    Joe Conway <mail@joeconway.com> — 2024-05-16T12:05:58Z

    On 5/15/24 21:45, Jonathan S. Katz wrote:
    > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
    > beta release takes some extra effort, I want to ensure all changes are
    > in with time to spare before release day.
    
    "You can find information about all of the features and changes found in
    PostgreSQL 17"
    
    Sounds repetitive; maybe:
    
    "Information about all of the features and changes in PostgreSQL 17 can 
    be found in the [release notes]"
    
    
    "more explicitly SIMD instructions" I think ought to be "more explicit 
    SIMD instructions"
    
    
    "PostgreSQL 17 includes a built-in collation provider that provides 
    similar semantics to the `C` collation provided by libc."
    
    I think that needs to mention UTF-8 encoding somehow, and "provided by 
    libc" is not really true; maybe:
    
    "PostgreSQL 17 includes a built-in collation provider that provides 
    similar sorting semantics to the `C` collation except with UTF-8 
    encoding rather than SQL_ASCII."
    
    
    -- 
    Joe Conway
    PostgreSQL Contributors Team
    RDS Open Source Databases
    Amazon Web Services: https://aws.amazon.com
    
    
    
    
    

11. Re: PostgreSQL 17 Beta 1 release announcement draft

    Joe Conway <mail@joeconway.com> — 2024-05-16T14:04:02Z

    On 5/16/24 08:05, Joe Conway wrote:
    > On 5/15/24 21:45, Jonathan S. Katz wrote:
    >> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
    >> beta release takes some extra effort, I want to ensure all changes are
    >> in with time to spare before release day.
    
    "`EXPLAIN` can now show how much time is spent for I/O block reads and 
    writes"
    
    Is that really EXPLAIN, or rather EXPLAIN ANALYZE?
    
    -- 
    Joe Conway
    PostgreSQL Contributors Team
    RDS Open Source Databases
    Amazon Web Services: https://aws.amazon.com
    
    
    
    
    

12. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:02:10Z

    On 5/15/24 10:42 PM, David Rowley wrote:
    > Thanks for writing that up. It's always exciting to see this each year.
    > 
    > On Thu, 16 May 2024 at 13:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >> * Please indicate if you believe there's a notable omission, or if we
    >> should omit any of these callouts
    > 
    > I'd say the streaming read stuff added in b5a9b18cd and subsequent
    > commits like b7b0f3f27 and 041b96802 are worth a mention. I'd be happy
    > to see this over the IS NOT NULL qual stuff that I worked on in there
    > or even the AVX512 bit counting. Speeding up a backwater aggregate
    > function is nice, but IMO, not compatible with reducing the number
    > reads.
    > There's some benchmarking in a youtube video:
    > https://youtu.be/QAYzWAlxCYc?si=L0UT6Lrf067ZBv46&t=237
    
    Nice! Definitely agree on including this - it wasn't initially clear to 
    me on the read of the release notes. I'll update it. Please see in the 
    next revision (will posted upthread), proposed text here for 
    convenience, as I'm not sure I'm appropriately capturing it:
    
    ==
    This release introduces adds an interface to stream I/O, and can show 
    performance improvements when performing sequential scans and running 
    [`ANALYZE`](https://www.postgresql.org/docs/17/sql-analyze.html).
    ==
    
    The AVX-512 bit counting showed solid impact[1] on the binary distance 
    functions in pgvector (I have to re-run again w/v17, as I seem to recall 
    seeing some numbers that boosted it 5-7x [but recall isn't 100% ;)]).
    
    >> * Please indicate if a description is confusing - I'm happy to rewrite
    >> to ensure it's clearer.
    >>
    >> Please provide feedback no later than Wed 2024-05-22 18:00 UTC.
    > 
    > The only other thing I saw from a quick read was a stray "the" in "the
    > copy proceed even if the there is an error inserting a row."
    
    Thanks!
    
    Jonathan
    
    [1] https://github.com/pgvector/pgvector/pull/519
    
    

13. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:07:07Z

    On 5/16/24 1:10 AM, Thom Brown wrote:
    > On Thu, May 16, 2024, 02:45 Jonathan S. Katz <jkatz@postgresql.org 
    > <mailto:jkatz@postgresql.org>> wrote:
    > 
    >     Hi,
    > 
    >     Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
    >     draft. This contains a user-facing summary of some of the features that
    >     will be available in the Beta, as well as a call to test. I've made an
    >     effort to group them logically around the different workflows they
    >     affect.
    > 
    >     A few notes:
    > 
    >     * The section with the features is not 80-char delimited. I will do
    >     that
    >     before the final copy
    > 
    >     * There is an explicit callout that we've added in the SQL/JSON
    >     features
    >     that were previously reverted in PG15. I want to ensure we're
    >     transparent about that, but also use it as a hook to get people testing.
    > 
    >     When reviewing:
    > 
    >     * Please check for correctness of feature descriptions, keeping in mind
    >     this is targeted for a general audience
    > 
    >     * Please indicate if you believe there's a notable omission, or if we
    >     should omit any of these callouts
    > 
    >     * Please indicate if a description is confusing - I'm happy to rewrite
    >     to ensure it's clearer.
    > 
    >     Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
    >     beta release takes some extra effort, I want to ensure all changes are
    >     in with time to spare before release day.
    > 
    > 
    > "Now as of PostgreSQL 17, you can now use parallel index builds for 
    > [BRIN](https://www.postgresql.org/docs/17/brin.html 
    > <https://www.postgresql.org/docs/17/brin.html>) indexes."
    > 
    > The 2nd "now" is redundant.
    > 
    > 
    > "Finally, PostgreSQL 17 adds more explicitly SIMD instructions, 
    > including AVX-512 support for the 
    > [`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html 
    > <https://www.postgresql.org/docs/17/functions-bitstring.html>) function."
    > 
    > Would "SIMD-explicit instructions" be better? Also, I know you may not 
    > be using markdown for the final version, but the bit_count backtick 
    > isn't matched by a closing backtick.
    > 
    > 
    > "[`COPY`](https://www.postgresql.org/docs/17/sql-copy.html 
    > <https://www.postgresql.org/docs/17/sql-copy.html>), used to efficiently 
    > bulk load data into PostgreSQL"
    > 
    > The "used to" makes me stumble into reading it as meaning "it previously 
    > could efficiently bulk load data".
    > 
    > Perhaps just add a "which is" before "used"?
    > 
    > 
    > "PostgreSQL 17 includes a built-in collation provider that provides 
    > similar semantics to the `C` collation provided by libc."
    > 
    > "provider", "provides", and "provided" feels too repetitive.
    > 
    > How about, "PostgreSQL 17 includes a built-in collation provider with 
    > semantics similar to the `C` collation offered by libc."?
    
    Thanks - I accepted (with modifications) most of the suggestions here. 
    I'll include in the next version of the draft.
    
    Jonathan
    
    

14. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:10:10Z

    On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
    > Hi,
    > 
    > On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
    >> Hi,
    >>
    >> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
    > 
    > Thanks for working on it!
    > 
    > I've one comment:
    > 
    >> PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into an operation.
    > 
    > Instead of "to give more insight into an operation", what about "to give more
    > insight about what a session is waiting for (should it be active)"?
    
    I put:
    
    "to give more in insight into why a session is blocked."
    
    Does that work?
    
    Jonathan
    
    

15. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:25:00Z

    On 5/16/24 6:41 AM, Jelte Fennema-Nio wrote:
    > On Thu, 16 May 2024 at 03:45, Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
    >> draft.
    > 
    > I think we can quickly mention c4ab7da6061 in the COPY paragraph, in
    > some benchmarks it improved perf by close to 2x. Something like this:
    > "has improved performance in PostgreSQL 17 when the source encoding
    > matches the destination encoding *and when sending large rows from
    > server to client*"
    
    (I'm going to make a note to test this with loading large vectors :) 
    I've modified the text to reflect this. Please see the new language 
    upthread.
    
    > Also, I think it's a bit weird to put the current COPY paragraph under
    > Developer Experience. I think if you want to keep it there instead of
    > move it to the per section, we should put the line about IGNORE_ERROR
    > first instead of the perf improvements. Now the IGNORE_ERROR addition
    > seems more of an afterthought.
    
    I don't agree with this. I think we want to push COPY as a developer 
    feature - I see a lot of people not utilizing COPY appropriate when it 
    would really benefit the performance of their app, and I think 
    emphasizing it as the way to do bulk loads (while touting that it's even 
    faster!) will help make it more apparent.
    
    > s/IGNORE_ERROR/ON_ERROR
    
    Thanks.
    
    > I think it would be good to clarify if the following applies when
    > upgrading from or to PostgreSQL 17:
    > "Starting with PostgreSQL 17, you no longer need to drop logical
    > replication slots when using pg_upgrade"
    
    Adjusted.
    
    > Finally, I personally would have included a lot more links for the new
    > items in this document. Some that would benefit from being a link
    > imho:
    > - pg_createsubscriber
    > - JSON_TABLE
    > - SQL/JSON constructor
    > - SQL/JSON query functions
    > - ON_ERROR
    > - sslnegotiation
    > - PQchangePassword
    > - pg_maintain
    
    I have to check if these have deep links or not, but I was planning to 
    make another pass once the copy (no pun intended) is closer to 
    finalized, so I don't have to constantly edit markdown.
    
    Jonathan
    

16. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:30:56Z

    On 5/16/24 8:05 AM, Joe Conway wrote:
    > On 5/15/24 21:45, Jonathan S. Katz wrote:
    >> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
    >> beta release takes some extra effort, I want to ensure all changes are
    >> in with time to spare before release day.
    > 
    > "You can find information about all of the features and changes found in
    > PostgreSQL 17"
    > 
    > Sounds repetitive; maybe:
    > 
    > "Information about all of the features and changes in PostgreSQL 17 can 
    > be found in the [release notes]"
    
    The first is active voice, the suggestion passive. However, I tightened 
    the language:
    
    You can find information about all of the PostgreSQL 17 features and 
    changes in the [release 
    notes](https://www.postgresql.org/docs/17/release-17.html):
    
    > "PostgreSQL 17 includes a built-in collation provider that provides 
    > similar semantics to the `C` collation provided by libc."
    > 
    > I think that needs to mention UTF-8 encoding somehow, and "provided by 
    > libc" is not really true; maybe:
    > 
    > "PostgreSQL 17 includes a built-in collation provider that provides 
    > similar sorting semantics to the `C` collation except with UTF-8 
    > encoding rather than SQL_ASCII."
    
    WFM. Taken verbatim.
    
    Thanks,
    
    Jonathan
    

17. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-19T21:34:56Z

    On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
    > Hi,
    > 
    > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
    > draft. This contains a user-facing summary of some of the features that 
    > will be available in the Beta, as well as a call to test. I've made an 
    > effort to group them logically around the different workflows they affect.
    > 
    > A few notes:
    > 
    > * The section with the features is not 80-char delimited. I will do that 
    > before the final copy
    > 
    > * There is an explicit callout that we've added in the SQL/JSON features 
    > that were previously reverted in PG15. I want to ensure we're 
    > transparent about that, but also use it as a hook to get people testing.
    > 
    > When reviewing:
    > 
    > * Please check for correctness of feature descriptions, keeping in mind 
    > this is targeted for a general audience
    > 
    > * Please indicate if you believe there's a notable omission, or if we 
    > should omit any of these callouts
    > 
    > * Please indicate if a description is confusing - I'm happy to rewrite 
    > to ensure it's clearer.
    > 
    > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
    > beta release takes some extra effort, I want to ensure all changes are 
    > in with time to spare before release day.
    
    Thanks for all the feedback to date. Please see the next revision. 
    Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    
    Thanks,
    
    Jonathan
    
    

18. Re: PostgreSQL 17 Beta 1 release announcement draft

    Erik Rijkers <er@xs4all.nl> — 2024-05-19T22:15:54Z

    Op 5/19/24 om 23:34 schreef Jonathan S. Katz:
    > On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
    >> Hi,
    >>
    >> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
    
    'This release introduces adds an interface'  should be:
    'This release adds an interface'
        (or 'introduces'; just not both...)
    
    Thanks,
    
    Erik Rijkers
    
    
    
    
    

19. Re: PostgreSQL 17 Beta 1 release announcement draft

    David Rowley <dgrowleyml@gmail.com> — 2024-05-19T23:24:06Z

    On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
    > Thanks for all the feedback to date. Please see the next revision.
    > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    
    Thanks for the updates.
    
    > [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL, and with PostgreSQL 17 shows a 2x performance improvement when loading large rows.
    
    The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
    So I think "exporting" or "sending large rows to the client"  rather
    than "loading".
    
    There's also a stray "with" in that sentence.
    
    David
    
    
    
    

20. Re: PostgreSQL 17 Beta 1 release announcement draft

    John Naylor <johncnaylorls@gmail.com> — 2024-05-20T06:58:48Z

    Hi Jon,
    
    Regarding vacuum "has shown up to a 6x improvement in overall time to
    complete its work" -- I believe I've seen reported numbers close to
    that only 1) when measuring the index phase in isolation or maybe 2)
    the entire vacuum of unlogged tables with one, perfectly-correlated
    index (testing has less variance with WAL out of the picture). I
    believe tables with many indexes would show a lot of improvement, but
    I'm not aware of testing that case specifically. Can you clarify where
    6x came from?
    
    
    
    

21. Re: PostgreSQL 17 Beta 1 release announcement draft

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-05-20T09:34:32Z

    Hi,
    
    On Sun, May 19, 2024 at 05:10:10PM -0400, Jonathan S. Katz wrote:
    > On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
    > > Hi,
    > > 
    > > On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
    > > > Hi,
    > > > 
    > > > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
    > > 
    > > Thanks for working on it!
    > > 
    > > I've one comment:
    > > 
    > > > PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into an operation.
    > > 
    > > Instead of "to give more insight into an operation", what about "to give more
    > > insight about what a session is waiting for (should it be active)"?
    > 
    > I put:
    > 
    > "to give more in insight into why a session is blocked."
    
    Thanks!
    
    > 
    > Does that work?
    > 
    
    I think using "waiting" is better (as the view is "pg_wait_events" and the
    join with pg_stat_activity would be on the "wait_event_type" and "wait_event"
    columns).
    
    The reason I mentioned "should it be active" is because wait_event and wait_event_type
    could be non empty in pg_stat_activity while the session is not in an active state
    anymore (then not waiting).
    
    A right query would be like the one in [1]:
    
    "
    SELECT a.pid, a.wait_event, w.description
      FROM pg_stat_activity a JOIN
           pg_wait_events w ON (a.wait_event_type = w.type AND
                                a.wait_event = w.name)
      WHERE a.wait_event is NOT NULL and a.state = 'active';
    "
    
    means filtering on the "active" state too, and that's what the description
    proposal I made was trying to highlight.
    
    [1]: https://www.postgresql.org/docs/devel/monitoring-stats.html
    
    Regards,
    
    -- 
    Bertrand Drouvot
    PostgreSQL Contributors Team
    RDS Open Source Databases
    Amazon Web Services: https://aws.amazon.com
    
    
    
    

22. Re: PostgreSQL 17 Beta 1 release announcement draft

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-20T10:08:32Z

    On 2024-May-19, Jonathan S. Katz wrote:
    
    > ### Query and Operational Performance Improvements
    
    In this section I'd add mention the new GUCs to control SLRU memory
    size, which is going to be a huge performance boon for cases where the
    current fixed-size buffers cause bottlenecks.  Perhaps something like
    
    "Increase scalability of transaction, subtransaction and multixact
    shared memory buffer handling, and make their buffer sizes configurable".
    
    I don't know if we have any published numbers of the performance
    improvement achieved, but with this patch (or ancestor ones) some
    systems go from completely unoperational to working perfectly fine.
    Maybe the best link is here
    https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS
    though exactly which GUC affects any particular user is workload-
    dependant, so I'm not sure how best to do it.
    
    > ### Developer Experience
    
    I think this section should also include the libpq query cancellation
    improvements Jelte wrote.  Maybe something like "On the client side,
    PostgreSQL 17 provides better support for asynchronous and more secure
    query cancellation routines in libpq."  --> link to
    https://www.postgresql.org/docs/17/libpq-cancel.html
    
    -- 
    Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
    "At least to kernel hackers, who really are human, despite occasional
    rumors to the contrary" (LWN.net)
    
    
    
    

23. Re: PostgreSQL 17 Beta 1 release announcement draft

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-20T10:11:37Z

    On 2024-May-16, David Rowley wrote:
    
    > On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
    > > I am trying to open the 17 docs but it looks removed. Getting
    > > following message "Page not found"
    > >
    > > https://www.postgresql.org/docs/17/
    > 
    > It's called "devel" for "development" until we branch sometime before July:
    > 
    > https://www.postgresql.org/docs/devel/
    
    Hmm, but that would mean that the Beta1 announce would ship full of
    links that will remain broken until July.  I'm not sure what the
    workflow for this is, but I hope the /17/ URLs would become valid with
    beta1, later this week.
    
    -- 
    Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
    "No renuncies a nada. No te aferres a nada."
    
    
    
    

24. Re: PostgreSQL 17 Beta 1 release announcement draft

    David Rowley <dgrowleyml@gmail.com> — 2024-05-20T10:32:36Z

    On Mon, 20 May 2024 at 22:11, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
    >
    > On 2024-May-16, David Rowley wrote:
    >
    > > On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
    > > > I am trying to open the 17 docs but it looks removed. Getting
    > > > following message "Page not found"
    > > >
    > > > https://www.postgresql.org/docs/17/
    > >
    > > It's called "devel" for "development" until we branch sometime before July:
    > >
    > > https://www.postgresql.org/docs/devel/
    >
    > Hmm, but that would mean that the Beta1 announce would ship full of
    > links that will remain broken until July.  I'm not sure what the
    > workflow for this is, but I hope the /17/ URLs would become valid with
    > beta1, later this week.
    
    I didn't quite click that it was Jonathan's links that were being
    complained about.
    
    I don't know how the website picks up where to link the doc page for a
    given version.  I see from e0b82fc8e8 that the PACKAGE_VERSION was
    changed from 16devel to 16beta1. Does the website have something that
    extracts "devel" from the former and "16" from the latter?  I see the
    release announcement for 16beta1 had /16/ links per [1].  So, I guess
    it works. I just don't know how.
    
    David
    
    [1] https://www.postgresql.org/about/news/postgresql-16-beta-1-released-2643/
    
    
    
    

25. Re: PostgreSQL 17 Beta 1 release announcement draft

    Thom Brown <thom@linux.com> — 2024-05-20T11:15:32Z

    On Mon, 20 May 2024 at 00:24, David Rowley <dgrowleyml@gmail.com> wrote:
    
    > On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org>
    > wrote:
    > > Thanks for all the feedback to date. Please see the next revision.
    > > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    >
    > Thanks for the updates.
    >
    > > [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to
    > efficiently bulk load data into PostgreSQL, and with PostgreSQL 17 shows a
    > 2x performance improvement when loading large rows.
    >
    > The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
    > So I think "exporting" or "sending large rows to the client"  rather
    > than "loading".
    >
    > There's also a stray "with" in that sentence.
    >
    
    Are you referring to the "with" in "and with PostgreSQL 17"? If so, it
    looks valid to me.
    -- 
    Thom
    

26. Re: PostgreSQL 17 Beta 1 release announcement draft

    David Rowley <dgrowleyml@gmail.com> — 2024-05-20T11:31:57Z

    On Mon, 20 May 2024 at 23:16, Thom Brown <thom@linux.com> wrote:
    >
    > On Mon, 20 May 2024 at 00:24, David Rowley <dgrowleyml@gmail.com> wrote:
    >>
    >> On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >> > Thanks for all the feedback to date. Please see the next revision.
    >> > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    >>
    >> Thanks for the updates.
    >>
    >> > [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL, and with PostgreSQL 17 shows a 2x performance improvement when loading large rows.
    >>
    >> The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
    >> So I think "exporting" or "sending large rows to the client"  rather
    >> than "loading".
    >>
    >> There's also a stray "with" in that sentence.
    >
    >
    > Are you referring to the "with" in "and with PostgreSQL 17"? If so, it looks valid to me.
    
    Yes that one.  It sounds wrong to me, but that's from a British
    English point of view. I'm continuing to learn the subtle differences
    with American English. Maybe this is one.
    
    It would make sense to me if it was "and with PostgreSQL 17, a 2x
    ...". From my point of view either "with" shouldn't be there or
    "shows" could be replaced with a comma. However, if you're ok with it,
    I'll say no more. I know this is well into your territory.
    
    David
    
    
    
    

27. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-20T11:44:11Z

    On 5/19/24 7:24 PM, David Rowley wrote:
    > On Mon, 20 May 2024 at 09:35, Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >> Thanks for all the feedback to date. Please see the next revision.
    >> Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    > 
    > Thanks for the updates.
    > 
    >> [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to efficiently bulk load data into PostgreSQL, and with PostgreSQL 17 shows a 2x performance improvement when loading large rows.
    > 
    > The 2x thing mentioned by Jelte is for COPY TO rather than COPY FROM.
    > So I think "exporting" or "sending large rows to the client"  rather
    > than "loading".
    
    Thanks for the clarification - I've edited it as such. That also brings 
    up a good point to highlight that COPY is not just for loading (since my 
    bias is to do loads these days :) Now it reads:
    
    [`COPY`](https://www.postgresql.org/docs/17/sql-copy.html) is used to 
    efficiently bulk load and export data from PostgreSQL, and now with 
    PostgreSQL 17 you may see up to a 2x performance improvement when 
    exporting large rows.
    
    > There's also a stray "with" in that sentence.
    
    Thanks, fixed.
    
    Jonathan
    
    

28. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-20T11:47:54Z

    On 5/20/24 2:58 AM, John Naylor wrote:
    > Hi Jon,
    > 
    > Regarding vacuum "has shown up to a 6x improvement in overall time to
    > complete its work" -- I believe I've seen reported numbers close to
    > that only 1) when measuring the index phase in isolation or maybe 2)
    > the entire vacuum of unlogged tables with one, perfectly-correlated
    > index (testing has less variance with WAL out of the picture). I
    > believe tables with many indexes would show a lot of improvement, but
    > I'm not aware of testing that case specifically. Can you clarify where
    > 6x came from?
    
    Sawada-san showed me the original context, but I can't rapidly find it 
    in the thread. Sawada-san, can you please share the numbers behind this?
    
    We can adjust the claim - but I'd like to ensure we highlight how the 
    changes to vacuum will visibly impact users.
    
    Jonathan
    

29. Re: PostgreSQL 17 Beta 1 release announcement draft

    jian he <jian.universality@gmail.com> — 2024-05-20T12:31:38Z

    On Mon, May 20, 2024 at 5:35 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >
    > On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
    > > Hi,
    > >
    > > Attached is a copy of the PostgreSQL 17 Beta 1 release announcement
    > > draft. This contains a user-facing summary of some of the features that
    > > will be available in the Beta, as well as a call to test. I've made an
    > > effort to group them logically around the different workflows they affect.
    > >
    > > A few notes:
    > >
    > > * The section with the features is not 80-char delimited. I will do that
    > > before the final copy
    > >
    > > * There is an explicit callout that we've added in the SQL/JSON features
    > > that were previously reverted in PG15. I want to ensure we're
    > > transparent about that, but also use it as a hook to get people testing.
    > >
    > > When reviewing:
    > >
    > > * Please check for correctness of feature descriptions, keeping in mind
    > > this is targeted for a general audience
    > >
    > > * Please indicate if you believe there's a notable omission, or if we
    > > should omit any of these callouts
    > >
    > > * Please indicate if a description is confusing - I'm happy to rewrite
    > > to ensure it's clearer.
    > >
    > > Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the
    > > beta release takes some extra effort, I want to ensure all changes are
    > > in with time to spare before release day.
    >
    > Thanks for all the feedback to date. Please see the next revision.
    > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    >
    > Thanks,
    >
    > Jonathan
    >
    
    release note (https://momjian.us/pgsql_docs/release-17.html)
    is
    "Add jsonpath methods to convert JSON values to other JSON data types
    (Jeevan Chalke)"
    
    
    >> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to different data types.
    so, I am not sure this is 100% correct.
    
    Maybe we can rephrase it like:
    
    >> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to other JSON data types.
    
    
    
    

30. Re: PostgreSQL 17 Beta 1 release announcement draft

    Masahiko Sawada <sawada.mshk@gmail.com> — 2024-05-20T13:40:32Z

    On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >
    > On 5/20/24 2:58 AM, John Naylor wrote:
    > > Hi Jon,
    > >
    > > Regarding vacuum "has shown up to a 6x improvement in overall time to
    > > complete its work" -- I believe I've seen reported numbers close to
    > > that only 1) when measuring the index phase in isolation or maybe 2)
    > > the entire vacuum of unlogged tables with one, perfectly-correlated
    > > index (testing has less variance with WAL out of the picture). I
    > > believe tables with many indexes would show a lot of improvement, but
    > > I'm not aware of testing that case specifically. Can you clarify where
    > > 6x came from?
    >
    > Sawada-san showed me the original context, but I can't rapidly find it
    > in the thread. Sawada-san, can you please share the numbers behind this?
    >
    
    I referenced the numbers that I measured during the development[1]
    (test scripts are here[2]). IIRC I used unlogged tables and indexes,
    and these numbers were the entire vacuum execution time including heap
    scanning, index vacuuming and heap vacuuming.
    
    FYI today I've run the same script with PG17 and measured the
    execution times. Here are results:
    
    monotonically ordered int column index:
    system usage: CPU: user: 1.72 s, system: 0.47 s, elapsed: 2.20 s
    
    uuid column index:
    system usage: CPU: user: 3.62 s, system: 0.89 s, elapsed: 4.52 s
    
    int & uuid indexes in parallel:
    system usage: CPU: user: 2.24 s, system: 0.44 s, elapsed: 5.01 s
    
    These numbers are better than ones I measured with v62 patch set as we
    now introduced some optimization into tidstore (8a1b31e6 and f35bd9b).
    
    Regards,
    
    [1] https://www.postgresql.org/message-id/CAD21AoBci3Hujzijubomo1tdwH3XtQ9F89cTNQ4bsQijOmqnEw%40mail.gmail.com
    [2] https://www.postgresql.org/message-id/CANWCAZYqWibTRCWs5mV57mLj1A0nbKX-eV5G%2Bd-KmBOGHTVY-w%40mail.gmail.com
    
    -- 
    Masahiko Sawada
    Amazon Web Services: https://aws.amazon.com
    
    
    
    

31. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-20T16:44:23Z

    On 5/20/24 8:31 AM, jian he wrote:
    
    > release note (https://momjian.us/pgsql_docs/release-17.html)
    > is
    > "Add jsonpath methods to convert JSON values to other JSON data types
    > (Jeevan Chalke)"
    > 
    > 
    >>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to different data types.
    > so, I am not sure this is 100% correct.
    > 
    > Maybe we can rephrase it like:
    > 
    >>> Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to other JSON data types.
    
    The release note goes on to state:
    
    ==
    The jsonpath methods are .bigint(), .boolean(), .date(), 
    .decimal([precision [, scale]]), .integer(), .number(), .string(), 
    .time(), .time_tz(), .timestamp(), and .timestamp_tz().
    ==
    
    And reviewing the docs[1], these are converted to a PostgreSQL native 
    types, and not JSON types (additionally a bunch of those are not JSON 
    types).
    
    Jeevan: can you please confirm that this work converts into the 
    PostgreSQL native types?
    
    Thanks,
    
    Jonathan
    
    [1] https://www.postgresql.org/docs/devel/functions-json.html
    

32. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-20T16:45:39Z

    On 5/20/24 6:32 AM, David Rowley wrote:
    > On Mon, 20 May 2024 at 22:11, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
    >>
    >> On 2024-May-16, David Rowley wrote:
    >>
    >>> On Thu, 16 May 2024 at 17:37, zaidagilist <zaidagilist@gmail.com> wrote:
    >>>> I am trying to open the 17 docs but it looks removed. Getting
    >>>> following message "Page not found"
    >>>>
    >>>> https://www.postgresql.org/docs/17/
    >>>
    >>> It's called "devel" for "development" until we branch sometime before July:
    >>>
    >>> https://www.postgresql.org/docs/devel/
    >>
    >> Hmm, but that would mean that the Beta1 announce would ship full of
    >> links that will remain broken until July.  I'm not sure what the
    >> workflow for this is, but I hope the /17/ URLs would become valid with
    >> beta1, later this week.
    > 
    > I didn't quite click that it was Jonathan's links that were being
    > complained about.
    > 
    > I don't know how the website picks up where to link the doc page for a
    > given version.  I see from e0b82fc8e8 that the PACKAGE_VERSION was
    > changed from 16devel to 16beta1. Does the website have something that
    > extracts "devel" from the former and "16" from the latter?  I see the
    > release announcement for 16beta1 had /16/ links per [1].  So, I guess
    > it works. I just don't know how.
    
    The tl;dr is that the /17/ links will be available on release day. I've 
    validated the current links using the /devel/ heading.
    
    Jonathan
    
    

33. Re: PostgreSQL 17 Beta 1 release announcement draft

    John Naylor <johncnaylorls@gmail.com> — 2024-05-21T10:40:46Z

    On Mon, May 20, 2024 at 8:41 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
    >
    > On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
    > >
    > > On 5/20/24 2:58 AM, John Naylor wrote:
    > > > Hi Jon,
    > > >
    > > > Regarding vacuum "has shown up to a 6x improvement in overall time to
    > > > complete its work" -- I believe I've seen reported numbers close to
    > > > that only 1) when measuring the index phase in isolation or maybe 2)
    > > > the entire vacuum of unlogged tables with one, perfectly-correlated
    > > > index (testing has less variance with WAL out of the picture). I
    > > > believe tables with many indexes would show a lot of improvement, but
    > > > I'm not aware of testing that case specifically. Can you clarify where
    > > > 6x came from?
    > >
    > > Sawada-san showed me the original context, but I can't rapidly find it
    > > in the thread. Sawada-san, can you please share the numbers behind this?
    > >
    >
    > I referenced the numbers that I measured during the development[1]
    > (test scripts are here[2]). IIRC I used unlogged tables and indexes,
    > and these numbers were the entire vacuum execution time including heap
    > scanning, index vacuuming and heap vacuuming.
    
    Thanks for confirming.
    
    The wording "has a new internal data structure that reduces memory
    usage and has shown up to a 6x improvement in overall time to complete
    its work" is specific for runtime, and the memory use is less
    specific. Unlogged tables are not the norm, so I'd be cautious of
    reporting numbers specifically designed (for testing) to isolate the
    thing that changed.
    
    I'm wondering if it might be both more impressive-sounding and more
    realistic for the average user experience to reverse that: specific on
    memory, and less specific on speed. The best-case memory reduction
    occurs for table update patterns that are highly localized, such as
    the most recently inserted records, and I'd say those are a lot more
    common than the use of unlogged tables.
    
    Maybe something like "has a new internal data structure that reduces
    overall time to complete its work and can use up to 20x less memory."
    
    Now, it is true that when dead tuples are sparse and evenly spaced
    (e.g. 1 every 100 pages), vacuum can now actually use more memory than
    v16. However, the nature of that scenario also means that the number
    of TIDs just can't get very big to begin with. In contrast, while the
    runtime improvement for normal (logged) tables is likely not
    earth-shattering, I believe it will always be at least somewhat
    faster, and never slower.
    
    
    
    

34. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-22T23:01:54Z

    On 5/20/24 5:34 AM, Bertrand Drouvot wrote:
    > Hi,
    > 
    > On Sun, May 19, 2024 at 05:10:10PM -0400, Jonathan S. Katz wrote:
    >> On 5/16/24 1:15 AM, Bertrand Drouvot wrote:
    >>> Hi,
    >>>
    >>> On Wed, May 15, 2024 at 09:45:35PM -0400, Jonathan S. Katz wrote:
    >>>> Hi,
    >>>>
    >>>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement draft.
    >>>
    >>> Thanks for working on it!
    >>>
    >>> I've one comment:
    >>>
    >>>> PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into an operation.
    >>>
    >>> Instead of "to give more insight into an operation", what about "to give more
    >>> insight about what a session is waiting for (should it be active)"?
    >>
    >> I put:
    >>
    >> "to give more in insight into why a session is blocked."
    > 
    > Thanks!
    > 
    >>
    >> Does that work?
    >>
    > 
    > I think using "waiting" is better (as the view is "pg_wait_events" and the
    > join with pg_stat_activity would be on the "wait_event_type" and "wait_event"
    > columns).
    > 
    > The reason I mentioned "should it be active" is because wait_event and wait_event_type
    > could be non empty in pg_stat_activity while the session is not in an active state
    > anymore (then not waiting).
    > 
    > A right query would be like the one in [1]:
    > 
    > "
    > SELECT a.pid, a.wait_event, w.description
    >    FROM pg_stat_activity a JOIN
    >         pg_wait_events w ON (a.wait_event_type = w.type AND
    >                              a.wait_event = w.name)
    >    WHERE a.wait_event is NOT NULL and a.state = 'active';
    > "
    > 
    > means filtering on the "active" state too, and that's what the description
    > proposal I made was trying to highlight.
    
    Thanks. As such I made it:
    
    "which provides descriptions about wait events and can be combined with 
    `pg_stat_activity` to give more insight into why an active session is 
    waiting."
    
    Jonathan
    

35. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-22T23:02:36Z

    On 5/19/24 6:15 PM, Erik Rijkers wrote:
    > Op 5/19/24 om 23:34 schreef Jonathan S. Katz:
    >> On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
    >>> Hi,
    >>>
    >>> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
    > 
    > 'This release introduces adds an interface'  should be:
    > 'This release adds an interface'
    >     (or 'introduces'; just not both...)
    
    Thanks; adjusted in the next copy.
    
    Jonathan
    
    

36. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-22T23:10:33Z

    On 5/20/24 6:08 AM, Alvaro Herrera wrote:
    > On 2024-May-19, Jonathan S. Katz wrote:
    > 
    >> ### Query and Operational Performance Improvements
    > 
    > In this section I'd add mention the new GUCs to control SLRU memory
    > size, which is going to be a huge performance boon for cases where the
    > current fixed-size buffers cause bottlenecks.  Perhaps something like
    > 
    > "Increase scalability of transaction, subtransaction and multixact
    > shared memory buffer handling, and make their buffer sizes configurable".
    > 
    > I don't know if we have any published numbers of the performance
    > improvement achieved, but with this patch (or ancestor ones) some
    > systems go from completely unoperational to working perfectly fine.
    > Maybe the best link is here
    > https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS
    > though exactly which GUC affects any particular user is workload-
    > dependant, so I'm not sure how best to do it.
    
    I originally had penciled in this change, but didn't have a good way of 
    describing it. The above solves that problem. I went with:
    
    "PostgreSQL 17 also includes configuration parameters that can control 
    scalability of [transaction, subtransaction and multixact 
    buffers](https://www.postgresql.org/docs/devel/runtime-config-resource.html#GUC-MULTIXACT-MEMBER-BUFFERS)."
    
    >> ### Developer Experience
    > 
    > I think this section should also include the libpq query cancellation
    > improvements Jelte wrote.  Maybe something like "On the client side,
    > PostgreSQL 17 provides better support for asynchronous and more secure
    > query cancellation routines in libpq."  --> link to
    > https://www.postgresql.org/docs/17/libpq-cancel.html
    
    I went with:
    
    PostgreSQL 17 also provides better support for [asynchronous and more 
    secure query cancellation 
    routines](https://www.postgresql.org/docs/17/libpq-cancel.html), which 
    drivers can adopt using the libpq API.
    
    Thanks,
    
    Jonathan
    
    

37. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-22T23:16:33Z

    On 5/21/24 6:40 AM, John Naylor wrote:
    > On Mon, May 20, 2024 at 8:41 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
    >>
    >> On Mon, May 20, 2024 at 8:47 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
    >>>
    >>> On 5/20/24 2:58 AM, John Naylor wrote:
    >>>> Hi Jon,
    >>>>
    >>>> Regarding vacuum "has shown up to a 6x improvement in overall time to
    >>>> complete its work" -- I believe I've seen reported numbers close to
    >>>> that only 1) when measuring the index phase in isolation or maybe 2)
    >>>> the entire vacuum of unlogged tables with one, perfectly-correlated
    >>>> index (testing has less variance with WAL out of the picture). I
    >>>> believe tables with many indexes would show a lot of improvement, but
    >>>> I'm not aware of testing that case specifically. Can you clarify where
    >>>> 6x came from?
    >>>
    >>> Sawada-san showed me the original context, but I can't rapidly find it
    >>> in the thread. Sawada-san, can you please share the numbers behind this?
    >>>
    >>
    >> I referenced the numbers that I measured during the development[1]
    >> (test scripts are here[2]). IIRC I used unlogged tables and indexes,
    >> and these numbers were the entire vacuum execution time including heap
    >> scanning, index vacuuming and heap vacuuming.
    > 
    > Thanks for confirming.
    > 
    > The wording "has a new internal data structure that reduces memory
    > usage and has shown up to a 6x improvement in overall time to complete
    > its work" is specific for runtime, and the memory use is less
    > specific. Unlogged tables are not the norm, so I'd be cautious of
    > reporting numbers specifically designed (for testing) to isolate the
    > thing that changed.
    > 
    > I'm wondering if it might be both more impressive-sounding and more
    > realistic for the average user experience to reverse that: specific on
    > memory, and less specific on speed. The best-case memory reduction
    > occurs for table update patterns that are highly localized, such as
    > the most recently inserted records, and I'd say those are a lot more
    > common than the use of unlogged tables.
    > 
    > Maybe something like "has a new internal data structure that reduces
    > overall time to complete its work and can use up to 20x less memory."
    > 
    > Now, it is true that when dead tuples are sparse and evenly spaced
    > (e.g. 1 every 100 pages), vacuum can now actually use more memory than
    > v16. However, the nature of that scenario also means that the number
    > of TIDs just can't get very big to begin with. In contrast, while the
    > runtime improvement for normal (logged) tables is likely not
    > earth-shattering, I believe it will always be at least somewhat
    > faster, and never slower.
    
    Thanks for the feedback. I flipped it around, per your suggestion:
    
    "has a new internal data structure that has shown up to a 20x memory 
    reduction for vacuum, along with improvements in overall time to 
    complete its work."
    
    Thanks,
    
    Jonathan
    

38. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-22T23:17:39Z

    On 5/19/24 5:34 PM, Jonathan S. Katz wrote:
    > On 5/15/24 9:45 PM, Jonathan S. Katz wrote:
    >> Hi,
    >>
    >> Attached is a copy of the PostgreSQL 17 Beta 1 release announcement 
    >> draft. This contains a user-facing summary of some of the features 
    >> that will be available in the Beta, as well as a call to test. I've 
    >> made an effort to group them logically around the different workflows 
    >> they affect.
    >>
    >> A few notes:
    >>
    >> * The section with the features is not 80-char delimited. I will do 
    >> that before the final copy
    >>
    >> * There is an explicit callout that we've added in the SQL/JSON 
    >> features that were previously reverted in PG15. I want to ensure we're 
    >> transparent about that, but also use it as a hook to get people testing.
    >>
    >> When reviewing:
    >>
    >> * Please check for correctness of feature descriptions, keeping in 
    >> mind this is targeted for a general audience
    >>
    >> * Please indicate if you believe there's a notable omission, or if we 
    >> should omit any of these callouts
    >>
    >> * Please indicate if a description is confusing - I'm happy to rewrite 
    >> to ensure it's clearer.
    >>
    >> Please provide feedback no later than Wed 2024-05-22 18:00 UTC. As the 
    >> beta release takes some extra effort, I want to ensure all changes are 
    >> in with time to spare before release day.
    > 
    > Thanks for all the feedback to date. Please see the next revision. 
    > Again, please provide feedback no later than 2024-05-22 18:00 UTC.
    
    Thanks again everyone for all your feedback. Attached is the final(-ish, 
    as I'll do one more readthrough before release) draft of the release 
    announcement.
    
    If you catch something and are able to post it prior to 2024-05-23 12:00 
    UTC, I may be able to incorporate into the announcement.
    
    Thanks!
    
    Jonathan
    
    

39. Re: PostgreSQL 17 Beta 1 release announcement draft

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-05-23T04:21:08Z

    Hi,
    
    On Wed, May 22, 2024 at 07:01:54PM -0400, Jonathan S. Katz wrote:
    > Thanks. As such I made it:
    > 
    > "which provides descriptions about wait events and can be combined with
    > `pg_stat_activity` to give more insight into why an active session is
    > waiting."
    > 
    
    Thanks! Works for me.
    
    Regards,
    
    -- 
    Bertrand Drouvot
    PostgreSQL Contributors Team
    RDS Open Source Databases
    Amazon Web Services: https://aws.amazon.com
    
    
    
    

40. Re: PostgreSQL 17 Beta 1 release announcement draft

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-23T12:00:13Z

    Looks good.  Some minor changes:
    
    On 2024-May-22, Jonathan S. Katz wrote:
    
    > ### Query and Operational Performance Improvements
    > 
    > PostgreSQL 17 builds on recent releases and continues to improve performance across the entire system. [Vacuum](https://www.postgresql.org/docs/17/routine-vacuuming.html), the PostgreSQL process responsible for reclaiming storage, has a new internal data structure that has shown up to a 20x memory reduction for vacuum,
    
    This reads funny:
    "Vacuum ... has shown a memory reduction for vacuum"
    Maybe just removing the "for vacuum" words at the end of the phrase is a
    sufficient fix.
    
    > PostgreSQL 17 can now use both planner statistics and the sort order of [common table expressions](https://www.postgresql.org/docs/17/queries-with.html) (aka [`WITH` queries](https://www.postgresql.org/docs/17/queries-with.html)) to further
    
    Is usage of "aka" typical?  I would have expected "a.k.a." but maybe I'm
    just outdated.
    
    
    > Finally, PostgreSQL 17 adds more explicit SIMD instructions, including AVX-512 support for the [`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html) function.
    
    Note the lack of closing backtick in [`bit_count`].
    
    > ### Developer Experience
    > 
    > PostgreSQL 17 continues to build on the SQL/JSON standard, adding support for the `JSON_TABLE` features that can convert JSON to a standard PostgreSQL table, and SQL/JSON constructor (`JSON`, `JSON_SCALAR`, `JSON_SERIALIZE`) and query functions (`JSON_EXISTS`, `JSON_QUERY`, `JSON_VALUE`). Notably, these features were originally planned for the PostgreSQL 15 release but were reverted during the beta period due to design considerations, which is one reason we ask for you to help us test features during beta! Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to different data types.
    
    I'm not sure it's accurate to say that converting JSON values to
    different datatypes is part of the jsonpath implementation; as I
    understand, jsonpath is the representation used to search for elements
    within JSON values.  If you replace "including" with "and", the result
    seems reasonable.
    
    > PostgreSQL 17 adds a new connection parameter, `sslnegotation`, which allows PostgreSQL to perform direct TLS handshakes when using [ALPN](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation), eliminating a network roundtrip. PostgreSQL is registered as `postgresql` in the ALPN directory.
    
    Typo here "sslnegotation" missing an i, sslnegotiation.
    
    
    > PostgreSQL 17 normalizes the parameters for `CALL` in [`pg_stat_statements`](https://www.postgresql.org/docs/17/pgstatstatements.html), reducing the number of entries for frequently called stored procedures. Additionally, [`VACUUM` progress reporting](https://www.postgresql.org/docs/devel/progress-reporting.html#VACUUM-PROGRESS-REPORTING) now shows the progress of vacuuming indexes. PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into why an active session is waiting. Additionally, some information in the [`pg_stat_bgwriter`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW) view is now split out into the new [`pg_stat_checkpointer`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-CHECKPOINTER-VIEW) view.
    
    Note use of one link to "/devel/" here.
    
    -- 
    Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
    "Always assume the user will do much worse than the stupidest thing
    you can imagine."                                (Julien PUYDT)
    
    
    
    

41. Re: PostgreSQL 17 Beta 1 release announcement draft

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-05-23T12:22:49Z

    On 5/23/24 8:00 AM, Alvaro Herrera wrote:
    > Looks good.  Some minor changes:
    
    Thanks for this - right at the deadline! :D
    
    >> ### Query and Operational Performance Improvements
    >>
    >> PostgreSQL 17 builds on recent releases and continues to improve performance across the entire system. [Vacuum](https://www.postgresql.org/docs/17/routine-vacuuming.html), the PostgreSQL process responsible for reclaiming storage, has a new internal data structure that has shown up to a 20x memory reduction for vacuum,
    > 
    > This reads funny:
    > "Vacuum ... has shown a memory reduction for vacuum"
    > Maybe just removing the "for vacuum" words at the end of the phrase is a
    > sufficient fix.
    
    Fixed.
    
    >> PostgreSQL 17 can now use both planner statistics and the sort order of [common table expressions](https://www.postgresql.org/docs/17/queries-with.html) (aka [`WITH` queries](https://www.postgresql.org/docs/17/queries-with.html)) to further
    > 
    > Is usage of "aka" typical?  I would have expected "a.k.a." but maybe I'm
    > just outdated.
    
    Removed, just b/c I can't quickly verify this.
    
    >> Finally, PostgreSQL 17 adds more explicit SIMD instructions, including AVX-512 support for the [`bit_count](https://www.postgresql.org/docs/17/functions-bitstring.html) function.
    > 
    > Note the lack of closing backtick in [`bit_count`].
    
    Fixed.
    
    >> ### Developer Experience
    >>
    >> PostgreSQL 17 continues to build on the SQL/JSON standard, adding support for the `JSON_TABLE` features that can convert JSON to a standard PostgreSQL table, and SQL/JSON constructor (`JSON`, `JSON_SCALAR`, `JSON_SERIALIZE`) and query functions (`JSON_EXISTS`, `JSON_QUERY`, `JSON_VALUE`). Notably, these features were originally planned for the PostgreSQL 15 release but were reverted during the beta period due to design considerations, which is one reason we ask for you to help us test features during beta! Additionally, PostgreSQL 17 adds more functionality to its `jsonpath` implementation, including the ability to convert JSON values to different data types.
    > 
    > I'm not sure it's accurate to say that converting JSON values to
    > different datatypes is part of the jsonpath implementation; as I
    > understand, jsonpath is the representation used to search for elements
    > within JSON values.  If you replace "including" with "and", the result
    > seems reasonable.
    
    Fixed.
    
    >> PostgreSQL 17 adds a new connection parameter, `sslnegotation`, which allows PostgreSQL to perform direct TLS handshakes when using [ALPN](https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation), eliminating a network roundtrip. PostgreSQL is registered as `postgresql` in the ALPN directory.
    > 
    > Typo here "sslnegotation" missing an i, sslnegotiation.
    
    Fixed.
    
    >> PostgreSQL 17 normalizes the parameters for `CALL` in [`pg_stat_statements`](https://www.postgresql.org/docs/17/pgstatstatements.html), reducing the number of entries for frequently called stored procedures. Additionally, [`VACUUM` progress reporting](https://www.postgresql.org/docs/devel/progress-reporting.html#VACUUM-PROGRESS-REPORTING) now shows the progress of vacuuming indexes. PostgreSQL 17 also introduces a new view, [`pg_wait_events`](https://www.postgresql.org/docs/17/view-pg-wait-events.html), which provides descriptions about wait events and can be combined with `pg_stat_activity` to give more insight into why an active session is waiting. Additionally, some information in the [`pg_stat_bgwriter`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-BGWRITER-VIEW) view is now split out into the new [`pg_stat_checkpointer`](https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-CHECKPOINTER-VIEW) view.
    > 
    > Note use of one link to "/devel/" here.
    
    Fixed.
    
    Thanks!
    
    Jonathan