Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Revert support for ALTER TABLE ... MERGE/SPLIT PARTITION(S) commands

  2. When creating materialized views, use REFRESH to load data.

  3. Revert temporal primary keys and foreign keys

  4. Avoid needless large memcpys in libpq socket writing

  5. Enhance nbtree ScalarArrayOp execution.

  6. Introduce a non-recursive JSON parser

  7. Combine freezing and pruning steps in VACUUM

  8. Allow SIGINT to cancel psql database reconnections.

  9. Provide API for streaming relation data.

  10. Add hash support functions and hash opclass for contrib/ltree.

  11. Pull up ANY-SUBLINK with the necessary lateral support.

  12. Read WAL directly from WAL buffers.

  13. Introduce the dynamic shared memory registry.

  14. Add macros for looping through a List without a ListCell.

  15. Support +/- infinity in the interval data type.

  16. Extend ALTER OPERATOR to allow setting more optimization attributes.

  17. Consider cheap startup paths in add_paths_to_append_rel

  1. First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T04:03:50Z

    I have committed the first draft of the PG 17 release notes;  you can
see the results here:

	https://momjian.us/pgsql_docs/release-17.html

It will be improved until the final release.  The item count is 188,
which is similar to recent releases:

	release-10:  189
	release-11:  170
	release-12:  180
	release-13:  178
	release-14:  220
	release-15:  184
	release-16:  206
	release-17:  188

I welcome feedback.  For some reason it was an easier job than usual.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  2. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-09T04:44:47Z

    On Thu, 9 May 2024 at 16:04, Bruce Momjian <bruce@momjian.us> wrote:
> I welcome feedback.  For some reason it was an easier job than usual.

Thanks for working on that.

> +2023-11-02 [cac169d68] Increase DEFAULT_FDW_TUPLE_COST from 0.01 to 0.2

> +Double the default foreign data wrapper tuple cost (David Rowley, Umair Shahid)

That's 20x rather than 2x.

David

  3. Re: First draft of PG 17 release notes

    Muhammad Ikram <mmikram@gmail.com> — 2024-05-09T04:47:34Z

    Hi Bruce,

A minor formatting issue in the start below. Bullet is not required here.

E.1.1. Overview
<https://momjian.us/pgsql_docs/release-17.html#RELEASE-17-HIGHLIGHTS>

PostgreSQL 17 contains many new features and enhancements, including:

   -


The above items and other new features of PostgreSQL 17 are explained in
more detail in the sections below.
Regards,
Ikram



On Thu, May 9, 2024 at 9:45 AM David Rowley <dgrowleyml@gmail.com> wrote:

> On Thu, 9 May 2024 at 16:04, Bruce Momjian <bruce@momjian.us> wrote:
> > I welcome feedback.  For some reason it was an easier job than usual.
>
> Thanks for working on that.
>
> > +2023-11-02 [cac169d68] Increase DEFAULT_FDW_TUPLE_COST from 0.01 to 0.2
>
> > +Double the default foreign data wrapper tuple cost (David Rowley, Umair
> Shahid)
>
> That's 20x rather than 2x.
>
> David
>
>
>

-- 
Muhammad Ikram

  4. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-09T04:52:14Z

    On Thu, 9 May 2024 at 16:47, Muhammad Ikram <mmikram@gmail.com> wrote:
> A minor formatting issue in the start below. Bullet is not required here.

This is a placeholder for the highlight features of v17 will go.
Bruce tends not to decide what those are all by himself.

David

  5. Re: First draft of PG 17 release notes

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-05-09T04:53:38Z

    Hi,

On Thu, May 09, 2024 at 12:03:50AM -0400, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
> 	https://momjian.us/pgsql_docs/release-17.html

Thanks for working on that!
 
> I welcome feedback.

> Add system view pg_wait_events that reports wait event types (Michael Paquier) 

Michael is the committer for 1e68e43d3f, the author is me.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  6. Re: First draft of PG 17 release notes

    Masahiko Sawada <sawada.mshk@gmail.com> — 2024-05-09T05:17:12Z

    Hi,

On Thu, May 9, 2024 at 1:03 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html

Thank you for working on that!

I'd like to mention some of my works. I think we can add the vacuum
performance improvements by the following commits:

- Add template for adaptive radix tree (ee1b30f1)
- Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently (30e144287)
- Use TidStore for dead tuple TIDs storage during lazy vacuum (667e65aac)

Also, please consider the following item:

- Improve eviction algorithm in ReorderBuffer using max-heap for many
subtransactions (5bec1d6bc)

Finally, should we mention the following commit in the release note?
It's not a user-visible change but added a new regression test module.

- Add tests for XID wraparound (e255b646a)

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

  7. Re: First draft of PG 17 release notes

    Richard Guo <guofenglinux@gmail.com> — 2024-05-09T06:37:57Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html


Thanks for working on that.

For this item:


> Allow the optimizer to improve CTE plans by using the sort order of
> columns referenced in earlier CTE clauses (Jian Guo)


I think you mean a65724dfa.  The author should be 'Richard Guo'.

And I'm wondering if it is more accurate to state it as "Allow the
optimizer to improve plans for the outer query by leveraging the sort
order of a CTE's output."

I think maybe a similar revision can be applied to the item just above
this one.

Thanks
Richard

  8. Re: First draft of PG 17 release notes

    Aleksander Alekseev <aleksander@timescale.com> — 2024-05-09T09:18:44Z

    Hi,

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:

Thanks for working on this.

I believe the part of the 64-bit XIDs patchset that was delivered in
PG17 is worth highlighting in "E.1.3.10. Source Code" section:

4ed8f0913bfd
2cdf131c46e6
5a1dfde8334b
a60b8a58f435

All this can probably be summarized as one bullet "Index SLRUs by
64-bit integers rather than by 32-bit ones" where the authors are:
Maxim Orlov, Aleksander Alekseev, Alexander Korotkov, Teodor Sigaev,
Nikita Glukhov, Pavel Borisov, Yura Sokolov.

-- 
Best regards,
Aleksander Alekseev

  9. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-09T10:00:24Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>

another potential incompatibilities issue:
ALTER TABLE DROP PRIMARY KEY

see:
https://www.postgresql.org/message-id/202404181849.6frtmajobe27%40alvherre.pgsql

  10. Re: First draft of PG 17 release notes

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2024-05-09T10:22:06Z

    Bruce Momjian <bruce@momjian.us> writes:

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
> 	https://momjian.us/pgsql_docs/release-17.html

My name is listed twice in the "Improve psql tab completion" item.

- ilmari

  11. Re: First draft of PG 17 release notes

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2024-05-09T10:31:17Z

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> writes:

> Bruce Momjian <bruce@momjian.us> writes:
>
>> I have committed the first draft of the PG 17 release notes;  you can
>> see the results here:
>>
>> 	https://momjian.us/pgsql_docs/release-17.html
>
> My name is listed twice in the "Improve psql tab completion" item.

You can move one of them to "Track DEALLOCATE in pg_stat_statements",
which Michael and I co-authored.

- ilmari

  12. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-09T10:53:30Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>

* Add function pg_buffercache_evict() to allow shared buffer eviction
(Palak Chaturvedi, Thomas Munro)
* This is useful for testing.

this should put it on the section
< E.1.3.11. Additional Modules
?

Then I found out official release notes don't have <section> attributes,
so it doesn't matter?



<<
Allow ALTER OPERATOR to set more optimization attributes (Tommy Pavlicek)
This is useful for extensions.
<<
I think this commit title "Add hash support functions and hash opclass
for contrib/ltree."
 from [1] is more descriptive.
i am not 100% sure of the meaning of "This is useful for extensions."



[1] https://git.postgresql.org/cgit/postgresql.git/commit/?id=485f0aa85995340fb62113448c992ee48dc6fff1

  13. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-09T10:57:01Z

    > <<
> Allow ALTER OPERATOR to set more optimization attributes (Tommy Pavlicek)
> This is useful for extensions.
> <<

sorry,  I mean
<<
Allow the creation of hash indexes on ltree columns (Tommy Pavlicek)
This also enables hash join and hash aggregation on ltree columns.
<<

better description would be:
<<
Add hash support functions and hash opclass for contrib/ltree (Tommy Pavlicek)
This also enables hash join and hash aggregation on ltree columns.
<<

  14. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-09T11:49:55Z

    On Thu, May 9, 2024 at 6:53 PM jian he <jian.universality@gmail.com> wrote:
>
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html

< Add columns to pg_stats to report range histogram information (Egor
Rogov, Soumyadeep Chakraborty)
I think this applies to range type and multi range type, "range
histogram information" seems not very clear to me.
So maybe:
< Add columns to pg_stats to report range-type histogram information
(Egor Rogov, Soumyadeep Chakraborty)



Display length and bounds histograms in pg_stats
< Add new COPY option "ON_ERROR ignore" to discard error rows (Damir
Belyalov, Atsushi Torikoshi, Alex Shulgin, Jian He, Jian He, Yugo
Nagata)
duplicate name.

  15. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T13:08:52Z

    On Thu, May  9, 2024 at 04:44:47PM +1200, David Rowley wrote:
> On Thu, 9 May 2024 at 16:04, Bruce Momjian <bruce@momjian.us> wrote:
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> Thanks for working on that.
> 
> > +2023-11-02 [cac169d68] Increase DEFAULT_FDW_TUPLE_COST from 0.01 to 0.2
> 
> > +Double the default foreign data wrapper tuple cost (David Rowley, Umair Shahid)
> 
> That's 20x rather than 2x.

Oops, changed to:

	Increase the default foreign data wrapper tuple cost (David
	Rowley, Umair Shahid)

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  16. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T13:10:16Z

    On Thu, May  9, 2024 at 09:47:34AM +0500, Muhammad Ikram wrote:
> Hi Bruce,
> 
> A minor formatting issue in the start below. Bullet is not required here.
> 
> 
> E.1.1. Overview  
> 
> PostgreSQL 17 contains many new features and enhancements, including:
> 
>   • 
> 
> The above items and other new features of PostgreSQL 17 are explained in more
> detail in the sections below.

That is just a place-holder.  I changed the bullet text to be:

	TO BE COMPLETED LATER

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  17. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T13:11:10Z

    On Thu, May  9, 2024 at 04:52:14PM +1200, David Rowley wrote:
> On Thu, 9 May 2024 at 16:47, Muhammad Ikram <mmikram@gmail.com> wrote:
> > A minor formatting issue in the start below. Bullet is not required here.
> 
> This is a placeholder for the highlight features of v17 will go.
> Bruce tends not to decide what those are all by himself.

Yes, I already have so much of my opinion in the release notes that I
prefer others to make that list, and to make the Acknowledgments list
at the bottom.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  18. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T13:29:21Z

    On Thu, May  9, 2024 at 04:53:38AM +0000, Bertrand Drouvot wrote:
> Hi,
> 
> On Thu, May 09, 2024 at 12:03:50AM -0400, Bruce Momjian wrote:
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> > 
> > 	https://momjian.us/pgsql_docs/release-17.html
> 
> Thanks for working on that!
>  
> > I welcome feedback.
> 
> > Add system view pg_wait_events that reports wait event types (Michael Paquier) 
> 
> Michael is the committer for 1e68e43d3f, the author is me.

Wow, thank you for finding that.  The commit message is very clear so I
don't know how I made that mistake.  Fixed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  19. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T13:48:41Z

    On Thu, May  9, 2024 at 02:17:12PM +0900, Masahiko Sawada wrote:
> Hi,
> 
> On Thu, May 9, 2024 at 1:03 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> Thank you for working on that!
> 
> I'd like to mention some of my works. I think we can add the vacuum
> performance improvements by the following commits:
> 
> - Add template for adaptive radix tree (ee1b30f1)
> - Add TIDStore, to store sets of TIDs (ItemPointerData) efficiently (30e144287)
> - Use TidStore for dead tuple TIDs storage during lazy vacuum (667e65aac)

Okay, I reworded the item, added authors, and added the commits:

	<!--
	Author: John Naylor <john.naylor@postgresql.org>
	2024-03-07 [ee1b30f12] Add template for adaptive radix tree
	Author: Masahiko Sawada <msawada@postgresql.org>
	2024-03-21 [30e144287] Add TIDStore, to store sets of TIDs (ItemPointerData) ef
	Author: Masahiko Sawada <msawada@postgresql.org>
	2024-04-02 [667e65aac] Use TidStore for dead tuple TIDs storage during lazy vac
	Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
	2024-04-03 [6dbb49026] Combine freezing and pruning steps in VACUUM
	-->
	
	<listitem>
	<para>
	Allow vacuum to more efficiently remove and freeze tuples (John Naylor, Masahiko Sawada, Melanie Plageman)
	</para>
	</listitem>

> Also, please consider the following item:
> 
> - Improve eviction algorithm in ReorderBuffer using max-heap for many
> subtransactions (5bec1d6bc)

I looked at that item and I don't have a generic "make logical
replication apply faster" item to merge it into, and many
subtransactions seemed like enough of an edge-case that I didn't think
mentioning it make sense.  Can you see a good place to add it?

> Finally, should we mention the following commit in the release note?
> It's not a user-visible change but added a new regression test module.
> 
> - Add tests for XID wraparound (e255b646a)

I don't normally add testing infrastructure changes unless they are
major.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  20. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T14:17:03Z

    On Thu, May  9, 2024 at 02:37:57PM +0800, Richard Guo wrote:
> 
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> 
>     I have committed the first draft of the PG 17 release notes;  you can
>     see the results here:
> 
>             https://momjian.us/pgsql_docs/release-17.html
> 
> 
> Thanks for working on that.
> 
> For this item:
>  
> 
>     Allow the optimizer to improve CTE plans by using the sort order of
>     columns referenced in earlier CTE clauses (Jian Guo)
> 
> 
> I think you mean a65724dfa.  The author should be 'Richard Guo'.

Wow the CTE item above it was done by Jian Guo.  I probably copied the
text from the line above it, modified the description, but thought the
author's name was the same, but it was not.  Fixed.

> And I'm wondering if it is more accurate to state it as "Allow the
> optimizer to improve plans for the outer query by leveraging the sort
> order of a CTE's output."
>
> I think maybe a similar revision can be applied to the item just above
> this one.

Okay, I went with this text:

	<!--
	Author: Tom Lane <tgl@sss.pgh.pa.us>
	2023-11-17 [f7816aec2] Extract column statistics from CTE references, if possib
	-->
	
	<listitem>
	<para>
	Allow the optimizer to improve CTE plans by considering the statistics of columns referenced in earlier row output clauses (Jian Guo, Tom Lane)
	</para>
	</listitem>
	
	<!--
	Author: Tom Lane <tgl@sss.pgh.pa.us>
	2024-03-26 [a65724dfa] Propagate pathkeys from CTEs up to the outer query.
	-->
	
	<listitem>
	<para>
	Allow the optimizer to improve CTE plans by considering the sort order of columns referenced in earlier row output clauses (Richard Guo)
	</para>
	</listitem>

I did not use "leveraging" because I am concerned non-native English
speakers might find the term confusing.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  21. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T14:20:23Z

    On Thu, May  9, 2024 at 12:18:44PM +0300, Aleksander Alekseev wrote:
> Hi,
> 
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> > It will be improved until the final release.  The item count is 188,
> > which is similar to recent releases:
> 
> Thanks for working on this.
> 
> I believe the part of the 64-bit XIDs patchset that was delivered in
> PG17 is worth highlighting in "E.1.3.10. Source Code" section:
> 
> 4ed8f0913bfd
> 2cdf131c46e6
> 5a1dfde8334b
> a60b8a58f435
> 
> All this can probably be summarized as one bullet "Index SLRUs by
> 64-bit integers rather than by 32-bit ones" where the authors are:
> Maxim Orlov, Aleksander Alekseev, Alexander Korotkov, Teodor Sigaev,
> Nikita Glukhov, Pavel Borisov, Yura Sokolov.

Wow, I try to only list source code items that have some user-facing
impact, and I don't think these do.  I do realize how important they are
though.  This gets into the balance of mentioning items _users_ need to
know about, vs. important improvements that _we_ know about.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  22. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T14:49:37Z

    On Thu, May  9, 2024 at 06:00:24PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> 
> another potential incompatibilities issue:
> ALTER TABLE DROP PRIMARY KEY
> 
> see:
> https://www.postgresql.org/message-id/202404181849.6frtmajobe27%40alvherre.pgsql

I see it now, and I see Alvaro Herrera saying:

	https://www.postgresql.org/message-id/202404181849.6frtmajobe27%40alvherre.pgsql

	> I wonder is there any incompatibility issue, or do we need to say something
	> about the new behavior when dropping a key column?
	
-->	Umm, yeah, maybe we should document it in ALTER TABLE DROP PRIMARY KEY
-->	and in the release notes to note the different behavior.

However, I don't see it mentioned as a release note item in the commit
message or mentioned in our docs. I suppose the release note text would
be:

	Removing a PRIMARY KEY will remove the NOT NULL column specification

	Previously the NOT NULL specification would be retained.

Do we have agreement that we want this release note item?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  23. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T14:50:45Z

    On Thu, May  9, 2024 at 11:22:06AM +0100, Dagfinn Ilmari Mannsåker wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> 
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> > 	https://momjian.us/pgsql_docs/release-17.html
> 
> My name is listed twice in the "Improve psql tab completion" item.

You did such a great job I wanted to list you twice.  :-)  Actually, the
author list was so long I just didn't notice, fixed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  24. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T14:51:38Z

    On Thu, May  9, 2024 at 11:31:17AM +0100, Dagfinn Ilmari Mannsåker wrote:
> Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> writes:
> 
> > Bruce Momjian <bruce@momjian.us> writes:
> >
> >> I have committed the first draft of the PG 17 release notes;  you can
> >> see the results here:
> >>
> >> 	https://momjian.us/pgsql_docs/release-17.html
> >
> > My name is listed twice in the "Improve psql tab completion" item.
> 
> You can move one of them to "Track DEALLOCATE in pg_stat_statements",
> which Michael and I co-authored.

Yep, also my mistake, fixed.  My apologies.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  25. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T15:08:34Z

    On Thu, May  9, 2024 at 06:53:30PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> 
> * Add function pg_buffercache_evict() to allow shared buffer eviction
> (Palak Chaturvedi, Thomas Munro)
> * This is useful for testing.
> 
> this should put it on the section
> < E.1.3.11. Additional Modules
> ?

Oh, it is in the pg_buffercache module --- I should have realized that
from the name, fixed.

> Then I found out official release notes don't have <section> attributes,
> so it doesn't matter?

Uh, what are sections?  Did previous release notes have it?

> I think this commit title "Add hash support functions and hash opclass
> for contrib/ltree."
>  from [1] is more descriptive.

Uh, I don't think people know what hash support functions are, but they
know what hash indexes are, and maybe hash joins and hash aggregates. 
Why do you consider the commit text better?

> i am not 100% sure of the meaning of "This is useful for extensions."

The commit says:

	commit 2b5154beab7
	Author: Tom Lane <tgl@sss.pgh.pa.us>
	Date:   Fri Oct 20 12:28:38 2023 -0400
	
	    Extend ALTER OPERATOR to allow setting more optimization attributes.
	
	    Allow the COMMUTATOR, NEGATOR, MERGES, and HASHES attributes to be set
	    by ALTER OPERATOR.  However, we don't allow COMMUTATOR/NEGATOR to be
	    changed once set, nor allow the MERGES/HASHES flags to be unset once
	    set.  Changes like that might invalidate plans already made, and
	    dealing with the consequences seems like more trouble than it's worth.
-->	    The main use-case we foresee for this is to allow addition of missed
-->	    properties in extension update scripts, such as extending an existing
-->	    operator to support hashing.  So only transitions from not-set to set
	    states seem very useful.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  26. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T15:09:26Z

    On Thu, May  9, 2024 at 06:57:01PM +0800, jian he wrote:
> > <<
> > Allow ALTER OPERATOR to set more optimization attributes (Tommy Pavlicek)
> > This is useful for extensions.
> > <<
> 
> sorry,  I mean
> <<
> Allow the creation of hash indexes on ltree columns (Tommy Pavlicek)
> This also enables hash join and hash aggregation on ltree columns.
> <<
> 
> better description would be:
> <<
> Add hash support functions and hash opclass for contrib/ltree (Tommy Pavlicek)
> This also enables hash join and hash aggregation on ltree columns.
> <<

Yes, please see my previous email where I am asking why being more
specific is worse.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  27. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T15:12:14Z

    On Thu, May  9, 2024 at 07:49:55PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 6:53 PM jian he <jian.universality@gmail.com> wrote:
> >
> > On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > I have committed the first draft of the PG 17 release notes;  you can
> > > see the results here:
> > >
> > >         https://momjian.us/pgsql_docs/release-17.html
> 
> < Add columns to pg_stats to report range histogram information (Egor
> Rogov, Soumyadeep Chakraborty)
> I think this applies to range type and multi range type, "range
> histogram information" seems not very clear to me.
> So maybe:
> < Add columns to pg_stats to report range-type histogram information
> (Egor Rogov, Soumyadeep Chakraborty)

Yes, good point, done.

> Display length and bounds histograms in pg_stats

Uh, isn't that assumed?  Is this a detail worth mentioning?

> < Add new COPY option "ON_ERROR ignore" to discard error rows (Damir
> Belyalov, Atsushi Torikoshi, Alex Shulgin, Jian He, Jian He, Yugo
> Nagata)
> duplicate name.

Fixed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  28. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-09T15:26:44Z

    On Thu, May 9, 2024 at 11:12 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, May  9, 2024 at 07:49:55PM +0800, jian he wrote:
> > On Thu, May 9, 2024 at 6:53 PM jian he <jian.universality@gmail.com> wrote:
> > >
> > > On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > > I have committed the first draft of the PG 17 release notes;  you can
> > > > see the results here:
> > > >
> > > >         https://momjian.us/pgsql_docs/release-17.html
> >

E.1.3.1.5. Privileges
Add per-table GRANT permission MAINTAIN to control maintenance
operations (Nathan Bossart)

The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZE VIEW,
CLUSTER, and LOCK TABLE.

Add user-grantable role pg_maintain to control maintenance operations
(Nathan Bossart)

The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZE VIEW,
CLUSTER, and LOCK TABLE.

Allow roles with pg_monitor privileges to execute pg_current_logfile()
(Pavlo Golub, Nathan Bossart)
---------------
should be "REFRESH MATERIALIZED VIEW"?

also
"Allow roles with pg_monitor privileges to execute
pg_current_logfile() (Pavlo Golub, Nathan Bossart)"
"pg_monitor" is a predefined role, so technically, "with pg_monitor
privileges" is not correct?
--------------------------------------------------------------------------
Add function XMLText() to convert text to a single XML text node (Jim Jones)

XMLText()
should be
xmltext()
--------------------------------------------------------------------------
Add function to_regtypemod() to return the typemod of a string (David
Wheeler, Erik Wienhold)
I think this description does not mean the same thing as the doc[1]

[1] https://www.postgresql.org/docs/devel/functions-info.html#FUNCTIONS-INFO-CATALOG
--------------------------------------------------------------------------

Allow GROUP BY columns to be internally ordered to match ORDER BY
(Andrei Lepikhov, Teodor Sigaev)
This can be disabled using server variable enable_group_by_reordering.

Probably
`This can be disabled by setting the server variable
enable_group_by_reordering to false`.

  29. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T15:41:57Z

    On Thu, May  9, 2024 at 11:26:44PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 11:12 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Thu, May  9, 2024 at 07:49:55PM +0800, jian he wrote:
> > > On Thu, May 9, 2024 at 6:53 PM jian he <jian.universality@gmail.com> wrote:
> > > >
> > > > On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > > > > I have committed the first draft of the PG 17 release notes;  you can
> > > > > see the results here:
> > > > >
> > > > >         https://momjian.us/pgsql_docs/release-17.html
> > >
> 
> E.1.3.1.5. Privileges
> Add per-table GRANT permission MAINTAIN to control maintenance
> operations (Nathan Bossart)
> 
> The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZE VIEW,
> CLUSTER, and LOCK TABLE.
> 
> Add user-grantable role pg_maintain to control maintenance operations
> (Nathan Bossart)
> 
> The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZE VIEW,
> CLUSTER, and LOCK TABLE.
> 
> Allow roles with pg_monitor privileges to execute pg_current_logfile()
> (Pavlo Golub, Nathan Bossart)
> ---------------
> should be "REFRESH MATERIALIZED VIEW"?

Yes, fixed.

> also
> "Allow roles with pg_monitor privileges to execute
> pg_current_logfile() (Pavlo Golub, Nathan Bossart)"
> "pg_monitor" is a predefined role, so technically, "with pg_monitor
> privileges" is not correct?

Good point, new text:

	Allow roles with pg_monitor membership to execute pg_current_logfile() (Pavlo Golub, Nathan Bossart)

> --------------------------------------------------------------------------
> Add function XMLText() to convert text to a single XML text node (Jim Jones)
> 
> XMLText()
> should be
> xmltext()

Right, fixed.

> --------------------------------------------------------------------------
> Add function to_regtypemod() to return the typemod of a string (David
> Wheeler, Erik Wienhold)
> I think this description does not mean the same thing as the doc[1]

Yes, I see your point.  I changed the text to:

	Add function to_regtypemod() to return the type modifier of a
	type specification (David Wheeler, Erik Wienhold)


> [1] https://www.postgresql.org/docs/devel/functions-info.html#FUNCTIONS-INFO-CATALOG
> --------------------------------------------------------------------------
> 
> Allow GROUP BY columns to be internally ordered to match ORDER BY
> (Andrei Lepikhov, Teodor Sigaev)
> This can be disabled using server variable enable_group_by_reordering.
> 
> Probably
> `This can be disabled by setting the server variable
> enable_group_by_reordering to false`.

Uh, I usually don't go into that detail.  There will be a link to the
variable in about a month so users can look up its behavior.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  30. Re: First draft of PG 17 release notes

    Andrew Dunstan <andrew@dunslane.net> — 2024-05-09T16:10:11Z

    On 2024-05-09 Th 00:03, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
> 	https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
>
> I welcome feedback.  For some reason it was an easier job than usual.


  *

    Remove the ability to build Postgres with Visual Studio (Michael
    Paquier)

    Meson is now the only available Windows build method.


This is a category mistake. What was removed was the special code we had 
for building with VS, but not the ability to build with VS. You can 
build with VS using meson (see for example drongo on the buildfarm)


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  31. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T16:29:38Z

    On Thu, May  9, 2024 at 12:10:11PM -0400, Andrew Dunstan wrote:
> 
> On 2024-05-09 Th 00:03, Bruce Momjian wrote:
> 
>     I have committed the first draft of the PG 17 release notes;  you can
>     see the results here:
> 
>             https://momjian.us/pgsql_docs/release-17.html
> 
>     It will be improved until the final release.  The item count is 188,
>     which is similar to recent releases:
> 
>             release-10:  189
>             release-11:  170
>             release-12:  180
>             release-13:  178
>             release-14:  220
>             release-15:  184
>             release-16:  206
>             release-17:  188
> 
>     I welcome feedback.  For some reason it was an easier job than usual.
> 
> 
>   • Remove the ability to build Postgres with Visual Studio (Michael Paquier)
> 
>     Meson is now the only available Windows build method.
> 
> 
> This is a category mistake. What was removed was the special code we had for
> building with VS, but not the ability to build with VS. You can build with VS
> using meson (see for example drongo on the buildfarm)

Wow, okay, I am not surprised I was confused.  New text is:

	<!--
	Author: Michael Paquier <michael@paquier.xyz>
	2023-12-20 [1301c80b2] Remove MSVC scripts
	-->
	
	<listitem>
	<para>
	Remove the Microsoft Visual Studio Studio-specific Postgres build option (Michael Paquier)
	</para>
	
	<para>
	Meson is now the only method for Visual Studio builds.
	</para>
	</listitem>
	<!--
	Author: Michael Paquier <michael@paquier.xyz>
	2023-12-20 [1301c80b2] Remove MSVC scripts
	-->
	
	<listitem>
	<para>
	Remove the Microsoft Visual Studio Studio-specific Postgres build option (Michael Paquier)
	</para>
	
	<para>
	Meson is now the only method for Visual Studio builds.
	</para>
	</listitem>

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  32. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-09T18:40:00Z

    On 2024-May-09, Bruce Momjian wrote:

> However, I don't see it mentioned as a release note item in the commit
> message or mentioned in our docs. I suppose the release note text would
> be:
> 
> 	Removing a PRIMARY KEY will remove the NOT NULL column specification
> 
> 	Previously the NOT NULL specification would be retained.
> 
> Do we have agreement that we want this release note item?

Yes.  Maybe we want some others too (especially regarding inheritance,
but also regarding the way we handle the constraints internally), and
maybe in this one we want different wording.  How about something like
this:

  Removing a primary key constraint may change the nullability
  characteristic of the columns that the primary key covered.

  If explicit not-null constraints exist on the same column, then they
  continue to be /known not nullable/; otherwise they become /possibly
  nullable/.

This is largely based on the SQL standard's language of a column
descriptor having a "nullability characteristic", which for columns with
not-null or primary key constraints is "known not null".  I don't think
we use those terms anywhere.  I hope this isn't too confusing.

The standard's text on this, in section "4.13 Columns, fields, and
attributes", is

  Every column has a nullability characteristic that indicates whether
  the value from that column can be the null value. A nullability
  characteristic is either known not nullable or possibly nullable.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/

  33. Re: First draft of PG 17 release notes

    Thomas Munro <thomas.munro@gmail.com> — 2024-05-09T20:05:43Z

    On Thu, May 9, 2024 at 4:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> I welcome feedback.  For some reason it was an easier job than usual.

> 2024-01-25 [820b5af73] jit: Require at least LLVM 10.

> Require LLVM version 10 or later (Peter Eisentraut)

Peter reviewed, I authored, and I think you intend to list authors in
parentheses.

  34. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T20:35:54Z

    On Fri, May 10, 2024 at 08:05:43AM +1200, Thomas Munro wrote:
> On Thu, May 9, 2024 at 4:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> > 2024-01-25 [820b5af73] jit: Require at least LLVM 10.
> 
> > Require LLVM version 10 or later (Peter Eisentraut)
> 
> Peter reviewed, I authored, and I think you intend to list authors in
> parentheses.

Yes, my mistake, fixed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  35. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-09T23:54:22Z

    On Thu, May  9, 2024 at 08:40:00PM +0200, Álvaro Herrera wrote:
> On 2024-May-09, Bruce Momjian wrote:
> 
> > However, I don't see it mentioned as a release note item in the commit
> > message or mentioned in our docs. I suppose the release note text would
> > be:
> > 
> > 	Removing a PRIMARY KEY will remove the NOT NULL column specification
> > 
> > 	Previously the NOT NULL specification would be retained.
> > 
> > Do we have agreement that we want this release note item?
> 
> Yes.  Maybe we want some others too (especially regarding inheritance,
> but also regarding the way we handle the constraints internally), and
> maybe in this one we want different wording.  How about something like
> this:
> 
>   Removing a primary key constraint may change the nullability
>   characteristic of the columns that the primary key covered.
> 
>   If explicit not-null constraints exist on the same column, then they
>   continue to be /known not nullable/; otherwise they become /possibly
>   nullable/.
> 
> This is largely based on the SQL standard's language of a column
> descriptor having a "nullability characteristic", which for columns with
> not-null or primary key constraints is "known not null".  I don't think
> we use those terms anywhere.  I hope this isn't too confusing.

Yes, it was confusing, partly because it is using wording we don't use,
and partly because it is talking about what can go into the column,
rather than the visible column restriction NOT NULL.  I also think "may"
is too imprecise.

How about:

	Removing a primary key will remove a column's NOT NULL constraint
	if the constraint was added by the primary key
	
	Previously such NOT NULL constraints would remain after a primary
	key was removed.  A column-level NOT NULL constraint would not be
	emoved.

Here is the PG 16 output:

	CREATE TABLE test ( x INT CONSTRAINT test_pkey PRIMARY KEY );
	                Table "public.test"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
	 x      | integer |           | not null |
	Indexes:
	    "test_pkey" PRIMARY KEY, btree (x)
	
	CREATE TABLE test_with_not_null (x INT NOT NULL CONSTRAINT test_pkey_with_not_null PRIMARY KEY);
	         Table "public.test_with_not_null"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
	 x      | integer |           | not null |
	Indexes:
	    "test_pkey_with_not_null" PRIMARY KEY, btree (x)
	
	ALTER TABLE test DROP CONSTRAINT test_pkey;
	                Table "public.test"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
-->	 x      | integer |           | not null |
	
	ALTER TABLE test_with_not_null DROP CONSTRAINT test_pkey_with_not_null;
	         Table "public.test_with_not_null"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
-->	 x      | integer |           | not null |

Here is the output in PG 17:

	CREATE TABLE test ( x INT CONSTRAINT test_pkey PRIMARY KEY );
	                Table "public.test"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
	 x      | integer |           | not null |
	Indexes:
	    "test_pkey" PRIMARY KEY, btree (x)
	
	CREATE TABLE test_with_not_null (x INT NOT NULL CONSTRAINT test_pkey_with_not_null PRIMARY KEY);
	         Table "public.test_with_not_null"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
	 x      | integer |           | not null |
	Indexes:
	    "test_pkey_with_not_null" PRIMARY KEY, btree (x)
	
	ALTER TABLE test DROP CONSTRAINT test_pkey;
	                Table "public.test"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
-->	 x      | integer |           |          |
	
	ALTER TABLE test_with_not_null DROP CONSTRAINT test_pkey_with_not_null;
	         Table "public.test_with_not_null"
	 Column |  Type   | Collation | Nullable | Default
	--------+---------+-----------+----------+---------
-->	 x      | integer |           | not null |

Notice that the table without a _column_ NOT NULL removes the NOT NULL
designation after removing the primary key only in PG 17.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  36. Re: First draft of PG 17 release notes

    Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> — 2024-05-10T08:24:30Z

    On Thu, May 9, 2024 at 9:34 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>
>         release-10:  189
>         release-11:  170
>         release-12:  180
>         release-13:  178
>         release-14:  220
>         release-15:  184
>         release-16:  206
>         release-17:  188
>
> I welcome feedback.  For some reason it was an easier job than usual.

Thanks a lot for this work Bruce! It looks like commit
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=91f2cae7a4e664e9c0472b364c7db29d755ab151
is missing from daft release notes. Just curious to know if it's
intentional or a miss out.

-- 
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  37. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-10T13:50:01Z

    On Fri, May 10, 2024 at 01:54:30PM +0530, Bharath Rupireddy wrote:
> On Thu, May 9, 2024 at 9:34 AM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> > It will be improved until the final release.  The item count is 188,
> > which is similar to recent releases:
> >
> >         release-10:  189
> >         release-11:  170
> >         release-12:  180
> >         release-13:  178
> >         release-14:  220
> >         release-15:  184
> >         release-16:  206
> >         release-17:  188
> >
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> Thanks a lot for this work Bruce! It looks like commit
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=91f2cae7a4e664e9c0472b364c7db29d755ab151
> is missing from daft release notes. Just curious to know if it's
> intentional or a miss out.

I did not mention it because the commit didn't mention any performance
benefit and it seemed more like an internal change than something people
needed to know about.  I could reword and merge it into this item, if
you think I should:

	 Improve performance of heavily-contended WAL writes (Bharath Rupireddy) 

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  38. Re: First draft of PG 17 release notes

    Daniel Verite <daniel@manitou-mail.org> — 2024-05-10T16:29:11Z

    	Bruce Momjian wrote:

>  have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
>         https://momjian.us/pgsql_docs/release-17.html

In the psql items, I'd suggest mentioning

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=90f5178

For the short description, maybe something like that:

- Improve FETCH_COUNT to work with all queries (Daniel Vérité)
Previously, results would be fetched in chunks only for queries
that start with the SELECT keyword.


Best regards,
-- 
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite

  39. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-05-10T16:50:54Z

    On Thu, 9 May 2024 at 06:04, Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html

Great work!

There are two commits that I think would benefit from being listed
(but maybe they are already listed and I somehow missed them, or they
are left out on purpose for some reason):

- c4ab7da60617f020e8d75b1584d0754005d71830
- cafe1056558fe07cdc52b95205588fcd80870362

  40. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-10T19:47:04Z

    On Fri, May 10, 2024 at 06:29:11PM +0200, Daniel Verite wrote:
> 	Bruce Momjian wrote:
> 
> >  have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> > 
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> In the psql items, I'd suggest mentioning
> 
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=90f5178
> 
> For the short description, maybe something like that:
> 
> - Improve FETCH_COUNT to work with all queries (Daniel Vérité)
> Previously, results would be fetched in chunks only for queries
> that start with the SELECT keyword.

Agreed, patch attached and applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  41. Re: First draft of PG 17 release notes

    Maiquel Grassi <grassi@hotmail.com.br> — 2024-05-10T20:58:02Z

    Mhd

Enviado desde Outlook para Android<https://aka.ms/AAb9ysg>
________________________________
From: Bruce Momjian <bruce@momjian.us>
Sent: Friday, May 10, 2024 4:47:04 PM
To: Daniel Verite <daniel@manitou-mail.org>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: First draft of PG 17 release notes

On Fri, May 10, 2024 at 06:29:11PM +0200, Daniel Verite wrote:
>        Bruce Momjian wrote:
>
> >  have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
>
> In the psql items, I'd suggest mentioning
>
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=90f5178
>
> For the short description, maybe something like that:
>
> - Improve FETCH_COUNT to work with all queries (Daniel Vérité)
> Previously, results would be fetched in chunks only for queries
> that start with the SELECT keyword.

Agreed, patch attached and applied.

--
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  42. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-10T21:21:01Z

    On Fri, May 10, 2024 at 06:50:54PM +0200, Jelte Fennema-Nio wrote:
> On Thu, 9 May 2024 at 06:04, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> Great work!
> 
> There are two commits that I think would benefit from being listed
> (but maybe they are already listed and I somehow missed them, or they
> are left out on purpose for some reason):

I looked at both of these.   In both cases I didn't see why the user
would need to know these changes were made:

---------------------------------------------------------------------------

> - c4ab7da60617f020e8d75b1584d0754005d71830

	commit c4ab7da6061
	Author: David Rowley <drowley@postgresql.org>
	Date:   Sun Apr 7 21:20:18 2024 +1200
	
	    Avoid needless large memcpys in libpq socket writing
	
	    Until now, when calling pq_putmessage to write new data to a libpq
	    socket, all writes are copied into a buffer and that buffer gets flushed
	    when full to avoid having to perform small writes to the socket.
	
	    There are cases where we must write large amounts of data to the socket,
	    sometimes larger than the size of the buffer.  In this case, it's
	    wasteful to memcpy this data into the buffer and flush it out, instead,
	    we can send it directly from the memory location that the data is already
	    stored in.
	
	    Here we adjust internal_putbytes() so that after having just flushed the
	    buffer to the socket, if the remaining bytes to send is as big or bigger
	    than the buffer size, we just send directly rather than needlessly
	    copying into the PqSendBuffer buffer first.
	
	    Examples of operations that write large amounts of data in one message
	    are; outputting large tuples with SELECT or COPY TO STDOUT and
	    pg_basebackup.
	
	    Author: Melih Mutlu
	    Reviewed-by: Heikki Linnakangas
	    Reviewed-by: Jelte Fennema-Nio
	    Reviewed-by: David Rowley
	    Reviewed-by: Ranier Vilela
	    Reviewed-by: Andres Freund
	    Discussion: https://postgr.es/m/CAGPVpCR15nosj0f6xe-c2h477zFR88q12e6WjEoEZc8ZYkTh3Q@mail.gmail.com

> - cafe1056558fe07cdc52b95205588fcd80870362

	commit cafe1056558
	Author: Robert Haas <rhaas@postgresql.org>
	Date:   Tue Apr 2 10:26:10 2024 -0400
	
	    Allow SIGINT to cancel psql database reconnections.
	
	    After installing the SIGINT handler in psql, SIGINT can no longer cancel
	    database reconnections. For instance, if the user starts a reconnection
	    and then needs to do some form of interaction (ie psql is polling),
	    there is no way to cancel the reconnection process currently.
	
	    Use PQconnectStartParams() in order to insert a cancel_pressed check
	    into the polling loop.
	
	    Tristan Partin, reviewed by Gurjeet Singh, Heikki Linnakangas, Jelte
	    Fennema-Nio, and me.
	
	    Discussion: http://postgr.es/m/D08WWCPVHKHN.3QELIKZJ2D9RZ@neon.tech

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  43. Re: First draft of PG 17 release notes

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-05-10T21:31:33Z

    Bruce Momjian <bruce@momjian.us> writes:
> On Fri, May 10, 2024 at 06:50:54PM +0200, Jelte Fennema-Nio wrote:
>> There are two commits that I think would benefit from being listed
>> (but maybe they are already listed and I somehow missed them, or they
>> are left out on purpose for some reason):

> I looked at both of these.   In both cases I didn't see why the user
> would need to know these changes were made:

I agree that the buffering change is not likely interesting, but
the fact that you can now control-C out of a psql "\c" command
is user-visible.  People might have internalized the fact that
it didn't work, or created complicated workarounds.

			regards, tom lane

  44. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-10T21:37:25Z

    On Fri, May 10, 2024 at 05:31:33PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Fri, May 10, 2024 at 06:50:54PM +0200, Jelte Fennema-Nio wrote:
> >> There are two commits that I think would benefit from being listed
> >> (but maybe they are already listed and I somehow missed them, or they
> >> are left out on purpose for some reason):
> 
> > I looked at both of these.   In both cases I didn't see why the user
> > would need to know these changes were made:
> 
> I agree that the buffering change is not likely interesting, but
> the fact that you can now control-C out of a psql "\c" command
> is user-visible.  People might have internalized the fact that
> it didn't work, or created complicated workarounds.

It was not clear to me what the user-visible behavior was with the
SIGINT control.  Yes, based on your details, it should be mentioned.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  45. Re: First draft of PG 17 release notes

    Andy Fan <zhihuifan1213@163.com> — 2024-05-11T05:27:25Z

    Hello Bruce,

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
> 	https://momjian.us/pgsql_docs/release-17.html

Thank you for working on this!

> I welcome feedback.  For some reason it was an easier job than usual.

Do you think we need to add the following 2 items?

- 9f133763961e280d8ba692bcad0b061b861e9138 this is an optimizer
  transform improvement.

- a8a968a8212ee3ef7f22795c834b33d871fac262 this is an optimizer costing
  improvement.

Both of them can generate a better plan on some cases. 

-- 
Best Regards
Andy Fan

  46. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-11T05:57:31Z

    On Sat, 11 May 2024 at 17:32, Andy Fan <zhihuifan1213@163.com> wrote:
> Do you think we need to add the following 2 items?
>
> - 9f133763961e280d8ba692bcad0b061b861e9138 this is an optimizer
>   transform improvement.

I think this should be in the release notes.

Suggest:

* Allow correlated IN subqueries to be transformed into joins (Andy
Fan, Tom Lane)

> - a8a968a8212ee3ef7f22795c834b33d871fac262 this is an optimizer costing
>   improvement.
>
> Both of them can generate a better plan on some cases.

I think this should be present too.

Suggest:

* Improve optimizer's ability to use cheap startup plans when querying
partitioned tables, inheritance parents and for UNION ALL (Andy Fan,
David Rowley)

Both under "E.1.3.1.1. Optimizer"

David

  47. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-05-11T13:57:49Z

    On Fri, 10 May 2024 at 23:31, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Bruce Momjian <bruce@momjian.us> writes:
> > I looked at both of these.   In both cases I didn't see why the user
> > would need to know these changes were made:
>
> I agree that the buffering change is not likely interesting, but
> the fact that you can now control-C out of a psql "\c" command
> is user-visible.  People might have internalized the fact that
> it didn't work, or created complicated workarounds.

The buffering change improved performance up to ~40% in some of the
benchmarks. The case it improves mostly is COPY of large rows and
streaming a base backup. That sounds user-visible enough to me to
warrant an entry imho.

  48. Re: First draft of PG 17 release notes

    Joe Conway <mail@joeconway.com> — 2024-05-11T14:24:39Z

    On 5/11/24 09:57, Jelte Fennema-Nio wrote:
> On Fri, 10 May 2024 at 23:31, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>
>> Bruce Momjian <bruce@momjian.us> writes:
>> > I looked at both of these.   In both cases I didn't see why the user
>> > would need to know these changes were made:
>>
>> I agree that the buffering change is not likely interesting, but
>> the fact that you can now control-C out of a psql "\c" command
>> is user-visible.  People might have internalized the fact that
>> it didn't work, or created complicated workarounds.
> 
> The buffering change improved performance up to ~40% in some of the
> benchmarks. The case it improves mostly is COPY of large rows and
> streaming a base backup. That sounds user-visible enough to me to
> warrant an entry imho.

+1

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  49. Re: First draft of PG 17 release notes

    Andrew Dunstan <andrew@dunslane.net> — 2024-05-11T19:32:55Z

    On 2024-05-09 Th 00:03, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
> 	https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
>
> I welcome feedback.  For some reason it was an easier job than usual.


I don't like blowing my own horn but I feel commit 3311ea86ed "Introduce 
a non-recursive JSON parser" should be in the release notes. This isn't 
something that's purely internal, but it could be used by an extension 
or a client program to parse JSON documents that are too large to handle 
with the existing API.

Maybe "Introduce an incremental JSON parser" would have been a better 
headline.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  50. Re: First draft of PG 17 release notes

    Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> — 2024-05-13T07:16:38Z

    On Fri, May 10, 2024 at 7:20 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> > Thanks a lot for this work Bruce! It looks like commit
> > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=91f2cae7a4e664e9c0472b364c7db29d755ab151
> > is missing from daft release notes. Just curious to know if it's
> > intentional or a miss out.
>
> I did not mention it because the commit didn't mention any performance
> benefit and it seemed more like an internal change than something people
> needed to know about.

Yes, it's an internal feature for someone not using Direct IO for WAL
and helps achieve things mentioned at
https://www.postgresql.org/message-id/flat/20230125211540.zylu74dj2uuh3k7w%40awork3.anarazel.de#0cac0a0d219129e32329831adea05db5
(I'm hoping to target them for PG18). It starts to show visible
benefits if someone enables direct IO for WAL (for whatever reasons)
https://www.postgresql.org/message-id/CALj2ACV6rS%2B7iZx5%2BoAvyXJaN4AG-djAQeM1mrM%3DYSDkVrUs7g%40mail.gmail.com
and https://www.postgresql.org/message-id/20230127061745.46yu4ksitzociwkt%40awork3.anarazel.de.

I'm okay if 91f2cae7 is left out for the reason that Direct IO for WAL
isn't something used in production and debug_io_direct is a developer
option.

> I could reword and merge it into this item, if
> you think I should:
>
>          Improve performance of heavily-contended WAL writes (Bharath Rupireddy)

I think both the commits are for different purposes - one is for WAL
wrties, another is for WAL reads.

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  51. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-14T00:30:59Z

    On Fri, May 10, 2024 at 05:31:33PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Fri, May 10, 2024 at 06:50:54PM +0200, Jelte Fennema-Nio wrote:
> >> There are two commits that I think would benefit from being listed
> >> (but maybe they are already listed and I somehow missed them, or they
> >> are left out on purpose for some reason):
> 
> > I looked at both of these.   In both cases I didn't see why the user
> > would need to know these changes were made:
> 
> I agree that the buffering change is not likely interesting, but
> the fact that you can now control-C out of a psql "\c" command
> is user-visible.  People might have internalized the fact that
> it didn't work, or created complicated workarounds.

Agreed, attached patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  52. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-14T00:56:26Z

    On Sat, May 11, 2024 at 10:24:39AM -0400, Joe Conway wrote:
> On 5/11/24 09:57, Jelte Fennema-Nio wrote:
> > On Fri, 10 May 2024 at 23:31, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > > 
> > > Bruce Momjian <bruce@momjian.us> writes:
> > > > I looked at both of these.   In both cases I didn't see why the user
> > > > would need to know these changes were made:
> > > 
> > > I agree that the buffering change is not likely interesting, but
> > > the fact that you can now control-C out of a psql "\c" command
> > > is user-visible.  People might have internalized the fact that
> > > it didn't work, or created complicated workarounds.
> > 
> > The buffering change improved performance up to ~40% in some of the
> > benchmarks. The case it improves mostly is COPY of large rows and
> > streaming a base backup. That sounds user-visible enough to me to
> > warrant an entry imho.
> 
> +1

Attached patch applied.


-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  53. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-14T00:59:42Z

    On Sat, May 11, 2024 at 01:27:25PM +0800, Andy Fan wrote:
> 
> Hello Bruce,
> 
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> > 	https://momjian.us/pgsql_docs/release-17.html
> 
> Thank you for working on this!
> 
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> Do you think we need to add the following 2 items?
> 
> - 9f133763961e280d8ba692bcad0b061b861e9138 this is an optimizer
>   transform improvement.

It was unclear from the commit message exactly what user-visible
optimization this allowed.  Do you have details?

> - a8a968a8212ee3ef7f22795c834b33d871fac262 this is an optimizer costing
>   improvement.

Does this allow faster UNION ALL with LIMIT, perhaps?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  54. Re: First draft of PG 17 release notes

    Tender Wang <tndrwang@gmail.com> — 2024-05-14T02:22:35Z

    jian he <jian.universality@gmail.com> 于2024年5月9日周四 18:00写道：

> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
>
> another potential incompatibilities issue:
> ALTER TABLE DROP PRIMARY KEY
>
> see:
>
> https://www.postgresql.org/message-id/202404181849.6frtmajobe27%40alvherre.pgsql
>
>
Since Alvaro has reverted all changes to not-null constraints, so will not
have potential incompatibilities issue.
-- 
Tender Wang
OpenPie:  https://en.openpie.com/

  55. Re: First draft of PG 17 release notes

    Andy Fan <zhihuifan1213@163.com> — 2024-05-14T02:32:14Z

    Bruce Momjian <bruce@momjian.us> writes:

> On Sat, May 11, 2024 at 01:27:25PM +0800, Andy Fan wrote:
>> 
>> Hello Bruce,
>> 
>> > I have committed the first draft of the PG 17 release notes;  you can
>> > see the results here:
>> >
>> > 	https://momjian.us/pgsql_docs/release-17.html
>> 
>> Thank you for working on this!
>> 
>> > I welcome feedback.  For some reason it was an easier job than usual.
>> 
>> Do you think we need to add the following 2 items?
>> 
>> - 9f133763961e280d8ba692bcad0b061b861e9138 this is an optimizer
>>   transform improvement.
>
> It was unclear from the commit message exactly what user-visible
> optimization this allowed.  Do you have details?

Yes, It allows the query like "SELECT * FROM t1 WHERE t1.a in (SELECT a
FROM t2 WHERE t2.b = t1.b)" be pulled up a semi join, hence more join
methods / join orders are possible.

>
>> - a8a968a8212ee3ef7f22795c834b33d871fac262 this is an optimizer costing
>>   improvement.
>
> Does this allow faster UNION ALL with LIMIT, perhaps?

Yes, for example:  (subquery-1) UNION ALL (subquery-2) LIMIT n;

When planning the subquery-1 or subquery-2, limit N should be
considered. As a consequence, maybe hash join should be replaced with
Nested Loop. Before this commits, it is ignored if it is flatten into 
appendrel, and the "flatten" happens very often.

David provided a summary for the both commits in [1].

[1]
https://www.postgresql.org/message-id/CAApHDvqAQgq27LgYmJ85VVGTR0%3DhRW6HHq2oZgK0ZiYC_a%2BEww%40mail.gmail.com 

-- 
Best Regards
Andy Fan

  56. Re: First draft of PG 17 release notes

    Elena Indrupskaya <e.indrupskaya@postgrespro.ru> — 2024-05-14T10:34:56Z

    Hi everybody,


Being a technical writer, I attached a small patch that fixes minor 
language stuff.

Thank you.


Elena Indrupskaya

Postgres Professional Company

Moscow, Russia


On 09.05.2024 07:03, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
> 	https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
>
> I welcome feedback.  For some reason it was an easier job than usual.
>

  57. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-05-14T12:20:24Z

    On Tue, 14 May 2024 at 02:56, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Sat, May 11, 2024 at 10:24:39AM -0400, Joe Conway wrote:
> > On 5/11/24 09:57, Jelte Fennema-Nio wrote:
> > > The buffering change improved performance up to ~40% in some of the
> > > benchmarks. The case it improves mostly is COPY of large rows and
> > > streaming a base backup. That sounds user-visible enough to me to
> > > warrant an entry imho.
> >
> > +1
>
> Attached patch applied.

I think we shouldn't list this under the libpq changes and shouldn't
mention libpq in the description, since this patch changes
src/backend/libpq files instead of src/interfaces/libpq files. I think
it should be in the "General performance" section and describe the
change as something like the below:

Improve performance when transferring large blocks of data to a client

PS. I completely understand that this was not clear from the commit message.

  58. Re: First draft of PG 17 release notes

    Pantelis Theodosiou <ypercube@gmail.com> — 2024-05-14T13:58:41Z

    On Thu, May 9, 2024 at 5:03 AM Bruce Momjian <bruce@momjian.us> wrote
>
>
> I welcome feedback.  For some reason it was an easier job than usual.

This looks better if "more case" -> "more cases" :
>  Allow query nodes to be run in parallel in more case (Tom Lane)

  59. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-14T19:39:26Z

    On Thu, May 9, 2024 at 12:04 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html

I had two comments:

--------
I think the read stream item:

"Allow the grouping of file system reads with the new system variable
io_combine_limit"

Might be better if it mentions the effect, like:

"Reduce system calls by automatically merging reads up to io_combine_limit"
-------
For the vacuum feature:

"Allow vacuum to more efficiently remove and freeze tuples"

I think that we need to more clearly point out the implications of the
feature added by Sawada-san (and reviewed by John) in 667e65aac35497.
Vacuum no longer uses a fixed amount of memory for dead tuple TID
storage and it is not preallocated. This affects users as they may
want to change their configuration (and expectations).

If you make that item more specific to their work, you should also
remove my name, as the work I did on vacuum this release was unrelated
to their work on dead tuple TID storage.

The work Heikki and I did which culminated in 6dbb490261 mainly has
the impact of improving vacuum's performance (vacuum emits less WAL
and is more efficient). So you could argue for removing it from the
release notes if you are using the requirement that performance
improvements don't go in the release notes.

However, one of the preliminary commits for this f83d70976 does change
WAL format. There are three WAL records which no longer exist as
separate records. Do users care about this?

- Melanie

  60. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T00:37:19Z

    On Tue, May 14, 2024 at 10:32:14AM +0800, Andy Fan wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > It was unclear from the commit message exactly what user-visible
> > optimization this allowed.  Do you have details?
> 
> Yes, It allows the query like "SELECT * FROM t1 WHERE t1.a in (SELECT a
> FROM t2 WHERE t2.b = t1.b)" be pulled up a semi join, hence more join
> methods / join orders are possible.
> 
> 
> Yes, for example:  (subquery-1) UNION ALL (subquery-2) LIMIT n;
> 
> When planning the subquery-1 or subquery-2, limit N should be
> considered. As a consequence, maybe hash join should be replaced with
> Nested Loop. Before this commits, it is ignored if it is flatten into 
> appendrel, and the "flatten" happens very often.
> 
> David provided a summary for the both commits in [1].

Okay, attached patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  61. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T00:39:29Z

    On Sat, May 11, 2024 at 03:32:55PM -0400, Andrew Dunstan wrote:
> 
> On 2024-05-09 Th 00:03, Bruce Momjian wrote:
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> > 
> > 	https://momjian.us/pgsql_docs/release-17.html
> > 
> > It will be improved until the final release.  The item count is 188,
> > which is similar to recent releases:
> > 
> > 	release-10:  189
> > 	release-11:  170
> > 	release-12:  180
> > 	release-13:  178
> > 	release-14:  220
> > 	release-15:  184
> > 	release-16:  206
> > 	release-17:  188
> > 
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> 
> I don't like blowing my own horn but I feel commit 3311ea86ed "Introduce a
> non-recursive JSON parser" should be in the release notes. This isn't
> something that's purely internal, but it could be used by an extension or a
> client program to parse JSON documents that are too large to handle with the
> existing API.
> 
> Maybe "Introduce an incremental JSON parser" would have been a better
> headline.

Well, this gets into a level of detail that is beyond the average
reader.  I think at that level people will need to read the git logs or
review the code.  Do we use it for anything yet?

It could be put in the source code section but I try to only have
user-visible stuff there.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  62. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T00:43:33Z

    On Tue, May 14, 2024 at 01:34:56PM +0300, Elena Indrupskaya wrote:
> Being a technical writer, I attached a small patch that fixes minor language
> stuff.

You are absolutely correct.  Patch applied, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  63. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T00:47:10Z

    On Tue, May 14, 2024 at 02:20:24PM +0200, Jelte Fennema-Nio wrote:
> On Tue, 14 May 2024 at 02:56, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Sat, May 11, 2024 at 10:24:39AM -0400, Joe Conway wrote:
> > > On 5/11/24 09:57, Jelte Fennema-Nio wrote:
> > > > The buffering change improved performance up to ~40% in some of the
> > > > benchmarks. The case it improves mostly is COPY of large rows and
> > > > streaming a base backup. That sounds user-visible enough to me to
> > > > warrant an entry imho.
> > >
> > > +1
> >
> > Attached patch applied.
> 
> I think we shouldn't list this under the libpq changes and shouldn't
> mention libpq in the description, since this patch changes
> src/backend/libpq files instead of src/interfaces/libpq files. I think
> it should be in the "General performance" section and describe the
> change as something like the below:
> 
> Improve performance when transferring large blocks of data to a client
> 
> PS. I completely understand that this was not clear from the commit message.

Okay, I went with your wording.  Attached patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  64. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T00:48:21Z

    On Tue, May 14, 2024 at 02:58:41PM +0100, Pantelis Theodosiou wrote:
> On Thu, May 9, 2024 at 5:03 AM Bruce Momjian <bruce@momjian.us> wrote
> >
> >
> > I welcome feedback.  For some reason it was an easier job than usual.
> 
> This looks better if "more case" -> "more cases" :
> >  Allow query nodes to be run in parallel in more case (Tom Lane)

Yes, you are correct, fixed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  65. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T01:00:50Z

    On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:
> On Thu, May 9, 2024 at 12:04 AM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> I had two comments:
> 
> --------
> I think the read stream item:
> 
> "Allow the grouping of file system reads with the new system variable
> io_combine_limit"
> 
> Might be better if it mentions the effect, like:
> 
> "Reduce system calls by automatically merging reads up to io_combine_limit"

Uh, as I understand it, the reduced number of system calls is not the
value of the feature, but rather the ability to request a larger block
from the I/O subsystem.  Without it, you have to make a request and wait
for each request to finish.  I am open to new wording, but I am not sure
your new wording is accurate.

> -------
> For the vacuum feature:
> 
> "Allow vacuum to more efficiently remove and freeze tuples"
> 
> I think that we need to more clearly point out the implications of the
> feature added by Sawada-san (and reviewed by John) in 667e65aac35497.
> Vacuum no longer uses a fixed amount of memory for dead tuple TID
> storage and it is not preallocated. This affects users as they may
> want to change their configuration (and expectations).
> 
> If you make that item more specific to their work, you should also
> remove my name, as the work I did on vacuum this release was unrelated
> to their work on dead tuple TID storage.
> 
> The work Heikki and I did which culminated in 6dbb490261 mainly has
> the impact of improving vacuum's performance (vacuum emits less WAL
> and is more efficient). So you could argue for removing it from the
> release notes if you are using the requirement that performance
> improvements don't go in the release notes.
> 
> However, one of the preliminary commits for this f83d70976 does change
> WAL format. There are three WAL records which no longer exist as
> separate records. Do users care about this?

I don't think users really care about these details, just that it is
faster so they will not be surprised if there is a change.  I was
purposely vague to group multiple commits into the single item.  By
grouping them together, I got enough impact to warrant listing it.  If
you split it apart, it is harder to justify mentioning them.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  66. Re: First draft of PG 17 release notes

    Masahiko Sawada <sawada.mshk@gmail.com> — 2024-05-15T01:10:28Z

    On Thu, May 9, 2024 at 10:48 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, May  9, 2024 at 02:17:12PM +0900, Masahiko Sawada wrote:
> > Hi,
> >
>
> > Also, please consider the following item:
> >
> > - Improve eviction algorithm in ReorderBuffer using max-heap for many
> > subtransactions (5bec1d6bc)
>
> I looked at that item and I don't have a generic "make logical
> replication apply faster" item to merge it into, and many
> subtransactions seemed like enough of an edge-case that I didn't think
> mentioning it make sense.  Can you see a good place to add it?

I think that since many subtransactions cases are no longer becoming
edge-cases these days, we needed to improve that and it might be
helpful for users to mention it. How about the following item for
example?

Improve logical decoding performance in cases where there are many
subtransactions.

>
> > Finally, should we mention the following commit in the release note?
> > It's not a user-visible change but added a new regression test module.
> >
> > - Add tests for XID wraparound (e255b646a)
>
> I don't normally add testing infrastructure changes unless they are
> major.

I've seen we had such item, for example in PG14 release note:

Add a test module for the regular expression package (Tom Lane)

But if our policy has already changed, I'm okay with not mentioning
the xid_wraparound test in the PG17 release note.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

  67. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T02:02:11Z

    On Tue, May 14, 2024 at 10:22:35AM +0800, Tender Wang wrote:
> 
> 
> jian he <jian.universality@gmail.com> 于2024年5月9日周四 18:00写道：
> 
>     On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
>     >
>     > I have committed the first draft of the PG 17 release notes;  you can
>     > see the results here:
>     >
>     >         https://momjian.us/pgsql_docs/release-17.html
>     >
> 
>     another potential incompatibilities issue:
>     ALTER TABLE DROP PRIMARY KEY
> 
>     see:
>     https://www.postgresql.org/message-id/
>     202404181849.6frtmajobe27%40alvherre.pgsql
> 
> 
> 
> Since Alvaro has reverted all changes to not-null constraints, so will not have
> potential incompatibilities issue.

Agreed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  68. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-15T02:03:32Z

    On Wed, 15 May 2024 at 13:00, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:
> > "Reduce system calls by automatically merging reads up to io_combine_limit"
>
> Uh, as I understand it, the reduced number of system calls is not the
> value of the feature, but rather the ability to request a larger block
> from the I/O subsystem.  Without it, you have to make a request and wait
> for each request to finish.  I am open to new wording, but I am not sure
> your new wording is accurate.

I think you have the cause and effect backwards. There's no advantage
to reading 128KB if you only need 8KB.  It's the fact that doing
*larger* reads allows *fewer* reads that allows it to be more
efficient.  There are also the efficiency gains from fadvise
POSIX_FADV_WILLNEED. I'm unsure how to jam that into a short sentence.
Maybe; "Optimize reading of tables by allowing pages to be prefetched
and read in chunks up to io_combine_limit", or a bit more buzzy;
"Optimize reading of tables by allowing pages to be prefetched and
performing vectored reads in chunks up to io_combine_limit".

David

  69. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T02:06:17Z

    On Wed, May 15, 2024 at 02:03:32PM +1200, David Rowley wrote:
> On Wed, 15 May 2024 at 13:00, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:
> > > "Reduce system calls by automatically merging reads up to io_combine_limit"
> >
> > Uh, as I understand it, the reduced number of system calls is not the
> > value of the feature, but rather the ability to request a larger block
> > from the I/O subsystem.  Without it, you have to make a request and wait
> > for each request to finish.  I am open to new wording, but I am not sure
> > your new wording is accurate.
> 
> I think you have the cause and effect backwards. There's no advantage
> to reading 128KB if you only need 8KB.  It's the fact that doing
> *larger* reads allows *fewer* reads that allows it to be more
> efficient.  There are also the efficiency gains from fadvise
> POSIX_FADV_WILLNEED. I'm unsure how to jam that into a short sentence.
> Maybe; "Optimize reading of tables by allowing pages to be prefetched
> and read in chunks up to io_combine_limit", or a bit more buzzy;
> "Optimize reading of tables by allowing pages to be prefetched and
> performing vectored reads in chunks up to io_combine_limit".

Yes, my point is that it is not the number of system calls or system
call overhead that is the advantage of this patch, but the ability to
request more of the I/O system in one call, which is not the same as
system calls.

I can use your wording, but how much prefetching to we enable now?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  70. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-15T02:20:11Z

    On Wed, May 15, 2024 at 10:10:28AM +0900, Masahiko Sawada wrote:
> > I looked at that item and I don't have a generic "make logical
> > replication apply faster" item to merge it into, and many
> > subtransactions seemed like enough of an edge-case that I didn't think
> > mentioning it make sense.  Can you see a good place to add it?
> 
> I think that since many subtransactions cases are no longer becoming
> edge-cases these days, we needed to improve that and it might be
> helpful for users to mention it. How about the following item for
> example?
> 
> Improve logical decoding performance in cases where there are many
> subtransactions.

Okay, item added in the attached applied patch.

> > > Finally, should we mention the following commit in the release note?
> > > It's not a user-visible change but added a new regression test module.
> > >
> > > - Add tests for XID wraparound (e255b646a)
> >
> > I don't normally add testing infrastructure changes unless they are
> > major.
> 
> I've seen we had such item, for example in PG14 release note:
> 
> Add a test module for the regular expression package (Tom Lane)
> 
> But if our policy has already changed, I'm okay with not mentioning
> the xid_wraparound test in the PG17 release note.

Uh, that PG 14 test suite was huge and flushed out a lot of bugs, not
only in our regex code but I think in the TCL/Henry Spencer regex
library we inherited.

We add 10-40 tests every year, and how many do I mention in the release
notes?  You had to go back to PG 14 to find one.  We have not changed
our release note "test item" criteria --- I only mention tests that are
significant to our userbase.  I think that test suite was significant to
anyone using the TCL/Henry Spencer regex library.

If you want your test mentioned, you have to explain why it is useful
for users to know about it, or the value it brings them.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  71. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-15T02:24:15Z

    On Wed, 15 May 2024 at 14:06, Bruce Momjian <bruce@momjian.us> wrote:
> I can use your wording, but how much prefetching to we enable now?

I believe the read stream API is used for Seq Scan, ANALYZE and
pg_prewarm().  fadvise() is used when the next buffer that's required
is not in shared buffers on any build that has defined
HAVE_DECL_POSIX_FADVISE.

David

  72. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-15T08:38:20Z

    On 2024-May-14, Bruce Momjian wrote:

> On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:

> > I think that we need to more clearly point out the implications of the
> > feature added by Sawada-san (and reviewed by John) in 667e65aac35497.
> > Vacuum no longer uses a fixed amount of memory for dead tuple TID
> > storage and it is not preallocated. This affects users as they may
> > want to change their configuration (and expectations).
> > 
> > If you make that item more specific to their work, you should also
> > remove my name, as the work I did on vacuum this release was unrelated
> > to their work on dead tuple TID storage.
> > 
> > The work Heikki and I did which culminated in 6dbb490261 mainly has
> > the impact of improving vacuum's performance (vacuum emits less WAL
> > and is more efficient). So you could argue for removing it from the
> > release notes if you are using the requirement that performance
> > improvements don't go in the release notes.
> 
> I don't think users really care about these details, just that it is
> faster so they will not be surprised if there is a change.  I was
> purposely vague to group multiple commits into the single item.  By
> grouping them together, I got enough impact to warrant listing it.  If
> you split it apart, it is harder to justify mentioning them.

I disagree with this.  IMO the impact of the Sawada/Naylor change is
likely to be enormous for people with large tables and large numbers of
tuples to clean up (I know we've had a number of customers in this
situation, I can't imagine any Postgres service provider that doesn't).
The fact that maintenance_work_mem is no longer capped at 1GB is very
important and I think we should mention that explicitly in the release
notes, as setting it higher could make a big difference in vacuum run
times.

I don't know what's the impact of the Plageman/Linnakangas work, but
since there are no user-visible consequences other than it being faster,
I agree it could be put more succintly, perhaps together as a sub-para
of the same item.

What about something like this?

<para>
 Lift the 1 GB allocation limit for vacuum memory usage for dead
 tuples, and make storage more compact and performant.
</para>
<para>
 This can reduce the number of index passes that vacuum has to perform
 for tables with many dead tuples, shortening vacuum times.
</para>
<para>
 Also, the WAL traffic caused by vacuum has been made more compact.
</para>
   

> > However, one of the preliminary commits for this f83d70976 does
> > change WAL format. There are three WAL records which no longer exist
> > as separate records. Do users care about this?

I don't think so.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"You don't solve a bad join with SELECT DISTINCT" #CupsOfFail
https://twitter.com/connor_mc_d/status/1431240081726115845

  73. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-15T11:17:50Z

    On Wed, 15 May 2024 at 20:38, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2024-May-14, Bruce Momjian wrote:
> > I don't think users really care about these details, just that it is
> > faster so they will not be surprised if there is a change.  I was
> > purposely vague to group multiple commits into the single item.  By
> > grouping them together, I got enough impact to warrant listing it.  If
> > you split it apart, it is harder to justify mentioning them.
>
> I disagree with this.  IMO the impact of the Sawada/Naylor change is
> likely to be enormous for people with large tables and large numbers of
> tuples to clean up (I know we've had a number of customers in this
> situation, I can't imagine any Postgres service provider that doesn't).
> The fact that maintenance_work_mem is no longer capped at 1GB is very
> important and I think we should mention that explicitly in the release
> notes, as setting it higher could make a big difference in vacuum run
> times.

I very much agree with Alvaro here. IMO, this should be on the
highlight feature list for v17. Prior to this, having to do multiple
index scans because of filling maintenance_work_mem was a performance
tragedy. If there were enough dead tuples to have filled
maintenance_work_mem, then the indexes are large. Having to scan
multiple large indexes multiple times isn't good use of I/O and CPU.
As far as I understand it, this work means it'll be unlikely that a
well-configured server will ever have to do multiple index passes. I
don't think "enormous impact" is an exaggeration here.

David

  74. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-15T13:13:14Z

    On Wed, May 15, 2024 at 4:38 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2024-May-14, Bruce Momjian wrote:
>
> > On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:
>
> > > I think that we need to more clearly point out the implications of the
> > > feature added by Sawada-san (and reviewed by John) in 667e65aac35497.
> > > Vacuum no longer uses a fixed amount of memory for dead tuple TID
> > > storage and it is not preallocated. This affects users as they may
> > > want to change their configuration (and expectations).
> > >
> > > If you make that item more specific to their work, you should also
> > > remove my name, as the work I did on vacuum this release was unrelated
> > > to their work on dead tuple TID storage.
> > >
> > > The work Heikki and I did which culminated in 6dbb490261 mainly has
> > > the impact of improving vacuum's performance (vacuum emits less WAL
> > > and is more efficient). So you could argue for removing it from the
> > > release notes if you are using the requirement that performance
> > > improvements don't go in the release notes.
> >
> > I don't think users really care about these details, just that it is
> > faster so they will not be surprised if there is a change.  I was
> > purposely vague to group multiple commits into the single item.  By
> > grouping them together, I got enough impact to warrant listing it.  If
> > you split it apart, it is harder to justify mentioning them.
>
> I disagree with this.  IMO the impact of the Sawada/Naylor change is
> likely to be enormous for people with large tables and large numbers of
> tuples to clean up (I know we've had a number of customers in this
> situation, I can't imagine any Postgres service provider that doesn't).
> The fact that maintenance_work_mem is no longer capped at 1GB is very
> important and I think we should mention that explicitly in the release
> notes, as setting it higher could make a big difference in vacuum run
> times.
>
> I don't know what's the impact of the Plageman/Linnakangas work, but
> since there are no user-visible consequences other than it being faster,
> I agree it could be put more succintly, perhaps together as a sub-para
> of the same item.
>
> What about something like this?
>
> <para>
>  Lift the 1 GB allocation limit for vacuum memory usage for dead
>  tuples, and make storage more compact and performant.
> </para>
> <para>
>  This can reduce the number of index passes that vacuum has to perform
>  for tables with many dead tuples, shortening vacuum times.
> </para>
> <para>
>  Also, the WAL traffic caused by vacuum has been made more compact.
> </para>

I think this wording and organization makes sense. I hadn't thought of
using "traffic" to describe this, but I like it.

Also +1 on the Sawada/Naylor change being on the highlight section of
the release (as David suggested upthread).

- Melanie

  75. Re: First draft of PG 17 release notes

    Amit Kapila <amit.kapila16@gmail.com> — 2024-05-15T13:13:51Z

    On Wed, May 15, 2024 at 7:36 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Wed, May 15, 2024 at 02:03:32PM +1200, David Rowley wrote:
> > On Wed, 15 May 2024 at 13:00, Bruce Momjian <bruce@momjian.us> wrote:
> > >
> > > On Tue, May 14, 2024 at 03:39:26PM -0400, Melanie Plageman wrote:
> > > > "Reduce system calls by automatically merging reads up to io_combine_limit"
> > >
> > > Uh, as I understand it, the reduced number of system calls is not the
> > > value of the feature, but rather the ability to request a larger block
> > > from the I/O subsystem.  Without it, you have to make a request and wait
> > > for each request to finish.  I am open to new wording, but I am not sure
> > > your new wording is accurate.
> >
> > I think you have the cause and effect backwards. There's no advantage
> > to reading 128KB if you only need 8KB.  It's the fact that doing
> > *larger* reads allows *fewer* reads that allows it to be more
> > efficient.  There are also the efficiency gains from fadvise
> > POSIX_FADV_WILLNEED. I'm unsure how to jam that into a short sentence.
> > Maybe; "Optimize reading of tables by allowing pages to be prefetched
> > and read in chunks up to io_combine_limit", or a bit more buzzy;
> > "Optimize reading of tables by allowing pages to be prefetched and
> > performing vectored reads in chunks up to io_combine_limit".
>
> Yes, my point is that it is not the number of system calls or system
> call overhead that is the advantage of this patch, but the ability to
> request more of the I/O system in one call, which is not the same as
> system calls.
>
> I can use your wording, but how much prefetching to we enable now?
>

Shouldn't we need to include commit
b5a9b18cd0bc6f0124664999b31a00a264d16913 with this item?

-- 
With Regards,
Amit Kapila.

  76. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-16T02:39:18Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>

>> Add local I/O block read/write timing statistics columns of pg_stat_statement (Nazir Bilal Yavuz)
>> The new columns are "local_blk_read_time" and "local_blk_write_time".
here, "pg_stat_statement" should be "pg_stat_statements"?


>> Add optional fourth parameter to pg_stat_statements_reset() to allow for the resetting of only min/max statistics (Andrei Zubkov)
>> This parameter defaults to "false".
here, "parameter",  should be "argument"?

maybe
>> Add optional fourth boolean argument (minmax_only) to pg_stat_statements_reset() to allow for the resetting of only min/max statistics (Andrei Zubkov)
>> This argument defaults to "false".
----------------------------------------------------------------
in section: E.1.2. Migration to Version 17

>> Rename I/O block read/write timing statistics columns of pg_stat_statement (Nazir Bilal Yavuz)
>> This renames "blk_read_time" to "shared_blk_read_time", and "blk_write_time" to "shared_blk_write_time".

"pg_stat_statement" should be "pg_stat_statements"?

also, we only mentioned, pg_stat_statements some columns name changed
in "E.1.2. Migration to Version 17"
but if you look at the release note pg_stat_statements section,
we added a bunch of columns, which are far more incompatibile than
just colunm name changes.

not sure we need add these in section "E.1.2. Migration to Version 17"

  77. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-16T02:48:27Z

    On Wed, May 15, 2024 at 09:13:14AM -0400, Melanie Plageman wrote:
> I think this wording and organization makes sense. I hadn't thought of
> using "traffic" to describe this, but I like it.
> 
> Also +1 on the Sawada/Naylor change being on the highlight section of
> the release (as David suggested upthread).

Agreed, I went with the attached applied patch.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  78. Re: First draft of PG 17 release notes

    David G. Johnston <david.g.johnston@gmail.com> — 2024-05-16T02:53:48Z

    On Wednesday, May 15, 2024, jian he <jian.universality@gmail.com> wrote:

> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
>
> in section: E.1.2. Migration to Version 17
>
> >> Rename I/O block read/write timing statistics columns of
> pg_stat_statement (Nazir Bilal Yavuz)
> >> This renames "blk_read_time" to "shared_blk_read_time", and
> "blk_write_time" to "shared_blk_write_time".
>
> we only mentioned, pg_stat_statements some columns name changed
> in "E.1.2. Migration to Version 17"
> but if you look at the release note pg_stat_statements section,
> we added a bunch of columns, which are far more incompatibile than
> just colunm name changes.
>
> not sure we need add these in section "E.1.2. Migration to Version 17"
>
>
New columns are not a migration issue since nothing being migrated forward
ever referenced them.  Its the ones that existing code knows about that
we’ve removed (including renames) that matter from a migration perspective.

David J.

  79. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-16T02:55:47Z

    On Thu, May 16, 2024 at 10:39:18AM +0800, jian he wrote:
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> 
> >> Add local I/O block read/write timing statistics columns of pg_stat_statement (Nazir Bilal Yavuz)
> >> The new columns are "local_blk_read_time" and "local_blk_write_time".
> here, "pg_stat_statement" should be "pg_stat_statements"?

Agreed.

> >> Add optional fourth parameter to pg_stat_statements_reset() to allow for the resetting of only min/max statistics (Andrei Zubkov)
> >> This parameter defaults to "false".
> here, "parameter",  should be "argument"?
> 
> maybe
> >> Add optional fourth boolean argument (minmax_only) to pg_stat_statements_reset() to allow for the resetting of only min/max statistics (Andrei Zubkov)
> >> This argument defaults to "false".

Sure.

> ----------------------------------------------------------------
> in section: E.1.2. Migration to Version 17
> 
> >> Rename I/O block read/write timing statistics columns of pg_stat_statement (Nazir Bilal Yavuz)
> >> This renames "blk_read_time" to "shared_blk_read_time", and "blk_write_time" to "shared_blk_write_time".
> 
> "pg_stat_statement" should be "pg_stat_statements"?

Yes, fixed.

> also, we only mentioned, pg_stat_statements some columns name changed
> in "E.1.2. Migration to Version 17"
> but if you look at the release note pg_stat_statements section,
> we added a bunch of columns, which are far more incompatibile than
> just colunm name changes.
> 
> not sure we need add these in section "E.1.2. Migration to Version 17"

Well, new columns don't cause breakage like renamed columns, which is
why I only put renames/removed columns in the migration section.

Also, thanks everyone for the release notes feedback.  In some cases I
made a mistake, and in some cases I misjudged the item.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  80. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-16T03:35:17Z

    On Thu, 16 May 2024 at 14:48, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Wed, May 15, 2024 at 09:13:14AM -0400, Melanie Plageman wrote:
> > Also +1 on the Sawada/Naylor change being on the highlight section of
> > the release (as David suggested upthread).
>
> Agreed, I went with the attached applied patch.

+Allow vacuum to more efficiently store tuple references and remove
its memory limit (Masahiko Sawada, John Naylor)
+</para>

I don't want it to seem like I'm splitting hairs, but I'd drop the "
and remove its memory limit"

+<para>
+Specifically, maintenance_work_mem and autovacuum_work_mem can now be
configured to use more than one gigabyte of memory.  WAL traffic
caused by vacuum is also more compact.

I'd say the first sentence above should be written as:

"Additionally, vacuum no longer silently imposes a 1GB tuple reference
limit even when maintenance_work_mem or autovacuum_work_mem are set to
higher values"

It's not "Specifically" as the "more efficiently store tuple
references" isn't the same thing as removing the 1GB cap. Also, there
was never a restriction in configuring maintenance_work_mem or
autovacuum_work_mem  to values higher than 1GB. The restriction was
that vacuum was unable to utilize anything more than that.

David

  81. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-16T03:48:02Z

    Hi,

On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> I disagree with this.  IMO the impact of the Sawada/Naylor change is
> likely to be enormous for people with large tables and large numbers of
> tuples to clean up (I know we've had a number of customers in this
> situation, I can't imagine any Postgres service provider that doesn't).
> The fact that maintenance_work_mem is no longer capped at 1GB is very
> important and I think we should mention that explicitly in the release
> notes, as setting it higher could make a big difference in vacuum run
> times.

+many.

We're having this debate every release. I think the ongoing reticence to note
performance improvements in the release notes is hurting Postgres.

For one, performance improvements are one of the prime reason users
upgrade. Without them being noted anywhere more dense than the commit log,
it's very hard to figure out what improved for users. A halfway widely
applicable performance improvement is far more impactful than many of the
feature changes we do list in the release notes.

For another, it's also very frustrating for developers that focus on
performance. The reticence to note their work, while noting other, far
smaller, things in the release notes, pretty much tells us that our work isn't
valued.

Greetings,

Andres Freund

  82. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-16T08:29:38Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>

>> Add jsonpath methods to convert JSON values to different data types (Jeevan Chalke)
>> The jsonpath methods are .bigint(), .boolean(), .date(), .decimal([precision [, scale]]), .integer(), .number(), .string(), .time(), .time_tz(), .timestamp(), and .timestamp_tz().

I think it's slightly incorrect.

for example:
select jsonb_path_query('"2023-08-15"', '$.date()');
I think it does is trying to validate json value "2023-08-15" can be a
date value, if so, print json string out, else error out.


"convert JSON values to different data types"
meaning that we are converting json values to another data type, date?

  83. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-05-16T09:49:24Z

    On Thu, 16 May 2024 at 05:48, Andres Freund <andres@anarazel.de> wrote:
> We're having this debate every release. I think the ongoing reticence to note
> performance improvements in the release notes is hurting Postgres.
>
> For one, performance improvements are one of the prime reason users
> upgrade. Without them being noted anywhere more dense than the commit log,
> it's very hard to figure out what improved for users. A halfway widely
> applicable performance improvement is far more impactful than many of the
> feature changes we do list in the release notes.
>
> For another, it's also very frustrating for developers that focus on
> performance. The reticence to note their work, while noting other, far
> smaller, things in the release notes, pretty much tells us that our work isn't
> valued.

+1 to the general gist of listing every perf improvement **and memory
usage reduction** in the release notes. Most of them are already
grouped together in a dedicated "General performance" section anyway,
having that section be big would only be good imho to show that we're
committed to improving perf.

I think one thing would make this a lot easier though is if commits
that knowlingy impact perf would clearly say so in the commit message,
because now it's sometimes hard to spot as someone not deeply involved
with the specific patch. e.g. c4ab7da606 doesn't mention performance
at all, so I'm not surprised it wasn't listed initially. And while
667e65aac3 states that multiple rounds of heap scanning is now
extremely rare, it doesn't explicitly state what the kind of perf
impact can be expected because of that.

Maybe something like introducing a common "Perf-Improvement: true"
marker in the commit message and when doing so add a clear paragraph
explaining the expected perf impact perf impact. Another option could
be to add a "User Impact" section to the commit message, where an
author could add their suggestion for a release note entry. So
basically this suggestion boils down to more clearly mentioning user
impact in commit messages, instead of mostly/only including
technical/implementation details.

  84. Re: First draft of PG 17 release notes

    Joe Conway <mail@joeconway.com> — 2024-05-16T12:09:56Z

    On 5/15/24 23:48, Andres Freund wrote:
> On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
>> I disagree with this.  IMO the impact of the Sawada/Naylor change is
>> likely to be enormous for people with large tables and large numbers of
>> tuples to clean up (I know we've had a number of customers in this
>> situation, I can't imagine any Postgres service provider that doesn't).
>> The fact that maintenance_work_mem is no longer capped at 1GB is very
>> important and I think we should mention that explicitly in the release
>> notes, as setting it higher could make a big difference in vacuum run
>> times.
> 
> +many.
> 
> We're having this debate every release. I think the ongoing reticence to note
> performance improvements in the release notes is hurting Postgres.
> 
> For one, performance improvements are one of the prime reason users
> upgrade. Without them being noted anywhere more dense than the commit log,
> it's very hard to figure out what improved for users. A halfway widely
> applicable performance improvement is far more impactful than many of the
> feature changes we do list in the release notes.

many++

> For another, it's also very frustrating for developers that focus on
> performance. The reticence to note their work, while noting other, far
> smaller, things in the release notes, pretty much tells us that our work isn't
> valued.

agreed

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  85. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-16T13:09:11Z

    On Wed, May 15, 2024 at 11:48 PM Andres Freund <andres@anarazel.de> wrote:
>
> On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> > I disagree with this.  IMO the impact of the Sawada/Naylor change is
> > likely to be enormous for people with large tables and large numbers of
> > tuples to clean up (I know we've had a number of customers in this
> > situation, I can't imagine any Postgres service provider that doesn't).
> > The fact that maintenance_work_mem is no longer capped at 1GB is very
> > important and I think we should mention that explicitly in the release
> > notes, as setting it higher could make a big difference in vacuum run
> > times.
>
> +many.
>
> We're having this debate every release. I think the ongoing reticence to note
> performance improvements in the release notes is hurting Postgres.
>
> For one, performance improvements are one of the prime reason users
> upgrade. Without them being noted anywhere more dense than the commit log,
> it's very hard to figure out what improved for users. A halfway widely
> applicable performance improvement is far more impactful than many of the
> feature changes we do list in the release notes.

The practical reason this matters to users is that they want to change
their configuration or expectations in response to performance
improvements.

And also, as Jelte mentions upthread, describing performance
improvements made each release in Postgres makes it clear that we are
consistently improving it.

> For another, it's also very frustrating for developers that focus on
> performance. The reticence to note their work, while noting other, far
> smaller, things in the release notes, pretty much tells us that our work isn't
> valued.

+many

- Melanie

  86. Re: First draft of PG 17 release notes

    Peter Geoghegan <pg@bowt.ie> — 2024-05-16T14:55:25Z

    On Wed, May 15, 2024 at 11:48 PM Andres Freund <andres@anarazel.de> wrote:
> On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> > I disagree with this.  IMO the impact of the Sawada/Naylor change is
> > likely to be enormous for people with large tables and large numbers of
> > tuples to clean up (I know we've had a number of customers in this
> > situation, I can't imagine any Postgres service provider that doesn't).
> > The fact that maintenance_work_mem is no longer capped at 1GB is very
> > important and I think we should mention that explicitly in the release
> > notes, as setting it higher could make a big difference in vacuum run
> > times.
>
> +many.

TIDStore/the lifting of the maintenance_work_mem cap is likely to make
the performance of VACUUM a lot more predictable, overall. While most
VACUUM operations don't hit the limit, the limit is disproportionately
involved in cases where (for whatever reason) vacuuming becomes a long
and painful process. Even if you as a user never run into such a
problem, you still spend time worrying about it, and/or taking
measures to make sure it doesn't affect you.

The justification for not including mention of these items is that
they're not very relevant to users. I find that hard to square with
what does get included. For example, the "Source Code" section is full
of highly niche items. Items that are low impact, even for users
that'll benefit the most. Also, "Monitoring" often mentions monitoring
improvements that expose low-level implementation details (e.g. SLRU
statistics), even though there's a good chance that Bruce wouldn't
include an item for some improvement to the SLRU subsystem itself.

If somebody puts in an enormous amount of effort to get a big
performance improvement over the line, then ISTM that that effort is a
useful signal when the time comes to write the release notes (at least
up to a point). For example, Masahiko and John spent about 2 years on
the TIDStore thing, on and off. These things do not happen in a vacuum
(no pun intended). Common sense tells me that they went to those
lengths precisely because they understood that it very much was
relevant to users. That belief would have been reinforced by both
experience, and by discussion on the list during the development of
the feature.

To be fair to Bruce, it probably really is true that most individual
users won't care about (say) TIDStore. But it's probably also true
that most individual users don't care about the release notes, or at
most skim the major items.

-- 
Peter Geoghegan

  87. Re: First draft of PG 17 release notes

    Andrew Dunstan <andrew@dunslane.net> — 2024-05-16T15:50:20Z

    On 2024-05-14 Tu 20:39, Bruce Momjian wrote:
> On Sat, May 11, 2024 at 03:32:55PM -0400, Andrew Dunstan wrote:
>> On 2024-05-09 Th 00:03, Bruce Momjian wrote:
>>> I have committed the first draft of the PG 17 release notes;  you can
>>> see the results here:
>>>
>>> 	https://momjian.us/pgsql_docs/release-17.html
>>>
>>> It will be improved until the final release.  The item count is 188,
>>> which is similar to recent releases:
>>>
>>> 	release-10:  189
>>> 	release-11:  170
>>> 	release-12:  180
>>> 	release-13:  178
>>> 	release-14:  220
>>> 	release-15:  184
>>> 	release-16:  206
>>> 	release-17:  188
>>>
>>> I welcome feedback.  For some reason it was an easier job than usual.
>>
>> I don't like blowing my own horn but I feel commit 3311ea86ed "Introduce a
>> non-recursive JSON parser" should be in the release notes. This isn't
>> something that's purely internal, but it could be used by an extension or a
>> client program to parse JSON documents that are too large to handle with the
>> existing API.
>>
>> Maybe "Introduce an incremental JSON parser" would have been a better
>> headline.
> Well, this gets into a level of detail that is beyond the average
> reader.  I think at that level people will need to read the git logs or
> review the code.  Do we use it for anything yet?


Yes, certainly, it's used in handling backup manifests. Without it we 
can't handle huge manifests. See commits ea7b4e9a2a and 222e11a10a.

Other uses are in the works.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  88. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-05-17T13:22:59Z

    On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>

This thread mentioned performance.
actually this[1] refactored some interval aggregation related functions,
which will make these two aggregate function: avg(interval), sum(interval)
run faster, especially avg(interval).
see [2].
well, I guess, this is a kind of niche performance improvement to be
mentioned separately.


these 3 items need to be removed, because of
https://git.postgresql.org/cgit/postgresql.git/commit/?id=8aee330af55d8a759b2b73f5a771d9d34a7b887f

>> Add stratnum GiST support function (Paul A. Jungwirth)

>> Allow foreign keys to reference WITHOUT OVERLAPS primary keys (Paul A. Jungwirth)
>> The keyword PERIOD is used for this purpose.

>> Allow PRIMARY KEY and UNIQUE constraints to use WITHOUT OVERLAPS for non-overlapping exclusion constraints (Paul A. Jungwirth)


[1] https://git.postgresql.org/cgit/postgresql.git/commit/?id=519fc1bd9e9d7b408903e44f55f83f6db30742b7
[2] https://www.postgresql.org/message-id/CAEZATCUJ0xjyQUL7SHKxJ5a%2BDm5pjoq-WO3NtkDLi6c76rh58w%40mail.gmail.com

  89. Re: First draft of PG 17 release notes

    Daniel Verite <daniel@manitou-mail.org> — 2024-05-17T13:42:44Z

    	Bruce Momjian wrote:

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
> 	https://momjian.us/pgsql_docs/release-17.html

About the changes in collations:

<quote>
  Create a "builtin" collation provider similar to libc's C locale
  (Jeff Davis)

  It uses a "C" locale which is identical but independent of
  libc, but it allows the use of non-"C" collations like "en_US"
  and "C.UTF-8" with the "C" locale, which libc does not. MORE?
</quote>

The new builtin provider has two collations:
* ucs_basic which is 100% identical to "C". It was introduced
several versions ago and the v17 novelty is simply to change
its pg_collation.collprovider from 'c' to 'b'.

* pg_c_utf8 which sorts like "C" but is Unicode-aware for
the rest, which makes it quite different from "C".
It's also different from the other UTF-8 collations that could
be used up to v17 in that it does not depend on an external
library, making it free from the collation OS-upgrade risks.

The part that is concretely of interest to users is the introduction
of pg_c_utf8. As described in [1]:

<quote>
pg_c_utf8

 This collation sorts by Unicode code point values rather than
 natural language order. For the functions lower, initcap, and
 upper, it uses Unicode simple case mapping. For pattern
 matching (including regular expressions), it uses the POSIX
 Compatible variant of Unicode Compatibility Properties. Behavior
 is efficient and stable within a Postgres major version. This
 collation is only available for encoding UTF8.
</quote>

I'd suggest that the relnote entry should be more like a condensed
version of that description, without mentioning en_US or C.UTF-8,
whose existence and semantics are OS-dependent, contrary to pg_c_utf8.


[1] https://www.postgresql.org/docs/devel/collation.html

Best regards,
-- 
Daniel Vérité
https://postgresql.verite.pro/
Twitter: @DanielVerite

  90. Re: First draft of PG 17 release notes

    Jeff Davis <pgsql@j-davis.com> — 2024-05-17T20:30:03Z

    On Thu, 2024-05-09 at 00:03 -0400, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
>         https://momjian.us/pgsql_docs/release-17.html

For this item:

    Create a "builtin" collation provider similar to libc's C
    locale (Jeff Davis)

    It uses a "C" locale which is identical but independent of
    libc, but it allows the use of non-"C" collations like "en_US"
    and "C.UTF-8" with the "C" locale, which libc does not. MORE? 

I suggest something more like:

    New, platform-independent "builtin" collation
    provider. (Jeff Davis)

    Currently, it offers the "C" and "C.UTF-8" locales. The
    "C.UTF-8" locale combines stable and fast code point order
    collation with Unicode character semantics.

Regards,
	Jeff Davis

  91. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T14:40:50Z

    On Thu, May 16, 2024 at 03:35:17PM +1200, David Rowley wrote:
> On Thu, 16 May 2024 at 14:48, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Wed, May 15, 2024 at 09:13:14AM -0400, Melanie Plageman wrote:
> > > Also +1 on the Sawada/Naylor change being on the highlight section of
> > > the release (as David suggested upthread).
> >
> > Agreed, I went with the attached applied patch.
> 
> +Allow vacuum to more efficiently store tuple references and remove
> its memory limit (Masahiko Sawada, John Naylor)
> +</para>
> 
> I don't want it to seem like I'm splitting hairs, but I'd drop the "
> and remove its memory limit"
> 
> +<para>
> +Specifically, maintenance_work_mem and autovacuum_work_mem can now be
> configured to use more than one gigabyte of memory.  WAL traffic
> caused by vacuum is also more compact.
> 
> I'd say the first sentence above should be written as:
> 
> "Additionally, vacuum no longer silently imposes a 1GB tuple reference
> limit even when maintenance_work_mem or autovacuum_work_mem are set to
> higher values"
> 
> It's not "Specifically" as the "more efficiently store tuple
> references" isn't the same thing as removing the 1GB cap. Also, there
> was never a restriction in configuring maintenance_work_mem or
> autovacuum_work_mem  to values higher than 1GB. The restriction was
> that vacuum was unable to utilize anything more than that.

Slightly adjusted wording patch attached and applied.

My deep apologies for the delay in addressing this.  I should have done
it sooner.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  92. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T14:59:47Z

    On Wed, May 15, 2024 at 08:48:02PM -0700, Andres Freund wrote:
> Hi,
> 
> On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> > I disagree with this.  IMO the impact of the Sawada/Naylor change is
> > likely to be enormous for people with large tables and large numbers of
> > tuples to clean up (I know we've had a number of customers in this
> > situation, I can't imagine any Postgres service provider that doesn't).
> > The fact that maintenance_work_mem is no longer capped at 1GB is very
> > important and I think we should mention that explicitly in the release
> > notes, as setting it higher could make a big difference in vacuum run
> > times.
> 
> +many.
> 
> We're having this debate every release. I think the ongoing reticence to note
> performance improvements in the release notes is hurting Postgres.
> 
> For one, performance improvements are one of the prime reason users
> upgrade. Without them being noted anywhere more dense than the commit log,
> it's very hard to figure out what improved for users. A halfway widely
> applicable performance improvement is far more impactful than many of the
> feature changes we do list in the release notes.

I agree the impact of performance improvements are often greater than
the average release note item.  However, if people expect Postgres to be
faster, is it important for them to know _why_ it is faster?

If we add a new flag to a command, people will want to know about it so
they can make use of it, or if there is a performance improvement that
allows new workloads, they will want to know about that too so they can
consider those workloads.

On the flip side, a performance improvement that makes everything 10%
faster has little behavioral change for users, and in fact I think we
get ~6% faster in every major release.

> For another, it's also very frustrating for developers that focus on
> performance. The reticence to note their work, while noting other, far
> smaller, things in the release notes, pretty much tells us that our work isn't
> valued.

Yes, but are we willing to add text that every user will have to read
just for this purpose?

One think we _could_ do is to create a generic performance release note
item saying performance has been improved in the following areas, with
no more details, but we can list the authors on the item.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  93. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T15:13:54Z

    On Thu, May 16, 2024 at 09:09:11AM -0400, Melanie Plageman wrote:
> On Wed, May 15, 2024 at 11:48 PM Andres Freund <andres@anarazel.de> wrote:
> >
> > On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> > > I disagree with this.  IMO the impact of the Sawada/Naylor change is
> > > likely to be enormous for people with large tables and large numbers of
> > > tuples to clean up (I know we've had a number of customers in this
> > > situation, I can't imagine any Postgres service provider that doesn't).
> > > The fact that maintenance_work_mem is no longer capped at 1GB is very
> > > important and I think we should mention that explicitly in the release
> > > notes, as setting it higher could make a big difference in vacuum run
> > > times.
> >
> > +many.
> >
> > We're having this debate every release. I think the ongoing reticence to note
> > performance improvements in the release notes is hurting Postgres.
> >
> > For one, performance improvements are one of the prime reason users
> > upgrade. Without them being noted anywhere more dense than the commit log,
> > it's very hard to figure out what improved for users. A halfway widely
> > applicable performance improvement is far more impactful than many of the
> > feature changes we do list in the release notes.
> 
> The practical reason this matters to users is that they want to change
> their configuration or expectations in response to performance
> improvements.
> 
> And also, as Jelte mentions upthread, describing performance
> improvements made each release in Postgres makes it clear that we are
> consistently improving it.
> 
> > For another, it's also very frustrating for developers that focus on
> > performance. The reticence to note their work, while noting other, far
> > smaller, things in the release notes, pretty much tells us that our work isn't
> > valued.
> 
> +many

Please see the email I just posted.  There are three goals we have to
adjust for:

1.  short release notes so they are readable
2.  giving people credit for performance improvements
3.  showing people Postgres cares about performance

I would like to achieve 2 & 3 without harming #1.  My experience is if I
am reading a long document, and I get to a section where I start to
wonder, "Why should I care about this?", I start to skim the rest of
the document.  I am particularly critical if I start to wonder, "Why
does the author _think_ I should care about this?" becasue it feels like
the author is writing for him/herself and not the audience.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  94. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T16:11:38Z

    On Thu, May 16, 2024 at 04:29:38PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> 
> >> Add jsonpath methods to convert JSON values to different data types (Jeevan Chalke)
> >> The jsonpath methods are .bigint(), .boolean(), .date(), .decimal([precision [, scale]]), .integer(), .number(), .string(), .time(), .time_tz(), .timestamp(), and .timestamp_tz().
> 
> I think it's slightly incorrect.
> 
> for example:
> select jsonb_path_query('"2023-08-15"', '$.date()');
> I think it does is trying to validate json value "2023-08-15" can be a
> date value, if so, print json string out, else error out.
> 
> 
> "convert JSON values to different data types"
> meaning that we are converting json values to another data type, date?

I see your point.  I have reworded it to be:

	Add jsonpath methods to convert JSON values to other JSON data
	types (Jeevan Chalke)

Does that help?  I think your example is causing confusion because once
JSON values enter the SQL data type space, they are strings.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  95. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T16:50:53Z

    On Thu, May 16, 2024 at 11:50:20AM -0400, Andrew Dunstan wrote:
> > Maybe "Introduce an incremental JSON parser" would have been a better
> > > headline.
> > Well, this gets into a level of detail that is beyond the average
> > reader.  I think at that level people will need to read the git logs or
> > review the code.  Do we use it for anything yet?
> 
> 
> Yes, certainly, it's used in handling backup manifests. Without it we can't
> handle huge manifests. See commits ea7b4e9a2a and 222e11a10a.
> 
> Other uses are in the works.

Okay, added in the attached applied patch.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  96. Re: First draft of PG 17 release notes

    Andrew Dunstan <andrew@dunslane.net> — 2024-05-18T20:37:54Z

    On 2024-05-18 Sa 12:50, Bruce Momjian wrote:
> On Thu, May 16, 2024 at 11:50:20AM -0400, Andrew Dunstan wrote:
>>> Maybe "Introduce an incremental JSON parser" would have been a better
>>>> headline.
>>> Well, this gets into a level of detail that is beyond the average
>>> reader.  I think at that level people will need to read the git logs or
>>> review the code.  Do we use it for anything yet?
>>
>> Yes, certainly, it's used in handling backup manifests. Without it we can't
>> handle huge manifests. See commits ea7b4e9a2a and 222e11a10a.
>>
>> Other uses are in the works.
> Okay, added in the attached applied patch.
>

Thanks


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  97. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T21:37:29Z

    On Fri, May 17, 2024 at 09:22:59PM +0800, jian he wrote:
> On Thu, May 9, 2024 at 12:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> >
> > It will be improved until the final release.  The item count is 188,
> > which is similar to recent releases:
> >
> 
> This thread mentioned performance.
> actually this[1] refactored some interval aggregation related functions,
> which will make these two aggregate function: avg(interval), sum(interval)
> run faster, especially avg(interval).
> see [2].
> well, I guess, this is a kind of niche performance improvement to be
> mentioned separately.
> 
> 
> these 3 items need to be removed, because of
> https://git.postgresql.org/cgit/postgresql.git/commit/?id=8aee330af55d8a759b2b73f5a771d9d34a7b887f
> 
> >> Add stratnum GiST support function (Paul A. Jungwirth)
> 
> >> Allow foreign keys to reference WITHOUT OVERLAPS primary keys (Paul A. Jungwirth)
> >> The keyword PERIOD is used for this purpose.
> 
> >> Allow PRIMARY KEY and UNIQUE constraints to use WITHOUT OVERLAPS for non-overlapping exclusion constraints (Paul A. Jungwirth)
> 
> 
> [1] https://git.postgresql.org/cgit/postgresql.git/commit/?id=519fc1bd9e9d7b408903e44f55f83f6db30742b7
> [2] https://www.postgresql.org/message-id/CAEZATCUJ0xjyQUL7SHKxJ5a%2BDm5pjoq-WO3NtkDLi6c76rh58w%40mail.gmail.com

Agreed, I have applied the attached patch to make the release notes
current.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  98. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-18T21:51:56Z

    On Fri, May 17, 2024 at 01:30:03PM -0700, Jeff Davis wrote:
> On Thu, 2024-05-09 at 00:03 -0400, Bruce Momjian wrote:
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> > 
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> For this item:
> 
>     Create a "builtin" collation provider similar to libc's C
>     locale (Jeff Davis)
> 
>     It uses a "C" locale which is identical but independent of
>     libc, but it allows the use of non-"C" collations like "en_US"
>     and "C.UTF-8" with the "C" locale, which libc does not. MORE? 
> 
> I suggest something more like:
> 
>     New, platform-independent "builtin" collation
>     provider. (Jeff Davis)
> 
>     Currently, it offers the "C" and "C.UTF-8" locales. The
>     "C.UTF-8" locale combines stable and fast code point order
>     collation with Unicode character semantics.

Okay, I went with the attached applied patch.  Adjustments?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  99. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-19T03:53:38Z

    On Sun, 19 May 2024 at 02:40, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, May 16, 2024 at 03:35:17PM +1200, David Rowley wrote:
> > "Additionally, vacuum no longer silently imposes a 1GB tuple reference
> > limit even when maintenance_work_mem or autovacuum_work_mem are set to
> > higher values"

> Slightly adjusted wording patch attached and applied.

Thanks for adjusting.

It's a minor detail, but I'll mention it because you went to the
effort to adjust it away from what I'd written...

I didn't make a random choice to use "or" between the two GUCs.
Changing it to "and", IMO, isn't an improvement.  Using "and" implies
that the silent limited was only imposed when both of these GUCs were
set >= 1GB. That's not true. For the case we're talking about here, if
autovacuum_work_mem is set to anything apart from -1 then the value of
maintenance_work_mem does not matter.

David

  100. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-20T00:12:31Z

    On Sun, May 19, 2024 at 03:53:38PM +1200, David Rowley wrote:
> On Sun, 19 May 2024 at 02:40, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Thu, May 16, 2024 at 03:35:17PM +1200, David Rowley wrote:
> > > "Additionally, vacuum no longer silently imposes a 1GB tuple reference
> > > limit even when maintenance_work_mem or autovacuum_work_mem are set to
> > > higher values"
> 
> > Slightly adjusted wording patch attached and applied.
> 
> Thanks for adjusting.
> 
> It's a minor detail, but I'll mention it because you went to the
> effort to adjust it away from what I'd written...

True.

> I didn't make a random choice to use "or" between the two GUCs.
> Changing it to "and", IMO, isn't an improvement.  Using "and" implies
> that the silent limited was only imposed when both of these GUCs were
> set >= 1GB. That's not true. For the case we're talking about here, if
> autovacuum_work_mem is set to anything apart from -1 then the value of
> maintenance_work_mem does not matter.

Okay, changed to "or".

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  101. Re: First draft of PG 17 release notes

    John Naylor <johncnaylorls@gmail.com> — 2024-05-20T06:23:02Z

    Hi Bruce, thanks for doing this again!

I'm a bit late to this discussion -- there's been a bit of churn in
the vacuum items, and some streams got crossed along the way. I've
attached an attempt to rectify this.

  102. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-20T13:37:15Z

    On Mon, May 20, 2024 at 01:23:02PM +0700, John Naylor wrote:
> Hi Bruce, thanks for doing this again!
> 
> I'm a bit late to this discussion -- there's been a bit of churn in
> the vacuum items, and some streams got crossed along the way. I've
> attached an attempt to rectify this.

Agreed, patch applied, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  103. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-20T18:35:37Z

    On Sat, May 18, 2024 at 11:13 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, May 16, 2024 at 09:09:11AM -0400, Melanie Plageman wrote:
> > On Wed, May 15, 2024 at 11:48 PM Andres Freund <andres@anarazel.de> wrote:
> > >
> > > On 2024-05-15 10:38:20 +0200, Alvaro Herrera wrote:
> > > > I disagree with this.  IMO the impact of the Sawada/Naylor change is
> > > > likely to be enormous for people with large tables and large numbers of
> > > > tuples to clean up (I know we've had a number of customers in this
> > > > situation, I can't imagine any Postgres service provider that doesn't).
> > > > The fact that maintenance_work_mem is no longer capped at 1GB is very
> > > > important and I think we should mention that explicitly in the release
> > > > notes, as setting it higher could make a big difference in vacuum run
> > > > times.
> > >
> > > +many.
> > >
> > > We're having this debate every release. I think the ongoing reticence to note
> > > performance improvements in the release notes is hurting Postgres.
> > >
> > > For one, performance improvements are one of the prime reason users
> > > upgrade. Without them being noted anywhere more dense than the commit log,
> > > it's very hard to figure out what improved for users. A halfway widely
> > > applicable performance improvement is far more impactful than many of the
> > > feature changes we do list in the release notes.
> >
> > The practical reason this matters to users is that they want to change
> > their configuration or expectations in response to performance
> > improvements.
> >
> > And also, as Jelte mentions upthread, describing performance
> > improvements made each release in Postgres makes it clear that we are
> > consistently improving it.
> >
> > > For another, it's also very frustrating for developers that focus on
> > > performance. The reticence to note their work, while noting other, far
> > > smaller, things in the release notes, pretty much tells us that our work isn't
> > > valued.
> >
> > +many
>
> Please see the email I just posted.  There are three goals we have to
> adjust for:
>
> 1.  short release notes so they are readable
> 2.  giving people credit for performance improvements
> 3.  showing people Postgres cares about performance

I agree with all three of these goals. I would even add to 3 "show
users Postgres is addressing their performance complaints". That in
particular makes me less excited about having a  "generic performance
release note item saying performance has been improved in the
following areas" (from your other email). I think that describing the
specific performance improvements is required to 1) allow users to
change expectations and configurations to take advantage of the
performance enhancements 2) ensure users know that their performance
concerns are being addressed.

> I would like to achieve 2 & 3 without harming #1.  My experience is if I
> am reading a long document, and I get to a section where I start to
> wonder, "Why should I care about this?", I start to skim the rest of
> the document.  I am particularly critical if I start to wonder, "Why
> does the author _think_ I should care about this?" becasue it feels like
> the author is writing for him/herself and not the audience.

I see what you are saying. We don't want to just end up with the whole
git log in the release notes. However, we know that not all users will
care about the same features. As someone said somewhere else in this
thread, presumably hackers spend time on features because some users
want them.

- Melanie

  104. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-20T18:40:42Z

    On Mon, May 20, 2024 at 02:35:37PM -0400, Melanie Plageman wrote:
> On Sat, May 18, 2024 at 11:13 AM Bruce Momjian <bruce@momjian.us> wrote:
> > Please see the email I just posted.  There are three goals we have to
> > adjust for:
> >
> > 1.  short release notes so they are readable
> > 2.  giving people credit for performance improvements
> > 3.  showing people Postgres cares about performance
> 
> I agree with all three of these goals. I would even add to 3 "show
> users Postgres is addressing their performance complaints". That in
> particular makes me less excited about having a  "generic performance
> release note item saying performance has been improved in the
> following areas" (from your other email). I think that describing the
> specific performance improvements is required to 1) allow users to
> change expectations and configurations to take advantage of the
> performance enhancements 2) ensure users know that their performance
> concerns are being addressed.

Well, as you can see, doing #2 & #3 works against accomplishing #1.

> > I would like to achieve 2 & 3 without harming #1.  My experience is if I
> > am reading a long document, and I get to a section where I start to
> > wonder, "Why should I care about this?", I start to skim the rest of
> > the document.  I am particularly critical if I start to wonder, "Why
> > does the author _think_ I should care about this?" becasue it feels like
> > the author is writing for him/herself and not the audience.
> 
> I see what you are saying. We don't want to just end up with the whole
> git log in the release notes. However, we know that not all users will
> care about the same features. As someone said somewhere else in this
> thread, presumably hackers spend time on features because some users
> want them.

I think we need as a separate section about performance improvements
that don't affect specific workloads.  Peter Eisentraut created an
Acknowledgements section at the bottom of the release notes, similar to
#2 above, so maybe someone else can add a performance internals section
too.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  105. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-20T18:47:28Z

    On Mon, May 20, 2024 at 9:37 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Mon, May 20, 2024 at 01:23:02PM +0700, John Naylor wrote:
> > Hi Bruce, thanks for doing this again!
> >
> > I'm a bit late to this discussion -- there's been a bit of churn in
> > the vacuum items, and some streams got crossed along the way. I've
> > attached an attempt to rectify this.
>
> Agreed, patch applied, thanks.

If "Allow vacuum to more efficiently remove and freeze tuples" stays
in the release notes, I would add Heikki's name. He wasn't listed as a
co-author on all of the commits that were part of this feature, but he
made a substantial investment in it and should be listed, IMO. Patch
attached.

- Melanie

  106. Re: First draft of PG 17 release notes

    Jeff Davis <pgsql@j-davis.com> — 2024-05-20T18:48:09Z

    On Sat, 2024-05-18 at 17:51 -0400, Bruce Momjian wrote:
> Okay, I went with the attached applied patch.  Adjustments?

I think it should have more emphasis on the actual new feature: a
platform-independent builtin collation provider and the C.UTF-8 locale.

The user can look at the documentation for comparison with libc.

Regards,
	Jeff Davis

  107. Re: First draft of PG 17 release notes

    amit <amitlangote09@gmail.com> — 2024-05-21T02:20:02Z

    On Thu, May 9, 2024 at 1:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>
>         release-10:  189
>         release-11:  170
>         release-12:  180
>         release-13:  178
>         release-14:  220
>         release-15:  184
>         release-16:  206
>         release-17:  188
>
> I welcome feedback.  For some reason it was an easier job than usual.

Thanks Bruce for working on this as always.

Failed to notice when I read the notes before:

<listitem>
<para>
Add SQL/JSON constructor functions JSON(), JSON_SCALAR(), and
JSON_SERIALIZE() (Amit Langote)
</para>
</listitem>

Should be:

<listitem>
<para>
Add SQL/JSON constructor functions JSON(), JSON_SCALAR(), and
JSON_SERIALIZE() (Nikita Glukhov, Teodor Sigaev, Oleg Bartunov,
Alexander Korotkov, Andrew Dunstan, Amit Langote)
</para>
</listitem>

Patch attached.

-- 
Thanks, Amit Langote

  108. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-21T16:27:20Z

    Hi,

On 2024-05-18 10:59:47 -0400, Bruce Momjian wrote:
> On Wed, May 15, 2024 at 08:48:02PM -0700, Andres Freund wrote:
> > +many.
> >
> > We're having this debate every release. I think the ongoing reticence to note
> > performance improvements in the release notes is hurting Postgres.
> >
> > For one, performance improvements are one of the prime reason users
> > upgrade. Without them being noted anywhere more dense than the commit log,
> > it's very hard to figure out what improved for users. A halfway widely
> > applicable performance improvement is far more impactful than many of the
> > feature changes we do list in the release notes.
>
> I agree the impact of performance improvements are often greater than
> the average release note item.  However, if people expect Postgres to be
> faster, is it important for them to know _why_ it is faster?

Yes, it very often is. Performance improvements typically aren't "make
everything 3% faster", they're more "make this special thing 20%
faster". Without know what got faster, users don't know if
a) the upgrade will improve their production situation
b) they need to change something to take advantage of the improvement


> On the flip side, a performance improvement that makes everything 10%
> faster has little behavioral change for users, and in fact I think we
> get ~6% faster in every major release.

I cannot recall many "make everything faster" improvements, if any.

And even if it's "make everything faster" - that's useful for users to know,
it might solve their production problem!  It's also good for PR.


Given how expensive postgres upgrades still are, we can't expect production
workloads to upgrade to every major version. The set of performance
improvements and feature additions between major versions can help users make
an informed decision.


Also, the release notes are also not just important to users. I often go back
and look in the release notes to see when some some important change was made,
because sometimes it's harder to find it in the git log, due to sheer
volume. And even just keeping up with all the changes between two releases is
hard, it's useful to be able to read the release notes and see what happened.


> > For another, it's also very frustrating for developers that focus on
> > performance. The reticence to note their work, while noting other, far
> > smaller, things in the release notes, pretty much tells us that our work isn't
> > valued.
>
> Yes, but are we willing to add text that every user will have to read
> just for this purpose?

Of course it's a tradeoff. We shouldn't verbosely note down every small
changes just because of the recognition, that'd make the release notes
unreadable. And it'd just duplicate the commit log. But that's not the same as
defaulting to not noting performance improvements, even if the performance
improvement is more impactful than many other features that are noted.


> One think we _could_ do is to create a generic performance release note
> item saying performance has been improved in the following areas, with
> no more details, but we can list the authors on the item.

To me that's the "General Performance" section. If somebody reading the
release notes doesn't care about performance, they can just skip that section
([1]).  I don't see why we wouldn't want to include the same level of detail
as for other changes.

Greetings,

Andres Freund

[1] I've wondered if we should have one more level of TOC on the release note
page, so it's easier to jump to specific sections.

  109. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-21T16:40:28Z

    Hi,

On 2024-05-18 11:13:54 -0400, Bruce Momjian wrote:
> Please see the email I just posted.  There are three goals we have to
> adjust for:
> 
> 1.  short release notes so they are readable
> 2.  giving people credit for performance improvements
> 3.  showing people Postgres cares about performance
> 
> I would like to achieve 2 & 3 without harming #1.  My experience is if I
> am reading a long document, and I get to a section where I start to
> wonder, "Why should I care about this?", I start to skim the rest of
> the document.

I agree keeping things reasonably short is important. But I don't think you're
evenly applying it as a goal.

Just skimming the notes from the end, I see
- an 8 entries long pg_stat_statements section
- multiple entries about "Create custom wait events for ..."
- three entries about adding --all to {reindexdb,vacuumdb,clusterdb}.
- an entry about adding long options to pg_archivecleanup
- two entries about grantable maintenance rights, once via pg_maintain, once
  per-table
- separate entries about pg_stat_reset_slru(), pg_stat_reset_shared("slru"),

If you're concerned about brevity, we can make things shorter without skipping
over most performance imporvements.


> I am particularly critical if I start to wonder, "Why
> does the author _think_ I should care about this?" becasue it feels like
> the author is writing for him/herself and not the audience.

FWIW, I think it's a good thing for somebody other than the author to have a
hand in writing a release notes entry for a change. The primary author(s) are
often too deep into some issue to have a good view of the right level of
detail and understandability.

Greetings,

Andres Freund

  110. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-21T16:51:09Z

    Hi,

On 2024-05-21 09:27:20 -0700, Andres Freund wrote:
> Also, the release notes are also not just important to users. I often go back
> and look in the release notes to see when some some important change was made,
> because sometimes it's harder to find it in the git log, due to sheer
> volume. And even just keeping up with all the changes between two releases is
> hard, it's useful to be able to read the release notes and see what happened.
>
> [...]
>
> [1] I've wondered if we should have one more level of TOC on the release note
> page, so it's easier to jump to specific sections.

Which reminds me: Eventually I'd like to add links to the most important
commits related to release note entries. We already do much of the work of
building that list of commits for each entry. That'd allow a reader to find
more details if interested.

Right one either has to open the sgml file (which no user will know to do), or
find the git entries manually. The latter of which is often hard, because the
git commits often will use different wording etc.

Admittedly doing so within the constraints of docbook and not wanting to
overly decrease density (both in .sgml and the resulting html) isn't a trivial
task.


That's entirely independent of my concern around noting performance
improvements in the release notes, of course.

Greetings,

Andres Freund

  111. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-21T16:55:01Z

    On 2024-May-21, Andres Freund wrote:

> Which reminds me: Eventually I'd like to add links to the most important
> commits related to release note entries. We already do much of the work of
> building that list of commits for each entry. That'd allow a reader to find
> more details if interested.

+1.  Several times I have had to open the SGML to fetch a commit ID and
build a URL to provide to someone.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"

  112. Re: First draft of PG 17 release notes

    Peter Geoghegan <pg@bowt.ie> — 2024-05-21T17:06:43Z

    On Tue, May 21, 2024 at 12:27 PM Andres Freund <andres@anarazel.de> wrote:
> > I agree the impact of performance improvements are often greater than
> > the average release note item.  However, if people expect Postgres to be
> > faster, is it important for them to know _why_ it is faster?
>
> Yes, it very often is.

Is it important for them to even read the release notes?

Bruce's arguments against listing performance items more often/with
greater prominence could just as easily be applied to other types of
features, in other areas. Performance is a feature (or a feature
category) -- no better or worse than any other category of feature.

> Performance improvements typically aren't "make
> everything 3% faster", they're more "make this special thing 20%
> faster". Without know what got faster, users don't know if
> a) the upgrade will improve their production situation
> b) they need to change something to take advantage of the improvement

Another important category of performance improvement is "make the
thing that was just unusable usable, for the first time ever".

Sometimes the baseline is unreasonably slow, so an improvement
effectively allows you as a user to do something that just wasn't
possible on previous versions. Other times it's addressed at something
that was very scary, like VACUUMs that need multiple rounds of index
vacuuming. Multiple rounds of index vacuuming are just woefully,
horribly inefficient, and are the single individual thing that can
make things far worse. Even if you didn't technically have that
problem before now, you did have the problem of having to worry about
it. So the work in question has sanded-down this really nasty sharp
edge. That's a substantial quality of life improvement for many users.

In short, many individual performance improvements are best thought of
as qualitative improvements, rather than quantitative improvements.

It doesn't help that there is a kind of pressure to present them as
quantitative improvements. For example, I was recently encouraged to
present my own Postgres 17 B-Tree work internally using some kind of
headline grabbing measure like "6x faster". That just seems silly to
me. I can contrive a case where it's faster by an arbitrarily large
amount. Much like how a selective index scan can be arbitrarily faster
than a sequential scan. Again, a qualitative improvement.

-- 
Peter Geoghegan

  113. Re: First draft of PG 17 release notes

    Robert Haas <robertmhaas@gmail.com> — 2024-05-21T17:50:58Z

    On Tue, May 21, 2024 at 12:27 PM Andres Freund <andres@anarazel.de> wrote:
> To me that's the "General Performance" section. If somebody reading the
> release notes doesn't care about performance, they can just skip that section
> ([1]).  I don't see why we wouldn't want to include the same level of detail
> as for other changes.

I'm relatively sure that we've had this argument in previous years and
essentially everyone but Bruce has agreed with the idea that
performance changes ought to be treated the same as any other kind of
improvement. The difficulty is that Bruce is the one doing the release
notes. I think it might help if someone were willing to prepare a
patch showing what they think specifically should be changed. Or maybe
Bruce would be willing to provide a list of all of the performance
improvements he doesn't think are worth release-noting or isn't sure
how to release-note, and someone else can then have a go at them.

Personally, I suspect that a part of the problem, other than the
inevitable fact that the person doing the work has a perspective on
how the work should be done with which not everyone will agree, is
that a lot of performance changes have commit messages that don't
really explain what the user impact is. For instance, consider
6dbb490261a6170a3fc3e326c6983ad63e795047 ("Combine freezing and
pruning steps in VACUUM"). It does actually say what the benefit is
("That reduces the overall amount of WAL generated") but the reader
could easily be left wondering whether that is really the selling
point. Does it also reduce CPU consumption? Is that more or less
important than the WAL reduction? Was the WAL reduction the motivation
for the work? Is the WAL reduction significant enough that this is a
feature in its own right, or is this just preparatory to some other
work? These kinds of ambiguities can exist for any commit, not just
performance commits, but I bet that on average the problem is worse
for performance-related commits.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  114. Re: First draft of PG 17 release notes

    Melanie Plageman <melanieplageman@gmail.com> — 2024-05-21T18:26:15Z

    On Tue, May 21, 2024 at 1:51 PM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Tue, May 21, 2024 at 12:27 PM Andres Freund <andres@anarazel.de> wrote:
> > To me that's the "General Performance" section. If somebody reading the
> > release notes doesn't care about performance, they can just skip that section
> > ([1]).  I don't see why we wouldn't want to include the same level of detail
> > as for other changes.
>
> I'm relatively sure that we've had this argument in previous years and
> essentially everyone but Bruce has agreed with the idea that
> performance changes ought to be treated the same as any other kind of
> improvement. The difficulty is that Bruce is the one doing the release
> notes. I think it might help if someone were willing to prepare a
> patch showing what they think specifically should be changed. Or maybe
> Bruce would be willing to provide a list of all of the performance
> improvements he doesn't think are worth release-noting or isn't sure
> how to release-note, and someone else can then have a go at them.
>
> Personally, I suspect that a part of the problem, other than the
> inevitable fact that the person doing the work has a perspective on
> how the work should be done with which not everyone will agree, is
> that a lot of performance changes have commit messages that don't
> really explain what the user impact is. For instance, consider
> 6dbb490261a6170a3fc3e326c6983ad63e795047 ("Combine freezing and
> pruning steps in VACUUM"). It does actually say what the benefit is
> ("That reduces the overall amount of WAL generated") but the reader
> could easily be left wondering whether that is really the selling
> point. Does it also reduce CPU consumption? Is that more or less
> important than the WAL reduction? Was the WAL reduction the motivation
> for the work? Is the WAL reduction significant enough that this is a
> feature in its own right, or is this just preparatory to some other
> work? These kinds of ambiguities can exist for any commit, not just
> performance commits, but I bet that on average the problem is worse
> for performance-related commits.

In Postgres development, we break larger projects into smaller ones
and then those smaller projects into multiple individual commits. Each
commit needs to stand alone and each subproject needs to have a
defensible benefit. One thing that is harder with performance-related
work than non-performance feature work is that there isn't always a
final "turn it on" commit. For example, let's say you are adding a new
view that tracks new stats of some kind. You do a bunch of refactoring
and small subprojects to make it possible to add the view. Then the
final commit that actually creates the view has obvious user value to
whoever is reading the log. For performance features, it doesn't
always work like this.

For the vacuum WAL volume reduction, there were a bunch of smaller
projects throughout the last development year that I worked on that
were committed by different people and with different individual
benefits. Some changes caused vacuum to do less visibility checks (so
less CPU usage), some changed WAL format in a way that saves some
space, and some, like the commit you mention, make vacuum emit less
WAL. That commit by itself doesn't contain all of the user benefits of
the whole project. I couldn't think of a good place to list all of the
commits together that were part of the same project. Perhaps you could
argue that they were not in fact part of the same project and instead
were just small individual changes -- none of which are individually
worth including in the release notes.

- Melanie

  115. Re: First draft of PG 17 release notes

    Robert Haas <robertmhaas@gmail.com> — 2024-05-21T18:53:39Z

    On Tue, May 21, 2024 at 2:26 PM Melanie Plageman
<melanieplageman@gmail.com> wrote:
> For the vacuum WAL volume reduction, there were a bunch of smaller
> projects throughout the last development year that I worked on that
> were committed by different people and with different individual
> benefits. Some changes caused vacuum to do less visibility checks (so
> less CPU usage), some changed WAL format in a way that saves some
> space, and some, like the commit you mention, make vacuum emit less
> WAL. That commit by itself doesn't contain all of the user benefits of
> the whole project. I couldn't think of a good place to list all of the
> commits together that were part of the same project. Perhaps you could
> argue that they were not in fact part of the same project and instead
> were just small individual changes -- none of which are individually
> worth including in the release notes.

Yeah, I think a lot of projects have this problem in one way or
another, but I think it may be worse for performance-related projects.

I wasn't intending to knock that particular commit, just to be clear,
or the commit message. I'm just saying that sometimes summarizing the
commit log may not be as easy as we'd hope.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  116. Re: First draft of PG 17 release notes

    Masahiko Sawada <sawada.mshk@gmail.com> — 2024-05-22T02:29:06Z

    Hi,

On Thu, May 9, 2024 at 1:03 PM Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html
>
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
>

I found a typo:

s/pg_statstatement/pg_stat_statement/

I've attached a patch to fix it.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

  117. Re: First draft of PG 17 release notes

    torikoshia <torikoshia@oss.nttdata.com> — 2024-05-22T12:25:41Z

    On 2024-05-09 13:03, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
> 	https://momjian.us/pgsql_docs/release-17.html
> 
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
> 
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
> 
> I welcome feedback.  For some reason it was an easier job than usual.

Thanks for working on this as always.

<para>
Allow pg_stat_reset_shared("slru") to clear SLRU statistics (Atsushi 
Torikoshi)
</para>

Considering someone may copy and paste this, 'slru' is better than 
"slru", isn't it?
I also found an older release note[1] used single quotes for this like:

   Add pg_stat_reset_shared('bgwriter') to reset the cluster-wide shared 
statistics for the background writer (Greg Smith)

[1] https://www.postgresql.org/docs/release/9.0.0/

-- 
Regards,

--
Atsushi Torikoshi
NTT DATA Group Corporation

  118. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T21:59:20Z

    On Mon, May 20, 2024 at 02:47:28PM -0400, Melanie Plageman wrote:
> On Mon, May 20, 2024 at 9:37 AM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Mon, May 20, 2024 at 01:23:02PM +0700, John Naylor wrote:
> > > Hi Bruce, thanks for doing this again!
> > >
> > > I'm a bit late to this discussion -- there's been a bit of churn in
> > > the vacuum items, and some streams got crossed along the way. I've
> > > attached an attempt to rectify this.
> >
> > Agreed, patch applied, thanks.
> 
> If "Allow vacuum to more efficiently remove and freeze tuples" stays
> in the release notes, I would add Heikki's name. He wasn't listed as a
> co-author on all of the commits that were part of this feature, but he
> made a substantial investment in it and should be listed, IMO. Patch
> attached.

Thanks, patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  119. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:04:09Z

    On Tue, May 21, 2024 at 09:27:20AM -0700, Andres Freund wrote:
> On 2024-05-18 10:59:47 -0400, Bruce Momjian wrote:
> > I agree the impact of performance improvements are often greater than
> > the average release note item.  However, if people expect Postgres to be
> > faster, is it important for them to know _why_ it is faster?
> 
> Yes, it very often is. Performance improvements typically aren't "make
> everything 3% faster", they're more "make this special thing 20%
> faster". Without know what got faster, users don't know if
> a) the upgrade will improve their production situation
> b) they need to change something to take advantage of the improvement

You might have seen in this thread, I do record commits that speed up
workloads that are user-visible, or specifically make new workloads
possible.  I assume that covers the items above, though I have to
determine this from the commit message.

> > On the flip side, a performance improvement that makes everything 10%
> > faster has little behavioral change for users, and in fact I think we
> > get ~6% faster in every major release.
> 
> I cannot recall many "make everything faster" improvements, if any.
> 
> And even if it's "make everything faster" - that's useful for users to know,
> it might solve their production problem!  It's also good for PR.

Again, it is down to having three goals for the release notes, and #1 is
having it readable/short, and 2 & 3 are for PR and acknowledging authors.

> Also, the release notes are also not just important to users. I often go back
> and look in the release notes to see when some some important change was made,
> because sometimes it's harder to find it in the git log, due to sheer
> volume. And even just keeping up with all the changes between two releases is
> hard, it's useful to be able to read the release notes and see what happened.

Well, I would say we need some _other_ way to record and perhaps
advertise such changes.

> > > For another, it's also very frustrating for developers that focus on
> > > performance. The reticence to note their work, while noting other, far
> > > smaller, things in the release notes, pretty much tells us that our work isn't
> > > valued.
> >
> > Yes, but are we willing to add text that every user will have to read
> > just for this purpose?
> 
> Of course it's a tradeoff. We shouldn't verbosely note down every small
> changes just because of the recognition, that'd make the release notes
> unreadable. And it'd just duplicate the commit log. But that's not the same as
> defaulting to not noting performance improvements, even if the performance
> improvement is more impactful than many other features that are noted.

Again, see above, I do mention performance improvements, but they have
to be user-visible or enable new workloads.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  120. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:05:13Z

    On Tue, May 21, 2024 at 09:51:09AM -0700, Andres Freund wrote:
> Hi,
> 
> On 2024-05-21 09:27:20 -0700, Andres Freund wrote:
> > Also, the release notes are also not just important to users. I often go back
> > and look in the release notes to see when some some important change was made,
> > because sometimes it's harder to find it in the git log, due to sheer
> > volume. And even just keeping up with all the changes between two releases is
> > hard, it's useful to be able to read the release notes and see what happened.
> >
> > [...]
> >
> > [1] I've wondered if we should have one more level of TOC on the release note
> > page, so it's easier to jump to specific sections.
> 
> Which reminds me: Eventually I'd like to add links to the most important
> commits related to release note entries. We already do much of the work of
> building that list of commits for each entry. That'd allow a reader to find
> more details if interested.

Yes, it would be cool if they could mouse-over a graphic next to each
release note item to get a popup to the commits.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  121. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:13:47Z

    On Tue, May 21, 2024 at 01:50:58PM -0400, Robert Haas wrote:
> On Tue, May 21, 2024 at 12:27 PM Andres Freund <andres@anarazel.de> wrote:
> > To me that's the "General Performance" section. If somebody reading the
> > release notes doesn't care about performance, they can just skip that section
> > ([1]).  I don't see why we wouldn't want to include the same level of detail
> > as for other changes.
> 
> I'm relatively sure that we've had this argument in previous years and
> essentially everyone but Bruce has agreed with the idea that
> performance changes ought to be treated the same as any other kind of
> improvement. The difficulty is that Bruce is the one doing the release
> notes. I think it might help if someone were willing to prepare a
> patch showing what they think specifically should be changed. Or maybe
> Bruce would be willing to provide a list of all of the performance
> improvements he doesn't think are worth release-noting or isn't sure
> how to release-note, and someone else can then have a go at them.

Well, developers do ask why their individual commits were not listed,
and I repeat the same thing, as I have done in this thread multiple
times.  You can probably look over this thread to find them, or in
previous years.

To be clear, it is performance improvements that don't have user-visible
changes or enable new workloads that I skip listing.  I will also note
that don't remember any user ever finding a performance boost, and then
coming to use and asking why we didn't list it --- this release note
review process seems to add all of those.

Maybe adding a section called "Internal Performance".  Here is our
General Performance contents:

	E.1.3.1.3. General Performance
	
	    Allow vacuum to more efficiently remove and freeze tuples (Melanie
	Plageman)
	
	    WAL traffic caused by vacuum is also more compact.
	
	    Allow vacuum to more efficiently store tuple references (Masahiko
	Sawada, John Naylor)
	
	    Additionally, vacuum is no longer silently limited to one gigabyte
	of memory when maintenance_work_mem or autovacuum_work_mem are higher.
	
	    Optimize vacuuming of relations with no indexes (Melanie Plageman)
	
	    Increase default vacuum_buffer_usage_limit to 2MB (Thomas Munro)
	
	    Improve performance when checking roles with many memberships
	(Nathan Bossart)
	
	    Improve performance of heavily-contended WAL writes (Bharath
	Rupireddy)
	
	    Improve performance when transferring large blocks of data to a
	client (Melih Mutlu)
	
	    Allow the grouping of file system reads with the new system variable
	io_combine_limit (Thomas Munro, Andres Freund, Melanie Plageman, Nazir
	Bilal Yavuz)

Do we think more user-invisible changes should be in there?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  122. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:15:04Z

    On Tue, May 21, 2024 at 02:26:15PM -0400, Melanie Plageman wrote:
> In Postgres development, we break larger projects into smaller ones
> and then those smaller projects into multiple individual commits. Each
> commit needs to stand alone and each subproject needs to have a
> defensible benefit. One thing that is harder with performance-related
> work than non-performance feature work is that there isn't always a
> final "turn it on" commit. For example, let's say you are adding a new
> view that tracks new stats of some kind. You do a bunch of refactoring
> and small subprojects to make it possible to add the view. Then the
> final commit that actually creates the view has obvious user value to
> whoever is reading the log. For performance features, it doesn't
> always work like this.
> 
> For the vacuum WAL volume reduction, there were a bunch of smaller
> projects throughout the last development year that I worked on that
> were committed by different people and with different individual
> benefits. Some changes caused vacuum to do less visibility checks (so
> less CPU usage), some changed WAL format in a way that saves some
> space, and some, like the commit you mention, make vacuum emit less
> WAL. That commit by itself doesn't contain all of the user benefits of
> the whole project. I couldn't think of a good place to list all of the
> commits together that were part of the same project. Perhaps you could
> argue that they were not in fact part of the same project and instead
> were just small individual changes -- none of which are individually
> worth including in the release notes.

I try and group them, but I am sure imperfectly.  It is very true that
infrastucture changes that enable later commits are often missed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  123. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:33:03Z

    On Tue, May 21, 2024 at 09:40:28AM -0700, Andres Freund wrote:
> Hi,
> 
> On 2024-05-18 11:13:54 -0400, Bruce Momjian wrote:
> > Please see the email I just posted.  There are three goals we have to
> > adjust for:
> > 
> > 1.  short release notes so they are readable
> > 2.  giving people credit for performance improvements
> > 3.  showing people Postgres cares about performance
> > 
> > I would like to achieve 2 & 3 without harming #1.  My experience is if I
> > am reading a long document, and I get to a section where I start to
> > wonder, "Why should I care about this?", I start to skim the rest of
> > the document.
> 
> I agree keeping things reasonably short is important. But I don't think you're
> evenly applying it as a goal.
> 
> Just skimming the notes from the end, I see
> - an 8 entries long pg_stat_statements section

What item did you want to remove?  Those are all user-visible changes.

> - multiple entries about "Create custom wait events for ..."

Well, those are all in different sections, so how can they be merged,
unless I create a "wait event section", I guess.

> - three entries about adding --all to {reindexdb,vacuumdb,clusterdb}.

The problem with merging these is that the "Specifically, --all can now
be used with" is different for all three of them.

> - an entry about adding long options to pg_archivecleanup

Well, that is a user-visible change.  Should it not be listed?

> - two entries about grantable maintenance rights, once via pg_maintain, once
>   per-table

Well, one is a GRANT and another is a role, so merging them seemed like
it would be too confusing.

> - separate entries about pg_stat_reset_slru(), pg_stat_reset_shared("slru"),

They are different functions with different detail text.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  124. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:39:41Z

    On Mon, May 20, 2024 at 11:48:09AM -0700, Jeff Davis wrote:
> On Sat, 2024-05-18 at 17:51 -0400, Bruce Momjian wrote:
> > Okay, I went with the attached applied patch.  Adjustments?
> 
> I think it should have more emphasis on the actual new feature: a
> platform-independent builtin collation provider and the C.UTF-8 locale.
> 
> The user can look at the documentation for comparison with libc.

Okay, changed with the attached applied patch.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  125. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:46:53Z

    On Tue, May 21, 2024 at 11:20:02AM +0900, Amit Langote wrote:
> Thanks Bruce for working on this as always.
> 
> Failed to notice when I read the notes before:
> 
> <listitem>
> <para>
> Add SQL/JSON constructor functions JSON(), JSON_SCALAR(), and
> JSON_SERIALIZE() (Amit Langote)
> </para>
> </listitem>
> 
> Should be:
> 
> <listitem>
> <para>
> Add SQL/JSON constructor functions JSON(), JSON_SCALAR(), and
> JSON_SERIALIZE() (Nikita Glukhov, Teodor Sigaev, Oleg Bartunov,
> Alexander Korotkov, Andrew Dunstan, Amit Langote)
> </para>
> </listitem>

Thanks, applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  126. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:48:31Z

    On Wed, May 22, 2024 at 11:29:06AM +0900, Masahiko Sawada wrote:
> I found a typo:
> 
> s/pg_statstatement/pg_stat_statement/
> 
> I've attached a patch to fix it.

Agreed, applied, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  127. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-22T22:50:53Z

    On Wed, May 22, 2024 at 09:25:41PM +0900, torikoshia wrote:
> Thanks for working on this as always.
> 
> <para>
> Allow pg_stat_reset_shared("slru") to clear SLRU statistics (Atsushi
> Torikoshi)
> </para>
> 
> Considering someone may copy and paste this, 'slru' is better than "slru",
> isn't it?
> I also found an older release note[1] used single quotes for this like:
> 
>   Add pg_stat_reset_shared('bgwriter') to reset the cluster-wide shared
> statistics for the background writer (Greg Smith)

Agreed, patch applied, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  128. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-23T01:34:10Z

    On Thu, 23 May 2024 at 10:04, Bruce Momjian <bruce@momjian.us> wrote:
> You might have seen in this thread, I do record commits that speed up
> workloads that are user-visible, or specifically make new workloads
> possible.  I assume that covers the items above, though I have to
> determine this from the commit message.

It sometimes is hard to write something specific in the commit message
about the actual performance increase.

For example, if a commit removes an O(N log2 N) algorithm and replaces
it with an O(1), you can't say there's an X% increase in performance
as the performance increase depends on the value of N.

Jelte did call me out for not mentioning enough detail about the
performance in c4ab7da60, but if I claimed any % of an increase, it
would have been specific to some workload.

What is the best way to communicate this stuff so it's easily
identifiable when you parse the commit messages?

David

  129. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-23T02:01:51Z

    On Thu, May 23, 2024 at 01:34:10PM +1200, David Rowley wrote:
> On Thu, 23 May 2024 at 10:04, Bruce Momjian <bruce@momjian.us> wrote:
> > You might have seen in this thread, I do record commits that speed up
> > workloads that are user-visible, or specifically make new workloads
> > possible.  I assume that covers the items above, though I have to
> > determine this from the commit message.
> 
> It sometimes is hard to write something specific in the commit message
> about the actual performance increase.
> 
> For example, if a commit removes an O(N log2 N) algorithm and replaces
> it with an O(1), you can't say there's an X% increase in performance
> as the performance increase depends on the value of N.
> 
> Jelte did call me out for not mentioning enough detail about the
> performance in c4ab7da60, but if I claimed any % of an increase, it
> would have been specific to some workload.
> 
> What is the best way to communicate this stuff so it's easily
> identifiable when you parse the commit messages?

This is why I think we need an "Internal Performance" section where we
can be clear about simple scaling improvements that might have no
user-visible explanation.  I would suggest putting it after our "Source
code" section.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  130. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-23T02:27:07Z

    On Thu, 23 May 2024 at 14:01, Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, May 23, 2024 at 01:34:10PM +1200, David Rowley wrote:
> > What is the best way to communicate this stuff so it's easily
> > identifiable when you parse the commit messages?
>
> This is why I think we need an "Internal Performance" section where we
> can be clear about simple scaling improvements that might have no
> user-visible explanation.  I would suggest putting it after our "Source
> code" section.

hmm, but that does not really answer my question. There's still a
communication barrier if you're parsing the commit messages are
committers don't clearly state some performance improvement numbers
for the reason I stated.

I also don't agree these should be left to "Source code" section. I
feel that section is best suited for extension authors who might care
about some internal API change.  I'm talking of stuff that makes some
user queries possibly N times (!) faster. Surely "Source Code" isn't
the place to talk about that?

David

  131. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-23T11:22:51Z

    Hello,

Regarding this item

: Allow the SLRU cache sizes to be configured (Andrey Borodin, Dilip Kumar)
: 
: The new server variables are commit_timestamp_buffers,
: multixact_member_buffers, multixact_offset_buffers, notify_buffers,
: serializable_buffers, subtransaction_buffers, and transaction_buffers.

I hereby request to be listed as third author of this feature.

Also, I'd like to suggest to make it more verbose, as details might be
useful to users.  Mention that scalability is improved, because
previously we've suggested to recompile with larger #defines, but to be
cautious because values too high degrade performance.  Also mention the
point that some of these grow with shared_buffers is user-visible enough
that it warrants an explicit mention.  How about like this:

: Allow the SLRU cache sizes to be configured and improve performance of
: larger caches
: (Andrey Borodin, Dilip Kumar, Álvaro Herrera)
: 
: The new server variables are commit_timestamp_buffers,
: multixact_member_buffers, multixact_offset_buffers, notify_buffers,
: serializable_buffers, subtransaction_buffers, and transaction_buffers.
: commit_timestamp_buffers, transaction_buffers and
: subtransaction_buffers scale up automatically with shared_buffers.


These three items

: Allow pg_stat_reset_shared() to reset all shared statistics (Atsushi Torikoshi)
: 
: This is done by passing NULL.
: 
: Allow pg_stat_reset_shared('slru') to clear SLRU statistics (Atsushi Torikoshi)
: 
: Now pg_stat_reset_shared(NULL) also resets SLRU statistics.
: 
: Allow pg_stat_reset_slru() to reset all SLRU statistics (Bharath Rupireddy)
: 
: The command pg_stat_reset_slru(NULL) already did this.

seem a bit repetitive.  (I think the first one is also wrong, because it
says you have to pass NULL, but in reality you can also not give an
argument and it works.)  Can we make them a single item?  Maybe
something like

: Improve reset routines for shared statistics (Atsushi Torikoshi, Bharath Rupireddy)
:
: Resetting all shared statistics can now be done with
: pg_stat_reset_shared() or pg_stat_reset_shared(NULL), while SLRU
: statistics can now be reset with pg_stat_reset_shared('slru'),
: pg_stat_reset_slru() and pg_stat_reset_slru(NULL).

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"Find a bug in a program, and fix it, and the program will work today.
Show the program how to find and fix a bug, and the program
will work forever" (Oliver Silfridge)

  132. Re: First draft of PG 17 release notes

    Jeff Davis <pgsql@j-davis.com> — 2024-05-23T15:30:25Z

    On Wed, 2024-05-22 at 18:39 -0400, Bruce Momjian wrote:
> On Mon, May 20, 2024 at 11:48:09AM -0700, Jeff Davis wrote:
> > On Sat, 2024-05-18 at 17:51 -0400, Bruce Momjian wrote:
> > > Okay, I went with the attached applied patch.  Adjustments?
> > 
> > I think it should have more emphasis on the actual new feature: a
> > platform-independent builtin collation provider and the C.UTF-8
> > locale.
> > 
> > The user can look at the documentation for comparison with libc.
> 
> Okay, changed with the attached applied patch.

Thank you, looks good to me.

Regards,
	Jeff Davis

  133. Re: First draft of PG 17 release notes

    Marcos Pegoraro <marcos@f10.com.br> — 2024-05-23T19:54:28Z

       -

   Rename SLRU columns in system view pg_stat_slru (Alvaro Herrera)

   The column names accepted by pg_stat_slru_rest() are also changed.

Is pg_stat_slru_rest() correct ?

  134. Re: First draft of PG 17 release notes

    Peter Geoghegan <pg@bowt.ie> — 2024-05-24T00:19:15Z

    On Wed, May 22, 2024 at 6:50 PM Bruce Momjian <bruce@momjian.us> wrote:
> Agreed, patch applied, thanks.

The item for my commit 5bf748b8 currently reads:

"Allow btree indexes to more efficiently find array matches"

I think that this isn't likely to mean much to most users. It seems
like it'd be a lot clearer if the wording was more in line with the
beta1 announcement, which talks about the work as an enhancement to
index scans that use an IN ( ) condition. Specifically referencing
IN() (as opposed to something about arrays or array conditions) is
likely to make the item much more understandable.

Referencing array matches makes me think of a GIN index on an array
column. While ScalarArrayOps do use an array under the cover, that's
mostly an implementation detail. At least it is to users that
exclusively use IN(), likely the majority (that's the SQL standard
syntax).

For context, the Postgres 9.2 release notes described the feature that
my work directly built on as follows:

"Allow indexed_col op ANY(ARRAY[...]) conditions to be used in plain
index scans and index-only scans"

This was a very accurate description of this earlier work. Similar
wording could be used now, but that doesn't seem great to me either.
Simply because this wording also doesn't reference IN() conditions in
index quals.

-- 
Peter Geoghegan

  135. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-24T03:04:43Z

    On Thu, May 23, 2024 at 02:27:07PM +1200, David Rowley wrote:
> On Thu, 23 May 2024 at 14:01, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Thu, May 23, 2024 at 01:34:10PM +1200, David Rowley wrote:
> > > What is the best way to communicate this stuff so it's easily
> > > identifiable when you parse the commit messages?
> >
> > This is why I think we need an "Internal Performance" section where we
> > can be clear about simple scaling improvements that might have no
> > user-visible explanation.  I would suggest putting it after our "Source
> > code" section.
> 
> hmm, but that does not really answer my question. There's still a
> communication barrier if you're parsing the commit messages are
> committers don't clearly state some performance improvement numbers
> for the reason I stated.

For a case where O(N^2) become O(N), we might not even know the
performance change since it is a micro-optimization.  That is why I
suggested we call it "Internal Performance".

> I also don't agree these should be left to "Source code" section. I
> feel that section is best suited for extension authors who might care
> about some internal API change.  I'm talking of stuff that makes some
> user queries possibly N times (!) faster. Surely "Source Code" isn't
> the place to talk about that?

I said a new section "after our 'Source code' section," not in the
source code section.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  136. Re: First draft of PG 17 release notes

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-05-24T03:11:10Z

    Bruce Momjian <bruce@momjian.us> writes:
> On Thu, May 23, 2024 at 02:27:07PM +1200, David Rowley wrote:
>> I also don't agree these should be left to "Source code" section. I
>> feel that section is best suited for extension authors who might care
>> about some internal API change.  I'm talking of stuff that makes some
>> user queries possibly N times (!) faster. Surely "Source Code" isn't
>> the place to talk about that?

> I said a new section "after our 'Source code' section," not in the
> source code section.

Surely, if we make this a separate section, it would come before
'Source code'?

I am not sure Bruce that you realize that your disregard for
performance improvements is shared by nobody.  Arguably,
performance is 90% of what we do these days, and it's also
90% of what users care about.

			regards, tom lane

  137. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-24T03:27:04Z

    On Thu, May 23, 2024 at 11:11:10PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Thu, May 23, 2024 at 02:27:07PM +1200, David Rowley wrote:
> >> I also don't agree these should be left to "Source code" section. I
> >> feel that section is best suited for extension authors who might care
> >> about some internal API change.  I'm talking of stuff that makes some
> >> user queries possibly N times (!) faster. Surely "Source Code" isn't
> >> the place to talk about that?
> 
> > I said a new section "after our 'Source code' section," not in the
> > source code section.
> 
> Surely, if we make this a separate section, it would come before
> 'Source code'?
> 
> I am not sure Bruce that you realize that your disregard for
> performance improvements is shared by nobody.  Arguably,
> performance is 90% of what we do these days, and it's also
> 90% of what users care about.

Please stop saying I don't document performance.  I have already
explained enough which performance items I choose.  Please address my
criteria or suggest new criteria.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  138. Re: First draft of PG 17 release notes

    Robert Haas <robertmhaas@gmail.com> — 2024-05-24T13:54:14Z

    On Thu, May 23, 2024 at 11:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> For a case where O(N^2) become O(N), we might not even know the
> performance change since it is a micro-optimization.  That is why I
> suggested we call it "Internal Performance".

I don't get this at all. If we can't tell the difference between
O(N^2) and O(N), then N was small enough that there wasn't any real
need to optimize in the first place. I think we should be assuming
that if somebody took the trouble to write a patch, the difference did
matter. Hence the change would be user-visible, and should be
documented.

"Internal Performance" doesn't make a lot of sense to me as a section
heading. What does "Internal" mean here as opposed to "General"? I
suspect you mean to imply that the user won't be able to tell the
difference, but I doubt that very much. We make performance
improvements because they are user-visible. If a performance
improvement is so miniscule that nobody would ever notice the
difference, well then I don't think it needs to be release-noted at
all, and we might have a few changes like that where people were
mostly aiming for code cleanliness. But in general, what people do is
look for something that's slow (for them) and try to make it faster.
So the presumption should be that a performance feature has a
meaningful impact on users, and then in rare cases we may decide
otherwise.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  139. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-24T17:50:28Z

    Hi,

On 2024-05-23 23:27:04 -0400, Bruce Momjian wrote:
> On Thu, May 23, 2024 at 11:11:10PM -0400, Tom Lane wrote:
> > Bruce Momjian <bruce@momjian.us> writes:
> > I am not sure Bruce that you realize that your disregard for
> > performance improvements is shared by nobody.  Arguably,
> > performance is 90% of what we do these days, and it's also
> > 90% of what users care about.
>
> Please stop saying I don't document performance.  I have already
> explained enough which performance items I choose.  Please address my
> criteria or suggest new criteria.

Bruce, just about everyone seems to disagree with your current approach. And
not just this year, this has been a discussion in most if not all release note
threads of the last few years.

People, including me, *have* addressed your criteria, but you just waved those
concerns away. It's hard to continue discussing criteria when it doesn't at
all feel like a conversation.

In the end, these are patches to the source code, I don't think you can just
wave away widespread disagreement with your changes. That's not how we do
postgres development.

Greetings,

Andres Freund

  140. Re: First draft of PG 17 release notes

    Andres Freund <andres@anarazel.de> — 2024-05-24T18:23:29Z

    Hi,

On 2024-05-22 18:33:03 -0400, Bruce Momjian wrote:
> On Tue, May 21, 2024 at 09:40:28AM -0700, Andres Freund wrote:
> > On 2024-05-18 11:13:54 -0400, Bruce Momjian wrote:
> > I agree keeping things reasonably short is important. But I don't think you're
> > evenly applying it as a goal.
> >
> > Just skimming the notes from the end, I see
> > - an 8 entries long pg_stat_statements section
>
> What item did you want to remove?  Those are all user-visible changes.

My point here was not that we necessarily need to remove those, but that their
impact to users is smaller than many of the performance impacts you disregard.


> > - multiple entries about "Create custom wait events for ..."
>
> Well, those are all in different sections, so how can they be merged,
> unless I create a "wait event section", I guess.

They're not, all are in "Additional Modules". Instead of

- Create custom wait events for postgres_fdw (Masahiro Ikeda)
- Create custom wait events for dblink (Masahiro Ikeda)
- Allow extensions to define custom wait events (Masahiro Ikeda)

I'd make it:

- Allow extensions to define custom wait events and create custom wait events
  for postgres_fdw, dblink (Masahiro Ikeda)


> > - three entries about adding --all to {reindexdb,vacuumdb,clusterdb}.
>
> The problem with merging these is that the "Specifically, --all can now
> be used with" is different for all three of them.

You said you were worried about the length of the release notes, because it
discourages users from actually reading the release notes, due to getting
bored. Having three instance of almost the same entry, with just minor changes
between them, seems to precisely endanger boring readers.

I'd probably just go for

- Add --all option to clusterdb, reindexdb, vacuumdb to process objects in all
  databases matching a pattern (Nathan Bossart)

or such. The precise details of how the option works for the different
commands doesn't need to be stated in the release notes, that's more of a
reference documentation thing. But if you want to include it, we can do
something like

  Specifically, --all can now be used with --table (all commands), --schema
  (reindexdb, vacuumdb), and --exclude-schema (reindexdb, vacuumdb).


> > - an entry about adding long options to pg_archivecleanup
>
> Well, that is a user-visible change.  Should it not be listed?

If you are concerned about the length of the release notes and as a
consequence not including more impactful performance changes, then no, it
shouldn't. It doesn't break anyones current scripts, it doesn't enable
anything new.


> > - two entries about grantable maintenance rights, once via pg_maintain, once
> >   per-table
>
> Well, one is a GRANT and another is a role, so merging them seemed like
> it would be too confusing.

I don't think it has to be.

Maybe something roughly like

- Allow granting the right to perform maintenance operations (Nathan Bossart)

  The permission can be granted on a per-table basis using the MAINTAIN
  privilege and on a system wide basis via the pg_maintain role.

  Operations that can be controlled are VACUUM, ANALYZE, REINDEX, REFRESH
  MATERIALIZED VIEW, CLUSTER, and LOCK TABLE.


I'm again mostly reacting to your concern that the release notes are getting
too boring to read. Repeated content, like in the current formulation, imo
does endanger that. Current it is:

- Add per-table GRANT permission MAINTAIN to control maintenance operations (Nathan Bossart)

  The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZED VIEW, CLUSTER, and LOCK TABLE.

- Add user-grantable role pg_maintain to control maintenance operations (Nathan Bossart)

  The operations are VACUUM, ANALYZE, REINDEX, REFRESH MATERIALIZED VIEW, CLUSTER, and LOCK TABLE.



> > - separate entries about pg_stat_reset_slru(), pg_stat_reset_shared("slru"),
>
> They are different functions with different detail text.

So what? You can change their text. Making it three entries makes it harder
for a reader that doesn't care about resetting stats to skip over the details.

Make it something like

- Improve control over resetting statistics (Atsushi Torikoshi, Bharath
  Rupireddy)

  pg_stat_reset_shared() can now reset all shared statistics, by passing NULL;
  pg_stat_reset_shared(NULL) also resets SLRU statistics;
  pg_stat_reset_shared("slru") resets SLRU statistics, which was already
  possible using pg_stat_reset_slru(NULL).

Greetings,

Andres Freund

  141. Re: First draft of PG 17 release notes

    Peter Geoghegan <pg@bowt.ie> — 2024-05-24T18:26:48Z

    On Fri, May 24, 2024 at 1:50 PM Andres Freund <andres@anarazel.de> wrote:
> Bruce, just about everyone seems to disagree with your current approach. And
> not just this year, this has been a discussion in most if not all release note
> threads of the last few years.

+1.

> People, including me, *have* addressed your criteria, but you just waved those
> concerns away. It's hard to continue discussing criteria when it doesn't at
> all feel like a conversation.

At one point on this thread, Bruce said "I am particularly critical if
I start to wonder, "Why does the author _think_ I should care about
this?" because it feels like the author is writing for him/herself and
not the audience."

Whenever this sort of thing has come up in the past, and I pushed
back, Bruce seemed to respond along these lines: he seemed to suggest
that there was some kind of conflict of interests involved. This isn't
completely unreasonable, of course -- my motivations aren't wholly
irrelevant. But for the most part they're *not* very relevant, and
wouldn't be even if Bruce's worst suspicions were actually true. In
principle it shouldn't matter that I'm biased, if I happen to be
correct in some relevant sense.

Everybody has some kind of bias. Even if my bias in these matters was
a significant factor (which I tend to doubt), I don't think that it's
fair to suggest that it's self-serving or careerist. My bias was
probably present before I even began work on the feature in question.
I tend to work on things because I believe that they're important --
it's not the other way around (at least not to a significant degree).

> In the end, these are patches to the source code, I don't think you can just
> wave away widespread disagreement with your changes. That's not how we do
> postgres development.

In lots of cases (a large minority of cases) the problem isn't even
really with the emphasis of one type of item over another/the
inclusion or non-inclusion of some individual item. It's actually a
problem with the information being presented in the most useful way.

Often I've suggested what I believe to be a better wording on the
merits (usually less obscure and more accessible language), only to be
met with the same sort of resistance from Bruce. If I've put a huge
amount of work into the item (as is usually the case), then I think
that I am at least entitled to a fair hearing.

I don't expect Bruce to meet me halfway, or even for him to meet me a
quarter of the way -- somebody has to be empowered to say no here
(even to very senior community members). I just don't think that he
has seriously considered my feedback in this area over the years. Not
always, not consistently, but often enough for it to seem like a real
problem.

--
Peter Geoghegan

  142. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T02:10:36Z

    On Fri, May 24, 2024 at 10:50:28AM -0700, Andres Freund wrote:
> Hi,
> 
> On 2024-05-23 23:27:04 -0400, Bruce Momjian wrote:
> > On Thu, May 23, 2024 at 11:11:10PM -0400, Tom Lane wrote:
> > > Bruce Momjian <bruce@momjian.us> writes:
> > > I am not sure Bruce that you realize that your disregard for
> > > performance improvements is shared by nobody.  Arguably,
> > > performance is 90% of what we do these days, and it's also
> > > 90% of what users care about.
> >
> > Please stop saying I don't document performance.  I have already
> > explained enough which performance items I choose.  Please address my
> > criteria or suggest new criteria.
> 
> Bruce, just about everyone seems to disagree with your current approach. And
> not just this year, this has been a discussion in most if not all release note
> threads of the last few years.
> 
> People, including me, *have* addressed your criteria, but you just waved those
> concerns away. It's hard to continue discussing criteria when it doesn't at
> all feel like a conversation.
> 
> In the end, these are patches to the source code, I don't think you can just
> wave away widespread disagreement with your changes. That's not how we do
> postgres development.

Well, let's start with a new section for PG 17 that lists these.  Is it
20 items, 50, or 150?  I have no idea, but without the user-visible
filter, I am unable to determine what not-included performance features
are worthy of the release notes.

Can someone do that?  There is no reason other committers can't change
the release notes.  Yes, I realize we are looking for a consistent
voice, but the new section can probably have its own style, and I can
make adjustments if desired.

Also, I think this has gone unaddressed so long because if we skip a
user-visible change, users complain, but I don't remember anyone
complaining about skipped performance changes.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  143. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T03:41:48Z

    On Fri, May 24, 2024 at 11:23:29AM -0700, Andres Freund wrote:
> Hi,
> 
> On 2024-05-22 18:33:03 -0400, Bruce Momjian wrote:
> > On Tue, May 21, 2024 at 09:40:28AM -0700, Andres Freund wrote:
> > > On 2024-05-18 11:13:54 -0400, Bruce Momjian wrote:
> > > I agree keeping things reasonably short is important. But I don't think you're
> > > evenly applying it as a goal.
> > >
> > > Just skimming the notes from the end, I see
> > > - an 8 entries long pg_stat_statements section
> >
> > What item did you want to remove?  Those are all user-visible changes.
> 
> My point here was not that we necessarily need to remove those, but that their
> impact to users is smaller than many of the performance impacts you disregard.

I liked all your detailed suggestions so applied the attached patch. 

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  144. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T03:49:03Z

    On Thu, May 23, 2024 at 01:22:51PM +0200, Álvaro Herrera wrote:
> Hello,
> 
> Regarding this item
> 
> : Allow the SLRU cache sizes to be configured (Andrey Borodin, Dilip Kumar)
> : 
> : The new server variables are commit_timestamp_buffers,
> : multixact_member_buffers, multixact_offset_buffers, notify_buffers,
> : serializable_buffers, subtransaction_buffers, and transaction_buffers.
> 
> I hereby request to be listed as third author of this feature.
> 
> Also, I'd like to suggest to make it more verbose, as details might be
> useful to users.  Mention that scalability is improved, because
> previously we've suggested to recompile with larger #defines, but to be
> cautious because values too high degrade performance.  Also mention the
> point that some of these grow with shared_buffers is user-visible enough
> that it warrants an explicit mention.  How about like this:
> 
> : Allow the SLRU cache sizes to be configured and improve performance of
> : larger caches
> : (Andrey Borodin, Dilip Kumar, Álvaro Herrera)
> : 
> : The new server variables are commit_timestamp_buffers,
> : multixact_member_buffers, multixact_offset_buffers, notify_buffers,
> : serializable_buffers, subtransaction_buffers, and transaction_buffers.
> : commit_timestamp_buffers, transaction_buffers and
> : subtransaction_buffers scale up automatically with shared_buffers.

Yes, I like that, patch applied.

> These three items
> 
> : Allow pg_stat_reset_shared() to reset all shared statistics (Atsushi Torikoshi)
> : 
> : This is done by passing NULL.
> : 
> : Allow pg_stat_reset_shared('slru') to clear SLRU statistics (Atsushi Torikoshi)
> : 
> : Now pg_stat_reset_shared(NULL) also resets SLRU statistics.
> : 
> : Allow pg_stat_reset_slru() to reset all SLRU statistics (Bharath Rupireddy)
> : 
> : The command pg_stat_reset_slru(NULL) already did this.
> 
> seem a bit repetitive.  (I think the first one is also wrong, because it
> says you have to pass NULL, but in reality you can also not give an
> argument and it works.)  Can we make them a single item?  Maybe
> something like
> 
> : Improve reset routines for shared statistics (Atsushi Torikoshi, Bharath Rupireddy)
> :
> : Resetting all shared statistics can now be done with
> : pg_stat_reset_shared() or pg_stat_reset_shared(NULL), while SLRU
> : statistics can now be reset with pg_stat_reset_shared('slru'),
> : pg_stat_reset_slru() and pg_stat_reset_slru(NULL).

Andres already suggested improvement for this, and I posted the applied
patch.  Can you see if that is good or can be improved?  Thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  145. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T03:50:34Z

    On Thu, May 23, 2024 at 04:54:28PM -0300, Marcos Pegoraro wrote:
>   • Rename SLRU columns in system view pg_stat_slru (Alvaro Herrera)
> 
>     The column names accepted by pg_stat_slru_rest() are also changed.
> 
> Is pg_stat_slru_rest() correct ? 

Oops, typo, fixed, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  146. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T03:57:01Z

    On Thu, May 23, 2024 at 08:19:15PM -0400, Peter Geoghegan wrote:
> On Wed, May 22, 2024 at 6:50 PM Bruce Momjian <bruce@momjian.us> wrote:
> > Agreed, patch applied, thanks.
> 
> The item for my commit 5bf748b8 currently reads:
> 
> "Allow btree indexes to more efficiently find array matches"
> 
> I think that this isn't likely to mean much to most users. It seems
> like it'd be a lot clearer if the wording was more in line with the
> beta1 announcement, which talks about the work as an enhancement to
> index scans that use an IN ( ) condition. Specifically referencing
> IN() (as opposed to something about arrays or array conditions) is
> likely to make the item much more understandable.
> 
> Referencing array matches makes me think of a GIN index on an array
> column. While ScalarArrayOps do use an array under the cover, that's
> mostly an implementation detail. At least it is to users that
> exclusively use IN(), likely the majority (that's the SQL standard
> syntax).
> 
> For context, the Postgres 9.2 release notes described the feature that
> my work directly built on as follows:
> 
> "Allow indexed_col op ANY(ARRAY[...]) conditions to be used in plain
> index scans and index-only scans"
> 
> This was a very accurate description of this earlier work. Similar
> wording could be used now, but that doesn't seem great to me either.
> Simply because this wording also doesn't reference IN() conditions in
> index quals.

Agreed.  I changed it to:

	Allow btree indexes to more efficiently find a set of values, such as
	those supplied by IN subqueries

Is that good?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  147. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-05-26T08:10:00Z

    On 2024-May-25, Bruce Momjian wrote:

> On Thu, May 23, 2024 at 01:22:51PM +0200, Álvaro Herrera wrote:

> > Can we make them a single item?  Maybe something like
> > 
> > : Improve reset routines for shared statistics (Atsushi Torikoshi, Bharath Rupireddy)
> > :
> > : Resetting all shared statistics can now be done with
> > : pg_stat_reset_shared() or pg_stat_reset_shared(NULL), while SLRU
> > : statistics can now be reset with pg_stat_reset_shared('slru'),
> > : pg_stat_reset_slru() and pg_stat_reset_slru(NULL).
> 
> Andres already suggested improvement for this, and I posted the applied
> patch.  Can you see if that is good or can be improved?  Thanks.

Yeah, looks good, thanks.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"No es bueno caminar con un hombre muerto"

  148. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-26T12:42:25Z

    On Sun, May 26, 2024 at 10:10:00AM +0200, Álvaro Herrera wrote:
> On 2024-May-25, Bruce Momjian wrote:
> 
> > On Thu, May 23, 2024 at 01:22:51PM +0200, Álvaro Herrera wrote:
> 
> > > Can we make them a single item?  Maybe something like
> > > 
> > > : Improve reset routines for shared statistics (Atsushi Torikoshi, Bharath Rupireddy)
> > > :
> > > : Resetting all shared statistics can now be done with
> > > : pg_stat_reset_shared() or pg_stat_reset_shared(NULL), while SLRU
> > > : statistics can now be reset with pg_stat_reset_shared('slru'),
> > > : pg_stat_reset_slru() and pg_stat_reset_slru(NULL).
> > 
> > Andres already suggested improvement for this, and I posted the applied
> > patch.  Can you see if that is good or can be improved?  Thanks.
> 
> Yeah, looks good, thanks.

Wow, that's great.  My head started to spin trying to make sense of how
those three entries connected.  Glad we were able to condense them, and
the new result is easier to read.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  149. Re: First draft of PG 17 release notes

    David Rowley <dgrowleyml@gmail.com> — 2024-05-28T02:44:28Z

    On Sun, 26 May 2024 at 15:57, Bruce Momjian <bruce@momjian.us> wrote:
> Agreed.  I changed it to:
>
>         Allow btree indexes to more efficiently find a set of values, such as
>         those supplied by IN subqueries
>
> Is that good?

I think this needs further adjustment.  An "IN subquery" is an IN
clause which contains a subquery.  As far as I understand it,
5bf748b86 does nothing to improve those. It's there to improve IN with
a set of values such as IN(1,2,3).

Maybe "IN subqueries" can be replaced with "an SQL IN clause".

David

  150. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-05-28T04:20:21Z

    On Tue, May 28, 2024 at 02:44:28PM +1200, David Rowley wrote:
> On Sun, 26 May 2024 at 15:57, Bruce Momjian <bruce@momjian.us> wrote:
> > Agreed.  I changed it to:
> >
> >         Allow btree indexes to more efficiently find a set of values, such as
> >         those supplied by IN subqueries
> >
> > Is that good?
> 
> I think this needs further adjustment.  An "IN subquery" is an IN
> clause which contains a subquery.  As far as I understand it,
> 5bf748b86 does nothing to improve those. It's there to improve IN with
> a set of values such as IN(1,2,3).
> 
> Maybe "IN subqueries" can be replaced with "an SQL IN clause".

Okay, I went with:

	Allow btree indexes to more efficiently find a set of values,
	such as those supplied by IN clauses using constants (Peter Geoghegan,
	Matthias van de Meent)


-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  151. Re: First draft of PG 17 release notes

    Dean Rasheed <dean.a.rasheed@gmail.com> — 2024-06-05T22:46:17Z

    On Thu, 9 May 2024 at 05:03, Bruce Momjian <bruce@momjian.us> wrote:
>
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
>
>         https://momjian.us/pgsql_docs/release-17.html

I noticed a couple more things. This item:

  Add functions to convert integers to hex and binary strings

should read:

  Add functions to convert integers to binary and octal strings


The "Improve psql tab completion" item should include this commit:

Author: Michael Paquier <michael@paquier.xyz>
2024-05-01 [2800fbb2b] Add tab completion for EXPLAIN (MEMORY|SERIALIZE)

and credit Jian He.

Regards,
Dean

  152. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-06-06T00:53:09Z

    On Wed, Jun  5, 2024 at 11:46:17PM +0100, Dean Rasheed wrote:
> On Thu, 9 May 2024 at 05:03, Bruce Momjian <bruce@momjian.us> wrote:
> >
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> >
> >         https://momjian.us/pgsql_docs/release-17.html
> 
> I noticed a couple more things. This item:
> 
>   Add functions to convert integers to hex and binary strings
> 
> should read:
> 
>   Add functions to convert integers to binary and octal strings
> 
> 
> The "Improve psql tab completion" item should include this commit:
> 
> Author: Michael Paquier <michael@paquier.xyz>
> 2024-05-01 [2800fbb2b] Add tab completion for EXPLAIN (MEMORY|SERIALIZE)
> 
> and credit Jian He.

Agreed, attached patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  153. Re: First draft of PG 17 release notes

    Matthias van de Meent <boekewurm+postgres@gmail.com> — 2024-07-05T17:51:38Z

    Hi,

I noticed that PG17's release note for commit cafe10565 is "Allow psql
connections to be canceled with control-C (Tristan Partin)", but this
summary seems wrong to me.

We already had ^C connection (query) cancellation for quite some time
before this patch. What's new with that patch, is that we now also can
cancel connection attempts with ^C while we're still connecting (i.e.,
we haven't yet authenticated and are trying to move the connection
state forward).
I think a better wording would be "Allow psql connection attempts to
be canceled with control-C (Tristan Partin)", or "Allow psql
connections to be canceled with control-C while psql is still
connecting (Tristan Partin)".

Kind regards,

Matthias van de Meent
Neon (https://neon.tech)

  154. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-07-05T20:53:02Z

    On Fri, Jul  5, 2024 at 07:51:38PM +0200, Matthias van de Meent wrote:
> Hi,
> 
> I noticed that PG17's release note for commit cafe10565 is "Allow psql
> connections to be canceled with control-C (Tristan Partin)", but this
> summary seems wrong to me.
> 
> We already had ^C connection (query) cancellation for quite some time
> before this patch. What's new with that patch, is that we now also can
> cancel connection attempts with ^C while we're still connecting (i.e.,
> we haven't yet authenticated and are trying to move the connection
> state forward).
> I think a better wording would be "Allow psql connection attempts to
> be canceled with control-C (Tristan Partin)", or "Allow psql
> connections to be canceled with control-C while psql is still
> connecting (Tristan Partin)".

I see your point.  I committed a change to use this wording:

      Allow psql connection attempts to be canceled with control-C
      (Tristan Partin)

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  155. Re: First draft of PG 17 release notes

    Kisoon Kwon <moxie2ks@gmail.com> — 2024-07-17T06:32:45Z

    Hi,

In the PG17 release notes, I noticed it is mentioned as
"pg_attribute.stxstattarget" which seems incorrect.
In my opinion, it should be "pg_statistic_ext.stxstattarget" because the
"stxstattarget" column is part of the "pg_statistic_ext" catalog.

Regards,
Kisoon Kwon
Bitnine Global (www.bitnine.net)

2024년 7월 6일 (토) 오전 5:53, Bruce Momjian <bruce@momjian.us>님이 작성:

> On Fri, Jul  5, 2024 at 07:51:38PM +0200, Matthias van de Meent wrote:
> > Hi,
> >
> > I noticed that PG17's release note for commit cafe10565 is "Allow psql
> > connections to be canceled with control-C (Tristan Partin)", but this
> > summary seems wrong to me.
> >
> > We already had ^C connection (query) cancellation for quite some time
> > before this patch. What's new with that patch, is that we now also can
> > cancel connection attempts with ^C while we're still connecting (i.e.,
> > we haven't yet authenticated and are trying to move the connection
> > state forward).
> > I think a better wording would be "Allow psql connection attempts to
> > be canceled with control-C (Tristan Partin)", or "Allow psql
> > connections to be canceled with control-C while psql is still
> > connecting (Tristan Partin)".
>
> I see your point.  I committed a change to use this wording:
>
>       Allow psql connection attempts to be canceled with control-C
>       (Tristan Partin)
>
> --
>   Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>   EDB                                      https://enterprisedb.com
>
>   Only you can decide what is important to you.
>
>
>

  156. Re: First draft of PG 17 release notes

    Yugo Nagata <nagata@sraoss.co.jp> — 2024-07-26T04:22:24Z

    Hi,

On Thu, 9 May 2024 00:03:50 -0400
Bruce Momjian <bruce@momjian.us> wrote:

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
> 	https://momjian.us/pgsql_docs/release-17.html
> 
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
> 
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
> 
> I welcome feedback.  For some reason it was an easier job than usual.

I found the following in the release notes:

 Change file boundary handling of two WAL file name functions 
 (Kyotaro Horiguchi, Andres Freund, Bruce Momjian)

 The functions pg_walfile_name() and pg_walfile_name_offset() used to report the previous 
 LSN segment number when  the LSN was on a file segment boundary; it now returns the LSN segment. 

It might be trivial, but, reading the associated commit message , I think it would be more explicit
for users to rewrite the last statement to

"it now returns the current LSN segment."

Regards,
Yugo Nagata



> 
> -- 
>   Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>   EDB                                      https://enterprisedb.com
> 
>   Only you can decide what is important to you.
> 
> 


-- 
Yugo Nagata <nagata@sraoss.co.jp>

  157. Re: First draft of PG 17 release notes

    Justin Pryzby <pryzby@telsasoft.com> — 2024-08-08T13:55:53Z

    > Add server variable huge_page_size to report the use of huge pages by

The new variable is huge_page_status; h_p_size is several years old.

BTW, I was surprised that these were included:

+2024-02-28 [363eb0599] Convert README to Markdown.
+2024-01-25 [7014c9a4b] Doc: improve documentation for jsonpath behavior.

-- 
Justin

  158. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-08-16T16:53:33Z

    On Wed, Jul 17, 2024 at 03:32:45PM +0900, Kisoon Kwon wrote:
> Hi,
> 
> In the PG17 release notes, I noticed it is mentioned as
> "pg_attribute.stxstattarget" which seems incorrect.
> In my opinion, it should be "pg_statistic_ext.stxstattarget" because the
> "stxstattarget" column is part of the "pg_statistic_ext" catalog.

You are right, fixed in the attached patch.  Sorry for the delay.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  159. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-08-16T17:02:19Z

    On Fri, Jul 26, 2024 at 01:22:24PM +0900, Yugo Nagata wrote:
> I found the following in the release notes:
> 
>  Change file boundary handling of two WAL file name functions 
>  (Kyotaro Horiguchi, Andres Freund, Bruce Momjian)
> 
>  The functions pg_walfile_name() and pg_walfile_name_offset() used to report the previous 
>  LSN segment number when  the LSN was on a file segment boundary; it now returns the LSN segment. 
> 
> It might be trivial, but, reading the associated commit message , I think it would be more explicit
> for users to rewrite the last statement to
> 
> "it now returns the current LSN segment."

Agreed, applied patch attached.  Sorry for the delay.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  160. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-08-16T17:20:28Z

    On Thu, Aug  8, 2024 at 08:55:53AM -0500, Justin Pryzby wrote:
> > Add server variable huge_page_size to report the use of huge pages by
> 
> The new variable is huge_page_status; h_p_size is several years old.

Fixed.  I created this mistake when I was adding links to the SGML file.

> BTW, I was surprised that these were included:
> 
> +2024-02-28 [363eb0599] Convert README to Markdown.
> +2024-01-25 [7014c9a4b] Doc: improve documentation for jsonpath behavior.

I try to mention significant doc changes, and have done so in the past.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  161. Re: First draft of PG 17 release notes

    Nathan Bossart <nathandbossart@gmail.com> — 2024-09-03T15:44:01Z

    While freely acknowledging that I am biased because I wrote it, I am a bit
surprised to see the DSM registry left out of the release notes (commit
8b2bcf3, docs are here [0]).  This feature is intended to allow modules to
allocate shared memory after startup, i.e., without requiring the module to
be loaded via shared_preload_libraries.  IMHO that is worth mentioning.

[0] https://www.postgresql.org/docs/devel/xfunc-c.html#XFUNC-SHARED-ADDIN-AFTER-STARTUP

-- 
nathan

  162. Re: First draft of PG 17 release notes

    jian he <jian.universality@gmail.com> — 2024-09-04T11:18:52Z

    hi.

Allow partitions to be merged using ALTER TABLE ... MERGE PARTITIONS
(Dmitry Koval)
Allow partitions to be split using ALTER TABLE ... SPLIT PARTITION
(Dmitry Koval)

also these two items got reverted? see
https://git.postgresql.org/cgit/postgresql.git/commit/?id=3890d90c1508125729ed20038d90513694fc3a7b

  163. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-06T01:49:48Z

    On Wed, Sep  4, 2024 at 07:18:52PM +0800, jian he wrote:
> hi.
> 
> Allow partitions to be merged using ALTER TABLE ... MERGE PARTITIONS
> (Dmitry Koval)
> Allow partitions to be split using ALTER TABLE ... SPLIT PARTITION
> (Dmitry Koval)
> 
> also these two items got reverted? see
> https://git.postgresql.org/cgit/postgresql.git/commit/?id=3890d90c1508125729ed20038d90513694fc3a7b

I don't see them in the PG 17 release notes at:

	https://www.postgresql.org/docs/17/release-17.html

I did just remove the tab complete comment for this though.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  164. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-06T01:51:25Z

    On Tue, Sep  3, 2024 at 10:44:01AM -0500, Nathan Bossart wrote:
> While freely acknowledging that I am biased because I wrote it, I am a bit
> surprised to see the DSM registry left out of the release notes (commit
> 8b2bcf3, docs are here [0]).  This feature is intended to allow modules to
> allocate shared memory after startup, i.e., without requiring the module to
> be loaded via shared_preload_libraries.  IMHO that is worth mentioning.
> 
> [0] https://www.postgresql.org/docs/devel/xfunc-c.html#XFUNC-SHARED-ADDIN-AFTER-STARTUP

That seems more infrastructure/extension author stuff which isn't
normally mentioned in the release notes.  I think such people really
need to look at all the commit messages.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  165. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-09-07T09:55:09Z

    On 2024-Sep-05, Bruce Momjian wrote:

> That seems more infrastructure/extension author stuff which isn't
> normally mentioned in the release notes.  I think such people really
> need to look at all the commit messages.

Are you saying all extension authors should be reading the complete git
log for every single major release?  That's a strange position to take.
Isn't this a good fit for "E.1.3.10. Source Code"?

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
<Schwern> It does it in a really, really complicated way
<crab> why does it need to be complicated?
<Schwern> Because it's MakeMaker.

  166. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-10T02:46:50Z

    On Sat, Sep  7, 2024 at 11:55:09AM +0200, Álvaro Herrera wrote:
> On 2024-Sep-05, Bruce Momjian wrote:
> 
> > That seems more infrastructure/extension author stuff which isn't
> > normally mentioned in the release notes.  I think such people really
> > need to look at all the commit messages.
> 
> Are you saying all extension authors should be reading the complete git
> log for every single major release?  That's a strange position to take.
> Isn't this a good fit for "E.1.3.10. Source Code"?

Yes.  There are so many changes at the source code level it is unwise to
try and get them into the main release notes.  If someone wants to
create an addendum, like was suggested for pure performance
improvements, that would make sense.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

  167. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-09-10T06:28:42Z

    On Tue, 10 Sept 2024 at 04:47, Bruce Momjian <bruce@momjian.us> wrote:
> Yes.  There are so many changes at the source code level it is unwise to
> try and get them into the main release notes.  If someone wants to
> create an addendum, like was suggested for pure performance
> improvements, that would make sense.

I agree that the release notes cannot fit every change. But I also
don't think any extension author reads the complete git commit log
every release, so taking the stance that they should be seems
unhelpful. And the "Source Code" section does exist so at some level
you seem to disagree with that too. So what is the way to decide that
something makes the cut for the "Source Code" section?

I think as an extension author there are usually three types of
changes that are relevant:
1. New APIs/hooks that are meant for extension authors
2. Stuff that causes my existing code to not compile anymore
3. Stuff that changes behaviour of existing APIs code in a
incompatible but silent way

For 1, I think adding them to the release notes makes total sense,
especially if the new APIs are documented not only in source code, but
also on the website. Nathan his change is of this type, so I agree
with him it should be in the release notes.

For 2, I'll be able to easily find the PG commit that caused the
compilation failure by grepping git history for the old API. So having
these changes in the release notes seems unnecessary.

For 3, it would be very useful if it would be in the release notes,
but I think in many cases it's hard to know what commits do this. So
unless it's obviously going to break a bunch of extensions silently, I
think we don't have to add such changes to the release notes.

  168. Re: First draft of PG 17 release notes

    Michael Banck <mbanck@gmx.net> — 2024-09-10T07:11:19Z

    On Thu, Sep 05, 2024 at 09:51:25PM -0400, Bruce Momjian wrote:
> On Tue, Sep  3, 2024 at 10:44:01AM -0500, Nathan Bossart wrote:
> > While freely acknowledging that I am biased because I wrote it, I am a bit
> > surprised to see the DSM registry left out of the release notes (commit
> > 8b2bcf3, docs are here [0]).  This feature is intended to allow modules to
> > allocate shared memory after startup, i.e., without requiring the module to
> > be loaded via shared_preload_libraries.  IMHO that is worth mentioning.
> > 
> > [0] https://www.postgresql.org/docs/devel/xfunc-c.html#XFUNC-SHARED-ADDIN-AFTER-STARTUP
> 
> That seems more infrastructure/extension author stuff which isn't
> normally mentioned in the release notes.

If I understand the feature correctly, it allows extensions to be just
CREATEd without having them to be added to shared_preload_libraries,
i.e. saving the organization an instance restart/downtime.

That seems important enough for end-users to know, even if they will
need to wait for extension authors to catch up to this (but I guess a
lot will).


Michael

  169. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-09-10T08:51:39Z

    On 2024-Sep-10, Jelte Fennema-Nio wrote:

> I think as an extension author there are usually three types of
> changes that are relevant:
>
> 1. New APIs/hooks that are meant for extension authors

> For 1, I think adding them to the release notes makes total sense,
> especially if the new APIs are documented not only in source code, but
> also on the website. Nathan his change is of this type, so I agree
> with him it should be in the release notes.

I agree.  The volume of such items should be pretty small.

> 3. Stuff that changes behaviour of existing APIs code in a
> incompatible but silent way

> For 3, it would be very useful if it would be in the release notes,
> but I think in many cases it's hard to know what commits do this. So
> unless it's obviously going to break a bunch of extensions silently, I
> think we don't have to add such changes to the release notes.

While we cannot be 100% vigilant (and it doesn't seem likely for
automated tools to detect this), we try to avoid API changes that would
still compile but behave incompatibly.  In many review discussions you
can see suggestions to change some function signature so that
third-party authors would be aware that they need to adapt their code to
new behavior, turning cases of (3) into (2).  I agree that these don't
need release notes items.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"XML!" Exclaimed C++.  "What are you doing here? You're not a programming
language."
"Tell that to the people who use me," said XML.
https://burningbird.net/the-parable-of-the-languages/

  170. Re: First draft of PG 17 release notes

    Masahiko Sawada <sawada.mshk@gmail.com> — 2024-09-10T16:52:42Z

    On Mon, Sep 9, 2024 at 11:29 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>
> On Tue, 10 Sept 2024 at 04:47, Bruce Momjian <bruce@momjian.us> wrote:
> > Yes.  There are so many changes at the source code level it is unwise to
> > try and get them into the main release notes.  If someone wants to
> > create an addendum, like was suggested for pure performance
> > improvements, that would make sense.
>
> I agree that the release notes cannot fit every change. But I also
> don't think any extension author reads the complete git commit log
> every release, so taking the stance that they should be seems
> unhelpful. And the "Source Code" section does exist so at some level
> you seem to disagree with that too. So what is the way to decide that
> something makes the cut for the "Source Code" section?
>
> I think as an extension author there are usually three types of
> changes that are relevant:
> 1. New APIs/hooks that are meant for extension authors
> 2. Stuff that causes my existing code to not compile anymore
> 3. Stuff that changes behaviour of existing APIs code in a
> incompatible but silent way
>
> For 1, I think adding them to the release notes makes total sense,
> especially if the new APIs are documented not only in source code, but
> also on the website. Nathan his change is of this type, so I agree
> with him it should be in the release notes.

+1. I think that the increment JSON parser that is already mentioned
in the release note would fall in this type too; it's not a feature
aimed just for extension authors, but it's kind of source and internal
changes IMO. Since the DSM registry feature is described in the doc, I
think it would make sense to have it in the release notes and probably
has a link to the "Requesting Shared Memory After Startup" section.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

  171. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-11T14:10:26Z

    On Tue, Sep 10, 2024 at 08:28:42AM +0200, Jelte Fennema-Nio wrote:
> I think as an extension author there are usually three types of
> changes that are relevant:
> 1. New APIs/hooks that are meant for extension authors
> 2. Stuff that causes my existing code to not compile anymore
> 3. Stuff that changes behaviour of existing APIs code in a
> incompatible but silent way
> 
> For 1, I think adding them to the release notes makes total sense,
> especially if the new APIs are documented not only in source code, but
> also on the website. Nathan his change is of this type, so I agree
> with him it should be in the release notes.
> 
> For 2, I'll be able to easily find the PG commit that caused the
> compilation failure by grepping git history for the old API. So having
> these changes in the release notes seems unnecessary.
> 
> For 3, it would be very useful if it would be in the release notes,
> but I think in many cases it's hard to know what commits do this. So
> unless it's obviously going to break a bunch of extensions silently, I
> think we don't have to add such changes to the release notes.

So, we are looking at this commit:

    commit b5a9b18cd0b
    Author: Thomas Munro <tmunro@postgresql.org>
    Date:   Wed Apr 3 00:17:06 2024 +1300

    Provide API for streaming relation data.

    Introduce an abstraction allowing relation data to be accessed as a
    stream of buffers, with an implementation that is more efficient than
    the equivalent sequence of ReadBuffer() calls.

    Client code supplies a callback that can say which block number it wants
    next, and then consumes individual buffers one at a time from the
    stream.  This division puts read_stream.c in control of how far ahead it
    can see and allows it to read clusters of neighboring blocks with
    StartReadBuffers().  It also issues POSIX_FADV_WILLNEED advice ahead of
    time when random access is detected.

    Other variants of I/O stream will be proposed in future work (for
    example to support recovery, whose LsnReadQueue device in
    xlogprefetcher.c is a distant cousin of this code and should eventually
    be replaced by this), but this basic API is sufficient for many common
    executor usage patterns involving predictable access to a single fork of
    a single relation.

    Several patches using this API are proposed separately.

    This stream concept is loosely based on ideas from Andres Freund on how
    we should pave the way for later work on asynchronous I/O.

You are right that I do mention changes specifically designed for the
use of extensions, but there is no mention in the commit message of its
use for extensions.  In fact, I thought this was too low-level to be of
use for extensions.  However, if people feel it should be added, we have
enough time to add it.

I also mention changes that are _likely_ to affect extensions, but not
all changes that could affect extensions.  An interesting idea would be
to report all function signature changes in each major release in some
way.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  172. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-11T14:12:58Z

    On Tue, Sep 10, 2024 at 09:52:42AM -0700, Masahiko Sawada wrote:
> On Mon, Sep 9, 2024 at 11:29 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> >
> > On Tue, 10 Sept 2024 at 04:47, Bruce Momjian <bruce@momjian.us> wrote:
> > > Yes.  There are so many changes at the source code level it is unwise to
> > > try and get them into the main release notes.  If someone wants to
> > > create an addendum, like was suggested for pure performance
> > > improvements, that would make sense.
> >
> > I agree that the release notes cannot fit every change. But I also
> > don't think any extension author reads the complete git commit log
> > every release, so taking the stance that they should be seems
> > unhelpful. And the "Source Code" section does exist so at some level
> > you seem to disagree with that too. So what is the way to decide that
> > something makes the cut for the "Source Code" section?
> >
> > I think as an extension author there are usually three types of
> > changes that are relevant:
> > 1. New APIs/hooks that are meant for extension authors
> > 2. Stuff that causes my existing code to not compile anymore
> > 3. Stuff that changes behaviour of existing APIs code in a
> > incompatible but silent way
> >
> > For 1, I think adding them to the release notes makes total sense,
> > especially if the new APIs are documented not only in source code, but
> > also on the website. Nathan his change is of this type, so I agree
> > with him it should be in the release notes.
> 
> +1. I think that the increment JSON parser that is already mentioned
> in the release note would fall in this type too; it's not a feature
> aimed just for extension authors, but it's kind of source and internal
> changes IMO. Since the DSM registry feature is described in the doc, I
> think it would make sense to have it in the release notes and probably
> has a link to the "Requesting Shared Memory After Startup" section.

This commit?

	commit 8b2bcf3f287
	Author: Nathan Bossart <nathan@postgresql.org>
	Date:   Fri Jan 19 14:24:36 2024 -0600
	
	    Introduce the dynamic shared memory registry.

Yes, we have time to add it.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  173. Re: First draft of PG 17 release notes

    Nathan Bossart <nathandbossart@gmail.com> — 2024-09-11T14:36:35Z

    On Wed, Sep 11, 2024 at 10:12:58AM -0400, Bruce Momjian wrote:
> On Tue, Sep 10, 2024 at 09:52:42AM -0700, Masahiko Sawada wrote:
>> On Mon, Sep 9, 2024 at 11:29 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>> > For 1, I think adding them to the release notes makes total sense,
>> > especially if the new APIs are documented not only in source code, but
>> > also on the website. Nathan his change is of this type, so I agree
>> > with him it should be in the release notes.
>> 
>> +1. I think that the increment JSON parser that is already mentioned
>> in the release note would fall in this type too; it's not a feature
>> aimed just for extension authors, but it's kind of source and internal
>> changes IMO. Since the DSM registry feature is described in the doc, I
>> think it would make sense to have it in the release notes and probably
>> has a link to the "Requesting Shared Memory After Startup" section.
> 
> This commit?
> 
> 	commit 8b2bcf3f287
> 	Author: Nathan Bossart <nathan@postgresql.org>
> 	Date:   Fri Jan 19 14:24:36 2024 -0600
> 	
> 	    Introduce the dynamic shared memory registry.
> 
> Yes, we have time to add it.

Yes, that's the one.

-- 
nathan

  174. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-09-11T17:50:40Z

    Hello,

I noticed that these two items in the current notes are separate:

<!--
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-03-25 [374c7a229] Allow specifying an access method for partitioned tables
Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
2024-03-28 [e2395cdbe] ALTER TABLE: rework determination of access method ID
-->

     <listitem>
      <para>
      Allow specification of partitioned <link linkend="tableam">table
      access methods</link> (Justin Pryzby, Soumyadeep Chakraborty,
      Michael Paquier)
      </para>
     </listitem>

<!--
Author: Michael Paquier <michael@paquier.xyz>
2024-03-08 [d61a6cad6] Add support for DEFAULT in ALTER TABLE .. SET ACCESS MET
-->

     <listitem>
      <para>
      Add <literal>DEFAULT</literal> setting for <literal>ALTER TABLE
      .. SET ACCESS METHOD</literal> (Michael Paquier)
      </para>
     </listitem>

They are very very closely related, so I suggest they should be
together as a single item.  Also, the first one is somewhat strangely
worded IMO (we don't have "partitioned table access methods" -- rather,
we have table access methods for partitioned tables).  Maybe something
like

* Improve ALTER TABLE ... SET ACCESS METHOD

  This command can now also be applied to partitioned tables, so that it
  can <link to "https://www.postgresql.org/docs/17/sql-altertable.html#SQL-ALTERTABLE-DESC-SET-ACCESS-METHOD">
  influence partitions created later</link>. (Justin, Soumyadeep, Michaël)

  In addition, it now accepts the value DEFAULT to reset a previously
  set value. (Michaël)


There's also a bunch of items on EXPLAIN, which could perhaps be grouped
in a single item with sub-paras for each individual change; I'd also
move it to the bottom of E.1.3.2.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"This is a foot just waiting to be shot"                (Andrew Dunstan)

  175. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-13T20:17:31Z

    On Wed, Sep 11, 2024 at 09:36:35AM -0500, Nathan Bossart wrote:
> On Wed, Sep 11, 2024 at 10:12:58AM -0400, Bruce Momjian wrote:
> > On Tue, Sep 10, 2024 at 09:52:42AM -0700, Masahiko Sawada wrote:
> >> On Mon, Sep 9, 2024 at 11:29 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> >> > For 1, I think adding them to the release notes makes total sense,
> >> > especially if the new APIs are documented not only in source code, but
> >> > also on the website. Nathan his change is of this type, so I agree
> >> > with him it should be in the release notes.
> >> 
> >> +1. I think that the increment JSON parser that is already mentioned
> >> in the release note would fall in this type too; it's not a feature
> >> aimed just for extension authors, but it's kind of source and internal
> >> changes IMO. Since the DSM registry feature is described in the doc, I
> >> think it would make sense to have it in the release notes and probably
> >> has a link to the "Requesting Shared Memory After Startup" section.
> > 
> > This commit?
> > 
> > 	commit 8b2bcf3f287
> > 	Author: Nathan Bossart <nathan@postgresql.org>
> > 	Date:   Fri Jan 19 14:24:36 2024 -0600
> > 	
> > 	    Introduce the dynamic shared memory registry.
> > 
> > Yes, we have time to add it.
> 
> Yes, that's the one.

Attached patch applied, with commit URL link.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  176. Re: First draft of PG 17 release notes

    Nathan Bossart <nathandbossart@gmail.com> — 2024-09-13T21:00:07Z

    On Fri, Sep 13, 2024 at 04:17:31PM -0400, Bruce Momjian wrote:
> Attached patch applied, with commit URL link.

Looks good, thanks.

-- 
nathan

  177. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-13T22:25:41Z

    On Wed, Sep 11, 2024 at 07:50:40PM +0200, Álvaro Herrera wrote:
> Hello,
> 
> I noticed that these two items in the current notes are separate:
> 
> <!--
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2024-03-25 [374c7a229] Allow specifying an access method for partitioned tables
> Author: Alvaro Herrera <alvherre@alvh.no-ip.org>
> 2024-03-28 [e2395cdbe] ALTER TABLE: rework determination of access method ID
> -->
> 
>      <listitem>
>       <para>
>       Allow specification of partitioned <link linkend="tableam">table
>       access methods</link> (Justin Pryzby, Soumyadeep Chakraborty,
>       Michael Paquier)
>       </para>
>      </listitem>
> 
> <!--
> Author: Michael Paquier <michael@paquier.xyz>
> 2024-03-08 [d61a6cad6] Add support for DEFAULT in ALTER TABLE .. SET ACCESS MET
> -->
> 
>      <listitem>
>       <para>
>       Add <literal>DEFAULT</literal> setting for <literal>ALTER TABLE
>       .. SET ACCESS METHOD</literal> (Michael Paquier)
>       </para>
>      </listitem>
> 
> They are very very closely related, so I suggest they should be
> together as a single item.  Also, the first one is somewhat strangely
> worded IMO (we don't have "partitioned table access methods" -- rather,
> we have table access methods for partitioned tables).  Maybe something
> like

Yes, agree, the wording needs improvement, patch attached.

> * Improve ALTER TABLE ... SET ACCESS METHOD
> 
>   This command can now also be applied to partitioned tables, so that it
>   can <link to "https://www.postgresql.org/docs/17/sql-altertable.html#SQL-ALTERTABLE-DESC-SET-ACCESS-METHOD">
>   influence partitions created later</link>. (Justin, Soumyadeep, Michaël)
> 
>   In addition, it now accepts the value DEFAULT to reset a previously
>   set value. (Michaël)

I moved the two items next to each other, but I am concerned combining
the partition feature with the DEFAULT features is just making it too
complicated to understand.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  178. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-14T13:38:46Z

    On Wed, Sep 11, 2024 at 07:50:40PM +0200, Álvaro Herrera wrote:
> There's also a bunch of items on EXPLAIN, which could perhaps be grouped
> in a single item with sub-paras for each individual change; I'd also
> move it to the bottom of E.1.3.2.

Oh, I hadn't noticed I have five EXPLAIN items --- that is enough to
make a new section, done at:

	https://momjian.us/pgsql_docs/release-17.html#RELEASE-17-EXPLAIN

I don't think I can combine the EXPLAIN items without making them too
complex.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  179. Re: First draft of PG 17 release notes

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-09-17T08:01:28Z

    On Wed, 11 Sept 2024 at 16:10, Bruce Momjian <bruce@momjian.us> wrote:
> You are right that I do mention changes specifically designed for the
> use of extensions, but there is no mention in the commit message of its
> use for extensions.  In fact, I thought this was too low-level to be of
> use for extensions.  However, if people feel it should be added, we have
> enough time to add it.

Another new API that is useful for extension authors is the following
one (I'm obviously biased since I'm the author, and I don't know if
there's still time):

commit 14dd0f27d7cd56ffae9ecdbe324965073d01a9ff
Author: Nathan Bossart <nathan@postgresql.org>
Date:   Thu Jan 4 16:09:34 2024 -0600

    Add macros for looping through a List without a ListCell.

    Many foreach loops only use the ListCell pointer to retrieve the
    content of the cell, like so:

        ListCell   *lc;

        foreach(lc, mylist)
        {
            int         myint = lfirst_int(lc);

            ...
        }

    This commit adds a few convenience macros that automatically
    declare the loop variable and retrieve the current cell's contents.
    This allows us to rewrite the previous loop like this:

        foreach_int(myint, mylist)
        {
            ...
        }

> An interesting idea would be
> to report all function signature changes in each major release in some
> way.

I think that might be useful, but it very much depends how long that
list gets. If it gets too long I think authors will just try to
compile and only look at the ones that break for them.

  180. Re: First draft of PG 17 release notes

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-09-17T08:19:29Z

    On 2024-Sep-11, Bruce Momjian wrote:

> An interesting idea would be to report all function signature changes
> in each major release in some way.

Hmm, extension authors are going to realize this as soon as they try to
compile, so it doesn't seem necessary.  Having useful APIs _added_ is a
different matter, because those might help them realize that they can
remove parts (or #ifdef-out for newer PG versions) of their code, or add
new features; there's no Clippit saying "it looks like you're compiling
for Postgres 18, would you like to ...?".

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"The important things in the world are problems with society that we don't
understand at all. The machines will become more complicated but they won't
be more complicated than the societies that run them."    (Freeman Dyson)

  181. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-18T21:33:18Z

    On Tue, Sep 17, 2024 at 10:01:28AM +0200, Jelte Fennema-Nio wrote:
> On Wed, 11 Sept 2024 at 16:10, Bruce Momjian <bruce@momjian.us> wrote:
> > You are right that I do mention changes specifically designed for the
> > use of extensions, but there is no mention in the commit message of its
> > use for extensions.  In fact, I thought this was too low-level to be of
> > use for extensions.  However, if people feel it should be added, we have
> > enough time to add it.
> 
> Another new API that is useful for extension authors is the following
> one (I'm obviously biased since I'm the author, and I don't know if
> there's still time):
> 
> commit 14dd0f27d7cd56ffae9ecdbe324965073d01a9ff
> Author: Nathan Bossart <nathan@postgresql.org>
> Date:   Thu Jan 4 16:09:34 2024 -0600
> 
>     Add macros for looping through a List without a ListCell.
> 
>     Many foreach loops only use the ListCell pointer to retrieve the
>     content of the cell, like so:
> 
>         ListCell   *lc;
> 
>         foreach(lc, mylist)
>         {
>             int         myint = lfirst_int(lc);
> 
>             ...
>         }
> 
>     This commit adds a few convenience macros that automatically
>     declare the loop variable and retrieve the current cell's contents.
>     This allows us to rewrite the previous loop like this:
> 
>         foreach_int(myint, mylist)
>         {
>             ...
>         }

Can someone else comment on the idea of adding this release note item? 
I don't feel confident in my ability to evaluate this.  I obviously did
not see it as significant the first time.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  182. Re: First draft of PG 17 release notes

    Nathan Bossart <nathandbossart@gmail.com> — 2024-09-19T01:09:27Z

    On Wed, Sep 18, 2024 at 05:33:18PM -0400, Bruce Momjian wrote:
> On Tue, Sep 17, 2024 at 10:01:28AM +0200, Jelte Fennema-Nio wrote:
>> Another new API that is useful for extension authors is the following
>> one (I'm obviously biased since I'm the author, and I don't know if
>> there's still time):
>> 
>>     Add macros for looping through a List without a ListCell.
> 
> Can someone else comment on the idea of adding this release note item? 
> I don't feel confident in my ability to evaluate this.  I obviously did
> not see it as significant the first time.

I'm not sure precisely what criteria you use to choose what goes in the
release notes, but this one seems like a judgement call to me.  My initial
reaction is that it shouldn't be included, but I do see some items with a
similar scope, such as "Remove some SPI macros."

-- 
nathan

  183. Re: First draft of PG 17 release notes

    Robert Haas <robertmhaas@gmail.com> — 2024-09-19T16:23:21Z

    On Wed, Sep 18, 2024 at 9:09 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> On Wed, Sep 18, 2024 at 05:33:18PM -0400, Bruce Momjian wrote:
> > On Tue, Sep 17, 2024 at 10:01:28AM +0200, Jelte Fennema-Nio wrote:
> >> Another new API that is useful for extension authors is the following
> >> one (I'm obviously biased since I'm the author, and I don't know if
> >> there's still time):
> >>
> >>     Add macros for looping through a List without a ListCell.
> >
> > Can someone else comment on the idea of adding this release note item?
> > I don't feel confident in my ability to evaluate this.  I obviously did
> > not see it as significant the first time.
>
> I'm not sure precisely what criteria you use to choose what goes in the
> release notes, but this one seems like a judgement call to me.  My initial
> reaction is that it shouldn't be included, but I do see some items with a
> similar scope, such as "Remove some SPI macros."

I wouldn't mention either this or "Remove some unused SPI macros".

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  184. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-19T17:04:40Z

    On Thu, Sep 19, 2024 at 12:23:21PM -0400, Robert Haas wrote:
> On Wed, Sep 18, 2024 at 9:09 PM Nathan Bossart <nathandbossart@gmail.com> wrote:
> > On Wed, Sep 18, 2024 at 05:33:18PM -0400, Bruce Momjian wrote:
> > > On Tue, Sep 17, 2024 at 10:01:28AM +0200, Jelte Fennema-Nio wrote:
> > >> Another new API that is useful for extension authors is the following
> > >> one (I'm obviously biased since I'm the author, and I don't know if
> > >> there's still time):
> > >>
> > >>     Add macros for looping through a List without a ListCell.
> > >
> > > Can someone else comment on the idea of adding this release note item?
> > > I don't feel confident in my ability to evaluate this.  I obviously did
> > > not see it as significant the first time.
> >
> > I'm not sure precisely what criteria you use to choose what goes in the
> > release notes, but this one seems like a judgement call to me.  My initial
> > reaction is that it shouldn't be included, but I do see some items with a
> > similar scope, such as "Remove some SPI macros."
> 
> I wouldn't mention either this or "Remove some unused SPI macros".

I mentioned the SPI macros because that could lead to breakage, and
there might be applications, as well as extensions, that use it.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  185. Re: First draft of PG 17 release notes

    Robert Haas <robertmhaas@gmail.com> — 2024-09-19T17:23:40Z

    On Thu, Sep 19, 2024 at 1:04 PM Bruce Momjian <bruce@momjian.us> wrote:
> I mentioned the SPI macros because that could lead to breakage, and
> there might be applications, as well as extensions, that use it.

Sure, this is all a judgement call. I don't think it's particularly
likely that many people are relying on those macros, though, and if
they are, they will mostly likely find out that they're gone when they
try to compile, rather than from reading the release notes. Likewise,
I feel that the new list iteration macros are both optional and minor,
so there's not really a reason to tell people about them. But opinions
will vary, and that's fine. I just mentioned my opinion since you
seemed to be asking. :-)

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  186. Re: First draft of PG 17 release notes

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-09-20T14:02:25Z

    On 5/9/24 12:03 AM, Bruce Momjian wrote:
> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:
> 
> 	release-17:  188
> 
> I welcome feedback.  For some reason it was an easier job than usual.

Attached is a proposal for the major features section. This borrows from 
the release announcement draft[1] and lists out features and themes that 
have broad user impact. This was a bit challenging for this release, 
because there are a lot of great features in PG17 that add up to a very 
special release.

Feedback welcome.

Thanks,

Jonathan

[1] 
https://git.postgresql.org/gitweb/?p=press.git;a=blob;f=releases/17/release.en.md;hb=HEAD

  187. Re: First draft of PG 17 release notes

    Laurenz Albe <laurenz.albe@cybertec.at> — 2024-09-20T16:55:56Z

    On Fri, 2024-09-20 at 10:02 -0400, Jonathan S. Katz wrote:
> Attached is a proposal for the major features section. This borrows from 
> the release announcement draft[1] and lists out features and themes that 
> have broad user impact. This was a bit challenging for this release, 
> because there are a lot of great features in PG17 that add up to a very 
> special release.
> 
> Feedback welcome.

I would have added the platform-independent binary collation provider.
And perhaps "pg_createsubscriber": that can be a game-changer for setting
up logical replication.

Yours,
Laurenz Albe

  188. Re: First draft of PG 17 release notes

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-09-20T17:47:43Z

    On 9/20/24 12:55 PM, Laurenz Albe wrote:
> On Fri, 2024-09-20 at 10:02 -0400, Jonathan S. Katz wrote:
>> Attached is a proposal for the major features section. This borrows from
>> the release announcement draft[1] and lists out features and themes that
>> have broad user impact. This was a bit challenging for this release,
>> because there are a lot of great features in PG17 that add up to a very
>> special release.
>>
>> Feedback welcome.
> 
> I would have added the platform-independent binary collation provider.
> And perhaps "pg_createsubscriber": that can be a game-changer for setting
> up logical replication.

I was on the fence about that, mostly because it'd make that sentence 
too much of a mouthful, but I do agree.

IIRC (didn't get to check) we did have a precedent for sublists in the 
major features, so I broke this one up. Please see attached.

Jonathan

  189. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-20T20:00:24Z

    On Fri, Sep 20, 2024 at 01:47:43PM -0400, Jonathan Katz wrote:
> On 9/20/24 12:55 PM, Laurenz Albe wrote:
> > On Fri, 2024-09-20 at 10:02 -0400, Jonathan S. Katz wrote:
> > > Attached is a proposal for the major features section. This borrows from
> > > the release announcement draft[1] and lists out features and themes that
> > > have broad user impact. This was a bit challenging for this release,
> > > because there are a lot of great features in PG17 that add up to a very
> > > special release.
> > > 
> > > Feedback welcome.
> > 
> > I would have added the platform-independent binary collation provider.
> > And perhaps "pg_createsubscriber": that can be a game-changer for setting
> > up logical replication.
> 
> I was on the fence about that, mostly because it'd make that sentence too
> much of a mouthful, but I do agree.
> 
> IIRC (didn't get to check) we did have a precedent for sublists in the major
> features, so I broke this one up. Please see attached.

Patch applied to PG 17.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  190. Re: First draft of PG 17 release notes

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-09-20T20:05:11Z

    Bruce Momjian <bruce@momjian.us> writes:
> Patch applied to PG 17.

I don't see a push?

			regards, tom lane

  191. Re: First draft of PG 17 release notes

    Laurenz Albe <laurenz.albe@cybertec.at> — 2024-09-20T20:13:57Z

    On Fri, 2024-09-20 at 13:47 -0400, Jonathan S. Katz wrote:
> Please see attached.

> +    <listitem>
> +     <para>
> +      Various query performance improvements, including to sequential reads
> +      using streaming I/O, write throughput under high concurrency, and
> +      searches over multiple values in a <link linkend="btree">btree</link>
> +      index.
> +     </para>
> +    </listitem>

Perhaps that last part could be "and searches over IN-lists in a b-tree index".
It might be technically less correct, but I'd expect that it gives more people
the right idea.

> +        <para>
> +         <link
> +          linkend="app-pgcreatesubscriber"><application>pg_createsubscriber</application></link>,
> +          a utility that logical replicas from physical standbys
> +        </para>

There's a verb missing: "a utility that *creates* logical replicas..."

> +        <para>
> +         <link
> +          linkend="pgupgrade"><application>pg_upgrade</application></link> now
> +          preserves replication slots on both publishers and subscribers
> +        </para>

I wonder if we should omit "on both publishers and subscribers".
It preserves replication slots anywhere, right?

> +    <listitem>
> +     <para>
> +      New client-side connection option, <link
> +      linkend="libpq-connect-sslnegotiation"><literal>sslnegotiation=direct</literal></link>,
> +      that allows direct TLS handshakes that avoids a round-trip negotation.
> +     </para>
> +    </listitem>

It should be "that avoid".  Plural.

Yours,
Laurenz Albe

  192. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-20T20:20:04Z

    On Fri, Sep 20, 2024 at 04:05:11PM -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > Patch applied to PG 17.
> 
> I don't see a push?

Push was delayed because my test script found some uncommitted files due
to earlier testing.  Should be fine now.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  193. Re: First draft of PG 17 release notes

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-09-20T20:21:13Z

    Laurenz Albe <laurenz.albe@cybertec.at> writes:
> [ assorted corrections ]

I fixed a couple of these before seeing your message.

			regards, tom lane

  194. Re: First draft of PG 17 release notes

    Yugo Nagata <nagata@sraoss.co.jp> — 2024-09-26T05:19:21Z

    On Thu, 9 May 2024 00:03:50 -0400
Bruce Momjian <bruce@momjian.us> wrote:

> I have committed the first draft of the PG 17 release notes;  you can
> see the results here:

I propose to improve the following description in "Migration to Version 17"
section by adding CREATE INDEX and CREATE MATERIALIZED VIEW into the command list.

     <para>
     Change functions to use a safe <xref linkend="guc-search-path"/>
     during maintenance operations (Jeff Davis)
     <ulink url="&commit_baseurl;2af07e2f7">&sect;</ulink>
     </para>

It is suggested in the thread [1] that users could not notice the behaviour
of CREATE INDEX is changed because the explicit command name is not listed in
the release notes. So, I think it is better to add CREATE INDEX and
CREATE MATERIALIZED VIEW into the command list. 

I've attached a patch.

[1] https://www.postgresql.org/message-id/flat/20240926125110.67e52f4f7a388af539367213%40sraoss.co.jp#71d4b5d6c842ba038e1e4e99c110b688

> 
> 	https://momjian.us/pgsql_docs/release-17.html
> 
> It will be improved until the final release.  The item count is 188,
> which is similar to recent releases:
> 
> 	release-10:  189
> 	release-11:  170
> 	release-12:  180
> 	release-13:  178
> 	release-14:  220
> 	release-15:  184
> 	release-16:  206
> 	release-17:  188
> 
> I welcome feedback.  For some reason it was an easier job than usual.
> 
> -- 
>   Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>   EDB                                      https://enterprisedb.com
> 
>   Only you can decide what is important to you.
> 
> 


-- 
Yugo Nagata <nagata@sraoss.co.jp>

  195. Re: First draft of PG 17 release notes

    Amit Kapila <amit.kapila16@gmail.com> — 2024-09-26T09:38:52Z

    On Sat, Sep 21, 2024 at 1:50 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Fri, Sep 20, 2024 at 04:05:11PM -0400, Tom Lane wrote:
> > Bruce Momjian <bruce@momjian.us> writes:
> > > Patch applied to PG 17.
> >
> > I don't see a push?
>
> Push was delayed because my test script found some uncommitted files due
> to earlier testing.  Should be fine now.
>

<para>
         <link
          linkend="app-pgcreatesubscriber"><application>pg_createsubscriber</application></link>,
          a utility that creates logical replicas from physical standbys
        </para>

This description is okay but according to me, the more compelling use
case is that this new utility helps to allow online upgrades of
physical replication setup as explained in the blog [1]. See the
section: "Upgrading Streaming (Physical) Replication Setup".

       </listitem>
       <listitem>
        <para>
         <link
          linkend="pgupgrade"><application>pg_upgrade</application></link> now
          preserves replication slots on both publishers and subscribers
        </para>

It is better to write the above statement as:
"pg_upgrade</application></link> now preserves replication slots on
publishers and full subscription's state on subscribers". This is
because replication slots are preserved on publishers. The subscribers
preserve the subscription state.

[1] - http://amitkapila16.blogspot.com/2024/09/online-upgrading-logical-and-physical.html

-- 
With Regards,
Amit Kapila.

  196. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-29T01:19:11Z

    On Thu, Sep 26, 2024 at 02:19:21PM +0900, Yugo Nagata wrote:
> On Thu, 9 May 2024 00:03:50 -0400
> Bruce Momjian <bruce@momjian.us> wrote:
> 
> > I have committed the first draft of the PG 17 release notes;  you can
> > see the results here:
> 
> I propose to improve the following description in "Migration to Version 17"
> section by adding CREATE INDEX and CREATE MATERIALIZED VIEW into the command list.
> 
>      <para>
>      Change functions to use a safe <xref linkend="guc-search-path"/>
>      during maintenance operations (Jeff Davis)
>      <ulink url="&commit_baseurl;2af07e2f7">&sect;</ulink>
>      </para>
> 
> It is suggested in the thread [1] that users could not notice the behaviour
> of CREATE INDEX is changed because the explicit command name is not listed in
> the release notes. So, I think it is better to add CREATE INDEX and
> CREATE MATERIALIZED VIEW into the command list. 
> 
> I've attached a patch.

It this a valid change?  Seems so.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  197. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-09-29T01:20:22Z

    On Thu, Sep 26, 2024 at 03:08:52PM +0530, Amit Kapila wrote:
> On Sat, Sep 21, 2024 at 1:50 AM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Fri, Sep 20, 2024 at 04:05:11PM -0400, Tom Lane wrote:
> > > Bruce Momjian <bruce@momjian.us> writes:
> > > > Patch applied to PG 17.
> > >
> > > I don't see a push?
> >
> > Push was delayed because my test script found some uncommitted files due
> > to earlier testing.  Should be fine now.
> >
> 
> <para>
>          <link
>           linkend="app-pgcreatesubscriber"><application>pg_createsubscriber</application></link>,
>           a utility that creates logical replicas from physical standbys
>         </para>
> 
> This description is okay but according to me, the more compelling use
> case is that this new utility helps to allow online upgrades of
> physical replication setup as explained in the blog [1]. See the
> section: "Upgrading Streaming (Physical) Replication Setup".
> 
>        </listitem>
>        <listitem>
>         <para>
>          <link
>           linkend="pgupgrade"><application>pg_upgrade</application></link> now
>           preserves replication slots on both publishers and subscribers
>         </para>
> 
> It is better to write the above statement as:
> "pg_upgrade</application></link> now preserves replication slots on
> publishers and full subscription's state on subscribers". This is
> because replication slots are preserved on publishers. The subscribers
> preserve the subscription state.

So, as I understand it, this preservation only happens when the _old_
Postgres version is 17+.  Do we want to try and explain that in the
Postgres 17 release notes?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  198. Re: First draft of PG 17 release notes

    Amit Kapila <amit.kapila16@gmail.com> — 2024-09-29T13:03:29Z

    On Sun, Sep 29, 2024 at 6:50 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Thu, Sep 26, 2024 at 03:08:52PM +0530, Amit Kapila wrote:
> > On Sat, Sep 21, 2024 at 1:50 AM Bruce Momjian <bruce@momjian.us> wrote:
> > >
> > > On Fri, Sep 20, 2024 at 04:05:11PM -0400, Tom Lane wrote:
> > > > Bruce Momjian <bruce@momjian.us> writes:
> > > > > Patch applied to PG 17.
> > > >
> > > > I don't see a push?
> > >
> > > Push was delayed because my test script found some uncommitted files due
> > > to earlier testing.  Should be fine now.
> > >
> >
> > <para>
> >          <link
> >           linkend="app-pgcreatesubscriber"><application>pg_createsubscriber</application></link>,
> >           a utility that creates logical replicas from physical standbys
> >         </para>
> >
> > This description is okay but according to me, the more compelling use
> > case is that this new utility helps to allow online upgrades of
> > physical replication setup as explained in the blog [1]. See the
> > section: "Upgrading Streaming (Physical) Replication Setup".
> >
> >        </listitem>
> >        <listitem>
> >         <para>
> >          <link
> >           linkend="pgupgrade"><application>pg_upgrade</application></link> now
> >           preserves replication slots on both publishers and subscribers
> >         </para>
> >
> > It is better to write the above statement as:
> > "pg_upgrade</application></link> now preserves replication slots on
> > publishers and full subscription's state on subscribers". This is
> > because replication slots are preserved on publishers. The subscribers
> > preserve the subscription state.
>
> So, as I understand it, this preservation only happens when the _old_
> Postgres version is 17+.
>

Yes.

>  Do we want to try and explain that in the
> Postgres 17 release notes?
>

It would be good if we can capture that information without bloating
the release document. However, this information is already present in
pg_upgrade docs, so users have a way to know the same even if we can't
mention it in the release notes.

-- 
With Regards,
Amit Kapila.

  199. Re: First draft of PG 17 release notes

    Yugo Nagata <nagata@sraoss.co.jp> — 2024-09-30T05:20:21Z

    On Sat, 28 Sep 2024 21:19:11 -0400
Bruce Momjian <bruce@momjian.us> wrote:

> On Thu, Sep 26, 2024 at 02:19:21PM +0900, Yugo Nagata wrote:
> > On Thu, 9 May 2024 00:03:50 -0400
> > Bruce Momjian <bruce@momjian.us> wrote:
> > 
> > > I have committed the first draft of the PG 17 release notes;  you can
> > > see the results here:
> > 
> > I propose to improve the following description in "Migration to Version 17"
> > section by adding CREATE INDEX and CREATE MATERIALIZED VIEW into the command list.
> > 
> >      <para>
> >      Change functions to use a safe <xref linkend="guc-search-path"/>
> >      during maintenance operations (Jeff Davis)
> >      <ulink url="&commit_baseurl;2af07e2f7">&sect;</ulink>
> >      </para>
> > 
> > It is suggested in the thread [1] that users could not notice the behaviour
> > of CREATE INDEX is changed because the explicit command name is not listed in
> > the release notes. So, I think it is better to add CREATE INDEX and
> > CREATE MATERIALIZED VIEW into the command list. 
> > 
> > I've attached a patch.
> 
> It this a valid change?  Seems so.

Yes. This change on CREATE INDEX was introduced by 2af07e2f7 together with
other commands, but it was missed to be mentioned in the commit message
although the description was added to the documentation.

The change on CEATE MATERIALIZED VIEW was introduced by a separate commit
b4da732fd, since which the REFRESH logic is used when creating a matview.
Should we add here a link to that commit, too?

Regards,
Yugo Nagata

> -- 
>   Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>   EDB                                      https://enterprisedb.com
> 
>   When a patient asks the doctor, "Am I going to die?", he means 
>   "Am I going to die soon?"


-- 
Yugo NAGATA <nagata@sraoss.co.jp>

  200. Re: First draft of PG 17 release notes

    Laurenz Albe <laurenz.albe@cybertec.at> — 2024-10-01T14:36:09Z

    I think that the removal of the "adminpack" extension should
be listed in the section "migration to v17" as an incompatibility.

I have seen one complaint that pg_upgrade fails if the extension
is installed, but a dump/restore would also throw an error.

Yours,
Laurenz Albe

  201. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-07T23:25:11Z

    On Mon, Sep 30, 2024 at 02:20:21PM +0900, Yugo NAGATA wrote:
> On Sat, 28 Sep 2024 21:19:11 -0400
> Bruce Momjian <bruce@momjian.us> wrote:
> 
> > On Thu, Sep 26, 2024 at 02:19:21PM +0900, Yugo Nagata wrote:
> > > On Thu, 9 May 2024 00:03:50 -0400
> > > Bruce Momjian <bruce@momjian.us> wrote:
> > > 
> > > > I have committed the first draft of the PG 17 release notes;  you can
> > > > see the results here:
> > > 
> > > I propose to improve the following description in "Migration to Version 17"
> > > section by adding CREATE INDEX and CREATE MATERIALIZED VIEW into the command list.
> > > 
> > >      <para>
> > >      Change functions to use a safe <xref linkend="guc-search-path"/>
> > >      during maintenance operations (Jeff Davis)
> > >      <ulink url="&commit_baseurl;2af07e2f7">&sect;</ulink>
> > >      </para>
> > > 
> > > It is suggested in the thread [1] that users could not notice the behaviour
> > > of CREATE INDEX is changed because the explicit command name is not listed in
> > > the release notes. So, I think it is better to add CREATE INDEX and
> > > CREATE MATERIALIZED VIEW into the command list. 
> > > 
> > > I've attached a patch.
> > 
> > It this a valid change?  Seems so.
> 
> Yes. This change on CREATE INDEX was introduced by 2af07e2f7 together with
> other commands, but it was missed to be mentioned in the commit message
> although the description was added to the documentation.
> 
> The change on CEATE MATERIALIZED VIEW was introduced by a separate commit
> b4da732fd, since which the REFRESH logic is used when creating a matview.
> Should we add here a link to that commit, too?

I developed the attached patch which adds the two commands and the
commit item.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  202. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-08T00:00:45Z

    On Mon, Oct  7, 2024 at 07:25:11PM -0400, Bruce Momjian wrote:
> > Yes. This change on CREATE INDEX was introduced by 2af07e2f7 together with
> > other commands, but it was missed to be mentioned in the commit message
> > although the description was added to the documentation.
> > 
> > The change on CEATE MATERIALIZED VIEW was introduced by a separate commit
> > b4da732fd, since which the REFRESH logic is used when creating a matview.
> > Should we add here a link to that commit, too?
> 
> I developed the attached patch which adds the two commands and the
> commit item.

Okay, updated commit after running src/tools/add_commit_links.pl.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  203. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-08T00:11:37Z

    On Tue, Oct  1, 2024 at 04:36:09PM +0200, Laurenz Albe wrote:
> I think that the removal of the "adminpack" extension should
> be listed in the section "migration to v17" as an incompatibility.
> 
> I have seen one complaint that pg_upgrade fails if the extension
> is installed, but a dump/restore would also throw an error.

Agreed. moved.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  204. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-08T00:55:42Z

    On Sun, Sep 29, 2024 at 06:33:29PM +0530, Amit Kapila wrote:
> > > It is better to write the above statement as:
> > > "pg_upgrade</application></link> now preserves replication slots on
> > > publishers and full subscription's state on subscribers". This is
> > > because replication slots are preserved on publishers. The subscribers
> > > preserve the subscription state.
> >
> > So, as I understand it, this preservation only happens when the _old_
> > Postgres version is 17+.
> >
> 
> Yes.
> 
> >  Do we want to try and explain that in the
> > Postgres 17 release notes?
> >
> 
> It would be good if we can capture that information without bloating
> the release document. However, this information is already present in
> pg_upgrade docs, so users have a way to know the same even if we can't
> mention it in the release notes.

I have developed the attached patch to mention it is "logical" slots,
and to mention its future use.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  205. Re: First draft of PG 17 release notes

    Amit Kapila <amit.kapila16@gmail.com> — 2024-10-08T06:27:19Z

    On Tue, Oct 8, 2024 at 6:25 AM Bruce Momjian <bruce@momjian.us> wrote:
>
> On Sun, Sep 29, 2024 at 06:33:29PM +0530, Amit Kapila wrote:
> > > > It is better to write the above statement as:
> > > > "pg_upgrade</application></link> now preserves replication slots on
> > > > publishers and full subscription's state on subscribers". This is
> > > > because replication slots are preserved on publishers. The subscribers
> > > > preserve the subscription state.
> > >
> > > So, as I understand it, this preservation only happens when the _old_
> > > Postgres version is 17+.
> > >
> >
> > Yes.
> >
> > >  Do we want to try and explain that in the
> > > Postgres 17 release notes?
> > >
> >
> > It would be good if we can capture that information without bloating
> > the release document. However, this information is already present in
> > pg_upgrade docs, so users have a way to know the same even if we can't
> > mention it in the release notes.
>
> I have developed the attached patch to mention it is "logical" slots,
> and to mention its future use.
>

LGTM. Thanks!

-- 
With Regards,
Amit Kapila.

  206. Re: First draft of PG 17 release notes

    Laurenz Albe <laurenz.albe@cybertec.at> — 2024-10-08T06:36:43Z

    On Mon, 2024-10-07 at 20:11 -0400, Bruce Momjian wrote:
> On Tue, Oct  1, 2024 at 04:36:09PM +0200, Laurenz Albe wrote:
> > I think that the removal of the "adminpack" extension should
> > be listed in the section "migration to v17" as an incompatibility.
> > 
> > I have seen one complaint that pg_upgrade fails if the extension
> > is installed, but a dump/restore would also throw an error.
> 
> Agreed. moved.

Thank you!

Yours,
Laurenz Albe

  207. Re: First draft of PG 17 release notes

    Yugo Nagata <nagata@sraoss.co.jp> — 2024-10-08T10:51:23Z

    On Mon, 7 Oct 2024 20:00:45 -0400
Bruce Momjian <bruce@momjian.us> wrote:

> On Mon, Oct  7, 2024 at 07:25:11PM -0400, Bruce Momjian wrote:
> > > Yes. This change on CREATE INDEX was introduced by 2af07e2f7 together with
> > > other commands, but it was missed to be mentioned in the commit message
> > > although the description was added to the documentation.
> > > 
> > > The change on CEATE MATERIALIZED VIEW was introduced by a separate commit
> > > b4da732fd, since which the REFRESH logic is used when creating a matview.
> > > Should we add here a link to that commit, too?
> > 
> > I developed the attached patch which adds the two commands and the
> > commit item.
> 
> Okay, updated commit after running src/tools/add_commit_links.pl.

LGTM. Thank you.

Regards,
Yugo Nagata

-- 
Yugo NAGATA <nagata@sraoss.co.jp>

  208. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-10T02:58:19Z

    On Tue, Oct  8, 2024 at 07:51:23PM +0900, Yugo NAGATA wrote:
> On Mon, 7 Oct 2024 20:00:45 -0400
> Bruce Momjian <bruce@momjian.us> wrote:
> 
> > On Mon, Oct  7, 2024 at 07:25:11PM -0400, Bruce Momjian wrote:
> > > > Yes. This change on CREATE INDEX was introduced by 2af07e2f7 together with
> > > > other commands, but it was missed to be mentioned in the commit message
> > > > although the description was added to the documentation.
> > > > 
> > > > The change on CEATE MATERIALIZED VIEW was introduced by a separate commit
> > > > b4da732fd, since which the REFRESH logic is used when creating a matview.
> > > > Should we add here a link to that commit, too?
> > > 
> > > I developed the attached patch which adds the two commands and the
> > > commit item.
> > 
> > Okay, updated commit after running src/tools/add_commit_links.pl.
> 
> LGTM. Thank you.

Thanks, patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"

  209. Re: First draft of PG 17 release notes

    Bruce Momjian <bruce@momjian.us> — 2024-10-10T03:06:09Z

    On Tue, Oct  8, 2024 at 11:57:19AM +0530, Amit Kapila wrote:
> On Tue, Oct 8, 2024 at 6:25 AM Bruce Momjian <bruce@momjian.us> wrote:
> >
> > On Sun, Sep 29, 2024 at 06:33:29PM +0530, Amit Kapila wrote:
> > > > > It is better to write the above statement as:
> > > > > "pg_upgrade</application></link> now preserves replication slots on
> > > > > publishers and full subscription's state on subscribers". This is
> > > > > because replication slots are preserved on publishers. The subscribers
> > > > > preserve the subscription state.
> > > >
> > > > So, as I understand it, this preservation only happens when the _old_
> > > > Postgres version is 17+.
> > > >
> > >
> > > Yes.
> > >
> > > >  Do we want to try and explain that in the
> > > > Postgres 17 release notes?
> > > >
> > >
> > > It would be good if we can capture that information without bloating
> > > the release document. However, this information is already present in
> > > pg_upgrade docs, so users have a way to know the same even if we can't
> > > mention it in the release notes.
> >
> > I have developed the attached patch to mention it is "logical" slots,
> > and to mention its future use.
> >
> 
> LGTM. Thanks!

Great, patch applied.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  When a patient asks the doctor, "Am I going to die?", he means 
  "Am I going to die soon?"