Thread

PostgreSQL 17 Release Management Team & Feature Freeze

Michael Paquier <michael@paquier.xyz> — 2024-03-18T02:50:25Z

Hi,

We are pleased to announce the Release Management Team (RMT) (cc'd)
for the PostgreSQL 17 release:
- Robert Haas
- Heikki Linnakangas
- Michael Paquier

You can find information about the responsibilities of the RMT here:
https://wiki.postgresql.org/wiki/Release_Management_Team

Additionally, the RMT has set the feature freeze to be **April 8, 2024
at 0:00 AoE** (see [1]).  This is the last time to commit features for
PostgreSQL 17.  In other words, no new PostgreSQL 17 feature can be
committed after April 8, 2024 at 0:00 AoE.  As mentioned last year in
[2], this uses the "standard" feature freeze date/time.

You can track open items for the PostgreSQL 17 release here:
https://wiki.postgresql.org/wiki/PostgreSQL_17_Open_Items

Please let us know if you have any questions.

On behalf of the PG17 RMT,

Michael

[1]: https://en.wikipedia.org/wiki/Anywhere_on_Earth
[2]: https://www.postgresql.org/message-id/9fbe60ec-fd1b-6ee0-240d-af7fc444223d@postgresql.org
--
Michael

RE: PostgreSQL 17 Release Management Team & Feature Freeze

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> — 2024-03-18T03:49:24Z

Dear Michael,

> We are pleased to announce the Release Management Team (RMT) (cc'd)
> for the PostgreSQL 17 release:
> - Robert Haas
> - Heikki Linnakangas
> - Michael Paquier

Thanks for managing the release of PostgreSQL to proceed the right direction!

> You can track open items for the PostgreSQL 17 release here:
> https://wiki.postgresql.org/wiki/PostgreSQL_17_Open_Items

I think the entry can be closed:

```
pg_upgrade with --link mode failing upgrade of publishers
Commit: 29d0a77fa660
Owner: Amit Kapila
```

The reported failure was only related with the test script, not the feature.
The issue has already been analyzed and the fix patch was pushed as f17529b710.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED
https://www.fujitsu.com/

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Michael Paquier <michael@paquier.xyz> — 2024-03-18T05:16:51Z

On Mon, Mar 18, 2024 at 03:49:24AM +0000, Hayato Kuroda (Fujitsu) wrote:
> I think the entry can be closed:
> 
> ```
> pg_upgrade with --link mode failing upgrade of publishers
> Commit: 29d0a77fa660
> Owner: Amit Kapila
> ```
> 
> The reported failure was only related with the test script, not the feature.
> The issue has already been analyzed and the fix patch was pushed as f17529b710.

If you think that this is OK, and as far as I can see this looks OK on
the thread, then this open item should be moved under "resolved before
17beta1", mentioning the commit involved in the fix.  Please see [1]
for examples.

[1]: https://wiki.postgresql.org/wiki/PostgreSQL_16_Open_Items#resolved_before_16beta1
--
Michael

RE: PostgreSQL 17 Release Management Team & Feature Freeze

Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com> — 2024-03-18T07:09:10Z

Dear Michael,

> If you think that this is OK, and as far as I can see this looks OK on
> the thread, then this open item should be moved under "resolved before
> 17beta1", mentioning the commit involved in the fix.  Please see [1]
> for examples.

OK, I understood that I could wait checking from you. Thanks.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED
https://www.fujitsu.com/

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Amit Kapila <amit.kapila16@gmail.com> — 2024-03-20T03:19:30Z

On Mon, Mar 18, 2024 at 12:39 PM Hayato Kuroda (Fujitsu)
<kuroda.hayato@fujitsu.com> wrote:
>
> > If you think that this is OK, and as far as I can see this looks OK on
> > the thread, then this open item should be moved under "resolved before
> > 17beta1", mentioning the commit involved in the fix.  Please see [1]
> > for examples.
>
> OK, I understood that I could wait checking from you. Thanks.
>

I don't think there is a need to wait here. The issue being tracked
was fixed, so I have updated the open items page accordingly.

-- 
With Regards,
Amit Kapila.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

David Rowley <dgrowleyml@gmail.com> — 2024-04-04T02:10:27Z

On Mon, 18 Mar 2024 at 15:50, Michael Paquier <michael@paquier.xyz> wrote:
> Additionally, the RMT has set the feature freeze to be **April 8, 2024
> at 0:00 AoE** (see [1]).  This is the last time to commit features for
> PostgreSQL 17.  In other words, no new PostgreSQL 17 feature can be
> committed after April 8, 2024 at 0:00 AoE.  As mentioned last year in
> [2], this uses the "standard" feature freeze date/time.

Someone asked me about this, so thought it might be useful to post here.

To express this as UTC, It's:

postgres=# select '2024-04-08 00:00-12:00' at time zone 'UTC';
      timezone
---------------------
 2024-04-08 12:00:00

Or, time remaining, relative to now:

select '2024-04-08 00:00-12:00' - now();

David

> [1]: https://en.wikipedia.org/wiki/Anywhere_on_Earth
> [2]: https://www.postgresql.org/message-id/9fbe60ec-fd1b-6ee0-240d-af7fc444223d@postgresql.org

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Michael Paquier <michael@paquier.xyz> — 2024-04-07T22:50:41Z

On Thu, Apr 04, 2024 at 03:10:27PM +1300, David Rowley wrote:
> Someone asked me about this, so thought it might be useful to post here.

I've received the same question.

> To express this as UTC, It's:
> 
> postgres=# select '2024-04-08 00:00-12:00' at time zone 'UTC';
>       timezone
> ---------------------
>  2024-04-08 12:00:00
> 
> Or, time remaining, relative to now:
> 
> select '2024-04-08 00:00-12:00' - now();

And, as of the moment of typing this email, I get:
=# select '2024-04-08 00:00-12:00' - now() as time_remaining;
 time_remaining
-----------------
 13:10:35.688134
(1 row)

So there is just a bit more than half a day remaining before the
feature freeze is in effect.
--
Michael

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Robert Haas <robertmhaas@gmail.com> — 2024-04-08T13:26:09Z

On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> And, as of the moment of typing this email, I get:
> =# select '2024-04-08 00:00-12:00' - now() as time_remaining;
>  time_remaining
> -----------------
>  13:10:35.688134
> (1 row)
>
> So there is just a bit more than half a day remaining before the
> feature freeze is in effect.

OK, so feature freeze is now in effect, then.

In the future, let's use GMT for these deadlines. There have got to be
a lot more people who can easily understand when a certain GMT
timestamp falls in their local timezone than there are people who can
easily understand when a certain AOE timestamp falls in their local
timezone.

And maybe we need to think of a way to further mitigate this crush of
last minute commits. e.g. In the last week, you can't have more
feature commits, or more lines of insertions in your commits, than you
did in the prior 3 weeks combined. I don't know. I think this mad rush
of last-minute commits is bad for the project.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tom Lane <tgl@sss.pgh.pa.us> — 2024-04-08T13:43:11Z

Robert Haas <robertmhaas@gmail.com> writes:
> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.

I was just about to pen an angry screed along the same lines.
The commit flux over the past couple days, and even the last
twelve hours, was flat-out ridiculous.  These patches weren't
ready a week ago, and I doubt they were ready now.

The RMT should feel free to exercise their right to require
revert "early and often", or we are going to be shipping a
very buggy release.

			regards, tom lane

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Melanie Plageman <melanieplageman@gmail.com> — 2024-04-08T14:26:58Z

On Mon, Apr 8, 2024 at 9:26 AM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> > And, as of the moment of typing this email, I get:
> > =# select '2024-04-08 00:00-12:00' - now() as time_remaining;
> >  time_remaining
> > -----------------
> >  13:10:35.688134
> > (1 row)
> >
> > So there is just a bit more than half a day remaining before the
> > feature freeze is in effect.
>
> OK, so feature freeze is now in effect, then.
>
> In the future, let's use GMT for these deadlines. There have got to be
> a lot more people who can easily understand when a certain GMT
> timestamp falls in their local timezone than there are people who can
> easily understand when a certain AOE timestamp falls in their local
> timezone.
>
> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.

What if we pick the actual feature freeze time randomly? That is,
starting on March 15th (or whenever but more than a week before), each
night someone from RMT generates a random number between $current_day
and April 8th. If the number matches $current_day, that day at
midnight is the feature freeze.

- Melanie

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Robert Treat <rob@xzilla.net> — 2024-04-08T14:41:17Z

On Mon, Apr 8, 2024 at 10:27 AM Melanie Plageman
<melanieplageman@gmail.com> wrote:
> On Mon, Apr 8, 2024 at 9:26 AM Robert Haas <robertmhaas@gmail.com> wrote:
> > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> > > And, as of the moment of typing this email, I get:
> > > =# select '2024-04-08 00:00-12:00' - now() as time_remaining;
> > >  time_remaining
> > > -----------------
> > >  13:10:35.688134
> > > (1 row)
> > >
> > > So there is just a bit more than half a day remaining before the
> > > feature freeze is in effect.
> >
> > OK, so feature freeze is now in effect, then.
> >
> > In the future, let's use GMT for these deadlines. There have got to be
> > a lot more people who can easily understand when a certain GMT
> > timestamp falls in their local timezone than there are people who can
> > easily understand when a certain AOE timestamp falls in their local
> > timezone.
> >
> > And maybe we need to think of a way to further mitigate this crush of
> > last minute commits. e.g. In the last week, you can't have more
> > feature commits, or more lines of insertions in your commits, than you
> > did in the prior 3 weeks combined. I don't know. I think this mad rush
> > of last-minute commits is bad for the project.
>
> What if we pick the actual feature freeze time randomly? That is,
> starting on March 15th (or whenever but more than a week before), each
> night someone from RMT generates a random number between $current_day
> and April 8th. If the number matches $current_day, that day at
> midnight is the feature freeze.
>

Unfortunately many humans are hardwired towards procrastination and
last minute heroics (it's one reason why deadline driven development
works even though in the long run it tends to be bad practice), and
historically was one of the driving factors in why we started doing
commitfests in the first place (you should have seen the mad dash of
commits before we had that process), so ISTM that it's not completely
avoidable...

That said, are you suggesting that the feature freeze deadline be
random, and also held in secret by the RMT, only to be announced after
the freeze time has passed? This feels weird, but might apply enough
deadline related pressure while avoiding last minute shenanigans.


Robert Treat
https://xzilla.net

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Laurenz Albe <laurenz.albe@cybertec.at> — 2024-04-08T14:42:05Z

On Mon, 2024-04-08 at 09:26 -0400, Robert Haas wrote:
> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.

I found that there are a lot of people who can only get going with a
pressing deadline.  But that's just an observation, not an excuse.

I don't know if additional rules will achieve anything here.  This can
only improve with buy-in from the committers, and that cannot be forced.

Yours,
Laurenz Albe

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Andrey Borodin <x4mmm@yandex-team.ru> — 2024-04-08T14:42:39Z


> On 8 Apr 2024, at 17:26, Melanie Plageman <melanieplageman@gmail.com> wrote:
> 
> What if we pick the actual feature freeze time randomly? That is,
> starting on March 15th (or whenever but more than a week before), each
> night someone from RMT generates a random number between $current_day
> and April 8th. If the number matches $current_day, that day at
> midnight is the feature freeze.

But this implies that actual date is not publicly known before feature freeze is in effect. Do I understand idea correctly?


Best regards, Andrey Borodin.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Heikki Linnakangas <hlinnaka@iki.fi> — 2024-04-08T14:42:41Z

On 08/04/2024 16:43, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> And maybe we need to think of a way to further mitigate this crush of
>> last minute commits. e.g. In the last week, you can't have more
>> feature commits, or more lines of insertions in your commits, than you
>> did in the prior 3 weeks combined. I don't know. I think this mad rush
>> of last-minute commits is bad for the project.
> 
> I was just about to pen an angry screed along the same lines.
> The commit flux over the past couple days, and even the last
> twelve hours, was flat-out ridiculous.  These patches weren't
> ready a week ago, and I doubt they were ready now.
> 
> The RMT should feel free to exercise their right to require
> revert "early and often", or we are going to be shipping a
> very buggy release.

Can you elaborate, which patches you think were not ready? Let's make 
sure to capture any concrete concerns in the Open Items list.

I agree the last-minute crunch felt more intense than in previous years. 
I'm guilty myself; I crunched the "direct SSL" patches in. My rationale 
for that one: It's been in a pretty settled state for a long time. There 
hasn't been any particular concerns about the design or the 
implementation. I haven't commit tit sooner because I was not 
comfortable with the lack of tests, especially the libpq parts. So I 
made a last minute dash to write the tests so that I'm comfortable with 
it, and I restructured the commits so that the tests and refactorings 
come first. The resulting code changes look the same they have for a 
long time, and I didn't uncover any significant new issues while doing 
that. I would not have felt comfortable committing it otherwise.

Yeah, I should have done that sooner, but realistically, there's nothing 
like a looming deadline as a motivator. One idea to avoid the mad rush 
in the future would be to make the feature freeze deadline more 
progressive. For example:

April 1: If you are still working on a feature that you still want to 
commit, you must explicitly flag it in the commitfest as "I plan to 
commit this very soon".

April 4: You must give a short status update about anything that you 
haven't committed yet, with an explanation of how you plan to proceed 
with it.

April 5-8: Mandatory daily status updates, explicit approval by the 
commitfest manager needed each day to continue working on it.

April 8: Hard feature freeze deadline

This would also give everyone more visibility, so that we're not all 
surprised by the last minute flood of commits.

-- 
Heikki Linnakangas
Neon (https://neon.tech)

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Melanie Plageman <melanieplageman@gmail.com> — 2024-04-08T14:52:28Z

On Mon, Apr 8, 2024 at 10:41 AM Robert Treat <rob@xzilla.net> wrote:
>
> On Mon, Apr 8, 2024 at 10:27 AM Melanie Plageman
> <melanieplageman@gmail.com> wrote:
> > On Mon, Apr 8, 2024 at 9:26 AM Robert Haas <robertmhaas@gmail.com> wrote:
> > > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> > > > And, as of the moment of typing this email, I get:
> > > > =# select '2024-04-08 00:00-12:00' - now() as time_remaining;
> > > >  time_remaining
> > > > -----------------
> > > >  13:10:35.688134
> > > > (1 row)
> > > >
> > > > So there is just a bit more than half a day remaining before the
> > > > feature freeze is in effect.
> > >
> > > OK, so feature freeze is now in effect, then.
> > >
> > > In the future, let's use GMT for these deadlines. There have got to be
> > > a lot more people who can easily understand when a certain GMT
> > > timestamp falls in their local timezone than there are people who can
> > > easily understand when a certain AOE timestamp falls in their local
> > > timezone.
> > >
> > > And maybe we need to think of a way to further mitigate this crush of
> > > last minute commits. e.g. In the last week, you can't have more
> > > feature commits, or more lines of insertions in your commits, than you
> > > did in the prior 3 weeks combined. I don't know. I think this mad rush
> > > of last-minute commits is bad for the project.
> >
> > What if we pick the actual feature freeze time randomly? That is,
> > starting on March 15th (or whenever but more than a week before), each
> > night someone from RMT generates a random number between $current_day
> > and April 8th. If the number matches $current_day, that day at
> > midnight is the feature freeze.
> >
>
> Unfortunately many humans are hardwired towards procrastination and
> last minute heroics (it's one reason why deadline driven development
> works even though in the long run it tends to be bad practice), and
> historically was one of the driving factors in why we started doing
> commitfests in the first place (you should have seen the mad dash of
> commits before we had that process), so ISTM that it's not completely
> avoidable...
>
> That said, are you suggesting that the feature freeze deadline be
> random, and also held in secret by the RMT, only to be announced after
> the freeze time has passed? This feels weird, but might apply enough
> deadline related pressure while avoiding last minute shenanigans.

Basically, yes. The RMT would find out each day whether or not that
day is the feature freeze but not tell anyone. Then they would push
some kind of feature freeze tag (do we already do a feature freeze
tag? I didn't think so) at 11:59 PM (in some timezone) that evening
and all commits that are feature commits after that are reverted.

I basically thought it would be a way for people to know that they
need to have their work done before April but keep them from waiting
until the actual last minute. The rationale for doing it this way is
it requires way less human involvement than some of the other methods
proposed. Figuring out how many commits each committer is allowed to
do based on past number of LOC,etc like Robert's suggestion sounds
like a lot of work. I was trying to think of a simple way to beat the
natural propensity people have toward procrastination.

But, an approach like Heikki suggested [1] with check-ins and
staggered deadlines is certainly much more principled. It just sounds
like it will require a lot of enforcement and oversight. And it might
be hard to ensure it doesn't end up being enforced only for some
people and not others. However, I suppose everyone is saying a mindset
shift is needed. And you can't usually shift a mindset with a
technical solution like I proposed (despite what Libertarians might
tell you about carbon offsets).

- Melanie

[1] https://www.postgresql.org/message-id/a5485b74-059a-4ea0-b445-7c393d6fe0de%40iki.fi

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Pavel Borisov <pashkin.elfe@gmail.com> — 2024-04-08T14:57:31Z

On Mon, 8 Apr 2024 at 18:42, Heikki Linnakangas <hlinnaka@iki.fi> wrote:

> On 08/04/2024 16:43, Tom Lane wrote:
> > Robert Haas <robertmhaas@gmail.com> writes:
> >> And maybe we need to think of a way to further mitigate this crush of
> >> last minute commits. e.g. In the last week, you can't have more
> >> feature commits, or more lines of insertions in your commits, than you
> >> did in the prior 3 weeks combined. I don't know. I think this mad rush
> >> of last-minute commits is bad for the project.
> >
> > I was just about to pen an angry screed along the same lines.
> > The commit flux over the past couple days, and even the last
> > twelve hours, was flat-out ridiculous.  These patches weren't
> > ready a week ago, and I doubt they were ready now.
> >
> > The RMT should feel free to exercise their right to require
> > revert "early and often", or we are going to be shipping a
> > very buggy release.
>
>
> Can you elaborate, which patches you think were not ready? Let's make
> sure to capture any concrete concerns in the Open Items list.
>
> I agree the last-minute crunch felt more intense than in previous years.
> I'm guilty myself; I crunched the "direct SSL" patches in. My rationale
> for that one: It's been in a pretty settled state for a long time. There
> hasn't been any particular concerns about the design or the
> implementation. I haven't commit tit sooner because I was not
> comfortable with the lack of tests, especially the libpq parts. So I
> made a last minute dash to write the tests so that I'm comfortable with
> it, and I restructured the commits so that the tests and refactorings
> come first. The resulting code changes look the same they have for a
> long time, and I didn't uncover any significant new issues while doing
> that. I would not have felt comfortable committing it otherwise.
>
> Yeah, I should have done that sooner, but realistically, there's nothing
> like a looming deadline as a motivator. One idea to avoid the mad rush
> in the future would be to make the feature freeze deadline more
> progressive. For example:
>
> April 1: If you are still working on a feature that you still want to
> commit, you must explicitly flag it in the commitfest as "I plan to
> commit this very soon".
>
> April 4: You must give a short status update about anything that you
> haven't committed yet, with an explanation of how you plan to proceed
> with it.
>
> April 5-8: Mandatory daily status updates, explicit approval by the
> commitfest manager needed each day to continue working on it.
>
> April 8: Hard feature freeze deadline
>
> This would also give everyone more visibility, so that we're not all
> surprised by the last minute flood of commits.
>

IMO the fact that people struggle to work on patches, and make them better,
etc. is an immense blessing for the Postgres community. Is the peak of
commits really a big problem provided we have 6 months before actual
release? I doubt March patches tend to be worse than the November ones.

People are different, so are the ways they feel motivation and inspiration.
This could be easily broken with bureaucratic decisions some of them, like
proposed counting of lines of code vs period of time look even little bit
repressive.

Let's remain an open community, support inspiration in each other, and
don't build an over-regulated corporation. I feel that Postgres will win if
people feel less limited by formal rules. Personally, I believe RMT has
enough experience and authority to stabilize and interact with authors if
questions arise.

Regards,
Pavel Borisov

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tom Lane <tgl@sss.pgh.pa.us> — 2024-04-08T14:59:44Z

Heikki Linnakangas <hlinnaka@iki.fi> writes:
> On 08/04/2024 16:43, Tom Lane wrote:
>> I was just about to pen an angry screed along the same lines.
>> The commit flux over the past couple days, and even the last
>> twelve hours, was flat-out ridiculous.  These patches weren't
>> ready a week ago, and I doubt they were ready now.

> Can you elaborate, which patches you think were not ready? Let's make 
> sure to capture any concrete concerns in the Open Items list.

[ shrug... ]  There were fifty-some commits on the last day,
some of them quite large, and you want me to finger particular ones?
I can't even have read them all yet.

> Yeah, I should have done that sooner, but realistically, there's nothing 
> like a looming deadline as a motivator. One idea to avoid the mad rush 
> in the future would be to make the feature freeze deadline more 
> progressive. For example:
> April 1: If you are still working on a feature that you still want to 
> commit, you must explicitly flag it in the commitfest as "I plan to 
> commit this very soon".
> April 4: You must give a short status update about anything that you 
> haven't committed yet, with an explanation of how you plan to proceed 
> with it.
> April 5-8: Mandatory daily status updates, explicit approval by the 
> commitfest manager needed each day to continue working on it.
> April 8: Hard feature freeze deadline

> This would also give everyone more visibility, so that we're not all 
> surprised by the last minute flood of commits.

Perhaps something like that could help, but it seems like a lot
of mechanism.  I think the real problem is just that committers
need to re-orient their thinking a little.  We must get *less*
willing to commit marginal patches, not more so, as the deadline
approaches.

			regards, tom lane

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tom Lane <tgl@sss.pgh.pa.us> — 2024-04-08T15:05:00Z

Pavel Borisov <pashkin.elfe@gmail.com> writes:
> IMO the fact that people struggle to work on patches, and make them better,
> etc. is an immense blessing for the Postgres community. Is the peak of
> commits really a big problem provided we have 6 months before actual
> release? I doubt March patches tend to be worse than the November ones.

Yes, it's a problem, and yes the average quality of last-minute
patches is visibly worse than that of patches committed in a less
hasty fashion.  We have been through this every year for the last
couple decades, seems like, and we keep re-learning that lesson
the hard way.  I'm just distressed at our utter failure to learn
from experience.

			regards, tom lane

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tomas Vondra <tomas.vondra@enterprisedb.com> — 2024-04-08T15:21:41Z

On 4/8/24 16:59, Tom Lane wrote:
> Heikki Linnakangas <hlinnaka@iki.fi> writes:
>> On 08/04/2024 16:43, Tom Lane wrote:
>>> I was just about to pen an angry screed along the same lines.
>>> The commit flux over the past couple days, and even the last
>>> twelve hours, was flat-out ridiculous.  These patches weren't
>>> ready a week ago, and I doubt they were ready now.
> 
>> Can you elaborate, which patches you think were not ready? Let's make 
>> sure to capture any concrete concerns in the Open Items list.
> 
> [ shrug... ]  There were fifty-some commits on the last day,
> some of them quite large, and you want me to finger particular ones?
> I can't even have read them all yet.
> 
>> Yeah, I should have done that sooner, but realistically, there's nothing 
>> like a looming deadline as a motivator. One idea to avoid the mad rush 
>> in the future would be to make the feature freeze deadline more 
>> progressive. For example:
>> April 1: If you are still working on a feature that you still want to 
>> commit, you must explicitly flag it in the commitfest as "I plan to 
>> commit this very soon".
>> April 4: You must give a short status update about anything that you 
>> haven't committed yet, with an explanation of how you plan to proceed 
>> with it.
>> April 5-8: Mandatory daily status updates, explicit approval by the 
>> commitfest manager needed each day to continue working on it.
>> April 8: Hard feature freeze deadline
> 
>> This would also give everyone more visibility, so that we're not all 
>> surprised by the last minute flood of commits.
> 
> Perhaps something like that could help, but it seems like a lot
> of mechanism.  I think the real problem is just that committers
> need to re-orient their thinking a little.  We must get *less*
> willing to commit marginal patches, not more so, as the deadline
> approaches.
> 

For me the main problem with the pre-freeze crush is that it leaves
pretty much no practical chance to do meaningful review/testing, and
some of the patches likely went through significant changes (at least
judging by the number of messages and patch versions in the associated
threads). That has to have a cost later ...

That being said, I'm not sure the "progressive deadline" proposed by
Heikki would improve that materially, and it seems like a lot of effort
to maintain etc. And even if someone updates the CF app with all the
details, does it even give others sufficient opportunity to review the
new patch versions? I don't think so. (It anything, it does not seem
fair to expect others to do last-minute reviews under pressure.)

Maybe it'd be better to start by expanding the existing rule about not
committing patches introduced for the first time in the last CF. What if
we said that patches in the last CF must not go through significant
changes, and if they do it'd mean the patch is moved to the next CF?
Perhaps with exceptions to be granted by the RMT when appropriate?

regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Joe Conway <mail@joeconway.com> — 2024-04-08T15:28:37Z

On 4/8/24 11:05, Tom Lane wrote:
> Pavel Borisov <pashkin.elfe@gmail.com> writes:
>> IMO the fact that people struggle to work on patches, and make them better,
>> etc. is an immense blessing for the Postgres community. Is the peak of
>> commits really a big problem provided we have 6 months before actual
>> release? I doubt March patches tend to be worse than the November ones.
> 
> Yes, it's a problem, and yes the average quality of last-minute
> patches is visibly worse than that of patches committed in a less
> hasty fashion.  We have been through this every year for the last
> couple decades, seems like, and we keep re-learning that lesson
> the hard way.  I'm just distressed at our utter failure to learn
> from experience.


I don't dispute that we could do better, and this is just a simplistic 
look based on "number of commits per day", but the attached does put it 
in perspective to some extent.

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Andres Freund <andres@anarazel.de> — 2024-04-08T15:29:48Z

Hi,

On 2024-04-08 09:26:09 -0400, Robert Haas wrote:
> On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.

I don't think it's very useful to paint a very broad brush here,
unfortunately. Some will just polish commits until the last minute, until the
the dot's on the i's really shine, others will continue picking up more CF
entries until the freeze is reached, others will push half baked stuff.  Of
course there will be an increased commit rate, but it does looks like there
was some stuff that looked somewhat rickety.

Greetings,

Andres

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Matthias van de Meent <boekewurm+postgres@gmail.com> — 2024-04-08T15:48:37Z

On Mon, 8 Apr 2024 at 17:21, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>
>
>
> On 4/8/24 16:59, Tom Lane wrote:
> > Heikki Linnakangas <hlinnaka@iki.fi> writes:
> >> On 08/04/2024 16:43, Tom Lane wrote:
> >>> I was just about to pen an angry screed along the same lines.
> >>> The commit flux over the past couple days, and even the last
> >>> twelve hours, was flat-out ridiculous.  These patches weren't
> >>> ready a week ago, and I doubt they were ready now.
> >
> >> Can you elaborate, which patches you think were not ready? Let's make
> >> sure to capture any concrete concerns in the Open Items list.
> >
> > [ shrug... ]  There were fifty-some commits on the last day,
> > some of them quite large, and you want me to finger particular ones?
> > I can't even have read them all yet.
> >
> >> Yeah, I should have done that sooner, but realistically, there's nothing
> >> like a looming deadline as a motivator. One idea to avoid the mad rush
> >> in the future would be to make the feature freeze deadline more
> >> progressive. For example:
> >> April 1: If you are still working on a feature that you still want to
> >> commit, you must explicitly flag it in the commitfest as "I plan to
> >> commit this very soon".
> >> April 4: You must give a short status update about anything that you
> >> haven't committed yet, with an explanation of how you plan to proceed
> >> with it.
> >> April 5-8: Mandatory daily status updates, explicit approval by the
> >> commitfest manager needed each day to continue working on it.
> >> April 8: Hard feature freeze deadline
> >
> >> This would also give everyone more visibility, so that we're not all
> >> surprised by the last minute flood of commits.
> >
> > Perhaps something like that could help, but it seems like a lot
> > of mechanism.  I think the real problem is just that committers
> > need to re-orient their thinking a little.  We must get *less*
> > willing to commit marginal patches, not more so, as the deadline
> > approaches.
> >
>
> For me the main problem with the pre-freeze crush is that it leaves
> pretty much no practical chance to do meaningful review/testing, and
> some of the patches likely went through significant changes (at least
> judging by the number of messages and patch versions in the associated
> threads). That has to have a cost later ...
>
> That being said, I'm not sure the "progressive deadline" proposed by
> Heikki would improve that materially, and it seems like a lot of effort
> to maintain etc. And even if someone updates the CF app with all the
> details, does it even give others sufficient opportunity to review the
> new patch versions? I don't think so. (It anything, it does not seem
> fair to expect others to do last-minute reviews under pressure.)
>
> Maybe it'd be better to start by expanding the existing rule about not
> committing patches introduced for the first time in the last CF.

I don't think adding more hurdles about what to include into the next
release is a good solution. Why the March CF, and not earlier? Or
later? How about unregistered patches? Changes to the docs? Bug fixes?
The March CF already has a submission deadline of "before march", so
that already puts a soft limit on the patches considered for the april
deadline.

> What if
> we said that patches in the last CF must not go through significant
> changes, and if they do it'd mean the patch is moved to the next CF?

I also think there is already a big issue with a lack of interest in
getting existing patches from non-committers committed, reducing the
set of patches that could be considered is just cheating the numbers
and discouraging people from contributing. For one, I know I have
motivation issues keeping up with reviewing other people's patches
when none (well, few, as of this CF) of my patches get reviewed
materially and committed. I don't see how shrinking the window of
opportunity for significant review from 9 to 7 months is going to help
there.

So, I think that'd be counter-productive, as this would get the
perverse incentive to band-aid over (architectural) issues to limit
churn inside the patch, rather than fix those issues in a way that's
appropriate for the project as a whole.

-Matthias

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Pavel Borisov <pashkin.elfe@gmail.com> — 2024-04-08T15:56:44Z

On Mon, 8 Apr 2024 at 19:48, Matthias van de Meent <
boekewurm+postgres@gmail.com> wrote:

> On Mon, 8 Apr 2024 at 17:21, Tomas Vondra <tomas.vondra@enterprisedb.com>
> wrote:
> >
> >
> >
> > On 4/8/24 16:59, Tom Lane wrote:
> > > Heikki Linnakangas <hlinnaka@iki.fi> writes:
> > >> On 08/04/2024 16:43, Tom Lane wrote:
> > >>> I was just about to pen an angry screed along the same lines.
> > >>> The commit flux over the past couple days, and even the last
> > >>> twelve hours, was flat-out ridiculous.  These patches weren't
> > >>> ready a week ago, and I doubt they were ready now.
> > >
> > >> Can you elaborate, which patches you think were not ready? Let's make
> > >> sure to capture any concrete concerns in the Open Items list.
> > >
> > > [ shrug... ]  There were fifty-some commits on the last day,
> > > some of them quite large, and you want me to finger particular ones?
> > > I can't even have read them all yet.
> > >
> > >> Yeah, I should have done that sooner, but realistically, there's
> nothing
> > >> like a looming deadline as a motivator. One idea to avoid the mad rush
> > >> in the future would be to make the feature freeze deadline more
> > >> progressive. For example:
> > >> April 1: If you are still working on a feature that you still want to
> > >> commit, you must explicitly flag it in the commitfest as "I plan to
> > >> commit this very soon".
> > >> April 4: You must give a short status update about anything that you
> > >> haven't committed yet, with an explanation of how you plan to proceed
> > >> with it.
> > >> April 5-8: Mandatory daily status updates, explicit approval by the
> > >> commitfest manager needed each day to continue working on it.
> > >> April 8: Hard feature freeze deadline
> > >
> > >> This would also give everyone more visibility, so that we're not all
> > >> surprised by the last minute flood of commits.
> > >
> > > Perhaps something like that could help, but it seems like a lot
> > > of mechanism.  I think the real problem is just that committers
> > > need to re-orient their thinking a little.  We must get *less*
> > > willing to commit marginal patches, not more so, as the deadline
> > > approaches.
> > >
> >
> > For me the main problem with the pre-freeze crush is that it leaves
> > pretty much no practical chance to do meaningful review/testing, and
> > some of the patches likely went through significant changes (at least
> > judging by the number of messages and patch versions in the associated
> > threads). That has to have a cost later ...
> >
> > That being said, I'm not sure the "progressive deadline" proposed by
> > Heikki would improve that materially, and it seems like a lot of effort
> > to maintain etc. And even if someone updates the CF app with all the
> > details, does it even give others sufficient opportunity to review the
> > new patch versions? I don't think so. (It anything, it does not seem
> > fair to expect others to do last-minute reviews under pressure.)
> >
> > Maybe it'd be better to start by expanding the existing rule about not
> > committing patches introduced for the first time in the last CF.
>
> I don't think adding more hurdles about what to include into the next
> release is a good solution. Why the March CF, and not earlier? Or
> later? How about unregistered patches? Changes to the docs? Bug fixes?
> The March CF already has a submission deadline of "before march", so
> that already puts a soft limit on the patches considered for the april
> deadline.
>
> > What if
> > we said that patches in the last CF must not go through significant
> > changes, and if they do it'd mean the patch is moved to the next CF?
>
> I also think there is already a big issue with a lack of interest in
> getting existing patches from non-committers committed, reducing the
> set of patches that could be considered is just cheating the numbers
> and discouraging people from contributing. For one, I know I have
> motivation issues keeping up with reviewing other people's patches
> when none (well, few, as of this CF) of my patches get reviewed
> materially and committed. I don't see how shrinking the window of
> opportunity for significant review from 9 to 7 months is going to help
> there.
>
> So, I think that'd be counter-productive, as this would get the
> perverse incentive to band-aid over (architectural) issues to limit
> churn inside the patch, rather than fix those issues in a way that's
> appropriate for the project as a whole.
>

I second your opinion, Mattias! I also feel that the management of the
review of other contibutor's patches participation is much more important
for a community as a whole. And this could make process of patches
proposals and improvement running, while motivating participation (in all
three roles of contributors, reviewers and committers), not vice versa.

Regards,
Pavel.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-04-08T16:07:25Z

On 2024-Apr-08, Robert Haas wrote:

> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.

Another idea is to run a patch triage around mid March 15th, with the
intention of punting patches to the next cycle early enough.  But rather
than considering each patch in its own merits, consider the responsible
_committers_ and the load that they are reasonably expected to handle:
determine which patches each committer deems his or her responsibility
for the rest of that March commitfest, and punt all the rest.  That way
we have a reasonably vetted amount of effort that each committer is
allowed to spend for the remainder of that commitfest.  Excesses should
be obvious enough and discouraged.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Jonathan S. Katz <jkatz@postgresql.org> — 2024-04-08T16:23:14Z

On 4/8/24 8:29 AM, Andres Freund wrote:
> Hi,
> 
> On 2024-04-08 09:26:09 -0400, Robert Haas wrote:
>> On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
>> And maybe we need to think of a way to further mitigate this crush of
>> last minute commits. e.g. In the last week, you can't have more
>> feature commits, or more lines of insertions in your commits, than you
>> did in the prior 3 weeks combined. I don't know. I think this mad rush
>> of last-minute commits is bad for the project.
> 
> I don't think it's very useful to paint a very broad brush here,
> unfortunately. Some will just polish commits until the last minute, until the
> the dot's on the i's really shine, others will continue picking up more CF
> entries until the freeze is reached, others will push half baked stuff.  Of
> course there will be an increased commit rate, but it does looks like there
> was some stuff that looked somewhat rickety.

I agree with Andres here (though decline to comment on the rickety-ness 
of the patches). I think overcoming human nature to be more proactive at 
a deadline is at least a NP-hard problem. This won't change if we adjust 
deadlines. I think it's better to ensure we're enforcing best practices 
for commits, and maybe that's a separate review to have.

As mentioned in different contexts, we do have several safeguards for a 
release:

* We have a (fairly long) beta period; this allows us to remove patches 
prior to GA and get in further testing.
* We have a RMT that (as Tom mentioned) can use its powers early and 
often to remove patches that are not up to our quality levels.
* We can evaluate the quality of the commits coming in and coach folks 
on what to do better.

I understand that a revert is costly, particularly the longer a commit 
stays in, and I do 100% agree we should maintain the high commit bar we 
have and not rush things in just so "they're in for feature freeze and 
we'll clean them up for beta." That's where best practices come in.

I tend to judge the release by the outcome: once we go GA, how buggy is 
the release? Did something during the release cycle (e.g. a sloppy 
commit during feature freeze, lack of testing) lead to a bug that 
warranted an out-of-cycle release? And yes, how we commit things at 
feature freeze / through the beta impact that - we should ensure we're 
still committing things that we would have committed at a least hectic 
time, but understand that the deadline is still a strong motivator co 
complete the work.

Thanks,

Jonathan

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Robert Haas <robertmhaas@gmail.com> — 2024-04-08T16:24:52Z

On Mon, Apr 8, 2024 at 11:48 AM Matthias van de Meent
<boekewurm+postgres@gmail.com> wrote:
> I also think there is already a big issue with a lack of interest in
> getting existing patches from non-committers committed, reducing the
> set of patches that could be considered is just cheating the numbers
> and discouraging people from contributing. For one, I know I have
> motivation issues keeping up with reviewing other people's patches
> when none (well, few, as of this CF) of my patches get reviewed
> materially and committed. I don't see how shrinking the window of
> opportunity for significant review from 9 to 7 months is going to help
> there.
>
> So, I think that'd be counter-productive, as this would get the
> perverse incentive to band-aid over (architectural) issues to limit
> churn inside the patch, rather than fix those issues in a way that's
> appropriate for the project as a whole.

I don't think you're wrong about any of this, but I don't think Tom
and I are wrong to be upset about the volume of last-minute commits,
either. There's a lot of this stuff that could have been committed a
month ago, or two months ago, or six months ago, and it just wasn't. A
certain amount of that is, as Heikki says, understandable and
expected. People procrastinate. But, if too many people procrastinate
too much, then it becomes a problem, and if you don't do anything
about that problem then, well, you still have one.

I don't want more barriers to getting stuff committed here either, but
I also don't want somebody whose 100-line patch is basically unchanged
since last July to commit it 19 hours before the feature freeze
deadline[1]. That's just making everyone's life more difficult. If
that patch happens to have been submitted by a non-committer, that
non-committer waited an extra 9 months for the commit, not knowing
whether it would actually happen, which like you say is demotivating.
And if it was the committer's own patch then it was probably going in
sooner or later, barring objections, so basically, they just deprived
the project of 9 months of in-tree testing that the patch could have
had basically for free. There's simply no world in which this kind of
behavior is actually helpful to committers, non-committers, reviews,
or the project in general.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

[1] This is a fictional example, I made up these numbers without
checking anything, but I think it's probably not far off some of what
actually happened.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Masahiko Sawada <sawada.mshk@gmail.com> — 2024-04-08T16:34:09Z

On Tue, Apr 9, 2024 at 12:30 AM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2024-04-08 09:26:09 -0400, Robert Haas wrote:
> > On Sun, Apr 7, 2024 at 6:50 PM Michael Paquier <michael@paquier.xyz> wrote:
> > And maybe we need to think of a way to further mitigate this crush of
> > last minute commits. e.g. In the last week, you can't have more
> > feature commits, or more lines of insertions in your commits, than you
> > did in the prior 3 weeks combined. I don't know. I think this mad rush
> > of last-minute commits is bad for the project.
>
> Some will just polish commits until the last minute, until the
> the dot's on the i's really shine, others will continue picking up more CF
> entries until the freeze is reached, others will push half baked stuff.

I agree with this part.

Aside from considering how to institute some rules for mitigating the
last-minute rush, it might also be a good idea to consider how to
improve testing the new commits during beta. FWIW in each year, after
feature freeze I personally pick some new features that I didn't get
involved with during the development and do intensive reviews in
April. It might be good if more people did things like that. That
might help finding half baked features earlier and improve the quality
in general. So for example, we list features that could require more
reviews (e.g. because of its volume, complexity, and a span of
influence etc.) and we do intensive reviews for these items. Each item
should be reviewed by other than the author and the committer. We may
want to set aside a specific period for intensive testing.

Regards,

-- 
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tomas Vondra <tomas.vondra@enterprisedb.com> — 2024-04-08T18:15:20Z

On 4/8/24 17:48, Matthias van de Meent wrote:
> On Mon, 8 Apr 2024 at 17:21, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>>
>> ...
>>
>> For me the main problem with the pre-freeze crush is that it leaves
>> pretty much no practical chance to do meaningful review/testing, and
>> some of the patches likely went through significant changes (at least
>> judging by the number of messages and patch versions in the associated
>> threads). That has to have a cost later ...
>>
>> That being said, I'm not sure the "progressive deadline" proposed by
>> Heikki would improve that materially, and it seems like a lot of effort
>> to maintain etc. And even if someone updates the CF app with all the
>> details, does it even give others sufficient opportunity to review the
>> new patch versions? I don't think so. (It anything, it does not seem
>> fair to expect others to do last-minute reviews under pressure.)
>>
>> Maybe it'd be better to start by expanding the existing rule about not
>> committing patches introduced for the first time in the last CF.
> 
> I don't think adding more hurdles about what to include into the next
> release is a good solution. Why the March CF, and not earlier? Or
> later? How about unregistered patches? Changes to the docs? Bug fixes?
> The March CF already has a submission deadline of "before march", so
> that already puts a soft limit on the patches considered for the april
> deadline.
> 
>> What if
>> we said that patches in the last CF must not go through significant
>> changes, and if they do it'd mean the patch is moved to the next CF?
> 
> I also think there is already a big issue with a lack of interest in
> getting existing patches from non-committers committed, reducing the
> set of patches that could be considered is just cheating the numbers
> and discouraging people from contributing. For one, I know I have
> motivation issues keeping up with reviewing other people's patches
> when none (well, few, as of this CF) of my patches get reviewed
> materially and committed. I don't see how shrinking the window of
> opportunity for significant review from 9 to 7 months is going to help
> there.
> 

I 100% understand how frustrating the lack of progress can be, and I
agree we need to do better. I tried to move a number of stuck patches
this CF, and I hope (and plan) to do more of this in the future.

But I don't quite see how would this rule modification change the
situation for non-committers. AFAIK we already have the rule that
(complex) patches should not appear in the last CF for the first time,
and I'd argue that a substantial rework of a complex patch is not that
far from a completely new patch. Sure, the reworks may be based on a
thorough review, so there's a lot of nuance. But still, is it possible
to properly review if it gets reworked at the very end of the CF?

> So, I think that'd be counter-productive, as this would get the
> perverse incentive to band-aid over (architectural) issues to limit
> churn inside the patch, rather than fix those issues in a way that's
> appropriate for the project as a whole.
> 

Surely those architectural shortcomings should be identified in a review
- which however requires time to do properly, and thus is an argument
for ensuring there's enough time for such review (which is in direct
conflict with the last-minute crush, IMO).

Once again, I 100% agree we need to do better in handling patches from
non-committers, absolutely no argument there. But does it require this
last-minute crush? I don't think so, it can't be at the expense of
proper review and getting it right. A complex patch needs to be
submitted early in the cycle, not in the last CF. If it's submitted
early, but does not receive enough interest/reviews, I think we need to
fix & improve that - not to rework/push it at the very end.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Jesper Pedersen <jesper.pedersen@comcast.net> — 2024-04-08T19:00:04Z

Hi,

On 4/8/24 14:15, Tomas Vondra wrote:
>I think we need to
> fix & improve that - not to rework/push it at the very end.
>

This is going to be very extreme...

Either a patch is ready for merge or it isn't - when 2 or more 
Committers agree on it then it can be merged - the policy have to be 
discussed of course.

However, development happens all through out the year, so having to wait 
for potential feedback during CommitFests windows can stop development 
during the other months - I'm talking about non-Committers here...

Having patches on -hackers@ is best, but maybe there is a hybrid model 
where they exists in pull requests, tested through CfBot, and merged 
when ready - no matter what month it is.

Pull requests could still have labels that ties them to a "CommitFest" 
to "high-light" them, but it would make it easier to have a much clearer 
cut-off date for feature freeze.

And, pull request labels are easily changed.

March is seeing a lot of last-minute changes...

Just something to think about.

Best regards,
  Jesper

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Jelte Fennema-Nio <postgres@jeltef.nl> — 2024-04-08T19:32:14Z

On Mon, 8 Apr 2024 at 20:15, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
> I 100% understand how frustrating the lack of progress can be, and I
> agree we need to do better. I tried to move a number of stuck patches
> this CF, and I hope (and plan) to do more of this in the future.
>
> But I don't quite see how would this rule modification change the
> situation for non-committers.

The problem that I feel I'm seeing is that committers mostly seem to
materially review large patchsets in the last two commit fests. This
might be not based in reality, but it is definitely how it feels to
me. If someone has stats on this, feel free to share.

I'll sketch a situation: There's a big patch that some non-committer
submitted that has been sitting on the mailinglist for 6 months or
more, only being reviewed by other non-committers, which the submitter
quickly addresses. Then in the final commit fest it is finally
reviewed by a committer, and they say it requires significant changes.
Right now, the submitter can decide to quickly address those changes,
and hope to keep the attention of this committer, to hopefully get it
merged before the deadline after probably a few more back and forths.
But this new rule would mean that those significant changes would be a
reason not to put it in the upcoming release. Which I expect would
first of all really feel like a slap in the face to the submitter,
because it's not their fault that those changes were only requested in
the last commit fest. This would then likely result in the submitter
not following up quickly (why make time right at this moment, if
you're suddenly going to have another full year), which would then
cause the committer to lose context of the patch and thus interest in
the patch. And then finally getting into the exact same situation next
year in the final commit fest, when some other committer didn't agree
with the redesign of the first one and request a new one pushing it
another year.

So yeah, I definitely agree with Matthias. I definitely feel like his
rule would seriously impact contributions made by non-committers.

Maybe a better solution to this problem would be to spread impactful
reviews by committers more evenly throughout the year. Then there
wouldn't be such a rush to address them in the last commit fest.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Robert Haas <robertmhaas@gmail.com> — 2024-04-08T19:49:51Z

On Mon, Apr 8, 2024 at 3:32 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> Maybe a better solution to this problem would be to spread impactful
> reviews by committers more evenly throughout the year. Then there
> wouldn't be such a rush to address them in the last commit fest.

Spreading activity of all sorts more evenly through the year should
definitely be the goal, I think. It's just not exactly clear how we
can do that.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tomas Vondra <tomas.vondra@enterprisedb.com> — 2024-04-08T22:34:10Z


On 4/8/24 21:32, Jelte Fennema-Nio wrote:
> On Mon, 8 Apr 2024 at 20:15, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>> I 100% understand how frustrating the lack of progress can be, and I
>> agree we need to do better. I tried to move a number of stuck patches
>> this CF, and I hope (and plan) to do more of this in the future.
>>
>> But I don't quite see how would this rule modification change the
>> situation for non-committers.
> 
> The problem that I feel I'm seeing is that committers mostly seem to
> materially review large patchsets in the last two commit fests. This
> might be not based in reality, but it is definitely how it feels to
> me. If someone has stats on this, feel free to share.
> 
> I'll sketch a situation: There's a big patch that some non-committer
> submitted that has been sitting on the mailinglist for 6 months or
> more, only being reviewed by other non-committers, which the submitter
> quickly addresses. Then in the final commit fest it is finally
> reviewed by a committer, and they say it requires significant changes.
> Right now, the submitter can decide to quickly address those changes,
> and hope to keep the attention of this committer, to hopefully get it
> merged before the deadline after probably a few more back and forths.
> But this new rule would mean that those significant changes would be a
> reason not to put it in the upcoming release. Which I expect would
> first of all really feel like a slap in the face to the submitter,
> because it's not their fault that those changes were only requested in
> the last commit fest. This would then likely result in the submitter
> not following up quickly (why make time right at this moment, if
> you're suddenly going to have another full year), which would then
> cause the committer to lose context of the patch and thus interest in
> the patch. And then finally getting into the exact same situation next
> year in the final commit fest, when some other committer didn't agree
> with the redesign of the first one and request a new one pushing it
> another year.
> 

FWIW I have no doubt this problem is very real. It has never been easy
to get stuff reviewed/committed, and I doubt it improved in last couple
years, considering how the traffic on pgsql-hackers exploded :-(

> So yeah, I definitely agree with Matthias. I definitely feel like his
> rule would seriously impact contributions made by non-committers.
> 
> Maybe a better solution to this problem would be to spread impactful
> reviews by committers more evenly throughout the year. Then there
> wouldn't be such a rush to address them in the last commit fest.

Right. I think that's mostly what I was aiming for, although I haven't
made it very clear/explicit. But yeah, if the consequence of the "rule"
was that some of the patches are neglected entirely, that'd be pretty
terrible - both for the project and for the contributors.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Andrew Dunstan <andrew@dunslane.net> — 2024-04-08T22:50:05Z

On 2024-04-08 Mo 12:07, Alvaro Herrera wrote:
> On 2024-Apr-08, Robert Haas wrote:
>
>> And maybe we need to think of a way to further mitigate this crush of
>> last minute commits. e.g. In the last week, you can't have more
>> feature commits, or more lines of insertions in your commits, than you
>> did in the prior 3 weeks combined. I don't know. I think this mad rush
>> of last-minute commits is bad for the project.
> Another idea is to run a patch triage around mid March 15th, with the
> intention of punting patches to the next cycle early enough.  But rather
> than considering each patch in its own merits, consider the responsible
> _committers_ and the load that they are reasonably expected to handle:
> determine which patches each committer deems his or her responsibility
> for the rest of that March commitfest, and punt all the rest.  That way
> we have a reasonably vetted amount of effort that each committer is
> allowed to spend for the remainder of that commitfest.  Excesses should
> be obvious enough and discouraged.
>

I quite like the triage idea. But I think there's also a case for being 
more a bit more flexible with those patches we don't throw out. A case 
close to my heart: I'd have been very sad if the NESTED piece of 
JSON_TABLE hadn't made the cut, which it did with a few hours to spare, 
and I would not have been alone, far from it. I'd have been happy to 
give Amit a few more days or a week if he needed it, for a significant 
headline feature.

I know there will be those who say it's the thin end of the wedge and 
rulez is rulez, but this is my view.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tom Lane <tgl@sss.pgh.pa.us> — 2024-04-08T23:26:39Z

Andrew Dunstan <andrew@dunslane.net> writes:
> I quite like the triage idea. But I think there's also a case for being 
> more a bit more flexible with those patches we don't throw out. A case 
> close to my heart: I'd have been very sad if the NESTED piece of 
> JSON_TABLE hadn't made the cut, which it did with a few hours to spare, 
> and I would not have been alone, far from it. I'd have been happy to 
> give Amit a few more days or a week if he needed it, for a significant 
> headline feature.

> I know there will be those who say it's the thin end of the wedge and 
> rulez is rulez, but this is my view.

You've certainly been around the project long enough to remember the
times in the past when we let the schedule slip for "one more big
patch".  It just about never worked out well, so I'm definitely in
favor of a hard deadline.  The trick is to control the tendency to
push in patches that are only almost-ready in order to nominally
meet the deadline.  (I don't pretend to be immune from that
temptation myself, but I think I resisted it better than some
this year.)

			regards, tom lane

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Matthias van de Meent <boekewurm+postgres@gmail.com> — 2024-04-09T09:25:06Z

On Mon, 8 Apr 2024 at 20:15, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>
>
> On 4/8/24 17:48, Matthias van de Meent wrote:
>> On Mon, 8 Apr 2024 at 17:21, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>>>
>>> Maybe it'd be better to start by expanding the existing rule about not
>>> committing patches introduced for the first time in the last CF.
>>
>> I don't think adding more hurdles about what to include into the next
>> release is a good solution. Why the March CF, and not earlier? Or
>> later? How about unregistered patches? Changes to the docs? Bug fixes?
>> The March CF already has a submission deadline of "before march", so
>> that already puts a soft limit on the patches considered for the april
>> deadline.
>>
>>> What if
>>> we said that patches in the last CF must not go through significant
>>> changes, and if they do it'd mean the patch is moved to the next CF?
>>
>> I also think there is already a big issue with a lack of interest in
>> getting existing patches from non-committers committed, reducing the
>> set of patches that could be considered is just cheating the numbers
>> and discouraging people from contributing. For one, I know I have
>> motivation issues keeping up with reviewing other people's patches
>> when none (well, few, as of this CF) of my patches get reviewed
>> materially and committed. I don't see how shrinking the window of
>> opportunity for significant review from 9 to 7 months is going to help
>> there.
>>
>
> I 100% understand how frustrating the lack of progress can be, and I
> agree we need to do better. I tried to move a number of stuck patches
> this CF, and I hope (and plan) to do more of this in the future.

Thanks in advance.

> But I don't quite see how would this rule modification change the
> situation for non-committers. AFAIK we already have the rule that
> (complex) patches should not appear in the last CF for the first time,
> and I'd argue that a substantial rework of a complex patch is not that
> far from a completely new patch. Sure, the reworks may be based on a
> thorough review, so there's a lot of nuance. But still, is it possible
> to properly review if it gets reworked at the very end of the CF?

Possible? Probably, if there is a good shared understanding of why the
previous patch version's approach didn't work well, and if the next
approach is well understood as well. But it's not likely, that I'll
agree with.

But the main issue I have with your suggestion is that the March
commitfest effectively contains all new patches starting from January,
and the leftovers not committed by February. If we start banning all
new patches and those with significant reworks from the March
commitfest, then combined with the lack of CF in May there wouldn't be
any chance for new patches in the first half of the year, and an
effective block on rewrites for 6 months- the next CF is only in July.
Sure, there is a bit of leeway there, as some patches get committed
before the commitfest they're registered in is marked active, but our
development workflow is already quite hostile to incidental
contributor's patches [^0], and increasing the periods in which
authors shouldn't expect their patches to be reviewed due to a major
release that's planned in >5 months is probably not going to help the
case.

> > So, I think that'd be counter-productive, as this would get the
> > perverse incentive to band-aid over (architectural) issues to limit
> > churn inside the patch, rather than fix those issues in a way that's
> > appropriate for the project as a whole.
> >
>
> Surely those architectural shortcomings should be identified in a review
> - which however requires time to do properly, and thus is an argument
> for ensuring there's enough time for such review (which is in direct
> conflict with the last-minute crush, IMO).
>
> Once again, I 100% agree we need to do better in handling patches from
> non-committers, absolutely no argument there. But does it require this
> last-minute crush? I don't think so, it can't be at the expense of
> proper review and getting it right.

Agreed on this, 100%, but as mentioned above, the March commitfest
contains more than just "last minute crushes" [^1]. I don't think we
should throw out the baby with the bathwater here.

> A complex patch needs to be
> submitted early in the cycle, not in the last CF. If it's submitted
> early, but does not receive enough interest/reviews, I think we need to
> fix & improve that - not to rework/push it at the very end.

Agree on all this, too.

-Matthias

[^0] (see e.g. the EXPLAIN SERIALIZE patch thread [0], where the
original author did not have the time capacity to maintain the
patchset over months:
https://www.postgresql.org/message-id/ca0adb0e-fa4e-c37e-1cd7-91170b18cae1@gmx.de

[^1] Are there metrics on how many of the committed patches this CF
were new, only registered this CF, and if they're more or less
distributed to the feature freeze when compared to longer-running
patchess? I can probably build these statistics by extracting the data
from the webpages, but that's quite tedious.
A manual count gets me 68 new patches (~50% of all committed
registered patches); distribution comparison isn't in my time budget.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Tomas Vondra <tomas.vondra@enterprisedb.com> — 2024-04-09T10:18:32Z

On 4/9/24 11:25, Matthias van de Meent wrote:
> On Mon, 8 Apr 2024 at 20:15, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>>
>>
>> On 4/8/24 17:48, Matthias van de Meent wrote:
>>> On Mon, 8 Apr 2024 at 17:21, Tomas Vondra <tomas.vondra@enterprisedb.com> wrote:
>>>>
>>>> Maybe it'd be better to start by expanding the existing rule about not
>>>> committing patches introduced for the first time in the last CF.
>>>
>>> I don't think adding more hurdles about what to include into the next
>>> release is a good solution. Why the March CF, and not earlier? Or
>>> later? How about unregistered patches? Changes to the docs? Bug fixes?
>>> The March CF already has a submission deadline of "before march", so
>>> that already puts a soft limit on the patches considered for the april
>>> deadline.
>>>
>>>> What if
>>>> we said that patches in the last CF must not go through significant
>>>> changes, and if they do it'd mean the patch is moved to the next CF?
>>>
>>> I also think there is already a big issue with a lack of interest in
>>> getting existing patches from non-committers committed, reducing the
>>> set of patches that could be considered is just cheating the numbers
>>> and discouraging people from contributing. For one, I know I have
>>> motivation issues keeping up with reviewing other people's patches
>>> when none (well, few, as of this CF) of my patches get reviewed
>>> materially and committed. I don't see how shrinking the window of
>>> opportunity for significant review from 9 to 7 months is going to help
>>> there.
>>>
>>
>> I 100% understand how frustrating the lack of progress can be, and I
>> agree we need to do better. I tried to move a number of stuck patches
>> this CF, and I hope (and plan) to do more of this in the future.
> 
> Thanks in advance.
> 
>> But I don't quite see how would this rule modification change the
>> situation for non-committers. AFAIK we already have the rule that
>> (complex) patches should not appear in the last CF for the first time,
>> and I'd argue that a substantial rework of a complex patch is not that
>> far from a completely new patch. Sure, the reworks may be based on a
>> thorough review, so there's a lot of nuance. But still, is it possible
>> to properly review if it gets reworked at the very end of the CF?
> 
> Possible? Probably, if there is a good shared understanding of why the
> previous patch version's approach didn't work well, and if the next
> approach is well understood as well. But it's not likely, that I'll
> agree with.
> 
> But the main issue I have with your suggestion is that the March
> commitfest effectively contains all new patches starting from January,
> and the leftovers not committed by February. If we start banning all
> new patches and those with significant reworks from the March
> commitfest, then combined with the lack of CF in May there wouldn't be
> any chance for new patches in the first half of the year, and an
> effective block on rewrites for 6 months- the next CF is only in July.
> Sure, there is a bit of leeway there, as some patches get committed
> before the commitfest they're registered in is marked active, but our
> development workflow is already quite hostile to incidental
> contributor's patches [^0], and increasing the periods in which
> authors shouldn't expect their patches to be reviewed due to a major
> release that's planned in >5 months is probably not going to help the
> case.
> 

But I don't think I suggested to ban such patches from the March CF
entirely. Surely those patches can still be submitted, reviewed, and
even reworked in the last CF. All I said is it might be better to treat
those patches as not committable by default. Sorry if that wasn't clear
enough ...

Would this be an additional restriction? I'm not quite sure. Surely if
the intent is to only commit patches that we agree are in "sufficiently
good" shape, and getting them into such shape requires time & reviews,
this would not be a significant change.

FWIW I'm not a huge fan of hard unbreakable rules, so there should be
some leeway when justified, but the bar would be somewhat higher (clear
consensus, RMT having a chance to say no, ...).

>>> So, I think that'd be counter-productive, as this would get the
>>> perverse incentive to band-aid over (architectural) issues to limit
>>> churn inside the patch, rather than fix those issues in a way that's
>>> appropriate for the project as a whole.
>>>
>>
>> Surely those architectural shortcomings should be identified in a review
>> - which however requires time to do properly, and thus is an argument
>> for ensuring there's enough time for such review (which is in direct
>> conflict with the last-minute crush, IMO).
>>
>> Once again, I 100% agree we need to do better in handling patches from
>> non-committers, absolutely no argument there. But does it require this
>> last-minute crush? I don't think so, it can't be at the expense of
>> proper review and getting it right.
> 
> Agreed on this, 100%, but as mentioned above, the March commitfest
> contains more than just "last minute crushes" [^1]. I don't think we
> should throw out the baby with the bathwater here.
> 
>> A complex patch needs to be
>> submitted early in the cycle, not in the last CF. If it's submitted
>> early, but does not receive enough interest/reviews, I think we need to
>> fix & improve that - not to rework/push it at the very end.
> 
> Agree on all this, too.
> 

OK

> -Matthias
> 
> 
> [^0] (see e.g. the EXPLAIN SERIALIZE patch thread [0], where the
> original author did not have the time capacity to maintain the
> patchset over months:
> https://www.postgresql.org/message-id/ca0adb0e-fa4e-c37e-1cd7-91170b18cae1@gmx.de
> 

Yeah, this is terrible :-(

> [^1] Are there metrics on how many of the committed patches this CF
> were new, only registered this CF, and if they're more or less
> distributed to the feature freeze when compared to longer-running
> patchess? I can probably build these statistics by extracting the data
> from the webpages, but that's quite tedious.
> A manual count gets me 68 new patches (~50% of all committed
> registered patches); distribution comparison isn't in my time budget.

I suppose there are, or would be possible to get from the CF app. And
perhaps it'd be good to look at some of this, I think a lot of the
discussion here is based on very subjective perceptions of how the
process works.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Andrew Dunstan <andrew@dunslane.net> — 2024-04-09T12:27:31Z

On 2024-04-08 Mo 19:26, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
>> I quite like the triage idea. But I think there's also a case for being
>> more a bit more flexible with those patches we don't throw out. A case
>> close to my heart: I'd have been very sad if the NESTED piece of
>> JSON_TABLE hadn't made the cut, which it did with a few hours to spare,
>> and I would not have been alone, far from it. I'd have been happy to
>> give Amit a few more days or a week if he needed it, for a significant
>> headline feature.
>> I know there will be those who say it's the thin end of the wedge and
>> rulez is rulez, but this is my view.
> You've certainly been around the project long enough to remember the
> times in the past when we let the schedule slip for "one more big
> patch".  It just about never worked out well, so I'm definitely in
> favor of a hard deadline.  The trick is to control the tendency to
> push in patches that are only almost-ready in order to nominally
> meet the deadline.  (I don't pretend to be immune from that
> temptation myself, but I think I resisted it better than some
> this year.)
>
> 			


If we want to change how things are working I suspect we probably need 
something a lot more radical than any of the suggestions I've seen 
floating around. I don't know what that might be, but ISTM we're not 
thinking boldly enough.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Bruce Momjian <bruce@momjian.us> — 2024-04-09T21:11:57Z

On Mon, Apr  8, 2024 at 09:32:14PM +0200, Jelte Fennema-Nio wrote:
> I'll sketch a situation: There's a big patch that some non-committer
> submitted that has been sitting on the mailinglist for 6 months or
> more, only being reviewed by other non-committers, which the submitter
> quickly addresses. Then in the final commit fest it is finally
> reviewed by a committer, and they say it requires significant changes.
> Right now, the submitter can decide to quickly address those changes,
> and hope to keep the attention of this committer, to hopefully get it
> merged before the deadline after probably a few more back and forths.
> But this new rule would mean that those significant changes would be a
> reason not to put it in the upcoming release. Which I expect would
> first of all really feel like a slap in the face to the submitter,
> because it's not their fault that those changes were only requested in
> the last commit fest. This would then likely result in the submitter
> not following up quickly (why make time right at this moment, if
> you're suddenly going to have another full year), which would then
> cause the committer to lose context of the patch and thus interest in
> the patch. And then finally getting into the exact same situation next
> year in the final commit fest, when some other committer didn't agree
> with the redesign of the first one and request a new one pushing it
> another year.

Yes, I can see this happening.  :-(

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Bruce Momjian <bruce@momjian.us> — 2024-04-09T21:16:27Z

On Mon, Apr  8, 2024 at 10:41:17AM -0400, Robert Treat wrote:
> Unfortunately many humans are hardwired towards procrastination and
> last minute heroics (it's one reason why deadline driven development
> works even though in the long run it tends to be bad practice), and
> historically was one of the driving factors in why we started doing
> commitfests in the first place (you should have seen the mad dash of
> commits before we had that process), so ISTM that it's not completely
> avoidable...
> 
> That said, are you suggesting that the feature freeze deadline be
> random, and also held in secret by the RMT, only to be announced after
> the freeze time has passed? This feels weird, but might apply enough
> deadline related pressure while avoiding last minute shenanigans.

Committing code is a hard job, no question.  However, committers have to
give up the idea that they should wait for brilliant ideas before
finalizing patches.  If you come up with a better idea later, great, but
don't wait to finalize patches.

I used to write college papers much too late because I expected some
brilliant idea to come to me, and it rarely happened.  I learned to
write the paper with the ideas I had, and if I come up with a better
idea later, I can add it to the end.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Robert Haas <robertmhaas@gmail.com> — 2024-04-10T13:51:00Z

On Tue, Apr 9, 2024 at 5:16 PM Bruce Momjian <bruce@momjian.us> wrote:
> Committing code is a hard job, no question.  However, committers have to
> give up the idea that they should wait for brilliant ideas before
> finalizing patches.  If you come up with a better idea later, great, but
> don't wait to finalize patches.

Completely agreed. If your ideas are too bad, you should just give up.
But if they're basically fine and you're just waiting for inspiration
to strike from above, you might as well get on with it. If it turns
out that you've broken something, well, that sucks, but it's still
better to get that out of the way in January ... or November ... or
August ... rather than waiting until March or April.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Re: PostgreSQL 17 Release Management Team & Feature Freeze

Ants Aasma <ants.aasma@cybertec.at> — 2024-04-10T14:37:28Z

On Mon, 8 Apr 2024 at 16:26, Robert Haas <robertmhaas@gmail.com> wrote:

> And maybe we need to think of a way to further mitigate this crush of
> last minute commits. e.g. In the last week, you can't have more
> feature commits, or more lines of insertions in your commits, than you
> did in the prior 3 weeks combined. I don't know. I think this mad rush
> of last-minute commits is bad for the project.
>

I think some part of this rush of commits could also be explained as a form
of entrainment[1]. Only patches reasonably close to commit will get picked
up with extra attention to get them ready before the deadline. After the
release hammer drops, the pool of remaining patches will have few patches
close to commit remaining. And to make matters worse the attention of
working on them will be spread thinner. When repeated, this pattern can be
self reinforcing.

If this hypothesis is true, maybe some forces could be introduced to
counteract this natural tendency. I don't have any bright ideas on how
exactly yet.

Ants

[1] Emergent synchronization of interacting oscillators, see:
https://en.wikipedia.org/wiki/Injection_locking#Entrainment
https://en.wikipedia.org/wiki/Entrainment_(biomusicology)