Thread
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Expose track_iotiming information via pg_stat_statements.
- 5b4f34661143 9.2.0 cited
-
Rewrite GiST support code for rangetypes.
- 80da9e68fdd7 9.2.0 cited
-
Clean up a couple of box gist helper functions.
- d50e1251946a 9.2.0 cited
-
Replace the "New Linear" GiST split algorithm for boxes and points with a
- 7f3bd86843e5 9.2.0 cited
-
Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T03:11:02Z
I have completed my draft of the 9.2 release notes, and committed it to git. I am waiting for our development docs to build, but after 40 minutes, I am still waiting: http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=guaibasaurus&dt=latest&stg=make-doc (Why is there no time zone shown in the date/time at the top?) I think it will eventually show up here: http://www.postgresql.org/docs/devel/static/release-9-2.html My private build is now online: http://momjian.us/tmp/pgsql/release-9-2.html I tested creation of the HISTORY file so Tom shouldn't need to fix missing markup tomorrow. :-) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T04:33:32Z
On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: > (Why is there no time zone shown in the date/time at the top?) I think > it will eventually show up here: > > http://www.postgresql.org/docs/devel/static/release-9-2.html The docs finally built 90 minutes after my commit, and the URL above is now working. (Does it always take this long to update?) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2012-05-10T05:02:43Z
On 05/10/2012 06:33 AM, Bruce Momjian wrote: > On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: >> (Why is there no time zone shown in the date/time at the top?) I think >> it will eventually show up here: >> >> http://www.postgresql.org/docs/devel/static/release-9-2.html > > The docs finally built 90 minutes after my commit, and the URL above is > now working. (Does it always take this long to update?) the developer docs builds are a byproduct of the snaptshot generation concept on borka.postgresql.org. For each snapshot we are doing a complete buildfarm-run to verify the basic integrity of the current code and only if that one succeeds we will generate a snapshot-tarball AND upload the docs to the main website. This whole process is not triggered by a commit but simply running on a fixed "every 4 hours" cycle. Stefan
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T05:08:02Z
On Thu, May 10, 2012 at 07:02:43AM +0200, Stefan Kaltenbrunner wrote: > On 05/10/2012 06:33 AM, Bruce Momjian wrote: > > On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: > >> (Why is there no time zone shown in the date/time at the top?) I think > >> it will eventually show up here: > >> > >> http://www.postgresql.org/docs/devel/static/release-9-2.html > > > > The docs finally built 90 minutes after my commit, and the URL above is > > now working. (Does it always take this long to update?) > > the developer docs builds are a byproduct of the snaptshot generation > concept on borka.postgresql.org. For each snapshot we are doing a > complete buildfarm-run to verify the basic integrity of the current code > and only if that one succeeds we will generate a snapshot-tarball AND > upload the docs to the main website. > This whole process is not triggered by a commit but simply running on a > fixed "every 4 hours" cycle. OK, good to know. Sometimes I need a quick build that I can show people, so I will just use my personal URL for those cases. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Erik Rijkers <er@xs4all.nl> — 2012-05-10T05:20:51Z
On Thu, May 10, 2012 06:33, Bruce Momjian wrote: > On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: >> >> http://www.postgresql.org/docs/devel/static/release-9-2.html > To "E.1.2.5. Monitoring" should be added: "Rename pg_stat_activity.current_query to query (Magnus Hagander)" And perhaps (same paragraph): "The previous query values are preserved, allowing for enhanced analysis." would be clearer as: "The last query values are preserved, allowing for enhanced analysis." Erik Rijkers -
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T05:29:11Z
Bruce Momjian <bruce@momjian.us> writes: > The docs finally built 90 minutes after my commit, and the URL above is > now working. (Does it always take this long to update?) I believe the new implementation of that stuff is that the devel docs are built whenever the buildfarm member guaibasaurus runs for HEAD, which it seems to do on an hourly schedule. This is definitely not as fast-responding as Peter's former custom script, but I'm not sure if it's worth thinking of another way. regards, tom lane
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T05:56:33Z
On Wed, May 9, 2012 at 11:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. Extra parens: Remove the spclocation field from pg_tablespace (Magnus Hagander, Tom Lane)) Reduce overhead of creating virtual transaction id locks ((Robert Haas, Jeff Davis) The antecedent of "these" is unclear: Allow backends to detect postmaster death via a pipe read failure, rather than polling (Peter Geoghegan, Heikki Linnakangas) These are internally called "latches". Missing comma: Cancel queries if clients get disconnected (Florian Pflug Greg Jaskiewicz) You mean "effect": Such casts have no affect. I think all three of these are the same thing: Avoid table and index rebuilds when NUMERIC, VARBIT, and temporal columns are changed in compatible ways (Noah Misch) Reduce need to rebuild indexes for various ALTER TABLE operations (Noah Misch) DUPLICATE? Avoid index rebuilds for no-rewrite ALTER TABLE / ALTER TYPE (Noah Misch) This feature wasn't committed at all: Parallel pg_dump (Robert Haas, Joachim Wieland) DETAILS? Yes, this is still true: This is currently unused. STILL TRUE? As a general comment, I think that your new policy of crediting the reviewer on every feature except when that reviewer is also a committer has produced a horrific mess. Just to pick one of many examples, consider this item: Add a security_barrier option for views (KaiGai Kohei, Noah Misch) Here is what the commit message says: Patch by KaiGai Kohei; original problem report by Heikki Linnakangas (in October 2009!). Review (in earlier versions) by Noah Misch and others. Design advice by Tom Lane and myself. Further review and cleanup by me. So there are four people mentioned in this commit message, and you've picked out two of them to credit, not on the basis of who did the most work, but rather on the basis of which ones happen to not be committers. The result is that, as I read through these release notes, one gets what I believe to be a very misleading notion of who developed which features. I don't object to not being credited on this one, but I don't think it makes sense to credit Noah and NOT credit me. As you have it, people who did little more than say "yep, looks fine to me" are credited almost equally with the people who wrote the code, while a committer who heavily revised the patch may not be mentioned at all, or sometimes (seemingly at random) they are. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -
Re: Draft release notes complete
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2012-05-10T07:50:14Z
On 10.05.2012 06:11, Bruce Momjian wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. Thanks! I committed a few trivial fixes, below are a few more I wasn't sure about: > * Add support for range data types (Jeff Davis, Tom Lane, Alexander Korotkov) > > The range data type records a lower and upper bound, and supports comparisons like contains, overlaps, and intersection. /s/comparisons/operations/ ? > * Allow a user to cancel queries in other owned sessions using pg_cancel_backend() (Magnus Hagander) > > Previously only the superuser could cancel queries. "other owned sessions" is a bit ambiguous. It reads to me like "possessed sessions" or "0wned sessions". It's not clear it means sessions owned by the same user. How about "... to cancel queries in his other sessions, using ..." ? Or: * Allow a non-superuser to cancel queries in another backend using pg_cancel_backend(), as long as the victim backend belongs to the same user Previously only the superuser could cancel queries. > * Change default names of triggers to fire action triggers before check triggers (Tom Lane) > > This allows default-named check triggers to check post-action rows. That's quite a mouthful :-). I don't understand what it means. > In psql tab completion, complete SQL key words based on COMP_KEYWORD_CASE setting and the perhaps case of the partially-supplied word (Peter Eisentraut, Fujii Masao) Which is correct spelling, "keyword" or "key word"? We seem to use both in the docs. "Keyword" sounds much better to me, but I think I might be more prone to write words together than native English speakers. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-10T10:18:08Z
On Thu, May 10, 2012 at 5:11 AM, Bruce Momjian <bruce@momjian.us> wrote: > (Why is there no time zone shown in the date/time at the top?) I think > it will eventually show up here: > > http://www.postgresql.org/docs/devel/static/release-9-2.html > Other than the comments others have specified: * Add libpq parameters for specifying the locations of server-side SSL files (Peter Eisentraut) Those are regular server side gucs and not libpq parameters. You certainly can't control the location of server-side files with libpq parameters.. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Thom Brown <thom@linux.com> — 2012-05-10T10:21:22Z
On 10 May 2012 04:11, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. I am waiting for our development docs to build, but after 40 > minutes, I am still waiting: > > http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=guaibasaurus&dt=latest&stg=make-doc > > (Why is there no time zone shown in the date/time at the top?) I think > it will eventually show up here: > > http://www.postgresql.org/docs/devel/static/release-9-2.html > > My private build is now online: > > http://momjian.us/tmp/pgsql/release-9-2.html > > I tested creation of the HISTORY file so Tom shouldn't need to fix > missing markup tomorrow. :-) Couple typo corrections attached. -- Thom
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T10:43:17Z
On 05/10/2012 01:29 AM, Tom Lane wrote: > Bruce Momjian<bruce@momjian.us> writes: >> The docs finally built 90 minutes after my commit, and the URL above is >> now working. (Does it always take this long to update?) > I believe the new implementation of that stuff is that the devel docs > are built whenever the buildfarm member guaibasaurus runs for HEAD, > which it seems to do on an hourly schedule. This is definitely not as > fast-responding as Peter's former custom script, but I'm not sure if > it's worth thinking of another way. > I don't see any reason it can't run more frequently, though. Currently a run takes 15 minutes or so. We could reduce that by making it skip some steps, and get it down to about 10 minutes. It would be perfectly reasonable to run every 5 minutes (it won't schedule concurrent runs - if the lock file is held by another run it exits gracefully). Of course, that's up to Magnus and Stefan. cheers andrew
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-10T10:49:51Z
On Thu, May 10, 2012 at 12:43 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > On 05/10/2012 01:29 AM, Tom Lane wrote: >> >> Bruce Momjian<bruce@momjian.us> writes: >>> >>> The docs finally built 90 minutes after my commit, and the URL above is >>> now working. (Does it always take this long to update?) >> >> I believe the new implementation of that stuff is that the devel docs >> are built whenever the buildfarm member guaibasaurus runs for HEAD, >> which it seems to do on an hourly schedule. This is definitely not as >> fast-responding as Peter's former custom script, but I'm not sure if >> it's worth thinking of another way. >> > > I don't see any reason it can't run more frequently, though. Currently a run > takes 15 minutes or so. We could reduce that by making it skip some steps, > and get it down to about 10 minutes. It would be perfectly reasonable to run > every 5 minutes (it won't schedule concurrent runs - if the lock file is > held by another run it exits gracefully). Of course, that's up to Magnus and > Stefan. If we can make it do *just* the docs, we can certainly run it a bit more often. But we don't want to make it run the full set of checks more or less continously, since the machine is shared with a number of other tasks... I don't think 5 minutes is anywhere near necessary even for the docs, but there is a lot of room between 5 minutes and 4 hours, so we can definitely shorten it. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T11:19:53Z
On 05/10/2012 06:49 AM, Magnus Hagander wrote: > On Thu, May 10, 2012 at 12:43 PM, Andrew Dunstan<andrew@dunslane.net> wrote: >> >> On 05/10/2012 01:29 AM, Tom Lane wrote: >>> Bruce Momjian<bruce@momjian.us> writes: >>>> The docs finally built 90 minutes after my commit, and the URL above is >>>> now working. (Does it always take this long to update?) >>> I believe the new implementation of that stuff is that the devel docs >>> are built whenever the buildfarm member guaibasaurus runs for HEAD, >>> which it seems to do on an hourly schedule. This is definitely not as >>> fast-responding as Peter's former custom script, but I'm not sure if >>> it's worth thinking of another way. >>> >> I don't see any reason it can't run more frequently, though. Currently a run >> takes 15 minutes or so. We could reduce that by making it skip some steps, >> and get it down to about 10 minutes. It would be perfectly reasonable to run >> every 5 minutes (it won't schedule concurrent runs - if the lock file is >> held by another run it exits gracefully). Of course, that's up to Magnus and >> Stefan. > If we can make it do *just* the docs, we can certainly run it a bit > more often. But we don't want to make it run the full set of checks > more or less continously, since the machine is shared with a number of > other tasks... > > I don't think 5 minutes is anywhere near necessary even for the docs, > but there is a lot of room between 5 minutes and 4 hours, so we can > definitely shorten it. > If you only want a docs build then a buildfarm animal is probably not a good choice. Do you want to divorce that from building validated snapshots? BTW, if there has been no change a buildfarm animal normally does no work (other than a git pull followed by the check for updates), which is why it's often safe to schedule it very frequently. However, if you need to schedule tasks at times when it's known not to be running then a sparse schedule makes sense. cheers andrew
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T11:25:35Z
On 05/09/2012 11:11 PM, Bruce Momjian wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. I am waiting for our development docs to build, but after 40 > minutes, I am still waiting: > > http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=guaibasaurus&dt=latest&stg=make-doc > > (Why is there no time zone shown in the date/time at the top?) I think > it will eventually show up here: > > http://www.postgresql.org/docs/devel/static/release-9-2.html > > My private build is now online: > > http://momjian.us/tmp/pgsql/release-9-2.html > > I tested creation of the HISTORY file so Tom shouldn't need to fix > missing markup tomorrow. :-) > This has broken my docs build because of this line: release-9.2.sgml:1946: Urbańnski, Steve Singer) with this error: openjade:/home/bf/bfr/root/HEAD/pgsql.9367/../pgsql/doc/src/sgml/release-9.2.sgml:1946:14:E: "324" is not a character number in the document character set cheers andrew -
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-10T12:11:54Z
On 10 May 2012 04:11, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. I am waiting for our development docs to build, but after 40 > minutes, I am still waiting: "Allow the bgwriter, walwriter, and statistics collector to sleep more efficiently during periods of inactivity (Peter Geoghegan, Heikki Linnakangas, Tom Lane)...This reduces CPU wake-ups." I think that there should be mention of why this is a good thing. When fully idle the server reaches less than a single wake-up per second, which I think is a nice, relevant fact. You should add the archiver and checkpointer to that list, though I suppose you could argue that the checkpointer, as a "new" auxiliary process, shouldn't count. I'm not really sure why you've listed Daniel Farina as a co-author of the pg_stat_statements normalisation feature. He did a good job of reviewing it, but he didn't actually contribute any code. Why can't we call group commit group commit (and for that matter, index-only scans index-only scans), so that people will understand that we are now competitive with other RDBMSs in this area? "Improve performance of WAL writes when multiple transactions commit at the same time" seems like a pretty bad description, since it doesn't make any reference to batching of commits. Also, I don't think that the placement of this as the second to last performance feature is commensurate with its actual importance. Still, the actual major feature list is a much more relevant indicator of how it is felt that individual features should be weighed, and of course that hasn't been decided upon yet. "Change pg_stat_statements' total_time column to be measured in milliseconds (Tom Lane)". Surely this should be under "pg_stat_statements"? Does "Make the visibility map crash-safe" really belong under "Performance"? It's not clear that this isn't just within comments that will be later removed, but I'd remove all references to "we". -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T12:24:11Z
On 05/10/2012 08:11 AM, Peter Geoghegan wrote: > I'm not really sure why you've listed Daniel Farina as a co-author of > the pg_stat_statements normalisation feature. He did a good job of > reviewing it, but he didn't actually contribute any code. It looks like reviewers have been given credit throughout. cheers andrew
-
Re: Draft release notes complete
Simon Riggs <simon@2ndquadrant.com> — 2012-05-10T12:25:54Z
On 10 May 2012 13:11, Peter Geoghegan <peter@2ndquadrant.com> wrote: > Why can't we call group commit group commit (and for that matter, > index-only scans index-only scans), so that people will understand > that we are now competitive with other RDBMSs in this area? "Improve > performance of WAL writes when multiple transactions commit at the > same time" seems like a pretty bad description, since it doesn't make > any reference to batching of commits. Also, I don't think that the > placement of this as the second to last performance feature is > commensurate with its actual importance. Agreed. Group Commit is a recognised term and also one where other DBMS, e.g. Marea have included that feature recently. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Vik Reykja <vikreykja@gmail.com> — 2012-05-10T12:28:39Z
On Thu, May 10, 2012 at 2:24 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > On 05/10/2012 08:11 AM, Peter Geoghegan wrote: > >> I'm not really sure why you've listed Daniel Farina as a co-author of the >> pg_stat_statements normalisation feature. He did a good job of reviewing >> it, but he didn't actually contribute any code. >> > > > It looks like reviewers have been given credit throughout. > Which could be good incentive to become more involved in reviewing for some people.
-
Re: Draft release notes complete
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2012-05-10T12:43:02Z
On 10.05.2012 13:21, Thom Brown wrote: > On 10 May 2012 04:11, Bruce Momjian<bruce@momjian.us> wrote: >> I have completed my draft of the 9.2 release notes, and committed it to >> git. > ... > > Couple typo corrections attached. Applied. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T12:45:21Z
On 05/10/2012 08:28 AM, Vik Reykja wrote: > On Thu, May 10, 2012 at 2:24 PM, Andrew Dunstan <andrew@dunslane.net > <mailto:andrew@dunslane.net>> wrote: > > > > On 05/10/2012 08:11 AM, Peter Geoghegan wrote: > > I'm not really sure why you've listed Daniel Farina as a > co-author of the pg_stat_statements normalisation feature. He > did a good job of reviewing it, but he didn't actually > contribute any code. > > > > It looks like reviewers have been given credit throughout. > > > Which could be good incentive to become more involved in reviewing for > some people. Right, but I think it would be good to identify them explicitly as reviewers if we're going to include the names. Otherwise it could enable people to claim authorship of something they did not in fact author, and even without that would dilute the claim to authorship of the actual author(s). cheers andrew
-
Re: Draft release notes complete
Josh Kupershmidt <schmiddy@gmail.com> — 2012-05-10T12:57:01Z
On Wed, May 9, 2012 at 8:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. I am waiting for our development docs to build, but after 40 > minutes, I am still waiting: This bit: Previously supplied years and year masks of less than four digits wrapped inconsistently. I first read as "Previously-supplied years..." instead of "Previously, years and year masks with...". In line with what Robert said, IMO he should be credited on pg_opfamily_is_visible(), particularly since it was his idea and code originally IIRC. And a few more I'm familiar with with, such as psql's \ir which he substantially revised, not to mention his much-appreciated assistance with all the psql comment-displaying under 'E.1.3.9.2.1. Comments'. Josh
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@commandprompt.com> — 2012-05-10T13:05:10Z
Excerpts from Andrew Dunstan's message of jue may 10 07:19:53 -0400 2012: > BTW, if there has been no change a buildfarm animal normally does no > work (other than a git pull followed by the check for updates), which is > why it's often safe to schedule it very frequently. However, if you need > to schedule tasks at times when it's known not to be running then a > sparse schedule makes sense. Magnus was trying to say that the physical machine has other VMs doing unrelated stuff, not that the BF animal VM itself had other things to do. -- Álvaro Herrera <alvherre@commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-10T13:12:57Z
On 10 May 2012 13:45, Andrew Dunstan <andrew@dunslane.net> wrote: > Right, but I think it would be good to identify them explicitly as reviewers > if we're going to include the names. +1. I think we should probably do more to credit reviewers. It's not uncommon for a reviewer to end up becoming a co-author, particularly if they're a committer, but it's a little misleading to add a reviewer after the feature description without qualifying that they are the reviewer. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@commandprompt.com> — 2012-05-10T13:20:32Z
Excerpts from Peter Geoghegan's message of jue may 10 09:12:57 -0400 2012: > On 10 May 2012 13:45, Andrew Dunstan <andrew@dunslane.net> wrote: > > Right, but I think it would be good to identify them explicitly as reviewers > > if we're going to include the names. > > +1. I think we should probably do more to credit reviewers. It's not > uncommon for a reviewer to end up becoming a co-author, particularly > if they're a committer, but it's a little misleading to add a reviewer > after the feature description without qualifying that they are the > reviewer. Agreed. What about crediting patch sponsors (other than the author's employer, I mean)? I remember crediting one in a commit message and being told it wasn't okay. Is it okay to credit them in the release notes? -- Álvaro Herrera <alvherre@commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T14:42:28Z
On Thu, May 10, 2012 at 9:12 AM, Peter Geoghegan <peter@2ndquadrant.com> wrote: > On 10 May 2012 13:45, Andrew Dunstan <andrew@dunslane.net> wrote: >> Right, but I think it would be good to identify them explicitly as reviewers >> if we're going to include the names. > > +1. I think we should probably do more to credit reviewers. It's not > uncommon for a reviewer to end up becoming a co-author, particularly > if they're a committer, but it's a little misleading to add a reviewer > after the feature description without qualifying that they are the > reviewer. Right. Plus Bruce has arbitrarily excluded committer-reviewers even when they substantially revised the patch as part of that review, and included non-committer-reviewers even when they did little more than say "good idea, +1". There are patches on that list where I did A LOT of work and am not credited, including some where other people did get credited for much less work. I don't feel a crying need to be credited on the maximum possible number of items, but it seems weird to see one group of people credited for what may well have been an hour's work while another group of people isn't credited even when they did two or three days worth of work. When we did the 9.1 release notes, reviewers weren't credited, and I sort of assumed that policy would be the same this time around. I also sort of assumed that the committer would be credited if the commit message stated that they had done substantial further work on the patch, but not if it said that they'd only done a little bit of work or none. Honestly, I don't really care what the standard for inclusion is, but it's so glaringly non-uniform right now that it really makes no sense. I think my own personal preference would be to remove all the reviewer names from individual items and list only the people who contributed significantly to the code, and then have a section at the bottom where we credit all the reviewers without reference to specific patches. Or maybe we should just remove all the names from the release notes, full stop, since it's pretty clear that we're on the verge of having the names take up more space than the items to which they refer. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T14:44:17Z
On Thu, May 10, 2012 at 09:20:32AM -0400, Alvaro Herrera wrote: > > Excerpts from Peter Geoghegan's message of jue may 10 09:12:57 -0400 2012: > > On 10 May 2012 13:45, Andrew Dunstan <andrew@dunslane.net> wrote: > > > Right, but I think it would be good to identify them explicitly as reviewers > > > if we're going to include the names. > > > > +1. I think we should probably do more to credit reviewers. It's not > > uncommon for a reviewer to end up becoming a co-author, particularly > > if they're a committer, but it's a little misleading to add a reviewer > > after the feature description without qualifying that they are the > > reviewer. > > Agreed. > > What about crediting patch sponsors (other than the author's employer, I > mean)? I remember crediting one in a commit message and being told it > wasn't okay. Is it okay to credit them in the release notes? No. We discussed crediting companies in the release notes, and that was agreed to be a bad idea, I think because the release notes live for so long, and because the release notes would end up being an advertisement. Can you imagine all our employers saying we should get their name into the release notes more? The big take-away is that the release notes are mostly for blame and to designate a go-to person for feature problems, not for giving credit, and especially not for company credit. There are just too many people reading those release notes for that to happen, and if we are going to go any direction, it would be to remove user names completely from the release notes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T15:04:47Z
Robert Haas <robertmhaas@gmail.com> writes: > When we did the 9.1 release notes, reviewers weren't credited, and I > sort of assumed that policy would be the same this time around. Yes. This seems to be a policy change that was made without notice or discussion, and I personally don't find it to be a good idea. I think the release notes should only credit the primary author(s) of a feature. Face it, most people don't care about that, so we should not be expending much space on it. regards, tom lane
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:16:02Z
On Thu, May 10, 2012 at 11:04:47AM -0400, Tom Lane wrote: > Robert Haas <robertmhaas@gmail.com> writes: > > When we did the 9.1 release notes, reviewers weren't credited, and I > > sort of assumed that policy would be the same this time around. > > Yes. This seems to be a policy change that was made without notice or > discussion, and I personally don't find it to be a good idea. I think > the release notes should only credit the primary author(s) of a feature. > Face it, most people don't care about that, so we should not be > expending much space on it. Agreed on just using the primary author. The first name is _always_ the primary author, so we can just go with that. I didn't want to do: (Tom Lane, Robert Haas; reviewers Bruce Momjian, Jeff Davis) That was too complicated. Should I make the change now? It is easy. Should we remove the names completely? We can consider going to a single name as a move toward removing names evantually. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:21:45Z
On Thu, May 10, 2012 at 07:20:51AM +0200, Erik Rijkers wrote: > On Thu, May 10, 2012 06:33, Bruce Momjian wrote: > > On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: > >> > >> http://www.postgresql.org/docs/devel/static/release-9-2.html > > > > To "E.1.2.5. Monitoring" should be added: > > "Rename pg_stat_activity.current_query to query (Magnus Hagander)" > > > > And perhaps (same paragraph): > > "The previous query values are preserved, allowing for enhanced analysis." > > would be clearer as: > > "The last query values are preserved, allowing for enhanced analysis." Thanks for the feedback. Adjustments made with the attached patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T15:23:11Z
Bruce Momjian <bruce@momjian.us> writes: > Should I make the change now? It is easy. Yes. > Should we remove the names completely? That would be a policy change too, and one that probably requires more leisurely consideration than we have time for today. regards, tom lane
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:24:39Z
On Thu, May 10, 2012 at 12:49:51PM +0200, Magnus Hagander wrote: > On Thu, May 10, 2012 at 12:43 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > > > > On 05/10/2012 01:29 AM, Tom Lane wrote: > >> > >> Bruce Momjian<bruce@momjian.us> writes: > >>> > >>> The docs finally built 90 minutes after my commit, and the URL above is > >>> now working. (Does it always take this long to update?) > >> > >> I believe the new implementation of that stuff is that the devel docs > >> are built whenever the buildfarm member guaibasaurus runs for HEAD, > >> which it seems to do on an hourly schedule. This is definitely not as > >> fast-responding as Peter's former custom script, but I'm not sure if > >> it's worth thinking of another way. > >> > > > > I don't see any reason it can't run more frequently, though. Currently a run > > takes 15 minutes or so. We could reduce that by making it skip some steps, > > and get it down to about 10 minutes. It would be perfectly reasonable to run > > every 5 minutes (it won't schedule concurrent runs - if the lock file is > > held by another run it exits gracefully). Of course, that's up to Magnus and > > Stefan. > > If we can make it do *just* the docs, we can certainly run it a bit > more often. But we don't want to make it run the full set of checks > more or less continously, since the machine is shared with a number of > other tasks... > > I don't think 5 minutes is anywhere near necessary even for the docs, > but there is a lot of room between 5 minutes and 4 hours, so we can > definitely shorten it. Do you want me to just setup a build on my machine like we did before; 5 minutes is no problem for me. I use the doc build to show patch submitters what their final work looks like, and anything more than a few minutes delay makes that useless. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T15:26:14Z
On Thu, May 10, 2012 at 11:16 AM, Bruce Momjian <bruce@momjian.us> wrote: >> Yes. This seems to be a policy change that was made without notice or >> discussion, and I personally don't find it to be a good idea. I think >> the release notes should only credit the primary author(s) of a feature. >> Face it, most people don't care about that, so we should not be >> expending much space on it. > > Agreed on just using the primary author. The first name is _always_ the > primary author, so we can just go with that. I didn't want to do: > > (Tom Lane, Robert Haas; reviewers Bruce Momjian, Jeff Davis) > > That was too complicated. > > Should I make the change now? It is easy. Should we remove the names > completely? We can consider going to a single name as a move toward > removing names evantually. There are some cases, like index-only scans, where I think it would be very hard to get down to one name, because four different people wrote code that ended up being part of that. Now you could probably get it down to just two by cutting Heikki (who isn't listed) and Ibrar (who is) but saying that only one of Tom and I did that feature would be quite misleading regardless of who you picked. Similarly, there are a couple of patches that I worked on with Simon where crediting only one of us would be wrong, regardless of which one you picked, and I think there are other cases of this involving other people as well. So I think a hard and fast rule of crediting exactly one person is not going to work, but limiting it to the primary author or authors is feasible. Honestly, I'm leaning more and more toward the view that we should just rip the names out entirely. I mean, look at something like sortsupport. That would never have gotten done without Peter Geoghegan's work on it, but the code *as committed* was half mine and half Tom's. So what are you going to do with that? It's weird to credit Peter and not Tom or I, and it's weird to credit Tom or I and not Peter, and it's even weird of you credit all three of us because any decision about who to put first is arguable and maybe wrong. The simplest solution to my mind is to credit no one, which at least has the advantage of being unarguably uniform. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-10T15:31:15Z
On May 10, 2012 5:24 PM, "Bruce Momjian" <bruce@momjian.us> wrote: > > On Thu, May 10, 2012 at 12:49:51PM +0200, Magnus Hagander wrote: > > On Thu, May 10, 2012 at 12:43 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > > > > > > > On 05/10/2012 01:29 AM, Tom Lane wrote: > > >> > > >> Bruce Momjian<bruce@momjian.us> writes: > > >>> > > >>> The docs finally built 90 minutes after my commit, and the URL above is > > >>> now working. (Does it always take this long to update?) > > >> > > >> I believe the new implementation of that stuff is that the devel docs > > >> are built whenever the buildfarm member guaibasaurus runs for HEAD, > > >> which it seems to do on an hourly schedule. This is definitely not as > > >> fast-responding as Peter's former custom script, but I'm not sure if > > >> it's worth thinking of another way. > > >> > > > > > > I don't see any reason it can't run more frequently, though. Currently a run > > > takes 15 minutes or so. We could reduce that by making it skip some steps, > > > and get it down to about 10 minutes. It would be perfectly reasonable to run > > > every 5 minutes (it won't schedule concurrent runs - if the lock file is > > > held by another run it exits gracefully). Of course, that's up to Magnus and > > > Stefan. > > > > If we can make it do *just* the docs, we can certainly run it a bit > > more often. But we don't want to make it run the full set of checks > > more or less continously, since the machine is shared with a number of > > other tasks... > > > > I don't think 5 minutes is anywhere near necessary even for the docs, > > but there is a lot of room between 5 minutes and 4 hours, so we can > > definitely shorten it. > > Do you want me to just setup a build on my machine like we did before; > 5 minutes is no problem for me. > > I use the doc build to show patch submitters what their final work looks > like, and anything more than a few minutes delay makes that useless. > Anything that runs off the main git repo would be useless there, since it would never show up prior to commit. If people want the main docs building more often that's not really a problem other than time - we just need to decouple it from the buildfarm and run a separate job for it. It's not rocket science.. /Magnus
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:32:18Z
On Thu, May 10, 2012 at 11:26:14AM -0400, Robert Haas wrote: > There are some cases, like index-only scans, where I think it would be > very hard to get down to one name, because four different people wrote > code that ended up being part of that. Now you could probably get it > down to just two by cutting Heikki (who isn't listed) and Ibrar (who > is) but saying that only one of Tom and I did that feature would be > quite misleading regardless of who you picked. Similarly, there are a > couple of patches that I worked on with Simon where crediting only one > of us would be wrong, regardless of which one you picked, and I think > there are other cases of this involving other people as well. So I > think a hard and fast rule of crediting exactly one person is not > going to work, but limiting it to the primary author or authors is > feasible. > > Honestly, I'm leaning more and more toward the view that we should > just rip the names out entirely. I mean, look at something like > sortsupport. That would never have gotten done without Peter > Geoghegan's work on it, but the code *as committed* was half mine and > half Tom's. So what are you going to do with that? It's weird to > credit Peter and not Tom or I, and it's weird to credit Tom or I and > not Peter, and it's even weird of you credit all three of us because > any decision about who to put first is arguable and maybe wrong. The > simplest solution to my mind is to credit no one, which at least has > the advantage of being unarguably uniform. Keep in mind that the reason I originally had names in the release notes was so I could remember who to email when something broke. That really isn't the case anymore. I agree that making these names give _credit_ is never going to work. Robert's example above is very clear on that. We could try cutting it down to one name and see if we have any problems with it. Robert is right that if you are thinking of this as "credit" it is never going to work. We will need to make some decision in the next few hours. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:34:34Z
On Thu, May 10, 2012 at 05:31:15PM +0200, Magnus Hagander wrote: > > I use the doc build to show patch submitters what their final work looks > > like, and anything more than a few minutes delay makes that useless. > > > > Anything that runs off the main git repo would be useless there, since it would > never show up prior to commit. I will commit something then send them a URL saying, "Hey, committed, look here for the results." > If people want the main docs building more often that's not really a problem > other than time - we just need to decouple it from the buildfarm and run a > separate job for it. It's not rocket science.. I do think we need to do that. The release note publication was stalled for 90 minutes just in this one case. The docs are still hard enough to build that I can imagine others would like to have quick feedback. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T15:45:08Z
Bruce Momjian <bruce@momjian.us> writes: > On Thu, May 10, 2012 at 11:26:14AM -0400, Robert Haas wrote: >> Honestly, I'm leaning more and more toward the view that we should >> just rip the names out entirely. > We will need to make some decision in the next few hours. I think this is a delicate question and we should *not* make a hasty decision. The release notes are almost certainly going to get worked over quite a bit between now and 9.2 final; there is no need to assume that the beta1 version has to reflect a final decision. I'd vote for starting a separate thread to solicit people's opinions on whether we need names in the release notes. Is there anybody on -hackers who would be offended, or would have a harder time persuading $BOSS to let them spend time on Postgres if they weren't mentioned in the release notes? There'd still be a policy of crediting people in commit messages of course, but it's not clear to me whether the release note mentions are important to anybody. regards, tom lane
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T15:46:20Z
On 05/10/2012 11:24 AM, Bruce Momjian wrote: > On Thu, May 10, 2012 at 12:49:51PM +0200, Magnus Hagander wrote: >> On Thu, May 10, 2012 at 12:43 PM, Andrew Dunstan<andrew@dunslane.net> wrote: >>> >>> On 05/10/2012 01:29 AM, Tom Lane wrote: >>>> Bruce Momjian<bruce@momjian.us> writes: >>>>> The docs finally built 90 minutes after my commit, and the URL above is >>>>> now working. (Does it always take this long to update?) >>>> I believe the new implementation of that stuff is that the devel docs >>>> are built whenever the buildfarm member guaibasaurus runs for HEAD, >>>> which it seems to do on an hourly schedule. This is definitely not as >>>> fast-responding as Peter's former custom script, but I'm not sure if >>>> it's worth thinking of another way. >>>> >>> I don't see any reason it can't run more frequently, though. Currently a run >>> takes 15 minutes or so. We could reduce that by making it skip some steps, >>> and get it down to about 10 minutes. It would be perfectly reasonable to run >>> every 5 minutes (it won't schedule concurrent runs - if the lock file is >>> held by another run it exits gracefully). Of course, that's up to Magnus and >>> Stefan. >> If we can make it do *just* the docs, we can certainly run it a bit >> more often. But we don't want to make it run the full set of checks >> more or less continously, since the machine is shared with a number of >> other tasks... >> >> I don't think 5 minutes is anywhere near necessary even for the docs, >> but there is a lot of room between 5 minutes and 4 hours, so we can >> definitely shorten it. > Do you want me to just setup a build on my machine like we did before; > 5 minutes is no problem for me. > > I use the doc build to show patch submitters what their final work looks > like, and anything more than a few minutes delay makes that useless. > It's been done the current way for quite a few months now. If you're only noticing it now is it really such an inconvenience? Having said that, I'm not at all opposed to reducing the lag time. cheers andrew
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T15:54:34Z
On Thu, May 10, 2012 at 11:46:20AM -0400, Andrew Dunstan wrote: > >>I don't think 5 minutes is anywhere near necessary even for the docs, > >>but there is a lot of room between 5 minutes and 4 hours, so we can > >>definitely shorten it. > >Do you want me to just setup a build on my machine like we did before; > >5 minutes is no problem for me. > > > >I use the doc build to show patch submitters what their final work looks > >like, and anything more than a few minutes delay makes that useless. > > > > It's been done the current way for quite a few months now. If you're > only noticing it now is it really such an inconvenience? Having said > that, I'm not at all opposed to reducing the lag time. Well, I am not applying doc patches much anymore; my point is that the first time I actually need to send out a URL of the docs, it wasn't sufficient for me. Yes, I can work around it, but it that what we want everyone to do? I don't remember this change being discussed anywhere I see. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T15:54:36Z
On 05/10/2012 11:32 AM, Bruce Momjian wrote: > On Thu, May 10, 2012 at 11:26:14AM -0400, Robert Haas wrote: >> There are some cases, like index-only scans, where I think it would be >> very hard to get down to one name, because four different people wrote >> code that ended up being part of that. Now you could probably get it >> down to just two by cutting Heikki (who isn't listed) and Ibrar (who >> is) but saying that only one of Tom and I did that feature would be >> quite misleading regardless of who you picked. Similarly, there are a >> couple of patches that I worked on with Simon where crediting only one >> of us would be wrong, regardless of which one you picked, and I think >> there are other cases of this involving other people as well. So I >> think a hard and fast rule of crediting exactly one person is not >> going to work, but limiting it to the primary author or authors is >> feasible. >> >> Honestly, I'm leaning more and more toward the view that we should >> just rip the names out entirely. I mean, look at something like >> sortsupport. That would never have gotten done without Peter >> Geoghegan's work on it, but the code *as committed* was half mine and >> half Tom's. So what are you going to do with that? It's weird to >> credit Peter and not Tom or I, and it's weird to credit Tom or I and >> not Peter, and it's even weird of you credit all three of us because >> any decision about who to put first is arguable and maybe wrong. The >> simplest solution to my mind is to credit no one, which at least has >> the advantage of being unarguably uniform. > Keep in mind that the reason I originally had names in the release notes > was so I could remember who to email when something broke. That really > isn't the case anymore. > > I agree that making these names give _credit_ is never going to work. > Robert's example above is very clear on that. > > We could try cutting it down to one name and see if we have any problems > with it. Robert is right that if you are thinking of this as "credit" > it is never going to work. > I don't really buy this at all. The fact that it's not perfect doesn't mean that it's wrong. Just about the only reward we give contributors is some kudos, and the more the better as far as I'm concerned. I'd almost like to see a "Credits" section of the release notes, but if we're not going to have that let's keep doing what we have been doing. cheers andrew
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T16:15:57Z
On Thu, May 10, 2012 at 11:54:36AM -0400, Andrew Dunstan wrote: > >We could try cutting it down to one name and see if we have any problems > >with it. Robert is right that if you are thinking of this as "credit" > >it is never going to work. > > > > > I don't really buy this at all. The fact that it's not perfect > doesn't mean that it's wrong. Just about the only reward we give > contributors is some kudos, and the more the better as far as I'm > concerned. I'd almost like to see a "Credits" section of the release > notes, but if we're not going to have that let's keep doing what we > have been doing. Well, the new change is that we now are listing reviewers, but frankly, we are doing a lot more collaborative work than we have in the past, meaning there is an increase in the number of names in this release, and it has been steadily growing even if we don't include the reviewers. I think the names are a balance between the release notes looking trim and professional, and giving credit to individuals, however imperfect. I think giving credit to companies is going too far away from trim/professional, and I think most agree on that. (I have suggested the company names belong mostly in the release announcement.) The question is how do we handle the explosion of names, and does it still look trim/professional? I think we are probably on the far edge of that with the 9.2 release notes. Putting names at the bottom or a "Credit" section just seems also too far away from trim/professional because there is no procedural reason for the names, and they are literally listed as "Credit". Let me also add I am embarassed at the number of pg_upgrade release note items with my name on them. They are all user-visible changes, so should be listed, but does having 9 pg_upgrade items out of 245 (4%) seem fair credit-wise? No. There's another unfair example. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T16:24:10Z
Andrew Dunstan <andrew@dunslane.net> writes: > This has broken my docs build because of this line: > release-9.2.sgml:1946: Urbańnski, Steve Singer) > with this error: > openjade:/home/bf/bfr/root/HEAD/pgsql.9367/../pgsql/doc/src/sgml/release-9.2.sgml:1946:14:E: "324" is not a character number in the document character set I get the same, and so do some of the buildfarm members. I've changed the text and added a note to release.sgml specifying not to use numeric character entities. regards, tom lane
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-10T16:28:35Z
On tor, 2012-05-10 at 17:31 +0200, Magnus Hagander wrote: > If people want the main docs building more often that's not really a > problem other than time - we just need to decouple it from the > buildfarm and run a separate job for it. It's not rocket science.. Many years ago, Bruce and myself in particular put in a lot of work to make the turnaround time on the docs build less than 5 minutes, based on various requests. I'm disappointed to learn that that was abandoned without discussion. We might as well just put the old job back.
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T16:34:36Z
On Thu, May 10, 2012 at 12:24:10PM -0400, Tom Lane wrote: > Andrew Dunstan <andrew@dunslane.net> writes: > > This has broken my docs build because of this line: > > > release-9.2.sgml:1946: Urbańnski, Steve Singer) > > > with this error: > > > openjade:/home/bf/bfr/root/HEAD/pgsql.9367/../pgsql/doc/src/sgml/release-9.2.sgml:1946:14:E: "324" is not a character number in the document character set > > I get the same, and so do some of the buildfarm members. I've changed > the text and added a note to release.sgml specifying not to use numeric > character entities. I does build on my Debian Squeeze toolchain and does render right, but I was very concerned about its use because I saw it referenced in only one URL: http://webdesign.about.com/library/bl_htmlcodes.htm I needed 'n' with an accent. I have remove that URL from release.sgml. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-10T16:40:29Z
On tor, 2012-05-10 at 12:24 -0400, Tom Lane wrote: > > > openjade:/home/bf/bfr/root/HEAD/pgsql.9367/../pgsql/doc/src/sgml/release-9.2.sgml:1946:14:E: "324" is not a character number in the document character set > > I get the same, and so do some of the buildfarm members. I've changed > the text and added a note to release.sgml specifying not to use > numeric character entities. The problem is not using numeric character entities, it's using a character not in the document character set, which is Latin 1.
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-10T16:44:06Z
On tor, 2012-05-10 at 10:44 -0400, Bruce Momjian wrote: > The big take-away is that the release notes are mostly for blame and > to designate a go-to person for feature problems, not for giving > credit, Then reviewers should be removed.
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T16:55:13Z
On Thu, May 10, 2012 at 07:40:29PM +0300, Peter Eisentraut wrote: > On tor, 2012-05-10 at 12:24 -0400, Tom Lane wrote: > > > > > openjade:/home/bf/bfr/root/HEAD/pgsql.9367/../pgsql/doc/src/sgml/release-9.2.sgml:1946:14:E: "324" is not a character number in the document character set > > > > I get the same, and so do some of the buildfarm members. I've changed > > the text and added a note to release.sgml specifying not to use > > numeric character entities. > > The problem is not using numeric character entities, it's using a > character not in the document character set, which is Latin 1. That's what I suspected. I now see they were saying you have to define the charset as Unicode, which we can't do. I updated the docs again to explain that. Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-10T16:55:37Z
On 5/10/12 9:44 AM, Peter Eisentraut wrote: > On tor, 2012-05-10 at 10:44 -0400, Bruce Momjian wrote: >> The big take-away is that the release notes are mostly for blame and >> to designate a go-to person for feature problems, not for giving >> credit, > > Then reviewers should be removed. I disagree. We're trying to get more reviewers, and encourage them to do more reviewing. Giving credit is a big part of that. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:02:34Z
On Thu, May 10, 2012 at 01:56:33AM -0400, Robert Haas wrote: > On Wed, May 9, 2012 at 11:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > > I have completed my draft of the 9.2 release notes, and committed it to > > git. > > Extra parens: > Remove the spclocation field from pg_tablespace (Magnus Hagander, Tom Lane)) > Reduce overhead of creating virtual transaction id locks ((Robert > Haas, Jeff Davis) Done. > The antecedent of "these" is unclear: > Allow backends to detect postmaster death via a pipe read failure, > rather than polling (Peter Geoghegan, Heikki Linnakangas) > These are internally called "latches". Fixed. > Missing comma: > Cancel queries if clients get disconnected (Florian Pflug Greg Jaskiewicz) Fixed by some else. > You mean "effect": > Such casts have no affect. Fixed. > > I think all three of these are the same thing: > Avoid table and index rebuilds when NUMERIC, VARBIT, and temporal > columns are changed in compatible ways (Noah Misch) > Reduce need to rebuild indexes for various ALTER TABLE operations > (Noah Misch) DUPLICATE? > Avoid index rebuilds for no-rewrite ALTER TABLE / ALTER TYPE (Noah Misch) Agreed, duplicates removed. > This feature wasn't committed at all: > Parallel pg_dump (Robert Haas, Joachim Wieland) DETAILS? I was confused because there were infrastructure commits menting the feature, so I thought it was lost somehow. > Yes, this is still true: > This is currently unused. STILL TRUE? OK, fixed. The attached patch includes all these fixes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:03:36Z
On Thu, May 10, 2012 at 01:56:33AM -0400, Robert Haas wrote: > As a general comment, I think that your new policy of crediting the > reviewer on every feature except when that reviewer is also a > committer has produced a horrific mess. Just to pick one of many > examples, consider this item: > > Add a security_barrier option for views (KaiGai Kohei, Noah Misch) > > Here is what the commit message says: > > Patch by KaiGai Kohei; original problem report by Heikki Linnakangas > (in October 2009!). Review (in earlier versions) by Noah Misch and > others. Design advice by Tom Lane and myself. Further review and > cleanup by me. > > So there are four people mentioned in this commit message, and you've > picked out two of them to credit, not on the basis of who did the most > work, but rather on the basis of which ones happen to not be > committers. The result is that, as I read through these release > notes, one gets what I believe to be a very misleading notion of who > developed which features. I don't object to not being credited on > this one, but I don't think it makes sense to credit Noah and NOT > credit me. As you have it, people who did little more than say "yep, > looks fine to me" are credited almost equally with the people who > wrote the code, while a committer who heavily revised the patch may > not be mentioned at all, or sometimes (seemingly at random) they are. I assumed reviewers mentioned in the commit messages made substantive suggestions on improving the patch, rather than just +1. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:14:00Z
On Thu, May 10, 2012 at 10:50:14AM +0300, Heikki Linnakangas wrote: > On 10.05.2012 06:11, Bruce Momjian wrote: > >I have completed my draft of the 9.2 release notes, and committed it to > >git. > > Thanks! I committed a few trivial fixes, below are a few more I > wasn't sure about: > > >* Add support for range data types (Jeff Davis, Tom Lane, Alexander Korotkov) > > > >The range data type records a lower and upper bound, and supports comparisons like contains, overlaps, and intersection. > > /s/comparisons/operations/ ? > > >* Allow a user to cancel queries in other owned sessions using pg_cancel_backend() (Magnus Hagander) > > > >Previously only the superuser could cancel queries. > > "other owned sessions" is a bit ambiguous. It reads to me like > "possessed sessions" or "0wned sessions". It's not clear it means > sessions owned by the same user. How about "... to cancel queries in > his other sessions, using ..." ? Or: > > * Allow a non-superuser to cancel queries in another backend using > pg_cancel_backend(), as long as the victim backend belongs to the > same user > > Previously only the superuser could cancel queries. > > > >* Change default names of triggers to fire action triggers before check triggers (Tom Lane) > > > >This allows default-named check triggers to check post-action rows. > > That's quite a mouthful :-). I don't understand what it means. > > >In psql tab completion, complete SQL key words based on COMP_KEYWORD_CASE setting and the perhaps case of the partially-supplied word (Peter Eisentraut, Fujii Masao) > > Which is correct spelling, "keyword" or "key word"? We seem to use > both in the docs. "Keyword" sounds much better to me, but I think I > might be more prone to write words together than native English > speakers. I have made adjustments based on your comments in the attached, applied patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:16:13Z
On Thu, May 10, 2012 at 12:18:08PM +0200, Magnus Hagander wrote: > On Thu, May 10, 2012 at 5:11 AM, Bruce Momjian <bruce@momjian.us> wrote: > > (Why is there no time zone shown in the date/time at the top?) I think > > it will eventually show up here: > > > > http://www.postgresql.org/docs/devel/static/release-9-2.html > > > > Other than the comments others have specified: > > * Add libpq parameters for specifying the locations of server-side SSL > files (Peter Eisentraut) > > Those are regular server side gucs and not libpq parameters. You > certainly can't control the location of server-side files with libpq > parameters.. But imagine how much fun we could have if we could! Anyway, fixed. ;-) Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:21:07Z
On Thu, May 10, 2012 at 09:55:37AM -0700, Josh Berkus wrote: > On 5/10/12 9:44 AM, Peter Eisentraut wrote: > > On tor, 2012-05-10 at 10:44 -0400, Bruce Momjian wrote: > >> The big take-away is that the release notes are mostly for blame and > >> to designate a go-to person for feature problems, not for giving > >> credit, > > > > Then reviewers should be removed. > > I disagree. We're trying to get more reviewers, and encourage them to > do more reviewing. Giving credit is a big part of that. OK, we are officially now "all over the map" on this! I did favor reviewers over committers for the purpose of encouragement. I am fine with anything that has the same or fewer names than we have now. I don't think anyone is arguing for more names, so at least we have a direction. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-10T17:29:30Z
On tor, 2012-05-10 at 09:55 -0700, Josh Berkus wrote: > On 5/10/12 9:44 AM, Peter Eisentraut wrote: > > On tor, 2012-05-10 at 10:44 -0400, Bruce Momjian wrote: > >> The big take-away is that the release notes are mostly for blame and > >> to designate a go-to person for feature problems, not for giving > >> credit, > > > > Then reviewers should be removed. > > I disagree. We're trying to get more reviewers, and encourage them to > do more reviewing. Giving credit is a big part of that. Are you disagreeing with Bruce's premise, my logic, or the conclusion?
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:44:23Z
On Thu, May 10, 2012 at 01:11:54PM +0100, Peter Geoghegan wrote: > On 10 May 2012 04:11, Bruce Momjian <bruce@momjian.us> wrote: > > I have completed my draft of the 9.2 release notes, and committed it to > > git. I am waiting for our development docs to build, but after 40 > > minutes, I am still waiting: > > "Allow the bgwriter, walwriter, and statistics collector to sleep more > efficiently during periods of inactivity (Peter Geoghegan, Heikki > Linnakangas, Tom Lane)...This reduces CPU wake-ups." > > I think that there should be mention of why this is a good thing. When > fully idle the server reaches less than a single wake-up per second, I added text that says it reduces power consuption on idle servers. > which I think is a nice, relevant fact. You should add the archiver > and checkpointer to that list, though I suppose you could argue that > the checkpointer, as a "new" auxiliary process, shouldn't count. I added the archiver and checkpointer to the list. Seems there is no doc section to link to for these processes. > Why can't we call group commit group commit (and for that matter, > index-only scans index-only scans), so that people will understand > that we are now competitive with other RDBMSs in this area? "Improve > performance of WAL writes when multiple transactions commit at the > same time" seems like a pretty bad description, since it doesn't make > any reference to batching of commits. Also, I don't think that the I didn't call it "group commit" because we have settings we used to regard as group commit: #commit_delay = 0 # range 0-100000, in microseconds #commit_siblings = 5 # range 1-1000 These are still there. Should they be removed? I updated the release docs to call the item "group commit" because I now don't see any reference to that term in our docs. > placement of this as the second to last performance feature is > commensurate with its actual importance. Still, the actual major I am really unclear on how the performance items should be listed in terms of importance, so I am ready for someone to tell me the proper order. > feature list is a much more relevant indicator of how it is felt that > individual features should be weighed, and of course that hasn't been > decided upon yet. > > "Change pg_stat_statements' total_time column to be measured in > milliseconds (Tom Lane)". Surely this should be under > "pg_stat_statements"? I had it above because it was a major incompatibility. I do have some incompatibilities, e.g. pg_upgrade, that I kept in their own section. Should I move it? Can we assume people will also look in per-module sections for incompatibility information? > Does "Make the visibility map crash-safe" really belong under "Performance"? Not sure where to move that to. Source Code doesn't seem right. I moved it lower in the performance section. > It's not clear that this isn't just within comments that will be later > removed, but I'd remove all references to "we". Fixed. Attached patch applied. Thanks. I do appreciate all the feedback. I think I got almost zero feedback on 9.1 and it was kind of weird. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T17:49:06Z
On Thu, May 10, 2012 at 05:57:01AM -0700, Josh Kupershmidt wrote: > On Wed, May 9, 2012 at 8:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > > I have completed my draft of the 9.2 release notes, and committed it to > > git. I am waiting for our development docs to build, but after 40 > > minutes, I am still waiting: > > This bit: > Previously supplied years and year masks of less than four digits > wrapped inconsistently. > > I first read as "Previously-supplied years..." instead of "Previously, > years and year masks with...". Good suggestion, I fixed that, and a few more, with the applied patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T17:51:28Z
On Thu, May 10, 2012 at 1:44 PM, Bruce Momjian <bruce@momjian.us> wrote: > Not sure where to move that to. Source Code doesn't seem right. I > moved it lower in the performance section. I'd just delete it. Instead, under index-only scans, I'd mention it in the detail text: "This is possible because the visibility map has been improved to be robust even in the face of database or system crashes. Various race conditions that could result in incorrect data in the visibility map have also been fixed." -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-10T18:15:09Z
>>> Then reviewers should be removed. >> >> I disagree. We're trying to get more reviewers, and encourage them to >> do more reviewing. Giving credit is a big part of that. > > Are you disagreeing with Bruce's premise, my logic, or the conclusion? Hah, good point. I'm disagreeing with the conclusion that reviewers should be removed, unless we're going to remove everyone *and* give them credit elsewhere. Which I would also be in favor of, I'm just not able to do the work right now. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Alexander Korotkov <aekorotkov@gmail.com> — 2012-05-10T18:22:58Z
"Improve GiST box and point index performance by producing better trees with less memory allocation overhead (Alexander Korotkov, Heikki Linnakangas, Kevin Grittner)" Is this note about following two commits? http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=7f3bd86843e5aad84585a57d3f6b80db3c609916 http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=d50e1251946a6e59092f0a84fc903532eb599a4f These improvements influence not only boxes and points but all geometrical datatypes. ------ With best regards, Alexander Korotkov.
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T18:29:47Z
On Thu, May 10, 2012 at 2:15 PM, Josh Berkus <josh@agliodbs.com> wrote: >>>> Then reviewers should be removed. >>> >>> I disagree. We're trying to get more reviewers, and encourage them to >>> do more reviewing. Giving credit is a big part of that. >> >> Are you disagreeing with Bruce's premise, my logic, or the conclusion? > > Hah, good point. I'm disagreeing with the conclusion that reviewers > should be removed, unless we're going to remove everyone *and* give them > credit elsewhere. Which I would also be in favor of, I'm just not able > to do the work right now. Well, the problem with the way it is right now is that we're giving similar amounts of credit for very different amounts of contribution, which IMHO is no good. I think that putting a "Credits" section at the bottom and listing contributors there would be a reasonable solution; I also think that crediting people on a web page or in some other place would be a fine solution. What we have right now manages to be both unfair and unreadable. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Christopher Browne <cbbrowne@gmail.com> — 2012-05-10T18:43:48Z
On Thu, May 10, 2012 at 12:55 PM, Josh Berkus <josh@agliodbs.com> wrote: > On 5/10/12 9:44 AM, Peter Eisentraut wrote: >> On tor, 2012-05-10 at 10:44 -0400, Bruce Momjian wrote: >>> The big take-away is that the release notes are mostly for blame and >>> to designate a go-to person for feature problems, not for giving >>> credit, >> >> Then reviewers should be removed. > > I disagree. We're trying to get more reviewers, and encourage them to > do more reviewing. Giving credit is a big part of that. As much as that's nice, I don't think that's quite enough reason to do so, at least not as a last minute afterthought in trying to finalize the release notes. On the other hand, if reviewers are considered extra "go-to" people for the purposes of 'blamecasting' if something goes wrong with a new feature, that's actually a fine reason to include them. If both the developer *and* the reviewer missed an issue, then *both* are "blameworthy," and if we have any features gone desperately wrong, both deserve to have appropriate things thrown at them. -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?"
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T19:07:33Z
On 05/10/2012 02:29 PM, Robert Haas wrote: > On Thu, May 10, 2012 at 2:15 PM, Josh Berkus<josh@agliodbs.com> wrote: >>>>> Then reviewers should be removed. >>>> I disagree. We're trying to get more reviewers, and encourage them to >>>> do more reviewing. Giving credit is a big part of that. >>> Are you disagreeing with Bruce's premise, my logic, or the conclusion? >> Hah, good point. I'm disagreeing with the conclusion that reviewers >> should be removed, unless we're going to remove everyone *and* give them >> credit elsewhere. Which I would also be in favor of, I'm just not able >> to do the work right now. > Well, the problem with the way it is right now is that we're giving > similar amounts of credit for very different amounts of contribution, > which IMHO is no good. I think that putting a "Credits" section at > the bottom and listing contributors there would be a reasonable > solution; I also think that crediting people on a web page or in some > other place would be a fine solution. What we have right now manages > to be both unfair and unreadable. > I don't really believe either of these. It's certainly not unreadable, and it's largely fair, although there may be some room for improvement. Moreover, until we have something better I'm strongly opposed to removing what we currently do (or have done in the past.) The important thing about the current mechanism is that it ties the contributor's name to a feature in the only place where we currently list features on a time basis. So if I (for example) want to put on my resume that I contributed adding new values to an enum in the 9.1 release, there is a really easy way for someone to check that that's true, without having to search commit logs, which aren't always wonderfully reliable either. If you want a little finer granularity, let me offer the following categories as a way of opening up discussion: Author: contributed a significant portion of the code of a feature (say, over 25%) Contributor: made a significant contribution to the code (say 10% or more?), but less than that of an author. Reviewer: did a significant review of the code but not a significant code contribution. These are intended as broad guidelines, rather than something to be nitpicked and litigated, but you should get the idea. cheers andrew -
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T19:23:23Z
Bruce Momjian <bruce@momjian.us> writes: > On Thu, May 10, 2012 at 01:56:33AM -0400, Robert Haas wrote: >> As a general comment, I think that your new policy of crediting the >> reviewer on every feature except when that reviewer is also a >> committer has produced a horrific mess. > I assumed reviewers mentioned in the commit messages made substantive > suggestions on improving the patch, rather than just +1. I wouldn't assume that. On most of what I committed from commitfest items, I just listed whoever was named in the CF entry. regards, tom lane
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-10T20:07:33Z
On Thu, May 10, 2012 at 3:07 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > The important thing about the current mechanism is that it ties the > contributor's name to a feature in the only place where we currently list > features on a time basis. So if I (for example) want to put on my resume > that I contributed adding new values to an enum in the 9.1 release, there is > a really easy way for someone to check that that's true, without having to > search commit logs, which aren't always wonderfully reliable either. If you > want a little finer granularity, let me offer the following categories as a > way of opening up discussion: > > Author: contributed a significant portion of the code of a feature > (say, over 25%) > Contributor: made a significant contribution to the code (say 10% or > more?), but less than that of an author. > Reviewer: did a significant review of the code but not a significant > code contribution. > > These are intended as broad guidelines, rather than something to be > nitpicked and litigated, but you should get the idea. Well, that would be fine, too. What I think is bizarre is that I got credit for some things I was barely involved in (like SP-gist) and no credit for other things I spent a LOT of time on (like security views and some of KaiGai's other stuff), and similarly for other people. Similarly, some things I am credited on involve very significant contributions from other people and others are cases where I did nearly all the work. I think it's weird to lump all those cases together without any distinction. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T20:16:01Z
Robert Haas <robertmhaas@gmail.com> writes: > Well, that would be fine, too. What I think is bizarre is that I got > credit for some things I was barely involved in (like SP-gist) and no > credit for other things I spent a LOT of time on (like security views > and some of KaiGai's other stuff), and similarly for other people. > Similarly, some things I am credited on involve very significant > contributions from other people and others are cases where I did > nearly all the work. I think it's weird to lump all those cases > together without any distinction. Well, you know, these are *draft* release notes. Feel free to correct them anywhere you believe they are inaccurate. I think the bigger issue here is that we don't seem to have consensus about whether to include reviewers' names. Bruce evidently thinks that's a good idea, else he wouldn't have done it, but I only recall one other person speaking in favor of it. Everybody else seems to think that it'll be too verbose. regards, tom lane
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@commandprompt.com> — 2012-05-10T20:18:30Z
Excerpts from Robert Haas's message of jue may 10 16:07:33 -0400 2012: > On Thu, May 10, 2012 at 3:07 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > The important thing about the current mechanism is that it ties the > > contributor's name to a feature in the only place where we currently list > > features on a time basis. So if I (for example) want to put on my resume > > that I contributed adding new values to an enum in the 9.1 release, there is > > a really easy way for someone to check that that's true, without having to > > search commit logs, which aren't always wonderfully reliable either. If you > > want a little finer granularity, let me offer the following categories as a > > way of opening up discussion: > > > > Author: contributed a significant portion of the code of a feature > > (say, over 25%) > > Contributor: made a significant contribution to the code (say 10% or > > more?), but less than that of an author. > > Reviewer: did a significant review of the code but not a significant > > code contribution. > > > > These are intended as broad guidelines, rather than something to be > > nitpicked and litigated, but you should get the idea. > > Well, that would be fine, too. What I think is bizarre is that I got > credit for some things I was barely involved in (like SP-gist) and no > credit for other things I spent a LOT of time on (like security views > and some of KaiGai's other stuff), and similarly for other people. > Similarly, some things I am credited on involve very significant > contributions from other people and others are cases where I did > nearly all the work. I think it's weird to lump all those cases > together without any distinction. It's been said elsewhere that adding all this to the release notes as found on the official docs would be too bulky. How about having a second copy of the release notes that contains authorship info as proposed by Andrew? Then the docs could have no names at all, and credit would be given by some other page in the website (to which the release notes would link). We could even have both be built from a single source, if we made inclusion depend on some DSSSL flag or something. (Obviously I'm not proposing doing this for beta1). -- Álvaro Herrera <alvherre@commandprompt.com> The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T20:50:56Z
On Thu, May 10, 2012 at 01:51:28PM -0400, Robert Haas wrote: > On Thu, May 10, 2012 at 1:44 PM, Bruce Momjian <bruce@momjian.us> wrote: > > Not sure where to move that to. Source Code doesn't seem right. I > > moved it lower in the performance section. > > I'd just delete it. Instead, under index-only scans, I'd mention it > in the detail text: "This is possible because the visibility map has > been improved to be robust even in the face of database or system > crashes. Various race conditions that could result in incorrect data > in the visibility map have also been fixed." OK, I merged it in in the attached, applied patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-10T21:57:26Z
> It's been said elsewhere that adding all this to the release notes as > found on the official docs would be too bulky. How about having a > second copy of the release notes that contains authorship info as > proposed by Andrew? Then the docs could have no names at all, and > credit would be given by some other page in the website (to which the > release notes would link). Personally, I'd love to have something really simple, with just 3 sections: "Contributors to PostgreSQL 9.2" (1) Specific Patch Contributors patch, original (co) author name (2) Patch reviewers list without specific patch affiliation (3) Other Contributors to this release anyone who contributed but isn't mentioned above I think if we get any more complicated, this is going to become a major chore for someone, and then it won't get done. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-10T22:15:12Z
Josh Berkus <josh@agliodbs.com> writes: >> It's been said elsewhere that adding all this to the release notes as >> found on the official docs would be too bulky. How about having a >> second copy of the release notes that contains authorship info as >> proposed by Andrew? Then the docs could have no names at all, and >> credit would be given by some other page in the website (to which the >> release notes would link). > Personally, I'd love to have something really simple, with just 3 sections: > "Contributors to PostgreSQL 9.2" > (1) Specific Patch Contributors > patch, original (co) author name > (2) Patch reviewers > list without specific patch affiliation > (3) Other Contributors to this release > anyone who contributed but isn't mentioned above > I think if we get any more complicated, this is going to become a major > chore for someone, and then it won't get done. The other problem with such an approach is that section (1) would be extremely duplicative of the main release-notes text. How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? regards, tom lane
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-10T22:15:50Z
> The other problem with such an approach is that section (1) would be > extremely duplicative of the main release-notes text. How about a > hybrid: we continue to identify patch authors as now, that is with names > attached to the feature/bugfix descriptions, and then have a separate > section "Other Contributors" to recognize patch reviewers and other > helpers? Sounds good to me. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-10T22:27:00Z
On Thu, May 10, 2012 at 04:16:01PM -0400, Tom Lane wrote: > Robert Haas <robertmhaas@gmail.com> writes: > > Well, that would be fine, too. What I think is bizarre is that I got > > credit for some things I was barely involved in (like SP-gist) and no > > credit for other things I spent a LOT of time on (like security views > > and some of KaiGai's other stuff), and similarly for other people. > > Similarly, some things I am credited on involve very significant > > contributions from other people and others are cases where I did > > nearly all the work. I think it's weird to lump all those cases > > together without any distinction. > > Well, you know, these are *draft* release notes. Feel free to correct > them anywhere you believe they are inaccurate. Yep. > I think the bigger issue here is that we don't seem to have consensus > about whether to include reviewers' names. Bruce evidently thinks > that's a good idea, else he wouldn't have done it, but I only recall one > other person speaking in favor of it. Everybody else seems to think > that it'll be too verbose. There were 2-3 who liked the reviewer names. The bottom line is it is easy to _remove_ names; it requires a lot of research to add them. One creative idea would be to keep the reviewer names as-is, but trim the release notes down to a single name just before final release. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-10T23:19:16Z
On 05/10/2012 06:15 PM, Tom Lane wrote: > How about a hybrid: we continue to identify patch authors as now, that > is with names attached to the feature/bugfix descriptions, and then > have a separate section "Other Contributors" to recognize patch > reviewers and other helpers? works for me. cheers andrew
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-11T03:46:56Z
On May 10, 2012, at 4:19 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > On 05/10/2012 06:15 PM, Tom Lane wrote: >> How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? > > works for me. Me, too. ...Robert
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-11T07:26:27Z
On Thu, May 10, 2012 at 6:28 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > On tor, 2012-05-10 at 17:31 +0200, Magnus Hagander wrote: >> If people want the main docs building more often that's not really a >> problem other than time - we just need to decouple it from the >> buildfarm and run a separate job for it. It's not rocket science.. > > Many years ago, Bruce and myself in particular put in a lot of work to > make the turnaround time on the docs build less than 5 minutes, based on > various requests. I'm disappointed to learn that that was abandoned > without discussion. We might as well just put the old job back. It was not "abandoned without discussion" in any way. First of all, the docs still build in 5 minutes. Second, the "5 minutes docs build" link on the website was removed in *2007*. At the request of Bruce, who maintained it. This request was (at least according to the commit message and form what I can remember) made in public on pgsql-www, and thus clearly open for discussion. At http://archives.postgresql.org/pgsql-www/2007-12/msg00212.php. Where Bruce claims the other one runs often enough, and nobody objects. Third, the regular docs build on the developer box (which I think ran once / hour?) *did not work* (prior to that it kind of work but often hung and failed, but at least it tried to run - at this point it stopped even trying). The current docs build replaced the case when we had *no developer docs updates at all*, by taking the reasonably quick and easy fix to run it as part of an existing buildfarm animal and upload the results. So where in all this was anything "abandoned"? Bruce already suggested putting the old job back on his box, which he's of course free to do. But since it took almost 5 years before anybody actually complained about that, maybe it's not really that big a problem? And it's already been agreed that increasing the dev docs build back up to maybe once an hour would make sense. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-11T07:55:11Z
On fre, 2012-05-11 at 09:26 +0200, Magnus Hagander wrote: > On Thu, May 10, 2012 at 6:28 PM, Peter Eisentraut <peter_e@gmx.net> wrote: > > On tor, 2012-05-10 at 17:31 +0200, Magnus Hagander wrote: > >> If people want the main docs building more often that's not really a > >> problem other than time - we just need to decouple it from the > >> buildfarm and run a separate job for it. It's not rocket science.. > > > > Many years ago, Bruce and myself in particular put in a lot of work to > > make the turnaround time on the docs build less than 5 minutes, based on > > various requests. I'm disappointed to learn that that was abandoned > > without discussion. We might as well just put the old job back. > > It was not "abandoned without discussion" in any way. > > First of all, the docs still build in 5 minutes. That is different from the turnaround time from the commit. > Second, the "5 minutes docs build" link on the website was removed in > *2007*. At the request of Bruce, who maintained it. This request was > (at least according to the commit message and form what I can > remember) made in public on pgsql-www, and thus clearly open for > discussion. At http://archives.postgresql.org/pgsql-www/2007-12/msg00212.php. > Where Bruce claims the other one runs often enough, and nobody > objects. You are misinterpreting this. The reason Bruce's link was removed was that the other (official) build was set to run at the same frequency, so Bruce's build was exactly redundant. The requirement/aspiration to have a few minutes turnaround time continued. > Third, the regular docs build on the developer box (which I think ran > once / hour?) *did not work* (prior to that it kind of work but often > hung and failed, but at least it tried to run - at this point it > stopped even trying). If you had any problems with how well they worked, we could have discussed this. It's fine if you want to change how they run, and I have no problem with how they are run now, but I just want to make clear what requirements led to the setup at the time. > So where in all this was anything "abandoned"? The ability to get a docs build in less than 5 minutes after commit.
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-11T09:32:04Z
On Fri, May 11, 2012 at 9:55 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On fre, 2012-05-11 at 09:26 +0200, Magnus Hagander wrote: >> On Thu, May 10, 2012 at 6:28 PM, Peter Eisentraut <peter_e@gmx.net> wrote: >> > On tor, 2012-05-10 at 17:31 +0200, Magnus Hagander wrote: >> >> If people want the main docs building more often that's not really a >> >> problem other than time - we just need to decouple it from the >> >> buildfarm and run a separate job for it. It's not rocket science.. >> > >> > Many years ago, Bruce and myself in particular put in a lot of work to >> > make the turnaround time on the docs build less than 5 minutes, based on >> > various requests. I'm disappointed to learn that that was abandoned >> > without discussion. We might as well just put the old job back. >> >> It was not "abandoned without discussion" in any way. >> >> First of all, the docs still build in 5 minutes. > > That is different from the turnaround time from the commit. > >> Second, the "5 minutes docs build" link on the website was removed in >> *2007*. At the request of Bruce, who maintained it. This request was >> (at least according to the commit message and form what I can >> remember) made in public on pgsql-www, and thus clearly open for >> discussion. At http://archives.postgresql.org/pgsql-www/2007-12/msg00212.php. >> Where Bruce claims the other one runs often enough, and nobody >> objects. > > You are misinterpreting this. The reason Bruce's link was removed was > that the other (official) build was set to run at the same frequency, so > Bruce's build was exactly redundant. The requirement/aspiration to have > a few minutes turnaround time continued. But the other (official) build was *not* set to run at the same frequency. It was set, according to that mail, to run frequently enough, but it did not run every 5 minutes. at least not the only cronjob I found back then. >> Third, the regular docs build on the developer box (which I think ran >> once / hour?) *did not work* (prior to that it kind of work but often >> hung and failed, but at least it tried to run - at this point it >> stopped even trying). > > If you had any problems with how well they worked, we could have > discussed this. It's fine if you want to change how they run, and I > have no problem with how they are run now, but I just want to make clear > what requirements led to the setup at the time. The entire machine they ran on *died*. Because it had been unmaintained for many years. and parts was silently upgraded where as other, incompatible, parts were not. We did actually leave the script around. It ran for months, failing at step one, and pretty much nobody complained. The docs build was *entirely* undocumented (other than the official cronjob which did *not* run every 5 minutes, but I guess you are saying there was a second, undocumented, cronjob that ran more often). But in the interest of actually being productive - what *is* the usecase for needing a 5 minute turnaround time? I don't buy the "check what a patch looks like", because that should be done *before* the commit, not after - so it's best verified by a local docs build anyway (which will also be faster). I'm sure we can put something in with a pretty quick turnaround again without too much strain on the system, but it does, as I mentioned before, require decoupling it from the buildfarm which means it's not just tweaking a config file. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Cédric Villemain <cedric@2ndquadrant.com> — 2012-05-11T10:08:27Z
Le jeudi 10 mai 2012 22:18:30, Alvaro Herrera a écrit : > It's been said elsewhere that adding all this to the release notes as > found on the official docs would be too bulky. How about having a > second copy of the release notes that contains authorship info as > proposed by Andrew? Then the docs could have no names at all, and > credit would be given by some other page in the website (to which the > release notes would link). Maybe we can update/upgrade the page [1] on the website and add per release authors/rewviewers/companies/... ? [1] http://www.postgresql.org/community/contributors/ -- Cédric Villemain +33 (0)6 20 30 22 52 http://2ndQuadrant.fr/ PostgreSQL: Support 24x7 - Développement, Expertise et Formation
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-11T12:56:55Z
On Thu, May 10, 2012 at 08:46:56PM -0700, Robert Haas wrote: > On May 10, 2012, at 4:19 PM, Andrew Dunstan <andrew@dunslane.net> wrote: > > On 05/10/2012 06:15 PM, Tom Lane wrote: > >> How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? > > > > works for me. > > Me, too. That does not work for me. There is no practical reason for a list of names to appear in the release notes. I suggest if we want to do that that we remove all names from the release notes (as Tom suggested), and create a wiki for credit, and link to that from the release announcement. That would allow us to put company names in there too. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-11T13:51:49Z
On 05/11/2012 08:56 AM, Bruce Momjian wrote: > On Thu, May 10, 2012 at 08:46:56PM -0700, Robert Haas wrote: >> On May 10, 2012, at 4:19 PM, Andrew Dunstan<andrew@dunslane.net> wrote: >>> On 05/10/2012 06:15 PM, Tom Lane wrote: >>>> How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? >>> works for me. >> Me, too. > That does not work for me. There is no practical reason for a list of > names to appear in the release notes. I suggest if we want to do that > that we remove all names from the release notes (as Tom suggested), and > create a wiki for credit, and link to that from the release > announcement. That would allow us to put company names in there too. > I gave you a reason. You might not agree with it but saying that it's no reason doesn't make it so. A wiki page will just be duplication, IMNSHO. cheers andrew
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-11T14:01:32Z
On Fri, May 11, 2012 at 09:51:49AM -0400, Andrew Dunstan wrote: > > > On 05/11/2012 08:56 AM, Bruce Momjian wrote: > >On Thu, May 10, 2012 at 08:46:56PM -0700, Robert Haas wrote: > >>On May 10, 2012, at 4:19 PM, Andrew Dunstan<andrew@dunslane.net> wrote: > >>>On 05/10/2012 06:15 PM, Tom Lane wrote: > >>>>How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? > >>>works for me. > >>Me, too. > >That does not work for me. There is no practical reason for a list of > >names to appear in the release notes. I suggest if we want to do that > >that we remove all names from the release notes (as Tom suggested), and > >create a wiki for credit, and link to that from the release > >announcement. That would allow us to put company names in there too. > > > > I gave you a reason. You might not agree with it but saying that > it's no reason doesn't make it so. A wiki page will just be > duplication, IMNSHO. I mean a reason from the reader/development-process perspective, not from the perspective of giving a some benefit to contributors. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-11T14:15:50Z
On Fri, May 11, 2012 at 10:01:32AM -0400, Bruce Momjian wrote: > On Fri, May 11, 2012 at 09:51:49AM -0400, Andrew Dunstan wrote: > > > > > > On 05/11/2012 08:56 AM, Bruce Momjian wrote: > > >On Thu, May 10, 2012 at 08:46:56PM -0700, Robert Haas wrote: > > >>On May 10, 2012, at 4:19 PM, Andrew Dunstan<andrew@dunslane.net> wrote: > > >>>On 05/10/2012 06:15 PM, Tom Lane wrote: > > >>>>How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? > > >>>works for me. > > >>Me, too. > > >That does not work for me. There is no practical reason for a list of > > >names to appear in the release notes. I suggest if we want to do that > > >that we remove all names from the release notes (as Tom suggested), and > > >create a wiki for credit, and link to that from the release > > >announcement. That would allow us to put company names in there too. > > > > > > > I gave you a reason. You might not agree with it but saying that > > it's no reason doesn't make it so. A wiki page will just be > > duplication, IMNSHO. > > I mean a reason from the reader/development-process perspective, not > from the perspective of giving a some benefit to contributors. Let me add that I am concerned about the lack of objectivity in many of the suggestions in this thread. This has prompted me to think that the temptation of having names on these release note items is just too great, and that the names should be removed. Let me put it this way --- the release notes are read by thousands of people. The benefit individuals gather from their names in the release notes is a small part of the overall value provided by the release notes to users. There was a practical need to have names on items in the past --- that need is no longer present. I predict that if we twist the release notes to have PR value for contributors, it will become a prepetual problem and will diminish the cohesiveness of our group. I am already personally upset by a few of the things I have seen on this thread. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-11T14:36:31Z
On 05/11/2012 10:15 AM, Bruce Momjian wrote: > On Fri, May 11, 2012 at 10:01:32AM -0400, Bruce Momjian wrote: >> On Fri, May 11, 2012 at 09:51:49AM -0400, Andrew Dunstan wrote: >>> >>> On 05/11/2012 08:56 AM, Bruce Momjian wrote: >>>> On Thu, May 10, 2012 at 08:46:56PM -0700, Robert Haas wrote: >>>>> On May 10, 2012, at 4:19 PM, Andrew Dunstan<andrew@dunslane.net> wrote: >>>>>> On 05/10/2012 06:15 PM, Tom Lane wrote: >>>>>>> How about a hybrid: we continue to identify patch authors as now, that is with names attached to the feature/bugfix descriptions, and then have a separate section "Other Contributors" to recognize patch reviewers and other helpers? >>>>>> works for me. >>>>> Me, too. >>>> That does not work for me. There is no practical reason for a list of >>>> names to appear in the release notes. I suggest if we want to do that >>>> that we remove all names from the release notes (as Tom suggested), and >>>> create a wiki for credit, and link to that from the release >>>> announcement. That would allow us to put company names in there too. >>>> >>> I gave you a reason. You might not agree with it but saying that >>> it's no reason doesn't make it so. A wiki page will just be >>> duplication, IMNSHO. >> I mean a reason from the reader/development-process perspective, not >> from the perspective of giving a some benefit to contributors. > Let me add that I am concerned about the lack of objectivity in many of > the suggestions in this thread. This has prompted me to think that the > temptation of having names on these release note items is just too > great, and that the names should be removed. > > Let me put it this way --- the release notes are read by thousands of > people. The benefit individuals gather from their names in the release > notes is a small part of the overall value provided by the release notes > to users. There was a practical need to have names on items in the past > --- that need is no longer present. > > I predict that if we twist the release notes to have PR value for > contributors, it will become a prepetual problem and will diminish the > cohesiveness of our group. I am already personally upset by a few of > the things I have seen on this thread. Well, I don't know what has changed that made it imperative in the past to have the names and makes it now redundant, nor what could possibly have upset you so much. Maybe I'm dense, but that's the truth of it. Now if someone is going to volunteer to build *AND* *MAINTAIN* a Credits page, that will be good. It would be even better if they would go back and do it historically. But just hoping that will happen and meantime removing the names from the notes seems to me a retrograde step. cheers andrew
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-11T14:37:25Z
Bruce Momjian <bruce@momjian.us> writes: > Let me add that I am concerned about the lack of objectivity in many of > the suggestions in this thread. This has prompted me to think that the > temptation of having names on these release note items is just too > great, and that the names should be removed. Er, what? I have not seen anything in this thread that merits such an accusation. > Let me put it this way --- the release notes are read by thousands of > people. The benefit individuals gather from their names in the release > notes is a small part of the overall value provided by the release notes > to users. There was a practical need to have names on items in the past > --- that need is no longer present. My recollection is that we have been putting the names on the items to (a) give credit where credit is due, and (b) show that Postgres has a large and growing development community. I had never heard the argument "remember whom to blame for a broken feature" until you raised it yesterday --- personally, I've always looked in the commit logs if I want to know something like that. So I don't see that there's really any change in the terms of discussion. It may be that the release notes aren't the best place for doing either (a) or (b), but I don't agree that we simply don't need to worry about either anymore. regards, tom lane
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-11T15:44:49Z
On 05/11/2012 05:32 AM, Magnus Hagander wrote: > > But in the interest of actually being productive - what *is* the > usecase for needing a 5 minute turnaround time? I don't buy the "check > what a patch looks like", because that should be done *before* the > commit, not after - so it's best verified by a local docs build anyway > (which will also be faster). > > I'm sure we can put something in with a pretty quick turnaround again > without too much strain on the system, but it does, as I mentioned > before, require decoupling it from the buildfarm which means it's not > just tweaking a config file. If it's of any use to you I have made some adjustments to the buildfarm code which would let you do *just* the docs build (and dist make if you want). It would still pull from git, and only do anything if there's a (relevant) change. So using that to set up a machine that would run every few minutes might work. Of course, building the docs can itself be fairly compute intensive, so you still might not want to run every few minutes if that's a limiting factor. cheers andrew
-
Re: Draft release notes complete
Jeff Janes <jeff.janes@gmail.com> — 2012-05-11T18:03:58Z
On Thu, May 10, 2012 at 10:44 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 10, 2012 at 01:11:54PM +0100, Peter Geoghegan wrote: > >> Why can't we call group commit group commit (and for that matter, >> index-only scans index-only scans), so that people will understand >> that we are now competitive with other RDBMSs in this area? "Improve >> performance of WAL writes when multiple transactions commit at the >> same time" seems like a pretty bad description, since it doesn't make >> any reference to batching of commits. Also, I don't think that the > > I didn't call it "group commit" because we have settings we used to > regard as group commit: My understanding of that patch is that is does not cause "group commit" to happen, but rather when a group commit does happen "naturally" it causes all members of the group to awaken more quickly/efficiently than they previously would have. > > #commit_delay = 0 # range 0-100000, in microseconds > #commit_siblings = 5 # range 1-1000 > > These are still there. Should they be removed? The new locking around releasing group commit waiters has, if anything, made these two more effective than before. But that isn't really saying much. It seems like these knobs are (and were) primarily useful for doing "stupid benchmark tricks" of little practical value. If there is an argument for removing them, I think it would revolve around either "They never really should have been there anyway", or "These days when people need far more commits per second than they have revolutions per second, they buy BBU or NVRAM". > I updated the release docs to call the item "group commit" because I now > don't see any reference to that term in our docs. I don't think I'd mention WAL writing at all, and just say that it improves the concurrency of locking around group commits. Cheers, Jeff
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-05-11T18:10:11Z
On fre, 2012-05-11 at 11:32 +0200, Magnus Hagander wrote: > > You are misinterpreting this. The reason Bruce's link was removed was > > that the other (official) build was set to run at the same frequency, so > > Bruce's build was exactly redundant. The requirement/aspiration to have > > a few minutes turnaround time continued. > > But the other (official) build was *not* set to run at the same > frequency. It was set, according to that mail, to run frequently > enough, but it did not run every 5 minutes. at least not the only > cronjob I found back then. I would love to see what cron job job you are referring to. Unfortunately, I don't have a backup, but I'm pretty sure at one point it ran every three minutes or so. > But in the interest of actually being productive - what *is* the > usecase for needing a 5 minute turnaround time? I don't exactly know, it was done at the request of users. A lot of people wanted to see the documentation of new checkins, just to learn about how the new features worked. As a general point, any delay time is going to raise questions, because it usually won't be easy to find out when things will happen. So the "human" maintenance effort will be lower if it runs as soon as possible.
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-12T00:37:58Z
On Fri, May 11, 2012 at 2:03 PM, Jeff Janes <jeff.janes@gmail.com> wrote: > On Thu, May 10, 2012 at 10:44 AM, Bruce Momjian <bruce@momjian.us> wrote: >> On Thu, May 10, 2012 at 01:11:54PM +0100, Peter Geoghegan wrote: >> >>> Why can't we call group commit group commit (and for that matter, >>> index-only scans index-only scans), so that people will understand >>> that we are now competitive with other RDBMSs in this area? "Improve >>> performance of WAL writes when multiple transactions commit at the >>> same time" seems like a pretty bad description, since it doesn't make >>> any reference to batching of commits. Also, I don't think that the >> >> I didn't call it "group commit" because we have settings we used to >> regard as group commit: > > My understanding of that patch is that is does not cause "group > commit" to happen, but rather when a group commit does happen > "naturally" it causes all members of the group to awaken more > quickly/efficiently than they previously would have. Right. It's not a new feature; it's a performance improvement. We've had group commit for a long time; it just didn't work very well before. And it's not batching the commits better; it's reducing the lock contention around realizing that the batched commit has happened. >> I updated the release docs to call the item "group commit" because I now >> don't see any reference to that term in our docs. > > I don't think I'd mention WAL writing at all, and just say that it > improves the concurrency of locking around group commits. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-12T12:55:30Z
On Fri, May 11, 2012 at 08:37:58PM -0400, Robert Haas wrote: > On Fri, May 11, 2012 at 2:03 PM, Jeff Janes <jeff.janes@gmail.com> wrote: > > On Thu, May 10, 2012 at 10:44 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> On Thu, May 10, 2012 at 01:11:54PM +0100, Peter Geoghegan wrote: > >> > >>> Why can't we call group commit group commit (and for that matter, > >>> index-only scans index-only scans), so that people will understand > >>> that we are now competitive with other RDBMSs in this area? "Improve > >>> performance of WAL writes when multiple transactions commit at the > >>> same time" seems like a pretty bad description, since it doesn't make > >>> any reference to batching of commits. Also, I don't think that the > >> > >> I didn't call it "group commit" because we have settings we used to > >> regard as group commit: > > > > My understanding of that patch is that is does not cause "group > > commit" to happen, but rather when a group commit does happen > > "naturally" it causes all members of the group to awaken more > > quickly/efficiently than they previously would have. > > Right. It's not a new feature; it's a performance improvement. We've > had group commit for a long time; it just didn't work very well > before. And it's not batching the commits better; it's reducing the > lock contention around realizing that the batched commit has happened. > > >> I updated the release docs to call the item "group commit" because I now > >> don't see any reference to that term in our docs. > > > > I don't think I'd mention WAL writing at all, and just say that it > > improves the concurrency of locking around group commits. > > +1. Updated with the attached patch. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-12T13:27:16Z
In summary, names on release note items potentially have the following beneficial effects: * Encouraging new developers/reviewers * Encouraging long-established developers * Showing appreciation to developers * Assisting future employment for developers * Helping developers get future funding * Assigning responsibility for features * Assigning blame for feature problems * Showing Postgres's increased developer base Many of these goals has already been mentioned. So the question is which of these is important? If we emphasize all of them, I am afraid the name list for each item will be too long to be acceptable. How many names on a single item is ideal? The activity of reviewers and their names on commit messages has greatly expanded the number of potential names per item. How much of a downside is having the names in the release notes? For example, we decided that company names shouldn't be on release note items, so there is a case where we decided names were more of a negative than a positive. Are there other negatives? Do other project release notes have developer names? How are these names perceived by our general readers? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Credit in the release notes WAS: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-12T22:42:48Z
> How many names on a single item is ideal? The activity of reviewers and > their names on commit messages has greatly expanded the number of > potential names per item. > > How much of a downside is having the names in the release notes? For > example, we decided that company names shouldn't be on release note > items, so there is a case where we decided names were more of a negative > than a positive. Are there other negatives? Do other project release > notes have developer names? How are these names perceived by our > general readers? The two paragraphs above show the main problem. Who gets listed on each item is a matter of some contention. For example, if Robert Haas reviews a patch, and makes substantial suggesitons and fixes to the patch, should he be listed on it as well? If so, how much work is required for someone to be listed if they're not the original author? What if we merge two patches, but take 90% of Patch A and only 10% of Patch B? etc. We can decide these things on a case-by-case basis, but it makes preparing the release notes a LOT more effort. Let's see what other OSS projects do: HTTPD: lists people in the release notes: http://www.apache.org/dist/httpd/CHANGES_2.4.2 Linux, as far as I can tell, does not do Release Notes as such. FreeBSD does not have people's names in release notes, but does put links to the relevant commitlog: http://www.freebsd.org/releases/8.2R/relnotes.html Drizzle puts author's names in the release notes: http://blog.drizzle.org/category/release/ LibreOffice has author names and companies in the release notes: http://www.libreoffice.org/download/3-4-new-features-and-fixes/ git does *not* put author names in the release notes: https://raw.github.com/gitster/git/master/Documentation/RelNotes/1.7.9.txt I don't know if any of the above are putting committer, or reviewer names in the release notes, but given that most items have a single name, I doubt it. So what common practice tells us is inconclusive ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Euler Taveira de Oliveira <euler@timbira.com> — 2012-05-13T00:11:49Z
On 12-05-2012 10:27, Bruce Momjian wrote: > How many names on a single item is ideal? The activity of reviewers and > their names on commit messages has greatly expanded the number of > potential names per item. > Main authors only. Reviewers should be mentioned only in the commit log. If I coded a feature and Bruce got the idea worked in another patch (that is better then mine), I think only Bruce should be credited in release notes (but I could be mentioned in the commit log as the feature designer). However, if I posted a patch and Robert improved that patch using only 30% of my work, I should be credited (as coauthor) because he used a considerable part of my work. I confess that I like the link for relevant commit log in the release notes. -- Euler Taveira de Oliveira - Timbira http://www.timbira.com.br/ PostgreSQL: Consultoria, Desenvolvimento, Suporte 24x7 e Treinamento
-
Re: Credit in the release notes WAS: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-13T01:02:42Z
On Sat, May 12, 2012 at 03:42:48PM -0700, Josh Berkus wrote: > > > How many names on a single item is ideal? The activity of reviewers and > > their names on commit messages has greatly expanded the number of > > potential names per item. > > > > How much of a downside is having the names in the release notes? For > > example, we decided that company names shouldn't be on release note > > items, so there is a case where we decided names were more of a negative > > than a positive. Are there other negatives? Do other project release > > notes have developer names? How are these names perceived by our > > general readers? > > The two paragraphs above show the main problem. > > Who gets listed on each item is a matter of some contention. For > example, if Robert Haas reviews a patch, and makes substantial > suggesitons and fixes to the patch, should he be listed on it as well? > If so, how much work is required for someone to be listed if they're not > the original author? What if we merge two patches, but take 90% of > Patch A and only 10% of Patch B? etc. One idea I just had was to optionally put developer names on section headings. That would remove my name from the nine pg_upgrade entries in the pg_upgrade section. We could put Tom Lane's name at the top of the optimizer section, and some of the server-side languages could be trimmed down this way. Should we go with a single developer per item, and then let people suggest corrections? With reviewers involved, and often multiple commit messages per release note item, the just isn't enough detail in git logs to reproduce this accurately. I also over-emphasized new developers/reviewers, but that seems to have distorted the other goals unacceptably. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-13T01:08:03Z
On Sat, May 12, 2012 at 09:11:49PM -0300, Euler Taveira wrote: > On 12-05-2012 10:27, Bruce Momjian wrote: > > How many names on a single item is ideal? The activity of reviewers and > > their names on commit messages has greatly expanded the number of > > potential names per item. > > > Main authors only. Reviewers should be mentioned only in the commit log. If I > coded a feature and Bruce got the idea worked in another patch (that is better > then mine), I think only Bruce should be credited in release notes (but I The old-timers are howling at how funny it is that I would develop a better patch than anyone! LOL > could be mentioned in the commit log as the feature designer). However, if I > posted a patch and Robert improved that patch using only 30% of my work, I > should be credited (as coauthor) because he used a considerable part of my work. > > I confess that I like the link for relevant commit log in the release notes. I _do_ have the git hash information in the git logs, so it is trivial to transfer that into the release notes, but be aware there are often multiple commits per feature, and I am unclear how we would interface those links into the SGML. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Credit in the release notes WAS: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-05-13T01:27:21Z
On 05/12/2012 09:02 PM, Bruce Momjian wrote: > On Sat, May 12, 2012 at 03:42:48PM -0700, Josh Berkus wrote: >>> How many names on a single item is ideal? The activity of reviewers and >>> their names on commit messages has greatly expanded the number of >>> potential names per item. >>> >>> How much of a downside is having the names in the release notes? For >>> example, we decided that company names shouldn't be on release note >>> items, so there is a case where we decided names were more of a negative >>> than a positive. Are there other negatives? Do other project release >>> notes have developer names? How are these names perceived by our >>> general readers? >> The two paragraphs above show the main problem. >> >> Who gets listed on each item is a matter of some contention. For >> example, if Robert Haas reviews a patch, and makes substantial >> suggesitons and fixes to the patch, should he be listed on it as well? >> If so, how much work is required for someone to be listed if they're not >> the original author? What if we merge two patches, but take 90% of >> Patch A and only 10% of Patch B? etc. > One idea I just had was to optionally put developer names on section > headings. That would remove my name from the nine pg_upgrade entries in > the pg_upgrade section. We could put Tom Lane's name at the top of the > optimizer section, and some of the server-side languages could be > trimmed down this way. Say you do eight and someone else does one. I just don't see any benefit in this. The fact that a name is repeated a few times really doesn't matter. > > Should we go with a single developer per item, and then let people > suggest corrections? With reviewers involved, and often multiple commit > messages per release note item, the just isn't enough detail in git logs > to reproduce this accurately. I also over-emphasized new > developers/reviewers, but that seems to have distorted the other goals > unacceptably. Most cases should be pretty clear. Most features have a single major commit. The author(s) mentioned there are who should be listed, IMNSHO. That might leave a handful of cases where more judgement is required. We seem to be in danger of overthinking this. cheers andrew
-
Re: Credit in the release notes WAS: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-13T01:35:48Z
On Sat, May 12, 2012 at 09:27:21PM -0400, Andrew Dunstan wrote: > >Should we go with a single developer per item, and then let people > >suggest corrections? With reviewers involved, and often multiple commit > >messages per release note item, the just isn't enough detail in git logs > >to reproduce this accurately. I also over-emphasized new > >developers/reviewers, but that seems to have distorted the other goals > >unacceptably. > > Most cases should be pretty clear. Most features have a single major > commit. The author(s) mentioned there are who should be listed, > IMNSHO. That might leave a handful of cases where more judgement is > required. > > We seem to be in danger of overthinking this. Results have just shown it isn't a simple case. It is unclear how important the reviewers were, and how much a committer rewrote the patch, and the significance of follow-on commits. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Credit in the release notes WAS: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-05-13T01:59:12Z
Bruce Momjian <bruce@momjian.us> writes: > On Sat, May 12, 2012 at 09:27:21PM -0400, Andrew Dunstan wrote: >> We seem to be in danger of overthinking this. > Results have just shown it isn't a simple case. It is unclear how > important the reviewers were, and how much a committer rewrote the > patch, and the significance of follow-on commits. I'm wondering how come this has suddenly gotten so complicated. We got through a dozen major releases without so much angst about how to credit people. I tend to think Andrew's right: we are overthinking this, and are in danger of instituting a set of bureaucratic rules that will result in endless arguments, without really making anybody happier than before. I haven't yet heard any very good argument for deviating from our past practice, which is to credit just the principal author(s) of each patch, not reviewers. regards, tom lane
-
Re: Credit in the release notes WAS: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-13T02:35:21Z
On Sat, May 12, 2012 at 09:59:12PM -0400, Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > On Sat, May 12, 2012 at 09:27:21PM -0400, Andrew Dunstan wrote: > >> We seem to be in danger of overthinking this. > > > Results have just shown it isn't a simple case. It is unclear how > > important the reviewers were, and how much a committer rewrote the > > patch, and the significance of follow-on commits. > > I'm wondering how come this has suddenly gotten so complicated. > We got through a dozen major releases without so much angst about > how to credit people. I tend to think Andrew's right: we are > overthinking this, and are in danger of instituting a set of > bureaucratic rules that will result in endless arguments, without > really making anybody happier than before. > > I haven't yet heard any very good argument for deviating from our > past practice, which is to credit just the principal author(s) > of each patch, not reviewers. Is that what people want? Reviewers are easily removed. What about committers who adjust the patch? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-13T04:37:52Z
On Sat, May 12, 2012 at 8:11 PM, Euler Taveira <euler@timbira.com> wrote: > On 12-05-2012 10:27, Bruce Momjian wrote: >> How many names on a single item is ideal? The activity of reviewers and >> their names on commit messages has greatly expanded the number of >> potential names per item. >> > Main authors only. Reviewers should be mentioned only in the commit log. If I > coded a feature and Bruce got the idea worked in another patch (that is better > then mine), I think only Bruce should be credited in release notes (but I > could be mentioned in the commit log as the feature designer). However, if I > posted a patch and Robert improved that patch using only 30% of my work, I > should be credited (as coauthor) because he used a considerable part of my work. Completely agreed. If we're going to include names in the release notes, I agree that this is the way to do it, and I think it's what we have done in prior releases. I tend to err on the side of crediting people in the commit message (of course, occasionally I forget someone who should have been included), but I also try to make it clear by the phrasing whose code got included and who contributed in some other way - e.g. by reporting the problem, coming up with the original idea, or reviewing. I do this in part because I assumed that we'd use that as the criteria for including names in the release notes, as we have done in prior releases. So if I write: Euler Taveira, reviewed by Bruce Momjian, substantially rewritten by me ...then I expect that to turn up in the release notes as (Euler Taveira, Robert Haas). If I write: Euler Taveira, reviewed by Bruce Momjian, with minor cleanup by me ...then I expect that to turn up as (Euler Taveira). And if I write something like: Inspired by a patch from Euler Taveira. Review (in earlier versions) by Bruce Momjian. ...then I expect that to turn up as (Robert Haas) or (Robert Haas, Euler Taveira). In doubtful cases, I think it's generally appropriate to err on the side of crediting the person who was the original driving force behind the patch, and also to err on the side of not crediting the committer. But if the committer chopped up the patch and committed something significantly different from the original, then they should be credited - or blamed - for the result. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Credit in the release notes WAS: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-13T04:42:41Z
On Sat, May 12, 2012 at 10:35 PM, Bruce Momjian <bruce@momjian.us> wrote: >> I haven't yet heard any very good argument for deviating from our >> past practice, which is to credit just the principal author(s) >> of each patch, not reviewers. > > Is that what people want? Reviewers are easily removed. +1 from me. > What about > committers who adjust the patch? It's usually pretty clear from the commit message whether the patch was adjusted a little bit (in which case, there is no need to credit the committer, any more than we'd credit Thom Brown for a patch in which he found doc typos) or substantially (in which case the committer should be credited). If it's not clear, take your best guess and someone can let you know if there's an issue. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Credit in the release notes WAS: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-13T06:34:35Z
>> I haven't yet heard any very good argument for deviating from our >> past practice, which is to credit just the principal author(s) >> of each patch, not reviewers. > > Is that what people want? Reviewers are easily removed. What about > committers who adjust the patch? Well, I still think we should give credit to the reviewers ... in a single section at the bottom, where we list each reviewers' name once. Reviewers for this release: ... -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-13T20:01:03Z
On 12 May 2012 01:37, Robert Haas <robertmhaas@gmail.com> wrote: > Right. It's not a new feature; it's a performance improvement. We've > had group commit for a long time; it just didn't work very well > before. And it's not batching the commits better; it's reducing the > lock contention around realizing that the batched commit has happened. This understanding of group commit is technically accurate, but the practical implications of the new code are that lots of people benefit from group commit, potentially to rather a large degree, where before they had exactly no benefit from group commit. We never officially called group commit group commit outside of git commit messages before. Therefore, it is sort of like we didn't have group commit before but now we do, and it's an implementation that's probably as effective as that of any of our competitors. It is for that reason that I suggested group commit get a more prominent billing, and that it actually be officially referred to as group commit. I'm glad that the release notes now actually refer to group commit. Now, I realise that as one of the authors of the feature I am open to accusations of lacking objectivity - clearly it isn't really my place to try and influence the feature's placement, and this is the last I will say on the matter unless someone else brings it up again. I just think that pedantically characterising this as an improvement to our existing group commit implementation within a document that will be read like a press release is a bad decision, especially since our competitors never had a group commit implementation that was far inferior to their current implementation. The assumption will be that it's a small improvement that's hardly worth noticing at all. As to the question of whether or not we should include author names at all, I personally wouldn't mind at all if we removed them, if that would prevent squabbles, which it probably would. However, that's only because I know that it wouldn't really affect my ability to work on Postgres during my working day. I don't know that the same thing can be said at all for people who don't work for one of the handful of Postgres companies. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-14T16:06:13Z
On Sun, May 13, 2012 at 09:01:03PM +0100, Peter Geoghegan wrote: > On 12 May 2012 01:37, Robert Haas <robertmhaas@gmail.com> wrote: > > Right. It's not a new feature; it's a performance improvement. We've > > had group commit for a long time; it just didn't work very well > > before. And it's not batching the commits better; it's reducing the > > lock contention around realizing that the batched commit has happened. > > This understanding of group commit is technically accurate, but the > practical implications of the new code are that lots of people benefit > from group commit, potentially to rather a large degree, where before > they had exactly no benefit from group commit. We never officially > called group commit group commit outside of git commit messages > before. Therefore, it is sort of like we didn't have group commit > before but now we do, and it's an implementation that's probably as > effective as that of any of our competitors. It is for that reason > that I suggested group commit get a more prominent billing, and that > it actually be officially referred to as group commit. I'm glad that > the release notes now actually refer to group commit. > > Now, I realise that as one of the authors of the feature I am open to > accusations of lacking objectivity - clearly it isn't really my place > to try and influence the feature's placement, and this is the last I > will say on the matter unless someone else brings it up again. I just > think that pedantically characterising this as an improvement to our > existing group commit implementation within a document that will be > read like a press release is a bad decision, especially since our > competitors never had a group commit implementation that was far > inferior to their current implementation. The assumption will be that > it's a small improvement that's hardly worth noticing at all. Thanks for the summary. I know we talk about group commit, but I wasn't aware that it had not been exposed to our general users. I agree we need to reword the item as you suggested. So this group commit happens even if users don't change these? #commit_delay = 0 # range 0-100000, in microseconds #commit_siblings = 5 # range 1-1000 So the new release item wording will be: Add group commit capability for sessions that commit at the same time This is the git commit message: Make group commit more effective. When a backend needs to flush the WAL, and someone else is already flushing the WAL, wait until it releases the WALInsertLock and check if we still need to do the flush or if the other backend already did the work for us, before acquiring WALInsertLock. This helps group commit, because when the WAL flush finishes, all the backends that were waiting for it can be woken up in one go, and the can all concurrently observe that they're done, rather than waking them up one by one in a cascading fashion. This is based on a new LWLock function, LWLockWaitUntilFree(), which has peculiar semantics. If the lock is immediately free, it grabs the lock and returns true. If it's not free, it waits until it is released, but then returns false without grabbing the lock. This is used in XLogFlush(), so that when the lock is acquired, the backend flushes the WAL, but if it's not, the backend first checks the current flush location before retrying. Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although this patch as committed ended up being very different from that. (Heikki Linnakangas) Is that commit message inaccurate? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-14T16:24:03Z
On May 14, 2012, at 9:06 AM, Bruce Momjian <bruce@momjian.us> wrote: > So the new release item wording will be: > > Add group commit capability for sessions that commit at the same > time > > This is the git commit message: > > Make group commit more effective. > > When a backend needs to flush the WAL, and someone else is already flushing > the WAL, wait until it releases the WALInsertLock and check if we still need > to do the flush or if the other backend already did the work for us, before > acquiring WALInsertLock. This helps group commit, because when the WAL flush > finishes, all the backends that were waiting for it can be woken up in one > go, and the can all concurrently observe that they're done, rather than > waking them up one by one in a cascading fashion. > > This is based on a new LWLock function, LWLockWaitUntilFree(), which has > peculiar semantics. If the lock is immediately free, it grabs the lock and > returns true. If it's not free, it waits until it is released, but then > returns false without grabbing the lock. This is used in XLogFlush(), so > that when the lock is acquired, the backend flushes the WAL, but if it's > not, the backend first checks the current flush location before retrying. > > Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although > this patch as committed ended up being very different from that. > > (Heikki Linnakangas) > > Is that commit message inaccurate? No, I think it's actually more accurate than the proposed release note wording. ...Robert
-
Re: Draft release notes complete
Jeff Janes <jeff.janes@gmail.com> — 2012-05-14T16:56:13Z
On Mon, May 14, 2012 at 9:06 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Sun, May 13, 2012 at 09:01:03PM +0100, Peter Geoghegan wrote: >> On 12 May 2012 01:37, Robert Haas <robertmhaas@gmail.com> wrote: >> > Right. It's not a new feature; it's a performance improvement. We've >> > had group commit for a long time; it just didn't work very well >> > before. And it's not batching the commits better; it's reducing the >> > lock contention around realizing that the batched commit has happened. >> >> This understanding of group commit is technically accurate, but the >> practical implications of the new code are that lots of people benefit >> from group commit, potentially to rather a large degree, where before >> they had exactly no benefit from group commit. We never officially >> called group commit group commit outside of git commit messages >> before. Therefore, it is sort of like we didn't have group commit >> before but now we do, and it's an implementation that's probably as >> effective as that of any of our competitors. It is for that reason >> that I suggested group commit get a more prominent billing, and that >> it actually be officially referred to as group commit. I'm glad that >> the release notes now actually refer to group commit. >> >> Now, I realise that as one of the authors of the feature I am open to >> accusations of lacking objectivity - clearly it isn't really my place >> to try and influence the feature's placement, and this is the last I >> will say on the matter unless someone else brings it up again. I just >> think that pedantically characterising this as an improvement to our >> existing group commit implementation within a document that will be >> read like a press release is a bad decision, especially since our >> competitors never had a group commit implementation that was far >> inferior to their current implementation. The assumption will be that >> it's a small improvement that's hardly worth noticing at all. > > Thanks for the summary. I know we talk about group commit, but I wasn't > aware that it had not been exposed to our general users. I agree we > need to reword the item as you suggested. So this group commit happens > even if users don't change these? > > #commit_delay = 0 # range 0-100000, in microseconds > #commit_siblings = 5 # range 1-1000 If a bunch of people are standing around waiting for a door to unlock and they are mutually aware of each other, it has never been the case (or at least not for years) that the first person through the door would systematically slam it in everyone else's face. Is this enough to qualify as "group commit"? If so, group commit has "always" (again, at least for years) been there. The new code simply makes it less likely that the group will trip over each others feet as they all stream through the door. The commit_delay settings cover the case where the door unlocks, and you open it, but then perhaps you stand there for an a few minutes holding it open in case someone else happens to show up. This is pretty much orthogonal to the prior case. You can not wait for new people to show up, but trip over the feet of the people already there. Or you can wait for new people to show up, then trip over them. Or not trip over them, with or without waiting for new arrival. (For the analogy to work, this particular door refuses to unlock more than once every 5 minutes. Maybe it is for a very slow elevator) > This is the git commit message: > > Make group commit more effective. > > When a backend needs to flush the WAL, and someone else is already flushing > the WAL, wait until it releases the WALInsertLock and check if we still need > to do the flush or if the other backend already did the work for us, before > acquiring WALInsertLock. This helps group commit, because when the WAL flush > finishes, all the backends that were waiting for it can be woken up in one > go, and the can all concurrently observe that they're done, rather than > waking them up one by one in a cascading fashion. > > This is based on a new LWLock function, LWLockWaitUntilFree(), which has > peculiar semantics. If the lock is immediately free, it grabs the lock and > returns true. If it's not free, it waits until it is released, but then > returns false without grabbing the lock. This is used in XLogFlush(), so > that when the lock is acquired, the backend flushes the WAL, but if it's > not, the backend first checks the current flush location before retrying. > > Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although > this patch as committed ended up being very different from that. > > (Heikki Linnakangas) > > Is that commit message inaccurate? I think the commit message is accurate, other than saying WALInsertLock where it meant WALWriteLock. Cheers, Jeff
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-14T19:21:37Z
On 14 May 2012 17:06, Bruce Momjian <bruce@momjian.us> wrote: > So this group commit happens > even if users don't change these? > > #commit_delay = 0 # range 0-100000, in microseconds > #commit_siblings = 5 # range 1-1000 Yes, that's right - the new group commit is not configurable, and is used all the time. I believe that very few people ever change these other settings in production, because it is very risky to tweak them. Read the description of them in Greg Smith's book for a good example of what I mean, or read the docs - "it is rare that adding delay by increasing this parameter will actually improve performance". There may actually be no one that's set commit_delay > 0 at all. All these settings do is insert a sleep of commit_delay microseconds, in the hope that the call to XLogFlush() will then find that doing work is unnecessary due to some other session having got there first. > So the new release item wording will be: > > Add group commit capability for sessions that commit at the same > time I'd say that's almost what I'd like to see, because commit_delay falls far short of the definition of group commit established by other RDBMSs, which is, I assume, why the term was never used for advocacy purposes. The old facility could result in batching of commits specifically by artificially adding latency so that after the delay the sessions found they could fastpath. While it worked under artificial conditions, I think it resulted in relatively small improvements next to new group commit's order-of-magnitude increase in throughput that was demonstrated for a maximally commit-bound workload. The mere ability to notice that an XLogFlush() call is unnecessary and fastpath out could be argued to be an aboriginal group commit, predating even commit_delay, as could skipping duplicate fsync() requests in XLogWrite(), which I think Jeff pointed out, but I don't think anyone actually takes this position. What I'd suggest is something like: """ Add group commit capability so that when a session commits, other, concurrent sessions with pending commits will later automatically check if their commit has been satisfied by the original request (or even some other, intervening request) before actually proceeding. This results in a large increase in transaction throughput for commit-bound workloads, as under these circumstances the actual number of flush requests will be considerably lower than that of earlier versions of PostgreSQL. """ Now, granted, the number of actual flush requests being so high wasn't a problem because of any actual number of follow-through duplicate fsync() system calls (or writes) - XLogWrite() notices them, and probably has forever (although I read suggestion that some MySQL flavours might have actually been stupid enough to do so pre group commit). But in order to be able to notice this, backends previously had to get the WALWriteLock exclusively. I don't think that these are the kind of qualifications that belong here though. People rightfully only care about the practical implications, and whether or not we're competitive in various respects, such as whether or not we tick the group commit box, and whether or not we have pretty graphs that are comparable to those of other database systems. > This is the git commit message: > > Make group commit more effective. > > When a backend needs to flush the WAL, and someone else is already flushing > the WAL, wait until it releases the WALInsertLock and check if we still need > to do the flush or if the other backend already did the work for us, before > acquiring WALInsertLock. This helps group commit, because when the WAL flush > finishes, all the backends that were waiting for it can be woken up in one > go, and the can all concurrently observe that they're done, rather than > waking them up one by one in a cascading fashion. > > This is based on a new LWLock function, LWLockWaitUntilFree(), which has > peculiar semantics. If the lock is immediately free, it grabs the lock and > returns true. If it's not free, it waits until it is released, but then > returns false without grabbing the lock. This is used in XLogFlush(), so > that when the lock is acquired, the backend flushes the WAL, but if it's > not, the backend first checks the current flush location before retrying. > > Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although > this patch as committed ended up being very different from that. > > (Heikki Linnakangas) > > Is that commit message inaccurate? I wouldn't say that it's inaccurate. However, if we were to deprecate commit_delay and commit_siblings, that would be a mere matter of removing this code (and the GUC stuff itself): if (CommitDelay > 0 && enableFsync && MinimumActiveBackends(CommitSiblings)) pg_usleep(CommitDelay); This code is just before one call (of several) to XLogFlush(). The guts of the new group commit are in that function itself. While my call to deprecate commit_delay + commit_siblings in another thread may have been premature (more on that later), this is basically orthogonal code. That said, most of the possible use-cases for "old" group commit may overlap with that of "new". So in that very limited sense only it is an improvement on the old group commit. Again, I don't think that that's the kind of subtle distinction that needs to be made here. This is a documented intended for the widest possible audience. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services -
Re: Draft release notes complete
Jeff Janes <jeff.janes@gmail.com> — 2012-05-14T19:29:19Z
On Mon, May 14, 2012 at 9:56 AM, Jeff Janes <jeff.janes@gmail.com> wrote: > On Mon, May 14, 2012 at 9:06 AM, Bruce Momjian <bruce@momjian.us> wrote: > >> This is the git commit message: >> >> Make group commit more effective. >> >> When a backend needs to flush the WAL, and someone else is already flushing >> the WAL, wait until it releases the WALInsertLock and check if we still need >> to do the flush or if the other backend already did the work for us, before >> acquiring WALInsertLock. This helps group commit, because when the WAL flush >> finishes, all the backends that were waiting for it can be woken up in one >> go, and the can all concurrently observe that they're done, rather than >> waking them up one by one in a cascading fashion. >> >> This is based on a new LWLock function, LWLockWaitUntilFree(), which has >> peculiar semantics. If the lock is immediately free, it grabs the lock and >> returns true. If it's not free, it waits until it is released, but then >> returns false without grabbing the lock. This is used in XLogFlush(), so >> that when the lock is acquired, the backend flushes the WAL, but if it's >> not, the backend first checks the current flush location before retrying. >> >> Original patch and benchmarking by Peter Geoghegan and Simon Riggs, although >> this patch as committed ended up being very different from that. >> >> (Heikki Linnakangas) >> >> Is that commit message inaccurate? > > I think the commit message is accurate, other than saying > WALInsertLock where it meant WALWriteLock. Sorry, wrong number of negations. "I think the commit message is accurate, other than" Cheers, Jeff
-
Re: Draft release notes complete
Merlin Moncure <mmoncure@gmail.com> — 2012-05-14T21:28:44Z
On Wed, May 9, 2012 at 10:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. The beta release announcement is on postgresql.org with a direct link to the release notes. The notes lead off with: NARRATIVE HERE. Major enhancements include: MAJOR LIST HERE That looks a little unfinished -- this is exactly where many people land to see what's coming with the new release. Need some help putting together the major feature list? merlin
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-15T16:51:33Z
On Mon, May 14, 2012 at 3:21 PM, Peter Geoghegan <peter@2ndquadrant.com> wrote: > The mere ability to notice that an XLogFlush() call is unnecessary and > fastpath out could be argued to be an aboriginal group commit, > predating even commit_delay, as could skipping duplicate fsync() > requests in XLogWrite(), which I think Jeff pointed out, but I don't > think anyone actually takes this position. Well, Tom appears to have to have he'd implemented group commit in 2002. http://archives.postgresql.org/pgsql-hackers/2002-10/msg00331.php More accurately, he seems to have thought that group commit was already there, and he'd improved it. So saying that we're getting it for the first time ten years later seems pretty odd to me. I don't deny that the new feature is a significant improvement under the right circumstances. But I still maintain it's an improvement of something that was already there, rather than something new. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-15T16:57:37Z
On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > On 05/11/2012 05:32 AM, Magnus Hagander wrote: >> >> >> But in the interest of actually being productive - what *is* the >> usecase for needing a 5 minute turnaround time? I don't buy the "check >> what a patch looks like", because that should be done *before* the >> commit, not after - so it's best verified by a local docs build anyway >> (which will also be faster). >> >> I'm sure we can put something in with a pretty quick turnaround again >> without too much strain on the system, but it does, as I mentioned >> before, require decoupling it from the buildfarm which means it's not >> just tweaking a config file. > > > If it's of any use to you I have made some adjustments to the buildfarm code > which would let you do *just* the docs build (and dist make if you want). It > would still pull from git, and only do anything if there's a (relevant) > change. So using that to set up a machine that would run every few minutes > might work. Of course, building the docs can itself be fairly compute > intensive, so you still might not want to run every few minutes if that's a > limiting factor. that would definitely be useful. Compute intensive is not really a problem, we can easily shape the box on that (and I think we already do). Do you have some details of what to do and how to do it to use that, so Stefan can set it up for us ? ;) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Greg Sabino Mullane <greg@turnstep.com> — 2012-05-15T16:59:58Z
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 Bruce wrote: > In summary, names on release note items potentially have the > following beneficial effects: > > * Encouraging new developers/reviewers > * Encouraging long-established developers > * Showing appreciation to developers > * Assisting future employment for developers > * Helping developers get future funding > * Assigning responsibility for features > * Showing Postgres's increased developer base The only important ones are: > * Assisting future employment for developers > * Helping developers get future funding > * Assigning responsibility for features > * Assigning blame for feature problems That last one is not very important either. If there is a bug, you report it. The original author may or may not handle it. A better way to state some of the above is: * Quick cross-reference of a person to a feature. If I claim to have written ON_ERROR_ROLLBACK, nobody should have to scroll back through git logs to confirm or deny. (For that matter, we should do everything possible to prevent anyone from using git log, especially non-developers, for any meta-information.) +1 to keep things they way they are. If you were significantly invested in [re]writing the patch, you get a name. Reviewers, I love you dearly, but you don't belong next to the patch. Group them all at the bottom if we must have them there. - -- Greg Sabino Mullane greg@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201205151259 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAk+yi3cACgkQvJuQZxSWSsiAcACfYC1HCxbMor/c0EJF6kn+XKc9 kOcAoMn0vnOJLa8+HVz5oWKAZxjkOtQi =eiUT -----END PGP SIGNATURE-----
-
Re: Draft release notes complete
Greg Sabino Mullane <greg@turnstep.com> — 2012-05-15T17:03:31Z
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 > I'd vote for starting a separate thread to solicit people's opinions > on whether we need names in the release notes. Is there anybody on > -hackers who would be offended, or would have a harder time persuading > $BOSS to let them spend time on Postgres if they weren't mentioned in > the release notes? There'd still be a policy of crediting people in > commit messages of course, but it's not clear to me whether the release > note mentions are important to anybody. Looks like this is mostly answered, and we obviously don't need another thread, but the answer to the above is "yes". Release notes are very public, plain text, easy to read, very archived and searchable. Commit messages might as well be a black hole as far as visibility to anyone not a developer in the project. - -- Greg Sabino Mullane greg@turnstep.com End Point Corporation http://www.endpoint.com/ PGP Key: 0x14964AC8 201205151301 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAk+yjFEACgkQvJuQZxSWSsi3gACgmikPzvshZPftTuEdmcB8/Ply 4vMAn1DxvG6hntfxJzWRDdPyWlP5X7WM =pUbl -----END PGP SIGNATURE-----
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-16T04:41:54Z
On 15 May 2012 17:51, Robert Haas <robertmhaas@gmail.com> wrote: > More accurately, he seems to have thought that group commit was > already there, and he'd improved it. So saying that we're getting it > for the first time ten years later seems pretty odd to me. Maybe it's odd, and maybe it's inconsistent with earlier terminology that was privately used, and maybe I'm just plain wrong. Nevertheless, it is my position that: 1. Group commit isn't a rigorously defined term, which sure is apparent by our confusion. So even if you're right, that's only by virtue of a precedent being set regarding the terminology, for which there could just as easily have been another precedent without there having to be substantive differences to the code, had things happened to go that way. 2. Group commit is associated in people's minds with results that look much like the results we can now show. It is my understanding that we couldn't show improvements like this before. So while group commit isn't rigorously defined, people have a certain vague set of expectations about it that we previously basically failed to meet. For these reasons, it may be timely and appropriate, from a purely advocacy point-of-view, to call our new group commit "group commit" in release notes and documentation, and announce it as a new feature. -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Jeff Janes <jeff.janes@gmail.com> — 2012-05-16T19:38:15Z
On Wed, May 9, 2012 at 8:11 PM, Bruce Momjian <bruce@momjian.us> wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. I am waiting for our development docs to build, but after 40 > minutes, I am still waiting: > > http://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=guaibasaurus&dt=latest&stg=make-doc > > (Why is there no time zone shown in the date/time at the top?) I think > it will eventually show up here: > > http://www.postgresql.org/docs/devel/static/release-9-2.html > For item: Improve COPY performance by adding tuples to the heap in batches (Heikki Linnakangas) I think we should point out that the batching only applies for COPY into unindexed tables. Nice as the feature is, that is pretty big limitation not to mention. Also, I wouldn't use "to the heap", as I think the main improvement is from the batching of the wal records, not the heap, and also because the basic improvement (get data into TABLES faster) can be understood by users who don't know what a heap is, so we should avoid referring to extraneous implementation details when they are not critical to understanding the feature. Thanks, Jeff
-
Re: Draft release notes complete
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2012-05-16T19:49:25Z
On 16.05.2012 22:38, Jeff Janes wrote: > For item: > Improve COPY performance by adding tuples to the heap in batches > (Heikki Linnakangas) > > I think we should point out that the batching only applies for COPY > into unindexed tables. Nice as the feature is, that is pretty big > limitation not to mention. No, it applies to indexed tables too. However, if there are indexes on the table, the overhead of updating the indexes will probably make any speedup in the heap insertions look tiny in comparison. The optimization doesn't apply if the table has BEFORE or INSTEAD OF triggers, or volatile DEFAULT expressions need to be evaluated. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-16T21:30:27Z
I will make the adjustments outlined below as soon as I can. --------------------------------------------------------------------------- On Sun, May 13, 2012 at 12:37:52AM -0400, Robert Haas wrote: > On Sat, May 12, 2012 at 8:11 PM, Euler Taveira <euler@timbira.com> wrote: > > On 12-05-2012 10:27, Bruce Momjian wrote: > >> How many names on a single item is ideal? The activity of reviewers and > >> their names on commit messages has greatly expanded the number of > >> potential names per item. > >> > > Main authors only. Reviewers should be mentioned only in the commit log. If I > > coded a feature and Bruce got the idea worked in another patch (that is better > > then mine), I think only Bruce should be credited in release notes (but I > > could be mentioned in the commit log as the feature designer). However, if I > > posted a patch and Robert improved that patch using only 30% of my work, I > > should be credited (as coauthor) because he used a considerable part of my work. > > Completely agreed. If we're going to include names in the release > notes, I agree that this is the way to do it, and I think it's what we > have done in prior releases. > > I tend to err on the side of crediting people in the commit message > (of course, occasionally I forget someone who should have been > included), but I also try to make it clear by the phrasing whose code > got included and who contributed in some other way - e.g. by reporting > the problem, coming up with the original idea, or reviewing. I do > this in part because I assumed that we'd use that as the criteria for > including names in the release notes, as we have done in prior > releases. So if I write: > > Euler Taveira, reviewed by Bruce Momjian, substantially rewritten by me > > ...then I expect that to turn up in the release notes as (Euler > Taveira, Robert Haas). If I write: > > Euler Taveira, reviewed by Bruce Momjian, with minor cleanup by me > > ...then I expect that to turn up as (Euler Taveira). And if I write > something like: > > Inspired by a patch from Euler Taveira. Review (in earlier versions) > by Bruce Momjian. > > ...then I expect that to turn up as (Robert Haas) or (Robert Haas, > Euler Taveira). > > In doubtful cases, I think it's generally appropriate to err on the > side of crediting the person who was the original driving force behind > the patch, and also to err on the side of not crediting the committer. > But if the committer chopped up the patch and committed something > significantly different from the original, then they should be > credited - or blamed - for the result. > > -- > Robert Haas > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-21T18:10:09Z
> For these reasons, it may be timely and appropriate, from a purely > advocacy point-of-view, to call our new group commit "group commit" in > release notes and documentation, and announce it as a new feature. First, shouldn't we be having this discussion on -advocacy? To date, I've been calling it "better group commit". -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Noah Misch <noah@leadboat.com> — 2012-05-22T01:54:44Z
On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: > I have completed my draft of the 9.2 release notes, and committed it to > git. Concerning "Have psql \copy use libpq's SendQuery()", SendQuery() is a psql-internal interface, not a libpq interface. The array statistics patch added new columns to the pg_stats view, and it moved existing tsvector most-common-element statistics to those new columns. Let's mention that as a (minor) incompatibility. Proposed changes attached. Thanks, nm
-
Re: Draft release notes complete
Robert Haas <robertmhaas@gmail.com> — 2012-05-22T14:21:48Z
On Mon, May 21, 2012 at 9:54 PM, Noah Misch <noah@leadboat.com> wrote: > On Wed, May 09, 2012 at 11:11:02PM -0400, Bruce Momjian wrote: >> I have completed my draft of the 9.2 release notes, and committed it to >> git. > > Concerning "Have psql \copy use libpq's SendQuery()", SendQuery() is a > psql-internal interface, not a libpq interface. > > The array statistics patch added new columns to the pg_stats view, and it > moved existing tsvector most-common-element statistics to those new columns. > Let's mention that as a (minor) incompatibility. > > Proposed changes attached. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-22T21:17:43Z
On Wed, May 16, 2012 at 05:30:27PM -0400, Bruce Momjian wrote: > > I will make the adjustments outlined below as soon as I can. Done and committed. --------------------------------------------------------------------------- > > On Sun, May 13, 2012 at 12:37:52AM -0400, Robert Haas wrote: > > On Sat, May 12, 2012 at 8:11 PM, Euler Taveira <euler@timbira.com> wrote: > > > On 12-05-2012 10:27, Bruce Momjian wrote: > > >> How many names on a single item is ideal? The activity of reviewers and > > >> their names on commit messages has greatly expanded the number of > > >> potential names per item. > > >> > > > Main authors only. Reviewers should be mentioned only in the commit log. If I > > > coded a feature and Bruce got the idea worked in another patch (that is better > > > then mine), I think only Bruce should be credited in release notes (but I > > > could be mentioned in the commit log as the feature designer). However, if I > > > posted a patch and Robert improved that patch using only 30% of my work, I > > > should be credited (as coauthor) because he used a considerable part of my work. > > > > Completely agreed. If we're going to include names in the release > > notes, I agree that this is the way to do it, and I think it's what we > > have done in prior releases. > > > > I tend to err on the side of crediting people in the commit message > > (of course, occasionally I forget someone who should have been > > included), but I also try to make it clear by the phrasing whose code > > got included and who contributed in some other way - e.g. by reporting > > the problem, coming up with the original idea, or reviewing. I do > > this in part because I assumed that we'd use that as the criteria for > > including names in the release notes, as we have done in prior > > releases. So if I write: > > > > Euler Taveira, reviewed by Bruce Momjian, substantially rewritten by me > > > > ...then I expect that to turn up in the release notes as (Euler > > Taveira, Robert Haas). If I write: > > > > Euler Taveira, reviewed by Bruce Momjian, with minor cleanup by me > > > > ...then I expect that to turn up as (Euler Taveira). And if I write > > something like: > > > > Inspired by a patch from Euler Taveira. Review (in earlier versions) > > by Bruce Momjian. > > > > ...then I expect that to turn up as (Robert Haas) or (Robert Haas, > > Euler Taveira). > > > > In doubtful cases, I think it's generally appropriate to err on the > > side of crediting the person who was the original driving force behind > > the patch, and also to err on the side of not crediting the committer. > > But if the committer chopped up the patch and committed something > > significantly different from the original, then they should be > > credited - or blamed - for the result. > > > > -- > > Robert Haas > > EnterpriseDB: http://www.enterprisedb.com > > The Enterprise PostgreSQL Company > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + It's impossible for everything to be true. + > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-22T21:26:45Z
On Thu, May 10, 2012 at 10:22:58PM +0400, Alexander Korotkov wrote: > "Improve GiST box and point index performance by producing better trees with > less memory allocation overhead (Alexander Korotkov, Heikki Linnakangas, Kevin > Grittner)" > Is this note about following two commits? > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > 7f3bd86843e5aad84585a57d3f6b80db3c609916 > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > d50e1251946a6e59092f0a84fc903532eb599a4f > These improvements influence not only boxes and points but all geometrical > datatypes. OK, new wording: Improve GiST geometric type index performance by producing better trees with less memory allocation overhead (Alexander Korotkov) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-22T21:27:47Z
On Wed, May 16, 2012 at 10:49:25PM +0300, Heikki Linnakangas wrote: > On 16.05.2012 22:38, Jeff Janes wrote: > >For item: > >Improve COPY performance by adding tuples to the heap in batches > >(Heikki Linnakangas) > > > >I think we should point out that the batching only applies for COPY > >into unindexed tables. Nice as the feature is, that is pretty big > >limitation not to mention. > > No, it applies to indexed tables too. However, if there are indexes > on the table, the overhead of updating the indexes will probably > make any speedup in the heap insertions look tiny in comparison. > > The optimization doesn't apply if the table has BEFORE or INSTEAD OF > triggers, or volatile DEFAULT expressions need to be evaluated. So, I will assume the existing wording is fine. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Alexander Korotkov <aekorotkov@gmail.com> — 2012-05-22T21:38:06Z
On Wed, May 23, 2012 at 1:26 AM, Bruce Momjian <bruce@momjian.us> wrote: > On Thu, May 10, 2012 at 10:22:58PM +0400, Alexander Korotkov wrote: > > "Improve GiST box and point index performance by producing better trees > with > > less memory allocation overhead (Alexander Korotkov, Heikki Linnakangas, > Kevin > > Grittner)" > > Is this note about following two commits? > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > > 7f3bd86843e5aad84585a57d3f6b80db3c609916 > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > > d50e1251946a6e59092f0a84fc903532eb599a4f > > These improvements influence not only boxes and points but all > geometrical > > datatypes. > > OK, new wording: > > Improve GiST geometric type index performance by producing better > trees with less memory allocation overhead (Alexander Korotkov) Thanks! Also, I've some notes about removing reviewers. "Improve GiST index build times (Alexander Korotkov)" I think Heikki Linnakangas should be also listed as author of that patch because he didn't only review and commit, but actually put his hands on code. Isn't my authorship of this patch lost now? http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=80da9e68fdd70b796b3a7de3821589513596c0f7 I think earlier this patch was taken into account in entry "Add support for range data types". Probably, we need separate entry for this patch? ------ With best regards, Alexander Korotkov.
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-23T03:00:15Z
On Wed, May 23, 2012 at 01:38:06AM +0400, Alexander Korotkov wrote: > On Wed, May 23, 2012 at 1:26 AM, Bruce Momjian <bruce@momjian.us> wrote: > > On Thu, May 10, 2012 at 10:22:58PM +0400, Alexander Korotkov wrote: > > "Improve GiST box and point index performance by producing better trees > with > > less memory allocation overhead (Alexander Korotkov, Heikki Linnakangas, > Kevin > > Grittner)" > > Is this note about following two commits? > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > > 7f3bd86843e5aad84585a57d3f6b80db3c609916 > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > > d50e1251946a6e59092f0a84fc903532eb599a4f > > These improvements influence not only boxes and points but all > geometrical > > datatypes. > > OK, new wording: > > Improve GiST geometric type index performance by producing better > trees with less memory allocation overhead (Alexander Korotkov) > > > Thanks! > > Also, I've some notes about removing reviewers. > "Improve GiST index build times (Alexander Korotkov)" > I think Heikki Linnakangas should be also listed as author of that patch > because he didn't only review and commit, but actually put his hands on code. OK, Heikki added. > Isn't my authorship of this patch lost now? > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h= > 80da9e68fdd70b796b3a7de3821589513596c0f7 > I think earlier this patch was taken into account in entry "Add support for > range data types". Probably, we need separate entry for this patch? I thought that was more of a Gist index improvement than a range type improvement, but I have added your name to the range type item. Thanks. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-24T21:34:22Z
On 21 May 2012 19:10, Josh Berkus <josh@agliodbs.com> wrote: > >> For these reasons, it may be timely and appropriate, from a purely >> advocacy point-of-view, to call our new group commit "group commit" in >> release notes and documentation, and announce it as a new feature. > > First, shouldn't we be having this discussion on -advocacy? Well, no, because this is a specific discussion about release notes. In any case, I've given up on the idea that we should market new group commit as "group commit". I believe that that would be a useful and fair way of representing the feature, but there doesn't seem to be any support for that view. In passing, I noticed this: """ E.1.3.12.2. pg_stat_statements Improve pg_stat_statements to aggregate similar queries (Peter Geoghegan, Tom Lane) Improve pg_stat_statements' handling of PREPARE/EXECUTE statements (Tom Lane) Add dirtied and written block counts to pg_stat_statements (Robert Haas) """ I think that the second entry should be listed as a bug fix, or a compatibility note, rather than an actual feature. At the very least, it should be listed after "Add dirtied and written block counts". I also think that we should separately list as a feature pg_stat_statements new ability to track I/O timings at the query granularity. Are we any closer to a list of major features? -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-24T21:57:19Z
On Thu, May 24, 2012 at 10:34:22PM +0100, Peter Geoghegan wrote: > In passing, I noticed this: > > """ > E.1.3.12.2. pg_stat_statements > > Improve pg_stat_statements to aggregate similar queries (Peter > Geoghegan, Tom Lane) > > Improve pg_stat_statements' handling of PREPARE/EXECUTE statements (Tom Lane) > > Add dirtied and written block counts to pg_stat_statements (Robert Haas) > """ > > I think that the second entry should be listed as a bug fix, or a > compatibility note, rather than an actual feature. At the very least, > it should be listed after "Add dirtied and written block counts". I > also think that we should separately list as a feature OK, item moved down. We have not have "bug fix" designation. You have a suggestion? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Peter Geoghegan <peter@2ndquadrant.com> — 2012-05-24T22:16:28Z
On 24 May 2012 22:57, Bruce Momjian <bruce@momjian.us> wrote: > OK, item moved down. We have not have "bug fix" designation. You have > a suggestion? I assumed you were going to put it beside the other compatibility note relating to pg_stat_statements, "Change pg_stat_statements' total_time column to be measured in milliseconds (Tom Lane)". The "Improve pg_stat_statements' handling of PREPARE/EXECUTE statements" is just a way of preventing SQL PREPARE and EXECUTE utility statements from being double counted in various ways as both utility statements and optimisable statements. No one actually noticed this before, and it wouldn't have been feasible to fix in back branches, I think. Here are the relevant comments: * If it's an EXECUTE statement, we don't track it and don't increment * the nesting level. This allows the cycles to be charged to the * underlying PREPARE instead (by the Executor hooks), which is much more * useful. * * We also don't track execution of PREPARE. If we did, we would get one * hash table entry for the PREPARE (with hash calculated from the query * string), and then a different one with the same query string (but hash * calculated from the query tree) would be used to accumulate costs of * ensuing EXECUTEs. This would be confusing, and inconsistent with other * cases where planning time is not included at all. Also, as I've said, this I/O timings thing certainly deserves to be separately listed as a new pg_stat_statements feature: http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=5b4f346611431361339253203d486789e4babb02 -- Peter Geoghegan http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training and Services
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-24T23:11:20Z
On Thu, May 24, 2012 at 11:16:28PM +0100, Peter Geoghegan wrote: > On 24 May 2012 22:57, Bruce Momjian <bruce@momjian.us> wrote: > > OK, item moved down. We have not have "bug fix" designation. You have > > a suggestion? > > I assumed you were going to put it beside the other compatibility note > relating to pg_stat_statements, "Change pg_stat_statements' total_time > column to be measured in milliseconds (Tom Lane)". > > The "Improve pg_stat_statements' handling of PREPARE/EXECUTE > statements" is just a way of preventing SQL PREPARE and EXECUTE > utility statements from being double counted in various ways as both > utility statements and optimisable statements. No one actually noticed > this before, and it wouldn't have been feasible to fix in back > branches, I think. Here are the relevant comments: > > * If it's an EXECUTE statement, we don't track it and don't increment > * the nesting level. This allows the cycles to be charged to the > * underlying PREPARE instead (by the Executor hooks), which is much more > * useful. > * > * We also don't track execution of PREPARE. If we did, we would get one > * hash table entry for the PREPARE (with hash calculated from the query > * string), and then a different one with the same query string (but hash > * calculated from the query tree) would be used to accumulate costs of > * ensuing EXECUTEs. This would be confusing, and inconsistent with other > * cases where planning time is not included at all. OK, I updated the wording on this, but don't see it as something incompatibile, in the same way that the others listed are incompatible. > > Also, as I've said, this I/O timings thing certainly deserves to be > separately listed as a new pg_stat_statements feature: > > http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=5b4f346611431361339253203d486789e4babb02 OK, I merged that into the existing item. Applied patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-05-25T17:45:35Z
On 5/24/12 2:34 PM, Peter Geoghegan wrote: > On 21 May 2012 19:10, Josh Berkus <josh@agliodbs.com> wrote: >> >>> For these reasons, it may be timely and appropriate, from a purely >>> advocacy point-of-view, to call our new group commit "group commit" in >>> release notes and documentation, and announce it as a new feature. >> >> First, shouldn't we be having this discussion on -advocacy? > > Well, no, because this is a specific discussion about release notes. True, but there's also the question of what we call this in the promotional materials. > In any case, I've given up on the idea that we should market new group > commit as "group commit". I believe that that would be a useful and > fair way of representing the feature, but there doesn't seem to be any > support for that view. What else would you call it? What's wrong with "Better Group Commit"? From my perspective, it's pretty simple: we had group commit before, but the new group commit is much better. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-05-31T15:55:18Z
On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: > On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan <andrew@dunslane.net> wrote: > > > > > > On 05/11/2012 05:32 AM, Magnus Hagander wrote: > >> > >> > >> But in the interest of actually being productive - what *is* the > >> usecase for needing a 5 minute turnaround time? I don't buy the "check > >> what a patch looks like", because that should be done *before* the > >> commit, not after - so it's best verified by a local docs build anyway > >> (which will also be faster). > >> > >> I'm sure we can put something in with a pretty quick turnaround again > >> without too much strain on the system, but it does, as I mentioned > >> before, require decoupling it from the buildfarm which means it's not > >> just tweaking a config file. > > > > > > If it's of any use to you I have made some adjustments to the buildfarm code > > which would let you do *just* the docs build (and dist make if you want). It > > would still pull from git, and only do anything if there's a (relevant) > > change. So using that to set up a machine that would run every few minutes > > might work. Of course, building the docs can itself be fairly compute > > intensive, so you still might not want to run every few minutes if that's a > > limiting factor. > > that would definitely be useful. Compute intensive is not really a > problem, we can easily shape the box on that (and I think we already > do). > > Do you have some details of what to do and how to do it to use that, > so Stefan can set it up for us ? ;) Where are we on building the development docs more frequently? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-05-31T15:58:58Z
On Thu, May 31, 2012 at 5:55 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: >> On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan <andrew@dunslane.net> wrote: >> > >> > >> > On 05/11/2012 05:32 AM, Magnus Hagander wrote: >> >> >> >> >> >> But in the interest of actually being productive - what *is* the >> >> usecase for needing a 5 minute turnaround time? I don't buy the "check >> >> what a patch looks like", because that should be done *before* the >> >> commit, not after - so it's best verified by a local docs build anyway >> >> (which will also be faster). >> >> >> >> I'm sure we can put something in with a pretty quick turnaround again >> >> without too much strain on the system, but it does, as I mentioned >> >> before, require decoupling it from the buildfarm which means it's not >> >> just tweaking a config file. >> > >> > >> > If it's of any use to you I have made some adjustments to the buildfarm code >> > which would let you do *just* the docs build (and dist make if you want). It >> > would still pull from git, and only do anything if there's a (relevant) >> > change. So using that to set up a machine that would run every few minutes >> > might work. Of course, building the docs can itself be fairly compute >> > intensive, so you still might not want to run every few minutes if that's a >> > limiting factor. >> >> that would definitely be useful. Compute intensive is not really a >> problem, we can easily shape the box on that (and I think we already >> do). >> >> Do you have some details of what to do and how to do it to use that, >> so Stefan can set it up for us ? ;) > > Where are we on building the development docs more frequently? Still waiting for details on how it works to set that up on the buildfarm client. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-08-30T01:25:11Z
On Thu, May 31, 2012 at 05:58:58PM +0200, Magnus Hagander wrote: > On Thu, May 31, 2012 at 5:55 PM, Bruce Momjian <bruce@momjian.us> wrote: > > On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: > >> On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan <andrew@dunslane.net> wrote: > >> > > >> > > >> > On 05/11/2012 05:32 AM, Magnus Hagander wrote: > >> >> > >> >> > >> >> But in the interest of actually being productive - what *is* the > >> >> usecase for needing a 5 minute turnaround time? I don't buy the "check > >> >> what a patch looks like", because that should be done *before* the > >> >> commit, not after - so it's best verified by a local docs build anyway > >> >> (which will also be faster). > >> >> > >> >> I'm sure we can put something in with a pretty quick turnaround again > >> >> without too much strain on the system, but it does, as I mentioned > >> >> before, require decoupling it from the buildfarm which means it's not > >> >> just tweaking a config file. > >> > > >> > > >> > If it's of any use to you I have made some adjustments to the buildfarm code > >> > which would let you do *just* the docs build (and dist make if you want). It > >> > would still pull from git, and only do anything if there's a (relevant) > >> > change. So using that to set up a machine that would run every few minutes > >> > might work. Of course, building the docs can itself be fairly compute > >> > intensive, so you still might not want to run every few minutes if that's a > >> > limiting factor. > >> > >> that would definitely be useful. Compute intensive is not really a > >> problem, we can easily shape the box on that (and I think we already > >> do). > >> > >> Do you have some details of what to do and how to do it to use that, > >> so Stefan can set it up for us ? ;) > > > > Where are we on building the development docs more frequently? > > Still waiting for details on how it works to set that up on the > buildfarm client. Where are we on this? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@2ndquadrant.com> — 2012-08-30T02:23:31Z
Excerpts from Bruce Momjian's message of mié ago 29 21:25:11 -0400 2012: > On Thu, May 31, 2012 at 05:58:58PM +0200, Magnus Hagander wrote: > > On Thu, May 31, 2012 at 5:55 PM, Bruce Momjian <bruce@momjian.us> wrote: > > > On Tue, May 15, 2012 at 12:57:37PM -0400, Magnus Hagander wrote: > > >> On Fri, May 11, 2012 at 11:44 AM, Andrew Dunstan <andrew@dunslane.net> wrote: > > >> > If it's of any use to you I have made some adjustments to the buildfarm code > > >> > which would let you do *just* the docs build (and dist make if you want). It > > >> > would still pull from git, and only do anything if there's a (relevant) > > >> > change. So using that to set up a machine that would run every few minutes > > >> > might work. Of course, building the docs can itself be fairly compute > > >> > intensive, so you still might not want to run every few minutes if that's a > > >> > limiting factor. > > >> > > >> that would definitely be useful. Compute intensive is not really a > > >> problem, we can easily shape the box on that (and I think we already > > >> do). > > >> > > >> Do you have some details of what to do and how to do it to use that, > > >> so Stefan can set it up for us ? ;) > > > > > > Where are we on building the development docs more frequently? > > > > Still waiting for details on how it works to set that up on the > > buildfarm client. > > Where are we on this? Waiting on Andrew. As far as I can see, we need to update the machine to release 4.7, and then install a "skip file" or something like that. Andrew, can you please explain how is that to be used? I don't see it documented anywhere. Please note that we have cron jobs that run every branch (we use run_build.pl for each branch separately, not run_branches.pl) and a signle build-farm.conf. It would be good if we could continue to use a single config file for all builds, including a build that *only* does the docs. If we can have a second file that only #includes the other one somehow and just tweaks %skip_step, that would work. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-08-30T03:20:51Z
On Wed, 2012-08-29 at 22:23 -0400, Alvaro Herrera wrote: > > > > Where are we on building the development docs more frequently? > > > > > > Still waiting for details on how it works to set that up on the > > > buildfarm client. > > > > Where are we on this? > > Waiting on Andrew. > > As far as I can see, we need to update the machine to release 4.7, and > then install a "skip file" or something like that. Andrew, can you > please explain how is that to be used? I don't see it documented > anywhere. Why does this need to be tied into the build farm? Someone can surely set up a script that just runs the docs build at every check-in, like it used to work. What's being proposed now just sounds like a lot of complication for little or no actual gain -- net loss in fact.
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-08-30T03:52:58Z
On 08/29/2012 11:20 PM, Peter Eisentraut wrote: > On Wed, 2012-08-29 at 22:23 -0400, Alvaro Herrera wrote: >>>>> Where are we on building the development docs more frequently? >>>> Still waiting for details on how it works to set that up on the >>>> buildfarm client. >>> Where are we on this? >> Waiting on Andrew. >> >> As far as I can see, we need to update the machine to release 4.7, and >> then install a "skip file" or something like that. Andrew, can you >> please explain how is that to be used? I don't see it documented >> anywhere. > Why does this need to be tied into the build farm? Someone can surely > set up a script that just runs the docs build at every check-in, like it > used to work. What's being proposed now just sounds like a lot of > complication for little or no actual gain -- net loss in fact. > > It doesn't just build the docs. It makes the dist snapshots too. And the old script often broke badly, IIRC. The current setup doesn't install anything if the build fails, which is a distinct improvement. And you can see the causes of any failure via the buildfarm logs. But I don't really have any stake in this, I just created a tiny Module to help Magnus do what he wanted. In answer to Alvaro's question, you can disable most steps via a command line switch --skip-steps, the value of which is a space separated list of names. I haven't documented it mainly because it was really developed for use in development. The allowed values are: install make make-doc install make-contrib install install-check contrib-install-check pl-install-check isolation-check check ecpg-check cheers andrew -
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-08-30T09:02:09Z
On Thu, Aug 30, 2012 at 5:20 AM, Peter Eisentraut <peter_e@gmx.net> wrote: > On Wed, 2012-08-29 at 22:23 -0400, Alvaro Herrera wrote: >> > > > Where are we on building the development docs more frequently? >> > > >> > > Still waiting for details on how it works to set that up on the >> > > buildfarm client. >> > >> > Where are we on this? >> >> Waiting on Andrew. >> >> As far as I can see, we need to update the machine to release 4.7, and >> then install a "skip file" or something like that. Andrew, can you >> please explain how is that to be used? I don't see it documented >> anywhere. > > Why does this need to be tied into the build farm? Someone can surely > set up a script that just runs the docs build at every check-in, like it > used to work. What's being proposed now just sounds like a lot of > complication for little or no actual gain -- net loss in fact. It doesn't have to. The gain is that we don't then end up conflicting with the buildfarm on when we run. Unlike the old server, we try to avoid running too many concurrent builds because it might kill other services on the same box - which happened quite frequently on the old one. In fact, it happened so frequently on the old one that people stopped caring... We also get the buildfarms functionality for collecting logs and reporting them. Just some more we don't have to build a new set of scripts for. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Peter Eisentraut <peter_e@gmx.net> — 2012-09-05T22:13:21Z
On 8/29/12 11:52 PM, Andrew Dunstan wrote: > >> Why does this need to be tied into the build farm? Someone can surely >> set up a script that just runs the docs build at every check-in, like it >> used to work. What's being proposed now just sounds like a lot of >> complication for little or no actual gain -- net loss in fact. > > It doesn't just build the docs. It makes the dist snapshots too. Thus making the turnaround time on a docs build even slower ... ? > And the old script often broke badly, IIRC. The script broke on occasion, but the main problem was that it wasn't monitored. Which is something that could have been fixed. > The current setup doesn't install > anything if the build fails, which is a distinct improvement. You mean it doesn't build the docs if the code build fails? Would that really be an improvement?
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-05T23:06:14Z
On 09/05/2012 06:13 PM, Peter Eisentraut wrote: > On 8/29/12 11:52 PM, Andrew Dunstan wrote: >>>> Why does this need to be tied into the build farm? Someone can surely >>> set up a script that just runs the docs build at every check-in, like it >>> used to work. What's being proposed now just sounds like a lot of >>> complication for little or no actual gain -- net loss in fact. >> It doesn't just build the docs. It makes the dist snapshots too. > Thus making the turnaround time on a docs build even slower ... ? A complete run of this process takes less than 15 minutes. And as I have pointed out elsewhere that could be reduced substantially by skipping certain steps. It's as simple as changing the command line in the crontab entry. The only reason there is a significant delay is that the administrators have chosen not to run the process more than once every 4 hours. That's a choice not dictated by the process they are using, but by other considerations concerning the machine it's being run on. Since I am not one of the admins and don't really want to take responsibility for it I am not going to second guess them. On the very rare occasions when I absolutely have to have the totally up to date docs I build them myself - it takes about 60 seconds on my modest hardware. cheers andrew
-
Re: Draft release notes complete
Tom Lane <tgl@sss.pgh.pa.us> — 2012-09-05T23:24:08Z
Andrew Dunstan <andrew@dunslane.net> writes: > The only reason there is a significant delay is that the administrators > have chosen not to run the process more than once every 4 hours. That's > a choice not dictated by the process they are using, but by other > considerations concerning the machine it's being run on. Since I am not > one of the admins and don't really want to take responsibility for it I > am not going to second guess them. On the very rare occasions when I > absolutely have to have the totally up to date docs I build them myself > - it takes about 60 seconds on my modest hardware. I think the argument for having a quick docs build service is not about the time needed, but the need to have all the appropriate tools installed. While I can understand that argument for J Random Hacker, I'm mystified why Bruce doesn't seem to have bothered to get a working SGML toolset installed. It's not like editing the docs is a one-shot task for him. regards, tom lane
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@2ndquadrant.com> — 2012-09-06T00:56:32Z
Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012: > Andrew Dunstan <andrew@dunslane.net> writes: > > The only reason there is a significant delay is that the administrators > > have chosen not to run the process more than once every 4 hours. That's > > a choice not dictated by the process they are using, but by other > > considerations concerning the machine it's being run on. Since I am not > > one of the admins and don't really want to take responsibility for it I > > am not going to second guess them. On the very rare occasions when I > > absolutely have to have the totally up to date docs I build them myself > > - it takes about 60 seconds on my modest hardware. > > I think the argument for having a quick docs build service is not about > the time needed, but the need to have all the appropriate tools > installed. While I can understand that argument for J Random Hacker, > I'm mystified why Bruce doesn't seem to have bothered to get a working > SGML toolset installed. It's not like editing the docs is a one-shot > task for him. As far as I understand, Bruce's concern is not about seeing the docs built himself, but having an HTML copy published somewhere that he can point people to, after applying some patch. To me, that's a perfectly legitimate reason to want to have them quickly. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-06T01:25:35Z
On Wed, Sep 5, 2012 at 09:56:32PM -0300, Alvaro Herrera wrote: > Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012: > > Andrew Dunstan <andrew@dunslane.net> writes: > > > The only reason there is a significant delay is that the administrators > > > have chosen not to run the process more than once every 4 hours. That's > > > a choice not dictated by the process they are using, but by other > > > considerations concerning the machine it's being run on. Since I am not > > > one of the admins and don't really want to take responsibility for it I > > > am not going to second guess them. On the very rare occasions when I > > > absolutely have to have the totally up to date docs I build them myself > > > - it takes about 60 seconds on my modest hardware. > > > > I think the argument for having a quick docs build service is not about > > the time needed, but the need to have all the appropriate tools > > installed. While I can understand that argument for J Random Hacker, > > I'm mystified why Bruce doesn't seem to have bothered to get a working > > SGML toolset installed. It's not like editing the docs is a one-shot > > task for him. > > As far as I understand, Bruce's concern is not about seeing the docs > built himself, but having an HTML copy published somewhere that he can > point people to, after applying some patch. To me, that's a perfectly > legitimate reason to want to have them quickly. Correct. I have always had a working SGML toolset. If we are not going to have the developer site run more often, I will just go back to setting up my own public doc build, like I used to do. I removed mine when the official one was more current/reliable --- if that has changed, I will return to my old setup, and publish my own URL for users to verify doc changes. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Josh Berkus <josh@agliodbs.com> — 2012-09-06T01:32:48Z
> Correct. I have always had a working SGML toolset. If we are not going > to have the developer site run more often, I will just go back to > setting up my own public doc build, like I used to do. I removed mine > when the official one was more current/reliable --- if that has changed, > I will return to my old setup, and publish my own URL for users to > verify doc changes. I guess I don't see why building every 4 hours is an issue? That's 6 times/day. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-06T01:33:35Z
On 09/05/2012 09:25 PM, Bruce Momjian wrote: > On Wed, Sep 5, 2012 at 09:56:32PM -0300, Alvaro Herrera wrote: >> Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012: >>> Andrew Dunstan <andrew@dunslane.net> writes: >>>> The only reason there is a significant delay is that the administrators >>>> have chosen not to run the process more than once every 4 hours. That's >>>> a choice not dictated by the process they are using, but by other >>>> considerations concerning the machine it's being run on. Since I am not >>>> one of the admins and don't really want to take responsibility for it I >>>> am not going to second guess them. On the very rare occasions when I >>>> absolutely have to have the totally up to date docs I build them myself >>>> - it takes about 60 seconds on my modest hardware. >>> I think the argument for having a quick docs build service is not about >>> the time needed, but the need to have all the appropriate tools >>> installed. While I can understand that argument for J Random Hacker, >>> I'm mystified why Bruce doesn't seem to have bothered to get a working >>> SGML toolset installed. It's not like editing the docs is a one-shot >>> task for him. >> As far as I understand, Bruce's concern is not about seeing the docs >> built himself, but having an HTML copy published somewhere that he can >> point people to, after applying some patch. To me, that's a perfectly >> legitimate reason to want to have them quickly. > Correct. I have always had a working SGML toolset. If we are not going > to have the developer site run more often, I will just go back to > setting up my own public doc build, like I used to do. I removed mine > when the official one was more current/reliable --- if that has changed, > I will return to my old setup, and publish my own URL for users to > verify doc changes. How often do you want? After all, <http://developer.postgresql.org/docs/postgres/index.html> is presumably going to keep pointing to where it now points. cheers andrew
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-06T01:35:36Z
On Wed, Sep 5, 2012 at 06:32:48PM -0700, Josh Berkus wrote: > > > Correct. I have always had a working SGML toolset. If we are not going > > to have the developer site run more often, I will just go back to > > setting up my own public doc build, like I used to do. I removed mine > > when the official one was more current/reliable --- if that has changed, > > I will return to my old setup, and publish my own URL for users to > > verify doc changes. > > I guess I don't see why building every 4 hours is an issue? That's 6 > times/day. I can't commit and send someone a URL showing the change because they might actually read their email in less than 4 hours. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-06T01:43:29Z
On Wed, Sep 5, 2012 at 09:33:35PM -0400, Andrew Dunstan wrote: > > On 09/05/2012 09:25 PM, Bruce Momjian wrote: > >On Wed, Sep 5, 2012 at 09:56:32PM -0300, Alvaro Herrera wrote: > >>Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012: > >>>Andrew Dunstan <andrew@dunslane.net> writes: > >>>>The only reason there is a significant delay is that the administrators > >>>>have chosen not to run the process more than once every 4 hours. That's > >>>>a choice not dictated by the process they are using, but by other > >>>>considerations concerning the machine it's being run on. Since I am not > >>>>one of the admins and don't really want to take responsibility for it I > >>>>am not going to second guess them. On the very rare occasions when I > >>>>absolutely have to have the totally up to date docs I build them myself > >>>>- it takes about 60 seconds on my modest hardware. > >>>I think the argument for having a quick docs build service is not about > >>>the time needed, but the need to have all the appropriate tools > >>>installed. While I can understand that argument for J Random Hacker, > >>>I'm mystified why Bruce doesn't seem to have bothered to get a working > >>>SGML toolset installed. It's not like editing the docs is a one-shot > >>>task for him. > >>As far as I understand, Bruce's concern is not about seeing the docs > >>built himself, but having an HTML copy published somewhere that he can > >>point people to, after applying some patch. To me, that's a perfectly > >>legitimate reason to want to have them quickly. > >Correct. I have always had a working SGML toolset. If we are not going > >to have the developer site run more often, I will just go back to > >setting up my own public doc build, like I used to do. I removed mine > >when the official one was more current/reliable --- if that has changed, > >I will return to my old setup, and publish my own URL for users to > >verify doc changes. > > How often do you want? After all, > <http://developer.postgresql.org/docs/postgres/index.html> is > presumably going to keep pointing to where it now points. Well, the old code checked every five minutes, and it rebuilt in 4 minutes, so there was a max of 10 minutes delay. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Stephen Frost <sfrost@snowman.net> — 2012-09-06T01:59:50Z
* Bruce Momjian (bruce@momjian.us) wrote: > > How often do you want? After all, > > <http://developer.postgresql.org/docs/postgres/index.html> is > > presumably going to keep pointing to where it now points. > > Well, the old code checked every five minutes, and it rebuilt in 4 > minutes, so there was a max of 10 minutes delay. I'm a bit mystified why we build them far *more* often than necessary.. Do we really commit documentation updates more than 6 times per day? Wouldn't it be reasonably straight-forward to set up a commit-hook that either kicks off a build itself, drops a file marker some place to signal a cron job to do it, or something similar? Have to agree with Bruce on this one, for my part. I wonder if the change to delay the crons was due to lack of proper locking or tracking, or perhaps a lack of a filter for just changes which would impact the documentation.. Thanks, Stephen
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-06T02:08:56Z
On Wed, Sep 5, 2012 at 09:59:50PM -0400, Stephen Frost wrote: > * Bruce Momjian (bruce@momjian.us) wrote: > > > How often do you want? After all, > > > <http://developer.postgresql.org/docs/postgres/index.html> is > > > presumably going to keep pointing to where it now points. > > > > Well, the old code checked every five minutes, and it rebuilt in 4 > > minutes, so there was a max of 10 minutes delay. > > I'm a bit mystified why we build them far *more* often than necessary.. > Do we really commit documentation updates more than 6 times per day? > Wouldn't it be reasonably straight-forward to set up a commit-hook that > either kicks off a build itself, drops a file marker some place to > signal a cron job to do it, or something similar? > > Have to agree with Bruce on this one, for my part. I wonder if the > change to delay the crons was due to lack of proper locking or > tracking, or perhaps a lack of a filter for just changes which would > impact the documentation.. What the script I donated did was to do a cvs update in the sgml directory and look for changes --- if it found them, it rebuilt. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-06T02:12:58Z
On 09/05/2012 09:59 PM, Stephen Frost wrote: > * Bruce Momjian (bruce@momjian.us) wrote: >>> How often do you want? After all, >>> <http://developer.postgresql.org/docs/postgres/index.html> is >>> presumably going to keep pointing to where it now points. >> Well, the old code checked every five minutes, and it rebuilt in 4 >> minutes, so there was a max of 10 minutes delay. > I'm a bit mystified why we build them far *more* often than necessary.. > Do we really commit documentation updates more than 6 times per day? > Wouldn't it be reasonably straight-forward to set up a commit-hook that > either kicks off a build itself, drops a file marker some place to > signal a cron job to do it, or something similar? > > Have to agree with Bruce on this one, for my part. I wonder if the > change to delay the crons was due to lack of proper locking or > tracking, or perhaps a lack of a filter for just changes which would > impact the documentation.. > > The buildfarm code does not run if there are no changes. The job runs, sees that there are no changes, and exits. And it has no problewm with collisions either. The code is guaranteed self-exclusionary. You can run it every minute from cron if you like and you will not get a collision. If it finds a running instance of itself it exits. Some people run the buildfarm script from cron every 15 minutes or so relying on the locking mechanism. And building the docs doesn't have a very high impact. And it takes about 2 minutes. So, many of the assumptions / speculations in your email are wrong ;-) cheers andrew
-
Re: Draft release notes complete
Stephen Frost <sfrost@snowman.net> — 2012-09-06T03:01:56Z
* Andrew Dunstan (andrew@dunslane.net) wrote: > The buildfarm code does not run if there are no changes. The job > runs, sees that there are no changes, and exits. Right, hence it makes great sense to use it for this (as opposed to Bruce's previous script or some other new one). While it might appear to be overkill, it actually does lots of useful and good things and integrates better with the existing setup anyway. Now that you've provided the magic sauce wrt --skip-steps, can we get an admin to implement a doc-only build that runs more frequently to update the dev docs..? Andrew, if we're going to rely on that, even just internally, perhaps we should go ahead and add documentation for it? Thanks, Stephen
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-06T03:33:35Z
On 09/05/2012 11:01 PM, Stephen Frost wrote: > * Andrew Dunstan (andrew@dunslane.net) wrote: >> The buildfarm code does not run if there are no changes. The job >> runs, sees that there are no changes, and exits. > Right, hence it makes great sense to use it for this (as opposed to > Bruce's previous script or some other new one). While it might appear > to be overkill, it actually does lots of useful and good things and > integrates better with the existing setup anyway. > > Now that you've provided the magic sauce wrt --skip-steps, can we get an > admin to implement a doc-only build that runs more frequently to update > the dev docs..? > > Andrew, if we're going to rely on that, even just internally, perhaps we > should go ahead and add documentation for it? > > You mean in my copious spare time? AIUI the only thing stopping the admins from doing what is wanted is a shortage of tuits. I suspect if we're all a tiny bit patient it will happen. But I guess they can speak for themselves. cheers andrew
-
Re: Draft release notes complete
Stephen Frost <sfrost@snowman.net> — 2012-09-06T03:44:20Z
* Andrew Dunstan (andrew@dunslane.net) wrote: > You mean in my copious spare time? If you're alright with the concept, then anyone can do it. I was looking more for your concurrence on the idea of documenting this explicitly (which also implies that it'll be supported, etc). I'd be happy to develop the actual patch to add the documentation. > AIUI the only thing stopping the admins from doing what is wanted is > a shortage of tuits. I suspect if we're all a tiny bit patient it > will happen. I agree that they'll now get to it, based off your explanation of how to use --skip-steps, and it'll be done and good. Thanks, Stephen
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-06T03:54:21Z
On 09/05/2012 11:44 PM, Stephen Frost wrote: > * Andrew Dunstan (andrew@dunslane.net) wrote: >> You mean in my copious spare time? > If you're alright with the concept, then anyone can do it. I was > looking more for your concurrence on the idea of documenting this > explicitly (which also implies that it'll be supported, etc). > > I'd be happy to develop the actual patch to add the documentation. > > Sure, go for it. The buildfarm code is entirely public, <https://github.com/PGBuildFarm/client-code> and the documentation lives on the wiki: <http://wiki.postgresql.org/wiki/PostgreSQL_Buildfarm_Howto> and an be edited by anyone with edit privs there. cheers andrew
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@2ndquadrant.com> — 2012-09-06T13:00:13Z
Excerpts from Andrew Dunstan's message of jue sep 06 00:33:35 -0300 2012: > > On 09/05/2012 11:01 PM, Stephen Frost wrote: > > * Andrew Dunstan (andrew@dunslane.net) wrote: > > Now that you've provided the magic sauce wrt --skip-steps, can we get an > > admin to implement a doc-only build that runs more frequently to update > > the dev docs..? > AIUI the only thing stopping the admins from doing what is wanted is a > shortage of tuits. I suspect if we're all a tiny bit patient it will happen. I can try to get it done sometime, yes. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Magnus Hagander <magnus@hagander.net> — 2012-09-07T13:57:11Z
On Thu, Sep 6, 2012 at 1:06 AM, Andrew Dunstan <andrew@dunslane.net> wrote: > > On 09/05/2012 06:13 PM, Peter Eisentraut wrote: >> >> On 8/29/12 11:52 PM, Andrew Dunstan wrote: >>>>> >>>>> Why does this need to be tied into the build farm? Someone can surely >>>> >>>> set up a script that just runs the docs build at every check-in, like it >>>> used to work. What's being proposed now just sounds like a lot of >>>> complication for little or no actual gain -- net loss in fact. >>> >>> It doesn't just build the docs. It makes the dist snapshots too. >> >> Thus making the turnaround time on a docs build even slower ... ? > > > > A complete run of this process takes less than 15 minutes. And as I have > pointed out elsewhere that could be reduced substantially by skipping > certain steps. It's as simple as changing the command line in the crontab > entry. Is it possible to run it only when the *docs* have changed, and not when it's just a code-commit? meaning, is the detection smart enough for that? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
-
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-07T16:50:44Z
On 09/07/2012 09:57 AM, Magnus Hagander wrote: > On Thu, Sep 6, 2012 at 1:06 AM, Andrew Dunstan <andrew@dunslane.net> wrote: >> >> A complete run of this process takes less than 15 minutes. And as I have >> pointed out elsewhere that could be reduced substantially by skipping >> certain steps. It's as simple as changing the command line in the crontab >> entry. > Is it possible to run it only when the *docs* have changed, and not > when it's just a code-commit? meaning, is the detection smart enough > for that? > > There is a filter mechanism used in detecting is a run is needed, and in modern versions of the client (Release 4.7, one version later than guaibasaurus is currently using) it lets you have both include and exclude filters. For example, you could have this config setting: trigger_include => qr(/doc/src/), and it would then only match changed files in the docs tree. It's a global mechanism, not per step. So it will run all the steps (other than those you have told it to skip) if it finds any files changed that match the filter conditions. If you do that you would probably want to have two animals, one doing docs builds only and running frequently, one doing the dist stuff much less frequently. cheers andrew -
Re: Draft release notes complete
Alvaro Herrera <alvherre@2ndquadrant.com> — 2012-09-07T19:07:49Z
Excerpts from Andrew Dunstan's message of vie sep 07 13:50:44 -0300 2012: > There is a filter mechanism used in detecting is a run is needed, and in > modern versions of the client (Release 4.7, one version later than > guaibasaurus is currently using) it lets you have both include and > exclude filters. For example, you could have this config setting: > > trigger_include => qr(/doc/src/), > > and it would then only match changed files in the docs tree. > > It's a global mechanism, not per step. So it will run all the steps > (other than those you have told it to skip) if it finds any files > changed that match the filter conditions. Sounds good. > If you do that you would probably want to have two animals, one doing > docs builds only and running frequently, one doing the dist stuff much > less frequently. What seems to make the most sense to me is to have a separate work directory for the buildfarm script to run, without setting up a whole buildfarm animal. That separate dir would build only the devel docs, triggered only by changes in doc/src, and would not do anything else. Thus we could leave guaibasaurus alone to do dist building. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2012-09-09T18:52:37Z
On 09/06/2012 12:13 AM, Peter Eisentraut wrote: > On 8/29/12 11:52 PM, Andrew Dunstan wrote: >>>> Why does this need to be tied into the build farm? Someone can surely >>> set up a script that just runs the docs build at every check-in, like it >>> used to work. What's being proposed now just sounds like a lot of >>> complication for little or no actual gain -- net loss in fact. >> >> It doesn't just build the docs. It makes the dist snapshots too. > > Thus making the turnaround time on a docs build even slower ... ? > >> And the old script often broke badly, IIRC. > > The script broke on occasion, but the main problem was that it wasn't > monitored. Which is something that could have been fixed. > >> The current setup doesn't install >> anything if the build fails, which is a distinct improvement. > > You mean it doesn't build the docs if the code build fails? Would that > really be an improvement? why would we want to publish docs for something that fails to build and/or fails to pass regression testing - to me code and the docs for it are a combined thing and there is no point in pushing docs for something that fails even basic testing... Stefan
-
Re: Draft release notes complete
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2012-09-09T18:56:44Z
On 09/06/2012 03:43 AM, Bruce Momjian wrote: > On Wed, Sep 5, 2012 at 09:33:35PM -0400, Andrew Dunstan wrote: >> >> On 09/05/2012 09:25 PM, Bruce Momjian wrote: >>> On Wed, Sep 5, 2012 at 09:56:32PM -0300, Alvaro Herrera wrote: >>>> Excerpts from Tom Lane's message of mié sep 05 20:24:08 -0300 2012: >>>>> Andrew Dunstan <andrew@dunslane.net> writes: >>>>>> The only reason there is a significant delay is that the administrators >>>>>> have chosen not to run the process more than once every 4 hours. That's >>>>>> a choice not dictated by the process they are using, but by other >>>>>> considerations concerning the machine it's being run on. Since I am not >>>>>> one of the admins and don't really want to take responsibility for it I >>>>>> am not going to second guess them. On the very rare occasions when I >>>>>> absolutely have to have the totally up to date docs I build them myself >>>>>> - it takes about 60 seconds on my modest hardware. >>>>> I think the argument for having a quick docs build service is not about >>>>> the time needed, but the need to have all the appropriate tools >>>>> installed. While I can understand that argument for J Random Hacker, >>>>> I'm mystified why Bruce doesn't seem to have bothered to get a working >>>>> SGML toolset installed. It's not like editing the docs is a one-shot >>>>> task for him. >>>> As far as I understand, Bruce's concern is not about seeing the docs >>>> built himself, but having an HTML copy published somewhere that he can >>>> point people to, after applying some patch. To me, that's a perfectly >>>> legitimate reason to want to have them quickly. >>> Correct. I have always had a working SGML toolset. If we are not going >>> to have the developer site run more often, I will just go back to >>> setting up my own public doc build, like I used to do. I removed mine >>> when the official one was more current/reliable --- if that has changed, >>> I will return to my old setup, and publish my own URL for users to >>> verify doc changes. >> >> How often do you want? After all, >> <http://developer.postgresql.org/docs/postgres/index.html> is >> presumably going to keep pointing to where it now points. > > Well, the old code checked every five minutes, and it rebuilt in 4 > minutes, so there was a max of 10 minutes delay. the new code gives you a lot more though - it makes sure that the code the docs refer to actually builds and passes testing, it uses the exact same toolchain and setup/infrastructure that we build the official snapshots/tarballs, the official PDFs and reuses an important piece of our environment - the buildfarm-client. I'm having a hard time understanding why getting a bit more frequency for the odd "docs only"+"need to show somebody the html and not the patch" requirement is really something we "need". Stefan
-
Re: Draft release notes complete
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2012-09-09T18:58:10Z
On 09/07/2012 06:50 PM, Andrew Dunstan wrote: > > On 09/07/2012 09:57 AM, Magnus Hagander wrote: >> On Thu, Sep 6, 2012 at 1:06 AM, Andrew Dunstan <andrew@dunslane.net> >> wrote: >>> >>> A complete run of this process takes less than 15 minutes. And as I have >>> pointed out elsewhere that could be reduced substantially by skipping >>> certain steps. It's as simple as changing the command line in the >>> crontab >>> entry. >> Is it possible to run it only when the *docs* have changed, and not >> when it's just a code-commit? meaning, is the detection smart enough >> for that? >> >> > > > There is a filter mechanism used in detecting is a run is needed, and in > modern versions of the client (Release 4.7, one version later than > guaibasaurus is currently using) it lets you have both include and > exclude filters. For example, you could have this config setting: > > trigger_include => qr(/doc/src/), > > and it would then only match changed files in the docs tree. > > It's a global mechanism, not per step. So it will run all the steps > (other than those you have told it to skip) if it finds any files > changed that match the filter conditions. > > If you do that you would probably want to have two animals, one doing > docs builds only and running frequently, one doing the dist stuff much > less frequently. hmm that might work, but it will only be a bandaid for what people really seem to advocate for ie "commit triggered" docs builds? Stefan
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-10T14:55:58Z
On Sun, Sep 9, 2012 at 08:52:37PM +0200, Stefan Kaltenbrunner wrote: > On 09/06/2012 12:13 AM, Peter Eisentraut wrote: > > On 8/29/12 11:52 PM, Andrew Dunstan wrote: > >>>> Why does this need to be tied into the build farm? Someone can surely > >>> set up a script that just runs the docs build at every check-in, like it > >>> used to work. What's being proposed now just sounds like a lot of > >>> complication for little or no actual gain -- net loss in fact. > >> > >> It doesn't just build the docs. It makes the dist snapshots too. > > > > Thus making the turnaround time on a docs build even slower ... ? > > > >> And the old script often broke badly, IIRC. > > > > The script broke on occasion, but the main problem was that it wasn't > > monitored. Which is something that could have been fixed. > > > >> The current setup doesn't install > >> anything if the build fails, which is a distinct improvement. > > > > You mean it doesn't build the docs if the code build fails? Would that > > really be an improvement? > > why would we want to publish docs for something that fails to build > and/or fails to pass regression testing - to me code and the docs for it > are a combined thing and there is no point in pushing docs for something > that fails even basic testing... Most of the cases I care about are doc-only commits. Frankly, there is a 99.9% chance thta if it was committed, it compiles. We are only displaying the docs, so why not just test for the docs. It is this kind of run-around that caused me to generate my own doc build in the past; maybe I need to return to doing my own doc build. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Alvaro Herrera <alvherre@2ndquadrant.com> — 2012-09-10T15:06:18Z
Excerpts from Bruce Momjian's message of lun sep 10 11:55:58 -0300 2012: > On Sun, Sep 9, 2012 at 08:52:37PM +0200, Stefan Kaltenbrunner wrote: > > why would we want to publish docs for something that fails to build > > and/or fails to pass regression testing - to me code and the docs for it > > are a combined thing and there is no point in pushing docs for something > > that fails even basic testing... > > Most of the cases I care about are doc-only commits. Frankly, there is > a 99.9% chance thta if it was committed, it compiles. We are only > displaying the docs, so why not just test for the docs. I see no reason for a code failure to cause the docs not to be refreshed, if they still build. Many buildfarm failures are platform dependencies that the original developer did not notice. That doesn't mean that the code "is utterly broken so much that docs suck and should not be published at all or we risk eternal embarrasment". Such failures tend to be short-lived anyway, and it's useful to be able to check that the docs are fine regardless of them. > It is this kind of run-around that caused me to generate my own doc > build in the past; maybe I need to return to doing my own doc build. You keep threatening with that. You are free, of course, to do anything you want, and no one will break sweat about it. I already said I will work on getting this up and running, but I can't give you a deadline for when it'll be working. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-10T15:19:00Z
On Mon, Sep 10, 2012 at 12:06:18PM -0300, Alvaro Herrera wrote: > > It is this kind of run-around that caused me to generate my own doc > > build in the past; maybe I need to return to doing my own doc build. > > You keep threatening with that. You are free, of course, to do anything > you want, and no one will break sweat about it. I already said I will > work on getting this up and running, but I can't give you a deadline for > when it'll be working. My point is that this frequent doc build feature was removed with no discussion, and adding it seems to be some herculean job that requires red tape only a government worker would love. I have already started working on updating my script for git --- should be done shortly, so you can remove my request. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-11T02:17:23Z
On Mon, Sep 10, 2012 at 11:19:00AM -0400, Bruce Momjian wrote: > On Mon, Sep 10, 2012 at 12:06:18PM -0300, Alvaro Herrera wrote: > > > It is this kind of run-around that caused me to generate my own doc > > > build in the past; maybe I need to return to doing my own doc build. > > > > You keep threatening with that. You are free, of course, to do anything > > you want, and no one will break sweat about it. I already said I will > > work on getting this up and running, but I can't give you a deadline for > > when it'll be working. > > My point is that this frequent doc build feature was removed with no > discussion, and adding it seems to be some herculean job that requires > red tape only a government worker would love. > > I have already started working on updating my script for git --- should > be done shortly, so you can remove my request. Here is my documentation build: http://momjian.postgresql.org/pgsql_docs/ It is updated every five minutes. (It checks git every 4 minutes, and the build takes 41 seconds.) -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Stefan Kaltenbrunner <stefan@kaltenbrunner.cc> — 2012-09-11T06:27:49Z
On 09/10/2012 05:19 PM, Bruce Momjian wrote: > On Mon, Sep 10, 2012 at 12:06:18PM -0300, Alvaro Herrera wrote: >>> It is this kind of run-around that caused me to generate my own doc >>> build in the past; maybe I need to return to doing my own doc build. >> >> You keep threatening with that. You are free, of course, to do anything >> you want, and no one will break sweat about it. I already said I will >> work on getting this up and running, but I can't give you a deadline for >> when it'll be working. > > My point is that this frequent doc build feature was removed with no > discussion, and adding it seems to be some herculean job that requires > red tape only a government worker would love. Not sure how you got that impression - but understand all requirements to something is usually key to implementing a solution, so discussing those requirements seems like a sensible thing to do. sysadmin is a volunteer effort and we do our best to deal with both keeping the existing infrastructure up and improving as we can but resources are limited and we need to consider the time/effort ration of stuff. Anyway alvaro clearly stated he would deal with it but obviously thatthat is not enough for your urgent demands so there is really not much we can do about it... > > I have already started working on updating my script for git --- should > be done shortly, so you can remove my request. ok Stefan
-
Re: Draft release notes complete
Bruce Momjian <bruce@momjian.us> — 2012-09-11T21:13:35Z
On Tue, Sep 11, 2012 at 08:27:49AM +0200, Stefan Kaltenbrunner wrote: > On 09/10/2012 05:19 PM, Bruce Momjian wrote: > > On Mon, Sep 10, 2012 at 12:06:18PM -0300, Alvaro Herrera wrote: > >>> It is this kind of run-around that caused me to generate my own doc > >>> build in the past; maybe I need to return to doing my own doc build. > >> > >> You keep threatening with that. You are free, of course, to do anything > >> you want, and no one will break sweat about it. I already said I will > >> work on getting this up and running, but I can't give you a deadline for > >> when it'll be working. > > > > My point is that this frequent doc build feature was removed with no > > discussion, and adding it seems to be some herculean job that requires > > red tape only a government worker would love. > > Not sure how you got that impression - but understand all requirements > to something is usually key to implementing a solution, so discussing > those requirements seems like a sensible thing to do. > sysadmin is a volunteer effort and we do our best to deal with both > keeping the existing infrastructure up and improving as we can but > resources are limited and we need to consider the time/effort ration of > stuff. > Anyway alvaro clearly stated he would deal with it but obviously > thatthat is not enough for your urgent demands so there is really not > much we can do about it... Don't know about "urgent", but I made this request in May: http://archives.postgresql.org/pgsql-hackers/2012-05/msg00480.php and at a certain point, waiting four months and discussing it repeatedly just isn't an efficient use of my time. It only took me 15 minutes to implement. I am guessing the complexity of the Postgres infrastructure just makes the job much harder to implement there. This is a good example of why some organizations like cloud services, where they can host things without waiting for the item to get to the top of the IT TODO list. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: Draft release notes complete
Stephen Frost <sfrost@snowman.net> — 2012-09-22T17:57:54Z
Andrew, Below is the patch that I mentioned at pgOpen. I'm pretty sure my silly github pull request got screwed up anyway, so probably best to ignore it. Regardless, please let me know what you think. I'd be happy to rework it to operate off of a single hash, though I think that would require having 'one true hash' of all possible steps and it kind of looked like you were trying to avoid that. Alvaro, assuming the patch is acceptable to everyone, it adds a "--only-steps" option, which would let you simply say: --only-steps=make-doc To build the docs using the buildfarm. Thanks, Stephen -- >8 -- Subject: [PATCH] Add --only-steps, improve --help and progress msgs Adds a new '--only-steps' option, intended to be used for debugging and for doc building (eg: --only=steps=make-doc). Also improved the --help message to have more specifics about how to use --skip-steps and --only-steps. Lastly, modified progress reporting to only report stages which are actually run, instead of listing all stages even if some aren't run. --- PGBuild/Options.pm | 6 +++- run_build.pl | 69 ++++++++++++++++++++++++++++++++++------------------ 2 files changed, 49 insertions(+), 26 deletions(-) diff --git a/PGBuild/Options.pm b/PGBuild/Options.pm index 64da7fc..05be6d5 100644 --- a/PGBuild/Options.pm +++ b/PGBuild/Options.pm @@ -22,7 +22,7 @@ BEGIN @option_list =qw( $forcerun $buildconf $keepall $help $quiet $from_source $from_source_clean $testmode - $test_mode $skip_steps $find_typedefs + $test_mode $skip_steps $only_steps $find_typedefs $nosend $nostatus $verbose ); } @@ -41,7 +41,8 @@ our ( $forcerun, $buildconf, $keepall, $help, $quiet, $from_source, $from_source_clean, $testmode,$test_mode, $skip_steps, - $find_typedefs,$nosend, $nostatus, $verbose, + $only_steps, $find_typedefs,$nosend, $nostatus, + $verbose, ); my (%standard_options); @@ -60,6 +61,7 @@ my (%standard_options); 'help' => \$help, 'quiet' => \$quiet, 'skip-steps=s' => \$skip_steps, + 'only-steps=s' => \$only_steps, ); $buildconf = "build-farm.conf"; # default value diff --git a/run_build.pl b/run_build.pl index 1848153..958318b 100755 --- a/run_build.pl +++ b/run_build.pl @@ -96,6 +96,13 @@ if ($skip_steps =~ /\S/) %skip_steps = map {$_ => 1} split(/\s+/,$skip_steps); } +my %only_steps; +$only_steps ||= ""; +if ($only_steps =~ /\S/) +{ + %only_steps = map {$_ => 1} split(/\s+/,$only_steps); +} + use vars qw($branch); my $explicit_branch = shift; $branch = $explicit_branch || 'HEAD'; @@ -598,29 +605,34 @@ configure(); # module configure has to wait until we have built and installed the base # so see below -print time_str(),"running make ...\n" if $verbose; +print time_str(),"running make ...\n" + if $verbose and !$skip_steps{'make'} and ($only_steps{'make'} or !$only_steps); make(); -print time_str(),"running make check ...\n" if $verbose; +print time_str(),"running make check ...\n" + if $verbose and !$skip_steps{'check'} and ($only_steps{'check'} or !$only_steps); make_check(); unless ($using_msvc) { - print time_str(),"running make contrib ...\n" if $verbose; + print time_str(),"running make contrib ...\n" + if $verbose and !$skip_steps{'make-contrib'} and ($only_steps{'make-contrib'} or !$only_steps); make_contrib(); } if (check_optional_step('build_docs')) { - print time_str(),"running make doc ...\n" if $verbose; + print time_str(),"running make doc ...\n" + if $verbose and !$skip_steps{'make-doc'} and ($only_steps{'make-doc'} or !$only_steps); make_doc(); } -print time_str(),"running make install ...\n" if $verbose; +print time_str(),"running make install ...\n" + if $verbose and !$skip_steps{'install'} and ($only_steps{'install'} or !$only_steps); make_install(); @@ -628,7 +640,7 @@ make_install(); unless ($using_msvc) { print time_str(),"running make contrib install ...\n" - if $verbose; + if $verbose and !$skip_steps{'install'} and ($only_steps{'install'} or !$only_steps); make_contrib_install(); } @@ -643,7 +655,7 @@ process_module_hooks('install'); foreach my $locale (@locales) { - last if $skip_steps{install}; + last if $skip_steps{'install'} or (!$only_steps{'install'} and $only_steps); print time_str(),"setting up db cluster ($locale)...\n" if $verbose; @@ -653,7 +665,8 @@ foreach my $locale (@locales) start_db($locale); - print time_str(),"running make installcheck ($locale)...\n" if $verbose; + print time_str(),"running make installcheck ($locale)...\n" + if $verbose and !$skip_steps{'install-check'} and ($only_steps{'install-check'} or !$only_steps); make_install_check($locale); @@ -668,7 +681,8 @@ foreach my $locale (@locales) stop_db($locale); start_db($locale); - print time_str(),"running make isolation check ...\n" if $verbose; + print time_str(),"running make isolation check ...\n" + if $verbose and !$skip_steps{'isolation-check'} and ($only_steps{'isolation-check'} or !$only_steps); make_isolation_check($locale); } @@ -686,7 +700,7 @@ foreach my $locale (@locales) start_db($locale); print time_str(),"running make PL installcheck ($locale)...\n" - if $verbose; + if $verbose and !$skip_steps{'pl-install-check'} and ($only_steps{'pl-install-check'} or !$only_steps); make_pl_install_check($locale); } @@ -698,7 +712,7 @@ foreach my $locale (@locales) start_db($locale); print time_str(),"running make contrib installcheck ($locale)...\n" - if $verbose; + if $verbose and !$skip_steps{'contrib-install-check'} and ($only_steps{'contrib-install-check'} or !$only_steps); make_contrib_install_check($locale); @@ -713,7 +727,8 @@ foreach my $locale (@locales) # ecpg checks are not supported in 8.1 and earlier if ($branch eq 'HEAD' || $branch gt 'REL8_2') { - print time_str(),"running make ecpg check ...\n" if $verbose; + print time_str(),"running make ecpg check ...\n" + if $verbose and !$skip_steps{'ecpg-check'} and ($only_steps{'ecpg-check'} or !$only_steps); make_ecpg_check(); } @@ -759,7 +774,13 @@ usage: $0 [options] [branch] --verbose[=n] = verbosity (default 1) 2 or more = huge output. --quiet = suppress normal error message --test = short for --nosend --nostatus --verbose --force - --skip-steps=list = skip certain steps + --skip-steps="step ..." = whitespace-separated list of steps to skip + valid steps are: make check make-contrib make-doc (optional) + install (includes contrib install) install-check + isolation-check pl-install-check + contrib-install-check ecpg-check + --only-steps="step ..." = opposite of skip; explicit list of steps to perform + eg: --only-steps="make-doc" Default branch is HEAD. Usually only the --config option should be necessary. @@ -870,7 +891,7 @@ sub check_make sub make { - return if $skip_steps{make}; + return if $skip_steps{'make'} or (!$only_steps{'make'} and $only_steps); my (@makeout); unless ($using_msvc) { @@ -894,7 +915,7 @@ sub make sub make_doc { - return if $skip_steps{'make-doc'}; + return if $skip_steps{'make-doc'} or (!$only_steps{'make-doc'} and $only_steps); my (@makeout); unless ($using_msvc) { @@ -915,7 +936,7 @@ sub make_doc sub make_install { - return if $skip_steps{install}; + return if $skip_steps{'install'} or (!$only_steps{'install'} and $only_steps); my @makeout; unless ($using_msvc) { @@ -977,7 +998,7 @@ sub make_contrib { # part of build under msvc - return if $skip_steps{'make-contrib'}; + return if $skip_steps{'make-contrib'} or (!$only_steps{'make-contrib'} and $only_steps); my $make_cmd = $make; $make_cmd = "$make -j $make_jobs" if ($make_jobs > 1 && ($branch eq 'HEAD' || $branch ge 'REL9_1')); @@ -991,7 +1012,7 @@ sub make_contrib sub make_contrib_install { - return if $skip_steps{'install'}; + return if $skip_steps{'install'} or (!$only_steps{'install'} and $only_steps); # part of install under msvc my @makeout = `cd $pgsql/contrib && $make install 2>&1`; @@ -1144,7 +1165,7 @@ sub get_stack_trace sub make_install_check { my $locale = shift; - return if $skip_steps{'install-check'}; + return if $skip_steps{'install-check'} or (!$only_steps{'install-check'} and $only_steps); my @checklog; unless ($using_msvc) { @@ -1187,7 +1208,7 @@ sub make_install_check sub make_contrib_install_check { my $locale = shift; - return if $skip_steps{'contrib-install-check'}; + return if $skip_steps{'contrib-install-check'} or (!$only_steps{'contrib-install-check'} and $only_steps); my @checklog; unless ($using_msvc) { @@ -1230,7 +1251,7 @@ sub make_contrib_install_check sub make_pl_install_check { my $locale = shift; - return if $skip_steps{'pl-install-check'}; + return if $skip_steps{'pl-install-check'} or (!$only_steps{'pl-install-check'} and $only_steps); my @checklog; unless ($using_msvc) { @@ -1276,7 +1297,7 @@ sub make_pl_install_check sub make_isolation_check { my $locale = shift; - return if $skip_steps{'isolation-check'}; + return if $skip_steps{'isolation-check'} or (!$only_steps{'isolation-check'} and $only_steps); my @makeout; unless ($using_msvc) { @@ -1325,7 +1346,7 @@ sub make_isolation_check sub make_check { - return if $skip_steps{check}; + return if $skip_steps{'check'} or (!$only_steps{'check'} and $only_steps); my @makeout; unless ($using_msvc) { @@ -1377,7 +1398,7 @@ sub make_check sub make_ecpg_check { - return if $skip_steps{'ecpg-check'}; + return if $skip_steps{'ecpg-check'} or (!$only_steps{'ecpg-check'} and $only_steps); my @makeout; my $ecpg_dir = "$pgsql/src/interfaces/ecpg"; if ($using_msvc) -- 1.7.9.1 -
Re: Draft release notes complete
Andrew Dunstan <andrew@dunslane.net> — 2012-09-23T20:27:31Z
On 09/22/2012 01:57 PM, Stephen Frost wrote: > Andrew, > > Below is the patch that I mentioned at pgOpen. I'm pretty sure my > silly github pull request got screwed up anyway, so probably best to > ignore it. Regardless, please let me know what you think. I'd be > happy to rework it to operate off of a single hash, though I think > that would require having 'one true hash' of all possible steps and > it kind of looked like you were trying to avoid that. I'm not sure it's a great advance, but I'll take a look. In any case please sent it as a proper MIME attachment. It did not come through clean. Alternatively, and probably better, put this on a topic branch that I can git-fetch (that's recommended practice for github pull requests too). cheers andrew