Horton

Thread

Commits

  1. Remove psql support for server versions preceding 9.2.

  2. Clean up some more freshly-dead code in pg_dump and pg_upgrade.

  3. Remove pg_dump's --no-synchronized-snapshots switch.

  4. Remove pg_upgrade support for upgrading from pre-9.2 servers.

  5. Remove pg_dump/pg_dumpall support for dumping from pre-9.2 servers.

  6. Suppress -Warray-bounds warning in 9.2's xlog.c.

  7. Suppress -Wformat-overflow warnings in 9.2's xml.c.

  8. Disable -Wsometimes-uninitialized warnings in the 9.2 branch.

  9. Fix function return type confusion

  10. Fix compiler warning

  11. Silence another gcc 11 warning.

  12. Suppress uninitialized-variable warning in guc.c.

  13. Suppress -Warray-parameter warnings in pgcrypto/sha2.c.

  14. Reformat imath.c macro to remove -Wmisleading-indentation warnings.

  15. Clean up compilation warnings coming from PL/Perl with clang-12~

  16. Make ecpg's rjulmdy() and rmdyjul() agree with their declarations.

  17. Use -Wno-format-truncation and -Wno-stringop-truncation, if available.

  18. Make pg_upgrade's test.sh less chatty.

  19. Add checks for valid multibyte character length in UtfToLocal, LocalToUtf.

  20. Use return instead of exit() in configure

  21. Add support for Visual Studio 2019 in build scripts

  1. pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-22T22:38:33Z

    While doing some desultory testing, I realized that the commit
I just pushed (92316a458) broke pg_dump against 8.0 servers:

$ pg_dump -p5480 -s regression
pg_dump: error: schema with OID 11 does not exist

The reason turns out to be something I'd long forgotten about: except
for the few "bootstrap" catalogs, our system catalogs didn't use to
have fixed OIDs.  That changed at 7c13781ee, but 8.0 predates that.
So when pg_dump reads a catalog on 8.0, it gets some weird number for
"tableoid", and the logic I just put into common.c's findNamespaceByOid
et al fails to find the resulting DumpableObjects.

So my first thought was just to revert 92316a458 and give up on it as
a bad idea.  However ... does anyone actually still care about being
able to dump from such ancient servers?  In addition to this issue,
I'm thinking of the discussion at [1] about wanting to use unnest()
in pg_dump, and of what we would need to do instead in pre-8.4 servers
that lack that.  Maybe it'd be better to move up pg_dump's minimum
supported server version to 8.4 or 9.0, and along the way whack a
few more lines of its backward-compatibility hacks.  If there is
anyone out there still using an 8.x server, they could use its
own pg_dump whenever they get around to migration.

Another idea would be to ignore "tableoid" and instead use the OIDs
we're expecting, but that's way too ugly for my taste, especially
given the rather thin argument for committing 92316a458 at all.

Anyway, I think the default answer is "revert 92316a458 and keep the
compatibility goalposts where they are".  But I wanted to open up a
discussion to see if anyone likes the other approach better.

			regards, tom lane

[1] https://www.postgresql.org/message-id/20211022055939.z6fihsm7hdzbjttf%40alap3.anarazel.de

  2. Re: pg_dump versus ancient server versions

    David G. Johnston <david.g.johnston@gmail.com> — 2021-10-22T23:00:19Z

    On Fri, Oct 22, 2021 at 3:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> Anyway, I think the default answer is "revert 92316a458 and keep the
> compatibility goalposts where they are".  But I wanted to open up a
> discussion to see if anyone likes the other approach better.
>
> [1]
> https://www.postgresql.org/message-id/20211022055939.z6fihsm7hdzbjttf%40alap3.anarazel.de
>
>
I'd rather drop legacy support than revert.  Even if the benefit of
92316a456 of is limited to refactoring the fact it was committed is enough
for me to feel it is a worthwhile improvement.  It's still yet another five
years before there won't be a supported release that can dump/restore this
- so 20 years for someone to have upgraded without having to go to the (not
that big a) hassle of installing an out-of-support version as a stop-over.

In short, IMO, the bar for this kind of situation should be 10 releases at
most - 5 of which would be in support at the time the patch goes in.  We
don't have to actively drop support of older stuff but anything older
shouldn't be preventing new commits.

David J.

  3. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-22T23:26:57Z

    On Fri, Oct 22, 2021 at 6:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> So my first thought was just to revert 92316a458 and give up on it as
> a bad idea.  However ... does anyone actually still care about being
> able to dump from such ancient servers?

I think I recently heard about an 8.4 server still out there in the
wild, but AFAICR it's been a long time since I've heard about anything
older.

It seems to me that if you're upgrading by a dozen server versions in
one shot, it's not a totally crazy idea that you might want to do it
in steps, or use the pg_dump for the version you have and then hack
the dump. I kind of wonder if there's really any hope of a pain-free
upgrade across that many versions anyway. There are things that can
bite you despite all the work we've put into pg_dump, like having
objects that depend on system objects whose definition has changed
over the years, plus implicit casting differences, operator precedence
changes, => getting deprecated, lots of GUC changes, etc. You are
going to be able to upgrade in the end, but it's probably going to
take some work. So I'm not really sure that giving up pg_dump
compatibility for versions that old is losing as much as it may seem.

Another thing to think about in that regard: how likely is that
PostgreSQL 7.4 and PostgreSQL 15 both compile and run on the same
operating system? I suspect the answer is "not very." I seem to recall
Greg Stark trying to compile really old versions of PostgreSQL for a
conference talk some years ago, and he got back to a point where it
just became impossible to make work on modern toolchains even with a
decent amount of hackery. One tends to think of C as about as static a
thing as can be, but that's kind of missing the point. On my laptop
for example, my usual configure invocation fails on 7.4 with:

checking for SSL_library_init in -lssl... no
configure: error: library 'ssl' is required for OpenSSL

In fact, I get that same failure on every branch older than 9.2. I
expect I could work around that by disabling SSL or finding an older
version of OpenSSL that works the way those branches expect, but that
might not be the only problem, either.  Now I understand you could
have PostgreSQL 15 on a new box and PostgreSQL 7.x on an ancient one
and connect via the network, and it would in all fairness be cool if
that Just Worked. But I suspect that even if that did happen in the
lab, reality wouldn't often be so kind.

--
Robert Haas
EDB: http://www.enterprisedb.com

  4. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-22T23:30:25Z

    "David G. Johnston" <david.g.johnston@gmail.com> writes:
> On Fri, Oct 22, 2021 at 3:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Anyway, I think the default answer is "revert 92316a458 and keep the
>> compatibility goalposts where they are".  But I wanted to open up a
>> discussion to see if anyone likes the other approach better.

> ... IMO, the bar for this kind of situation should be 10 releases at
> most - 5 of which would be in support at the time the patch goes in.  We
> don't have to actively drop support of older stuff but anything older
> shouldn't be preventing new commits.

Yeah.  I checked into when it was that we dropped pre-8.0 support
from pg_dump, and the answer is just about five years ago (64f3524e2).
So moving the bar forward by five releases isn't at all out of line.
8.4 would be eight years past EOL by the time v15 comes out.

One of the arguments for the previous change was that it was getting
very hard to build old releases on modern platforms, thus making it
hard to do any compatibility testing.  I believe the same is starting
to become true of the 8.x releases, though I've not tried personally
to build any of them in some time.  (The executables I'm using for
them date from 2014 or earlier, and have not been recompiled in
subsequent platform upgrades ...)  Anyway it's definitely not free
to continue to support old source server versions.

			regards, tom lane

  5. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-22T23:48:13Z

    Robert Haas <robertmhaas@gmail.com> writes:
> Another thing to think about in that regard: how likely is that
> PostgreSQL 7.4 and PostgreSQL 15 both compile and run on the same
> operating system? I suspect the answer is "not very." I seem to recall
> Greg Stark trying to compile really old versions of PostgreSQL for a
> conference talk some years ago, and he got back to a point where it
> just became impossible to make work on modern toolchains even with a
> decent amount of hackery.

Right.  The toolchains keep moving, even if the official language
definition doesn't.  For grins, I just checked out REL8_4_STABLE
on my M1 Mac, and found that it only gets this far:

checking test program... ok
checking whether long int is 64 bits... no
checking whether long long int is 64 bits... no
configure: error: Cannot find a working 64-bit integer type.

which turns out to be down to a configure-script issue we fixed
some years ago, ie using exit() without a prototype:

conftest.c:158:3: error: implicitly declaring library function 'exit' with type\
 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaratio\
n]
  exit(! does_int64_work());
  ^

I notice that the configure script is also selecting some warning
switches that this compiler doesn't much like, plus it doesn't
believe 2.6.x flex is usable.  So that's *at least* three things
that'd have to be hacked even to get to a successful configure run.

Individually such issues are (usually) not very painful, but when
you have to recreate all of them at once it's a daunting project.

So if I had to rebuild 8.4 from scratch right now, I would not be
a happy camper.  That seems like a good argument for not deeming
it to be something we still have to support.

			regards, tom lane

  6. Re: pg_dump versus ancient server versions

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-10-22T23:51:22Z

    On 2021-Oct-22, Robert Haas wrote:

> In fact, I get that same failure on every branch older than 9.2. I
> expect I could work around that by disabling SSL or finding an older
> version of OpenSSL that works the way those branches expect, but that
> might not be the only problem, either.

I just tried to build 9.1.  My config line there doesn't have ssl, but I
do get this in the compile stage:

gram.c:69:25: error: conflicting types for ‘base_yylex’
   69 | #define yylex           base_yylex
      |                         ^~~~~~~~~~
scan.c:15241:12: note: in expansion of macro ‘yylex’
15241 | extern int yylex \
      |            ^~~~~
In file included from /pgsql/source/REL9_1_STABLE/src/backend/parser/gram.y:60:
/pgsql/source/REL9_1_STABLE/src/include/parser/gramparse.h:66:12: note: previous declaration of ‘base_yylex’ was here
   66 | extern int base_yylex(YYSTYPE *lvalp, YYLTYPE *llocp,
      |            ^~~~~~~~~~
gram.c:69:25: error: conflicting types for ‘base_yylex’
   69 | #define yylex           base_yylex
      |                         ^~~~~~~~~~
scan.c:15244:21: note: in expansion of macro ‘yylex’
15244 | #define YY_DECL int yylex \
      |                     ^~~~~
scan.c:15265:1: note: in expansion of macro ‘YY_DECL’
15265 | YY_DECL
      | ^~~~~~~
In file included from /pgsql/source/REL9_1_STABLE/src/backend/parser/gram.y:60:
/pgsql/source/REL9_1_STABLE/src/include/parser/gramparse.h:66:12: note: previous declaration of ‘base_yylex’ was here
   66 | extern int base_yylex(YYSTYPE *lvalp, YYLTYPE *llocp,
      |            ^~~~~~~~~~
make[3]: *** [../../../src/Makefile.global:655: gram.o] Error 1

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/
"El Maquinismo fue proscrito so pena de cosquilleo hasta la muerte"
(Ijon Tichy en Viajes, Stanislaw Lem)

  7. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-24T20:45:20Z

    On Fri, Oct 22, 2021 at 7:51 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> I just tried to build 9.1.  My config line there doesn't have ssl, but I
> do get this in the compile stage:

Hmm.

You know, one thing we could think about doing is patching some of the
older branches to make them compile on modern machines. That would not
only be potentially useful for people who are upgrading from ancient
versions, but also for hackers trying to do research on the origin of
bugs or performance problems, and also for people who are trying to
maintain some kind of backward compatibility or other and want to test
against old versions.

I don't know whether that's really worth the effort and I expect Tom
will say that it's not. If he does say that, he may be right. But I
think if I were trying to extract my data from an old 7.4 database, I
think I'd find it a lot more useful if I could make 9.0 or 9.2 or
something compile and talk to it than if I had to use v15 and hope
that held together somehow. It doesn't really make sense to try to
keep compatibility of any sort with versions we can no longer test
against.

--
Robert Haas
EDB: http://www.enterprisedb.com

  8. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-24T21:46:17Z

    Robert Haas <robertmhaas@gmail.com> writes:
> You know, one thing we could think about doing is patching some of the
> older branches to make them compile on modern machines. That would not
> only be potentially useful for people who are upgrading from ancient
> versions, but also for hackers trying to do research on the origin of
> bugs or performance problems, and also for people who are trying to
> maintain some kind of backward compatibility or other and want to test
> against old versions.

Yeah.  We have done that in the past; I thought more than once,
but right now the only case I can find is d13f41d21/105f3ef49.
There are some other post-EOL commits in git, but I think the
others were mistakes from over-enthusiastic back-patching, while
that one was definitely an intentional portability fix for EOL'd
versions.

> I don't know whether that's really worth the effort and I expect Tom
> will say that it's not. If he does say ,that, he may be right.

Hmm ... I guess the question is how much work we feel like putting
into that, and how we'd track whether old branches still work,
and on what platforms.  It could easily turn into a time sink
that's not justified by the value.  I do see your point that there's
some value in it; I'm just not sure about the cost/benefit ratio.

One thing we could do that would help circumscribe the costs is to say
"we are not going to consider issues involving new compiler warnings
or bugs caused by more-aggressive optimization".  We could mechanize
that pretty effectively by changing configure shortly after a branch's
EOL to select -O0 and no extra warning flags, so that anyone building
from branch tip would get those switch choices.

(I have no idea what this might look like on the Windows side, but
I'm concerned by the fact that we seem to need fixes every time a
new Visual Studio major version comes out.)

			regards, tom lane

  9. Re: pg_dump versus ancient server versions

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-10-24T21:52:00Z

    On 2021-Oct-24, Robert Haas wrote:

> You know, one thing we could think about doing is patching some of the
> older branches to make them compile on modern machines. That would not
> only be potentially useful for people who are upgrading from ancient
> versions, but also for hackers trying to do research on the origin of
> bugs or performance problems, and also for people who are trying to
> maintain some kind of backward compatibility or other and want to test
> against old versions.

I think it is worth *some* effort, at least as far back as we want to
claim that we maintain pg_dump and/or psql compatibility, assuming it is
not too onerous.  For instance, I wouldn't want to clutter buildfarm or
CI dashboards with testing these branches, unless it is well isolated
from regular ones; we shouldn't commit anything that's too invasive; and
we shouldn't make any claims about supportability of these abandoned
branches.

As an example, I did backpatch one such fix to 8.3 (just over a year)
and 8.2 (four years) after they had closed -- see d13f41d21538 and
105f3ef492ab.

-- 
Álvaro Herrera           39°49'30"S 73°17'W  —  https://www.EnterpriseDB.com/
"Puedes vivir sólo una vez, pero si lo haces bien, una vez es suficiente"

  10. Re: pg_dump versus ancient server versions

    Laurenz Albe <laurenz.albe@cybertec.at> — 2021-10-25T08:29:19Z

    On Fri, 2021-10-22 at 19:26 -0400, Robert Haas wrote:
> On Fri, Oct 22, 2021 at 6:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > So my first thought was just to revert 92316a458 and give up on it as
> > a bad idea.  However ... does anyone actually still care about being
> > able to dump from such ancient servers?
> 
> I think I recently heard about an 8.4 server still out there in the
> wild, but AFAICR it's been a long time since I've heard about anything
> older.

I had a customer with 8.3 in the not too distant past, but that need not
stop the show.  If necessary, they can dump with 8.3 and restire that.

Yours,
Laurenz Albe

  11. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-25T12:29:24Z

    On 10/22/21 19:30, Tom Lane wrote:
> "David G. Johnston" <david.g.johnston@gmail.com> writes:
>> On Fri, Oct 22, 2021 at 3:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Anyway, I think the default answer is "revert 92316a458 and keep the
>>> compatibility goalposts where they are".  But I wanted to open up a
>>> discussion to see if anyone likes the other approach better.
>> ... IMO, the bar for this kind of situation should be 10 releases at
>> most - 5 of which would be in support at the time the patch goes in.  We
>> don't have to actively drop support of older stuff but anything older
>> shouldn't be preventing new commits.
> Yeah.  I checked into when it was that we dropped pre-8.0 support
> from pg_dump, and the answer is just about five years ago (64f3524e2).
> So moving the bar forward by five releases isn't at all out of line.
> 8.4 would be eight years past EOL by the time v15 comes out.
>
> One of the arguments for the previous change was that it was getting
> very hard to build old releases on modern platforms, thus making it
> hard to do any compatibility testing.  I believe the same is starting
> to become true of the 8.x releases, though I've not tried personally
> to build any of them in some time.  (The executables I'm using for
> them date from 2014 or earlier, and have not been recompiled in
> subsequent platform upgrades ...)  Anyway it's definitely not free
> to continue to support old source server versions.


But we don't need to build them on modern platforms, just run them on
modern platforms, ISTM.

Some months ago I built binaries all the way back to 7.2 that with a
little help run on modern Fedora and Ubuntu systems. I just upgraded my
Fedora system from 31 to 34 and they still run. See
<https://gitlab.com/adunstan/pg-old-bin> One of the intended use cases
was to test pg_dump against old versions.

I'm not opposed to us cutting off support for very old versions,
although I think we should only do that very occasionally (no more than
once every five years, say) unless there's a very good reason. I'm also
not opposed to us making small adjustments to allow us to build old
versions on modern platforms, but if we do that then we should probably
have some buildfarm support for it.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  12. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-25T12:55:37Z

    On Sun, Oct 24, 2021 at 5:46 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Hmm ... I guess the question is how much work we feel like putting
> into that, and how we'd track whether old branches still work,
> and on what platforms.  It could easily turn into a time sink
> that's not justified by the value.  I do see your point that there's
> some value in it; I'm just not sure about the cost/benefit ratio.

Right. Well, we could leave it up to people who care to decide how
much work they want to do, perhaps. But I do find it annoying that
pg_dump is supposed to maintain compatibility with server releases
that I can't easily build. Fortunately I don't patch pg_dump very
often, but if I did, it'd be very difficult for me to verify that
things work against really old versions. I know that you (Tom) do a
lot of work of this type though. In my opinion, if you find yourself
working on a project of this type and as part of that you do some
fixes to an older branch to make it compile, maybe you ought to commit
those so that the next person doesn't have the same problem. And maybe
when we add support for newer versions of OpenSSL or Windows, we ought
to consider back-patching those even to unsupported releases if
someone's willing to do the work. If they're not, they're not, but I
think we tend to strongly discourage commits to EOL branches, and I
think maybe we should stop doing that. Not that people should
routinely back-patch bug fixes, but stuff that makes it easier to
build seems fair game.

I don't think we need to worry too much about users getting the wrong
impression. People who want to know what is supported are going to
look at our web site for that information, and they are going to look
for releases. I can't rule out the possibility that someone is going
to build an updated version of 7.4 or 8.2 with whatever patches we
might choose to commit there, but they're unlikely to think that means
those are fully supported branches. And if they somehow do think that
despite all evidence to the contrary, we can just tell them that they
are mistaken.

> One thing we could do that would help circumscribe the costs is to say
> "we are not going to consider issues involving new compiler warnings
> or bugs caused by more-aggressive optimization".  We could mechanize
> that pretty effectively by changing configure shortly after a branch's
> EOL to select -O0 and no extra warning flags, so that anyone building
> from branch tip would get those switch choices.

I don't much like the idea of including -O0 because it seems like it
could be confusing. People might not realize that that the build
settings have been changed. I don't think that's really the problem
anyway: anybody who hits compiler warnings in older branches could
decide to fix them (and as long as it's a committer who will be
responsible for their own work, I think that's totally fine) or enable
-O0 locally. I routinely do that when I hit problems on older
branches, and it helps a lot, but the way I see it, that's such an
easy change that there's little reason to make it in the source code.
What's a lot more annoying is if the compile fails altogether, or you
can't even get past the configure step.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  13. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-25T12:59:56Z

    On Mon, Oct 25, 2021 at 8:29 AM Andrew Dunstan <andrew@dunslane.net> wrote:
> But we don't need to build them on modern platforms, just run them on
> modern platforms, ISTM.

I don't really agree with this.

> Some months ago I built binaries all the way back to 7.2 that with a
> little help run on modern Fedora and Ubuntu systems. I just upgraded my
> Fedora system from 31 to 34 and they still run. See
> <https://gitlab.com/adunstan/pg-old-bin> One of the intended use cases
> was to test pg_dump against old versions.

That's cool, but I don't have a Fedora or Ubuntu VM handy, and it does
seem like if people are working on testing against old versions, they
might even want to be able to recompile with debugging statements
added or something. So I think actually compiling is a lot better than
being able to get working binaries from someplace, even though the
latter is better than nothing.

> I'm not opposed to us cutting off support for very old versions,
> although I think we should only do that very occasionally (no more than
> once every five years, say) unless there's a very good reason. I'm also
> not opposed to us making small adjustments to allow us to build old
> versions on modern platforms, but if we do that then we should probably
> have some buildfarm support for it.

Yeah, I think having a small number of buildfarm animals testing very
old versions would be nice. Perhaps we can call them tyrannosaurus,
brontosaurus, triceratops, etc. :-)

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  14. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T14:23:40Z

    Robert Haas <robertmhaas@gmail.com> writes:
> Right. Well, we could leave it up to people who care to decide how
> much work they want to do, perhaps. But I do find it annoying that
> pg_dump is supposed to maintain compatibility with server releases
> that I can't easily build. Fortunately I don't patch pg_dump very
> often, but if I did, it'd be very difficult for me to verify that
> things work against really old versions. I know that you (Tom) do a
> lot of work of this type though. In my opinion, if you find yourself
> working on a project of this type and as part of that you do some
> fixes to an older branch to make it compile, maybe you ought to commit
> those so that the next person doesn't have the same problem.

Well, the answer to that so far is that I've never done such fixes.
I have the last released versions of old branches laying around,
and that's what I test against.  It's been sufficient so far, although
if I suddenly needed to do (say) SSL-enabled testing, that would be
a problem because I don't think I built with SSL for any of those
branches.

Because of that angle, I concur with your position that it'd really
be desirable to be able to build old versions on modern platforms.
Even if you've got an old executable, it might be misconfigured for
the purpose you have in mind.

> And maybe
> when we add support for newer versions of OpenSSL or Windows, we ought
> to consider back-patching those even to unsupported releases if
> someone's willing to do the work. If they're not, they're not, but I
> think we tend to strongly discourage commits to EOL branches, and I
> think maybe we should stop doing that. Not that people should
> routinely back-patch bug fixes, but stuff that makes it easier to
> build seems fair game.

What concerns me here is that we not get into a position where we're
effectively still maintaining EOL'd versions.  Looking at the git
history yesterday reminded me that we had such a situation back in
the early 7.x days.  I can see that I still occasionally made commits
into 7.1 and 7.2 years after the last releases of those branches,
which ended up being a complete waste of effort.  There was no policy
guiding what to back-patch into what branches, partly because we
didn't have a defined EOL policy then.  So I want to have a policy
(and a pretty tight one) before I'll go back to doing that.

Roughly speaking, I think the policy should be "no feature bug fixes,
not even security fixes, for EOL'd branches; only fixes that are
minimally necessary to make it build on newer platforms".  And
I want to have a sunset provision even for that.  Fixing every branch
forevermore doesn't scale.

There's also the question of how we get to a working state in the
first place -- as we found upthread, there's a fair-sized amount
of work to do just to restore buildability right now, for anything
that was EOL'd more than a year or two back.  I'm not volunteering
for that, but somebody would have to to get things off the ground.

Also, I concur with Andrew's point that we'd really have to have
buildfarm support.  However, this might not be as bad as it seems.
In principle we might just need to add resurrected branches back to
the branches_to_build list.  Given my view of what the back-patching
policy ought to be, a new build in an old branch might only be
required a couple of times a year, which would not be an undue
investment of buildfarm resources.  (Hmmm ... but disk space could
become a problem, particularly on older machines with not so much
disk.  Do we really need to maintain a separate checkout for each
branch?  It seems like a fresh checkout from the repo would be
little more expensive than the current copy-a-checkout process.)

			regards, tom lane

  15. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-25T14:40:29Z

    On Mon, Oct 25, 2021 at 10:23 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> What concerns me here is that we not get into a position where we're
> effectively still maintaining EOL'd versions.  Looking at the git
> history yesterday reminded me that we had such a situation back in
> the early 7.x days.  I can see that I still occasionally made commits
> into 7.1 and 7.2 years after the last releases of those branches,
> which ended up being a complete waste of effort.  There was no policy
> guiding what to back-patch into what branches, partly because we
> didn't have a defined EOL policy then.  So I want to have a policy
> (and a pretty tight one) before I'll go back to doing that.
>
> Roughly speaking, I think the policy should be "no feature bug fixes,
> not even security fixes, for EOL'd branches; only fixes that are
> minimally necessary to make it build on newer platforms".  And
> I want to have a sunset provision even for that.  Fixing every branch
> forevermore doesn't scale.

Sure, but you can ameliorate that a lot by just saying it's something
people have the *option* to do, not something anybody is *expected* to
do. I agree it's best if we continue to discourage back-patching bug
fixes into supported branches, but I also think we don't need to be
too stringent about this. What I think we don't want is, for example,
somebody working at company X deciding to back-patch all the bug fixes
that customers of company X cares about into our back-branches, but
not the other ones. But on the other hand if somebody is trying to
benchmark or test compatibility an old branch and it keeps crashing
because of some bug, telling them that they're not allowed to fix that
bug because it's not a sufficiently-minimal change to a dead branch is
kind of ridiculous. In other words, if you try to police every change
anyone wants to make, e.g. "well I know that would help YOU build on a
newer platform but it doesn't seem like it meets the criteria of the
minimum necessary change to make it build on a newer platform," then
you might as well just give up now. Nobody cares about the older
branches enough to put work into fixing whatever's wrong and then
having to argue about whether that work ought to be thrown away
anyway.

> There's also the question of how we get to a working state in the
> first place -- as we found upthread, there's a fair-sized amount
> of work to do just to restore buildability right now, for anything
> that was EOL'd more than a year or two back.  I'm not volunteering
> for that, but somebody would have to to get things off the ground.

Right.

> Also, I concur with Andrew's point that we'd really have to have
> buildfarm support.  However, this might not be as bad as it seems.
> In principle we might just need to add resurrected branches back to
> the branches_to_build list.  Given my view of what the back-patching
> policy ought to be, a new build in an old branch might only be
> required a couple of times a year, which would not be an undue
> investment of buildfarm resources.  (Hmmm ... but disk space could
> become a problem, particularly on older machines with not so much
> disk.  Do we really need to maintain a separate checkout for each
> branch?  It seems like a fresh checkout from the repo would be
> little more expensive than the current copy-a-checkout process.)

I suppose it would be useful if we had the ability to do new runs only
when the source code has changed...

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  16. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T15:00:42Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Oct 25, 2021 at 10:23 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Roughly speaking, I think the policy should be "no feature bug fixes,
>> not even security fixes, for EOL'd branches; only fixes that are
>> minimally necessary to make it build on newer platforms".  And
>> I want to have a sunset provision even for that.  Fixing every branch
>> forevermore doesn't scale.

> Sure, but you can ameliorate that a lot by just saying it's something
> people have the *option* to do, not something anybody is *expected* to
> do. I agree it's best if we continue to discourage back-patching bug
> fixes into supported branches, but I also think we don't need to be
> too stringent about this.

Actually, I think we do.  If I want to test against 7.4, ISTM I want
to test against the last released 7.4 version, not something with
arbitrary later changes.  Otherwise, what exactly is the point?

>> In principle we might just need to add resurrected branches back to
>> the branches_to_build list.  Given my view of what the back-patching
>> policy ought to be, a new build in an old branch might only be
>> required a couple of times a year, which would not be an undue
>> investment of buildfarm resources.

> I suppose it would be useful if we had the ability to do new runs only
> when the source code has changed...

Uh, don't we have that already?  I know you can configure a buildfarm
animal to force a run at least every-so-often, but it's not required,
and I don't think it's even the default.

			regards, tom lane

  17. Re: pg_dump versus ancient server versions

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-10-25T15:05:22Z

    On 2021-Oct-25, Tom Lane wrote:

> Roughly speaking, I think the policy should be "no feature bug fixes,
> not even security fixes, for EOL'd branches; only fixes that are
> minimally necessary to make it build on newer platforms".  And
> I want to have a sunset provision even for that.  Fixing every branch
> forevermore doesn't scale.

Agreed.  I think dropping such support at the same time we drop
psql/pg_dump support is a decent answer to that.  That meets the stated
purpose of being able to test such support, and also it moves forward
according to subjective choice per development needs.

> Also, I concur with Andrew's point that we'd really have to have
> buildfarm support.  However, this might not be as bad as it seems.
> In principle we might just need to add resurrected branches back to
> the branches_to_build list.

Well, we would add them to *some* list, but not to the one used by stock
BF members -- not only because of the diskspace issue but also because
of the time to build.  I suggest that we should have a separate
list-of-branches file that would only be used by BF members especially
configured to do so; and hopefully we won't allow more than a handful
animals to do that but rather a well-chosen subset, and also maybe allow
only GCC rather than try to support other compilers.  (There's no need
to ensure compilability on any Windows platform, for example.)

-- 
Álvaro Herrera              Valdivia, Chile  —  https://www.EnterpriseDB.com/
"Ed is the standard text editor."
      http://groups.google.com/group/alt.religion.emacs/msg/8d94ddab6a9b0ad3

  18. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-10-25T15:09:04Z

    On Mon, Oct 25, 2021 at 11:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Actually, I think we do.  If I want to test against 7.4, ISTM I want
> to test against the last released 7.4 version, not something with
> arbitrary later changes.  Otherwise, what exactly is the point?

1. You're free to check out any commit you like.

2. Nothing I said can reasonably be confused with "let's allow
arbitrary later changes."

> Uh, don't we have that already?  I know you can configure a buildfarm
> animal to force a run at least every-so-often, but it's not required,
> and I don't think it's even the default.

Oh, OK. I wonder how that plays with the buildfarm status page's
desire to drop old results that are more than 30 days old. I guess
you'd just need to force a run at least every 28 days or something.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  19. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-25T15:25:00Z

    On 10/25/21 11:09, Robert Haas wrote:
> On Mon, Oct 25, 2021 at 11:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Actually, I think we do.  If I want to test against 7.4, ISTM I want
>> to test against the last released 7.4 version, not something with
>> arbitrary later changes.  Otherwise, what exactly is the point?
> 1. You're free to check out any commit you like.
>
> 2. Nothing I said can reasonably be confused with "let's allow
> arbitrary later changes."
>
>> Uh, don't we have that already?  I know you can configure a buildfarm
>> animal to force a run at least every-so-often, but it's not required,
>> and I don't think it's even the default.


Yes, in fact its rather discouraged. The default is just to build when
there's a code change detected.


> Oh, OK. I wonder how that plays with the buildfarm status page's
> desire to drop old results that are more than 30 days old. I guess
> you'd just need to force a run at least every 28 days or something.
>

Well, we could do that, or we could modify the way the server does the
status. The table it's based on has the last 500 records for each branch
for each animal, so the data is there.


cheers


andrew




--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  20. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T15:26:26Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Oct 25, 2021 at 11:00 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Actually, I think we do.  If I want to test against 7.4, ISTM I want
>> to test against the last released 7.4 version, not something with
>> arbitrary later changes.  Otherwise, what exactly is the point?

> 1. You're free to check out any commit you like.

Yeah, and get something that won't build.  If there's any point
to this work at all, it has to be that we'll maintain the closest
possible buildable approximation to the last released version.

> Oh, OK. I wonder how that plays with the buildfarm status page's
> desire to drop old results that are more than 30 days old. I guess
> you'd just need to force a run at least every 28 days or something.

I don't think it's a problem.  If we haven't committed anything to
branch X in a month, it's likely not interesting.  It might be worth
having a way to get the website to show results further back than
a month, but that doesn't need to be in the default view.

			regards, tom lane

  21. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-25T15:28:04Z

    On 10/25/21 10:23, Tom Lane wrote:
>
> Also, I concur with Andrew's point that we'd really have to have
> buildfarm support.  However, this might not be as bad as it seems.
> In principle we might just need to add resurrected branches back to
> the branches_to_build list.  Given my view of what the back-patching
> policy ought to be, a new build in an old branch might only be
> required a couple of times a year, which would not be an undue
> investment of buildfarm resources.  (Hmmm ... but disk space could
> become a problem, particularly on older machines with not so much
> disk.  Do we really need to maintain a separate checkout for each
> branch?  It seems like a fresh checkout from the repo would be
> little more expensive than the current copy-a-checkout process.)


If you set it up with these settings then the disk space used is minimal:

     git_use_workdirs => 1,
     rm_worktrees => 1,

So I have this on crake:

    andrew@emma:root $ du -sh REL*/pgsql
    5.5M    REL_10_STABLE/pgsql
    5.6M    REL_11_STABLE/pgsql
    5.6M    REL_12_STABLE/pgsql
    5.6M    REL_13_STABLE/pgsql
    2.0M    REL_14_STABLE/pgsql
    2.6M    REL9_5_STABLE/pgsql
    5.5M    REL9_6_STABLE/pgsql


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  22. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-25T15:30:40Z

    On 10/25/21 11:05, Alvaro Herrera wrote:
>
>> Also, I concur with Andrew's point that we'd really have to have
>> buildfarm support.  However, this might not be as bad as it seems.
>> In principle we might just need to add resurrected branches back to
>> the branches_to_build list.
> Well, we would add them to *some* list, but not to the one used by stock
> BF members -- not only because of the diskspace issue but also because
> of the time to build.  I suggest that we should have a separate
> list-of-branches file that would only be used by BF members especially
> configured to do so; and hopefully we won't allow more than a handful
> animals to do that but rather a well-chosen subset, and also maybe allow
> only GCC rather than try to support other compilers.  (There's no need
> to ensure compilability on any Windows platform, for example.)


Well, we do build with gcc on Windows :-) But yes, maybe we should make
this a more opt-in process.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  23. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T15:33:17Z

    Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> On 2021-Oct-25, Tom Lane wrote:
>> Also, I concur with Andrew's point that we'd really have to have
>> buildfarm support.  However, this might not be as bad as it seems.
>> In principle we might just need to add resurrected branches back to
>> the branches_to_build list.

> Well, we would add them to *some* list, but not to the one used by stock
> BF members -- not only because of the diskspace issue but also because
> of the time to build.  I suggest that we should have a separate
> list-of-branches file that would only be used by BF members especially
> configured to do so; and hopefully we won't allow more than a handful
> animals to do that but rather a well-chosen subset, and also maybe allow
> only GCC rather than try to support other compilers.  (There's no need
> to ensure compilability on any Windows platform, for example.)

Meh.  I don't think that's a great approach, because then we're only
ensuring buildability on a rather static set of platforms.  The whole
point here is that when release N+1 of $your_favorite_platform arrives,
we want to know whether the old branches still build on it.  If the
default behavior for new buildfarm animals is to ignore the old branches,
we're much less likely to find that out.

It's also unclear to me why we'd leave Windows out of this discussion.
We keep saying we want to encourage Windows-based hackers to contribute,
so doesn't that require testing it on the same basis as other platforms?

			regards, tom lane

  24. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T15:38:51Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> On 10/25/21 10:23, Tom Lane wrote:
>> (Hmmm ... but disk space could
>> become a problem, particularly on older machines with not so much
>> disk.  Do we really need to maintain a separate checkout for each
>> branch?  It seems like a fresh checkout from the repo would be
>> little more expensive than the current copy-a-checkout process.)

> If you set it up with these settings then the disk space used is minimal:
>      git_use_workdirs => 1,
>      rm_worktrees => 1,

Maybe we should make those the defaults?  AFAICS the current
default setup uses circa 200MB per back branch, even between runs.
I'm not sure what that is buying us.

			regards, tom lane

  25. Re: pg_dump versus ancient server versions

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2021-10-25T16:25:04Z

    On 2021-Oct-25, Tom Lane wrote:

> It's also unclear to me why we'd leave Windows out of this discussion.
> We keep saying we want to encourage Windows-based hackers to contribute,
> so doesn't that require testing it on the same basis as other platforms?

Testing of in-support branches, sure -- I don't propose to break that.
But this is all about providing *some* server against which to test
client-side changes with, right?  Not to test the old servers
themselves.  Looking at Amit K's "Postgres person of the week" interview[1]
and remembering conversations with David Rowley, Windows hackers seem
perfectly familiar with getting Linux builds going, so we wouldn't need
to force MSVC fixes in order for them to have old servers available.

But anyway, I was thinking that the fixes required for MSVC buildability
were quite invasive, but on looking again they don't seem all that
bad[2], so I withdraw that comment.

I do think you have moved the goalposts: to reiterate what I said above,
I thought what we wanted was to have *some* server in order to test
client-side changes with; not to be able to get a server running on
every possible platform.  I'm not really on board with the idea that old
branches have to be buildable everywhere all the time.

[1] https://postgresql.life/post/amit_kapila/
[2] e.g., commit 2b1394fc2b52a2573d08aa626e7b49568f27464e

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/

  26. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T16:43:15Z

    Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> I do think you have moved the goalposts: to reiterate what I said above,
> I thought what we wanted was to have *some* server in order to test
> client-side changes with; not to be able to get a server running on
> every possible platform.  I'm not really on board with the idea that old
> branches have to be buildable everywhere all the time.

Agreed, that might be too much work compared to the value.  But if we're
to be selective about support for this, I'm unclear on how we decide
which platforms are supported --- and, more importantly, how we keep
that list up to date over time.

			regards, tom lane

  27. Re: pg_dump versus ancient server versions

    Andres Freund <andres@anarazel.de> — 2021-10-25T16:56:20Z

    Hi,

On 2021-10-22 19:30:25 -0400, Tom Lane wrote:
> Yeah.  I checked into when it was that we dropped pre-8.0 support
> from pg_dump, and the answer is just about five years ago (64f3524e2).
> So moving the bar forward by five releases isn't at all out of line.
> 8.4 would be eight years past EOL by the time v15 comes out.

I'd really like us to adopt a "default" policy on this. I think it's a waste
to spend time every few years arguing what exact versions to drop. I'd much
rather say that, unless there are concrete reasons to deviate from that, we
provide pg_dump compatibility for 5+3 releases, pg_upgrade for 5+1, and psql
for 5 releases or something like that.

It's fine to not actually spend the time to excise support for old versions
every release if not useful, but we should be able to "just do it" whenever
version compat is a meaningful hindrance.

Greetings,

Andres Freund

  28. Re: pg_dump versus ancient server versions

    Andres Freund <andres@anarazel.de> — 2021-10-25T17:06:57Z

    Hi,

On 2021-10-25 10:23:40 -0400, Tom Lane wrote:
> Also, I concur with Andrew's point that we'd really have to have
> buildfarm support.  However, this might not be as bad as it seems.
> In principle we might just need to add resurrected branches back to
> the branches_to_build list.  Given my view of what the back-patching
> policy ought to be, a new build in an old branch might only be
> required a couple of times a year, which would not be an undue
> investment of buildfarm resources.

FWIW, if helpful I could easily specify a few additional branches to some of
my buildfarm animals. Perhaps serinus/flaviventris (snapshot gcc wo/w
optimizations) so we'd see problems coming early? I could also add
recent-clang one.

I think doing this to a few designated animals is a better idea than wasting
cycles and space on a lot of animals.


> It seems like a fresh checkout from the repo would be little more expensive
> than the current copy-a-checkout process.)

I haven't looked in detail, but from what I've seen in the logs the
is-there-anything-new check is already not cheap, and does a checkout / update
of the git directory.

Greetings,

Andres Freund

  29. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T17:09:43Z

    Andres Freund <andres@anarazel.de> writes:
> On 2021-10-22 19:30:25 -0400, Tom Lane wrote:
>> Yeah.  I checked into when it was that we dropped pre-8.0 support
>> from pg_dump, and the answer is just about five years ago (64f3524e2).
>> So moving the bar forward by five releases isn't at all out of line.
>> 8.4 would be eight years past EOL by the time v15 comes out.

> I'd really like us to adopt a "default" policy on this. I think it's a waste
> to spend time every few years arguing what exact versions to drop. I'd much
> rather say that, unless there are concrete reasons to deviate from that, we
> provide pg_dump compatibility for 5+3 releases, pg_upgrade for 5+1, and psql
> for 5 releases or something like that.

I agree with considering something like that to be the minimum support
policy, but the actual changes need a bit more care.  For example, when
we last did this, the technical need was just to drop pre-7.4 versions,
but we chose to make the cutoff 8.0 on the grounds that that was more
understandable to users [1].  In the same way, I'm thinking of moving the
cutoff to 9.0 now, although 8.4 would be sufficient from a technical
standpoint.

OTOH, in the new world of one-part major versions, it's less clear that
there will be obvious division points for future cutoff changes.  Maybe
versions-divisible-by-five would work?  Or versions divisible by ten,
but experience so far suggests that we'll want to move the cutoff more
often than once every ten years.

			regards, tom lane

[1] https://www.postgresql.org/message-id/flat/2661.1475849167%40sss.pgh.pa.us

  30. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T17:14:19Z

    Andres Freund <andres@anarazel.de> writes:
> On 2021-10-25 10:23:40 -0400, Tom Lane wrote:
>> It seems like a fresh checkout from the repo would be little more expensive
>> than the current copy-a-checkout process.)

> I haven't looked in detail, but from what I've seen in the logs the
> is-there-anything-new check is already not cheap, and does a checkout / update
> of the git directory.

Yeah, you probably need a checkout to apply the rule about don't rebuild
after documentation-only changes.  But it seems like the case where the
branch tip hasn't moved at all could be optimized fairly easily.  I'm not
sure it's worth the trouble to add code for that given our current usage
of the buildfarm; but if we were to start tracking branches that only
change a couple of times a year, it would be.

			regards, tom lane

  31. Re: pg_dump versus ancient server versions

    Andres Freund <andres@anarazel.de> — 2021-10-25T17:17:20Z

    Hi,

On 2021-10-25 12:43:15 -0400, Tom Lane wrote:
> Agreed, that might be too much work compared to the value.  But if we're
> to be selective about support for this, I'm unclear on how we decide
> which platforms are supported --- and, more importantly, how we keep
> that list up to date over time.

I honestly think that if we just test on linux with a single distribution,
we're already covering most of the benefit. From memory there have been two
rough classes of doesn't-build-anymore:

1) New optimizations / warnings. At least between gcc and clang, within a year
   or two, most of the issues end up being visible with the other compiler
   too. These aren't particularly distribution / OS specific.

2) Library dependencies cause problems, like the ssl detection mentioned
   elsewhere in this thread. This is also not that OS dependent. It's also not
   that clear that we can do something about the issues with a reasonable
   amount of effort in all cases. It's easy enough if it's just a minor
   configure fix, but we'd not want to backpatch larger SSL changes or such.


Maybe there's also a case for building older releases with msvc, but that
seems like a pain due to the msvc project generation needing to support a
specific version of msvc.

Greetings,

Andres Freund

  32. Re: pg_dump versus ancient server versions

    Andres Freund <andres@anarazel.de> — 2021-10-25T17:24:13Z

    Hi,

On 2021-10-25 13:09:43 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I'd really like us to adopt a "default" policy on this. I think it's a waste
> > to spend time every few years arguing what exact versions to drop. I'd much
> > rather say that, unless there are concrete reasons to deviate from that, we
> > provide pg_dump compatibility for 5+3 releases, pg_upgrade for 5+1, and psql
> > for 5 releases or something like that.
> 
> I agree with considering something like that to be the minimum support
> policy, but the actual changes need a bit more care.  For example, when
> we last did this, the technical need was just to drop pre-7.4 versions,
> but we chose to make the cutoff 8.0 on the grounds that that was more
> understandable to users [1].  In the same way, I'm thinking of moving the
> cutoff to 9.0 now, although 8.4 would be sufficient from a technical
> standpoint.

I think that'd be less of a concern if we had a documented policy
somewhere. It'd not be hard to include a version table in that policy to make
it easier to understand. We could even add it to the table in
https://www.postgresql.org/support/versioning/ or something similar.


> OTOH, in the new world of one-part major versions, it's less clear that
> there will be obvious division points for future cutoff changes.  Maybe
> versions-divisible-by-five would work?

I think that's more confusing than helpful, because the support timeframes
then differ between releases. It's easier to just subtract a number of major
releases for from a specific major version. Especially if there's a table
somewhere.


> Or versions divisible by ten, but experience so far suggests that we'll want
> to move the cutoff more often than once every ten years.

Yes, I think that'd be quite a bit too restrictive.

Greetings,

Andres Freund

  33. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-25T20:29:11Z

    On 10/25/21 13:06, Andres Freund wrote:
> Hi,
>
> On 2021-10-25 10:23:40 -0400, Tom Lane wrote:
>> Also, I concur with Andrew's point that we'd really have to have
>> buildfarm support.  However, this might not be as bad as it seems.
>> In principle we might just need to add resurrected branches back to
>> the branches_to_build list.  Given my view of what the back-patching
>> policy ought to be, a new build in an old branch might only be
>> required a couple of times a year, which would not be an undue
>> investment of buildfarm resources.
> FWIW, if helpful I could easily specify a few additional branches to some of
> my buildfarm animals. Perhaps serinus/flaviventris (snapshot gcc wo/w
> optimizations) so we'd see problems coming early? I could also add
> recent-clang one.
>
> I think doing this to a few designated animals is a better idea than wasting
> cycles and space on a lot of animals.


Right now the server will only accept results for something in
branches_of_interest.txt. So we would need to modify that.


I tend to agree that we don't need a whole lot of cross platform testing
here.


>
>
>> It seems like a fresh checkout from the repo would be little more expensive
>> than the current copy-a-checkout process.)
> I haven't looked in detail, but from what I've seen in the logs the
> is-there-anything-new check is already not cheap, and does a checkout / update
> of the git directory.
>
>

If you have removed the work tree (with the "rm_worktrees => 1" setting)
then it restores it by doing a checkout. It then does a "git fetch", and
then as you say looks to see if there is anything new. If you know of a
better way to manage it then please let me know. On crake (which is
actually checking out four different repos) the checkout step typically
takes one or two seconds.


Copying the work tree can take a few seconds - to avoid that on
Unix/msys use vpath builds.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  34. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-10-25T22:52:18Z

    Anyway, to get back to the original point ...

No one has spoken against moving up the cutoff for pg_dump support,
so I did a very quick pass to see how much code could be removed.
The answer is right about 1000 lines, counting both pg_dump and
pg_upgrade, so it seems like it's worth doing independently of the
unnest() issue.

The attached is just draft-quality, because I don't really want
to pursue the point until after committing the pg_dump changes
being discussed in the other thread.  If I push this first it'll
break a lot of those patches.  (Admittedly, pushing those first
will break this one, but this one is a lot easier to re-do.)

BTW, while looking at pg_upgrade I chanced to notice
check_for_isn_and_int8_passing_mismatch(), which seems like it's
not well thought out at all.  It's right that contrib/isn will
not upgrade nicely if the target cluster has a different
float8_pass_by_value setting from the source.  What's wrong is
the assumption that no other extension has the same issue.
We invented and publicized the "LIKE type" option for CREATE TYPE
precisely so that people could build types that act just like isn,
so it seems pretty foolish to imagine that no one has done so.

I think we should nuke check_for_isn_and_int8_passing_mismatch()
and just refuse to upgrade if float8_pass_by_value differs, full stop.
I can see little practical need to allow that case.

			regards, tom lane

  35. Re: pg_dump versus ancient server versions

    Justin Pryzby <pryzby@telsasoft.com> — 2021-10-25T23:12:56Z

    On Mon, Oct 25, 2021 at 11:38:51AM -0400, Tom Lane wrote:
> Andrew Dunstan <andrew@dunslane.net> writes:
> > On 10/25/21 10:23, Tom Lane wrote:
> >> (Hmmm ... but disk space could
> >> become a problem, particularly on older machines with not so much
> >> disk.  Do we really need to maintain a separate checkout for each
> >> branch?  It seems like a fresh checkout from the repo would be
> >> little more expensive than the current copy-a-checkout process.)
> 
> > If you set it up with these settings then the disk space used is minimal:
> >      git_use_workdirs => 1,
> >      rm_worktrees => 1,
> 
> Maybe we should make those the defaults?  AFAICS the current
> default setup uses circa 200MB per back branch, even between runs.
> I'm not sure what that is buying us.

Maybe git's shared/"alternates" would be helpful to minimize the size of
.git/objects?

I'm not sure - it looks like the BF client does its own stuff with symlinks.
Is that for compatibility with old git ?
https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/SCM.pm

If you "clone" a local location, it uses hard links by default.
If you use --shared or --reference, then it uses references to the configured
"alternates", if any.

In both cases, .git/objects requires no additional space (but the "checked out"
copy still takes up however much space).

$ mkdir tmp
$ git clone --quiet ./postgresql tmp/pg
$ du -sh tmp/pg
492M    tmp/pg

$ rm -fr tmp/pg
$ git clone --quiet --shared ./postgresql tmp/pg
$ du -sh tmp/pg
124M    tmp/pg

-- 
Justin

  36. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-26T17:41:32Z

    On 10/25/21 19:12, Justin Pryzby wrote:
> On Mon, Oct 25, 2021 at 11:38:51AM -0400, Tom Lane wrote:
>> Andrew Dunstan <andrew@dunslane.net> writes:
>>> On 10/25/21 10:23, Tom Lane wrote:
>>>> (Hmmm ... but disk space could
>>>> become a problem, particularly on older machines with not so much
>>>> disk.  Do we really need to maintain a separate checkout for each
>>>> branch?  It seems like a fresh checkout from the repo would be
>>>> little more expensive than the current copy-a-checkout process.)
>>> If you set it up with these settings then the disk space used is minimal:
>>>      git_use_workdirs => 1,
>>>      rm_worktrees => 1,
>> Maybe we should make those the defaults?  AFAICS the current
>> default setup uses circa 200MB per back branch, even between runs.
>> I'm not sure what that is buying us.
> Maybe git's shared/"alternates" would be helpful to minimize the size of
> .git/objects?
>
> I'm not sure - it looks like the BF client does its own stuff with symlinks.
> Is that for compatibility with old git ?
> https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/SCM.pm


It's actually based on the git contrib script git-new-workdir. And using
it is the default (except on Windows, where it doesn't work due to
issues with symlinking plain files :-( )

Since what we have is not broken I'm not inclined to fix it.

The issue Tom was complaining about is different, namely the storage for
each branch's working tree. As I mentioned upthread, you can alleviate
that by setting "rm_worktrees => 1" in your config. That works
everywhere, including Windows, and will be the default in the next
buildfarm release.


cheers


andrew


-- 

Andrew Dunstan
EDB: https://www.enterprisedb.com

  37. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-10-26T17:59:34Z

    On 10/25/21 13:09, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2021-10-22 19:30:25 -0400, Tom Lane wrote:
>>> Yeah.  I checked into when it was that we dropped pre-8.0 support
>>> from pg_dump, and the answer is just about five years ago (64f3524e2).
>>> So moving the bar forward by five releases isn't at all out of line.
>>> 8.4 would be eight years past EOL by the time v15 comes out.
>> I'd really like us to adopt a "default" policy on this. I think it's a waste
>> to spend time every few years arguing what exact versions to drop. I'd much
>> rather say that, unless there are concrete reasons to deviate from that, we
>> provide pg_dump compatibility for 5+3 releases, pg_upgrade for 5+1, and psql
>> for 5 releases or something like that.
> I agree with considering something like that to be the minimum support
> policy, but the actual changes need a bit more care.  For example, when
> we last did this, the technical need was just to drop pre-7.4 versions,
> but we chose to make the cutoff 8.0 on the grounds that that was more
> understandable to users [1].  In the same way, I'm thinking of moving the
> cutoff to 9.0 now, although 8.4 would be sufficient from a technical
> standpoint.
>
> OTOH, in the new world of one-part major versions, it's less clear that
> there will be obvious division points for future cutoff changes.  Maybe
> versions-divisible-by-five would work?  Or versions divisible by ten,
> but experience so far suggests that we'll want to move the cutoff more
> often than once every ten years.
>
> 	


pg_upgrade claims to be able to operate on 8.4, which might be all the
better for some regular testing (which this could enable), so that seems
to me more like where the cutoff should be at least for this round.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  38. Re: pg_dump versus ancient server versions

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-12-02T10:01:47Z

    I was thinking a bit about formulating a policy for pg_dump backward
compatibility, based on the discussions in this thread.

Premises and preparatory thoughts:

- Users (and developers) want pg_dump to support server versions that
   are much older than non-EOL versions.

- Less critically, much-longer backward compatibility has also
   historically been provided for psql, so keeping those two the same
   would make sense.

- The policy for other client-side tools (list at [0]) is less clear
   and arguably less important.  I suggest we focus on pg_dump and psql
   first, and then we can decide for the rest whether they want to
   match a longer window, a shorter window, or a different policy
   altogether (e.g., ecpg).

- If we are going to maintain compatibility with very old server
   versions, we need to make sure the older server versions can at
   least still be built while an allegedly-compatible client tool is
   under support.

[0]: https://www.postgresql.org/docs/devel/reference-client.html

Proposal:

* pg_dump and psql will maintain compatibility with servers at least
   ten major releases back.

This assumes a yearly major release cadence.

I use the count of major releases here instead of some number of
years, as was previously discussed, for two reasons.  First, it makes
computing the cutoff easier, because you are not bothered by whether
some old release was released a few weeks before or after the
equivalent date in the current year for the new release.  Second,
there is no ambiguity about what happens during the lifetime of a
major release: If major release $NEW supports major release $OLD at
the time of $NEW's release, then that stays the same for the whole
life of $NEW; we don't start dropping support for $OLD in $NEW.5
because a year has passed.

I say "at least" because I wouldn't go around aggressively removing
support for old releases.  If $NEW is supposed to support 9.5 but
there is code that says `if (version > 9.4)`, I would not s/9.4/9.5/
that unless that code is touched for other reasons.

Then ...

* We keep old major release branches buildable as long as a new major
   release that has support for that old release is under support.

Buildable for this purpose means just enough that you can use it to
test pg_dump and psql.  This probably includes being able to run make
installcheck and use pg_dump and psql against the regression database.
It does not require support for any additional build-time options that
are not required for this purpose (e.g., new OpenSSL releases).
Conversely, it should be buildable with default compiler options.  For
example, if it fails to build and test cleanly unless you use -O0,
that should be fixed.  Fixes in very-old branches should normally be
backpatches that have stabilized in under-support branches.  Changes
that silence compiler warnings in newer compilers are by themselves
not considered a backpatch-worthy fix.

(In some cases, the support window of typical compilers should be
considered.  If adding support for a very new compiler with new
aggressive optimizations turns out to be too invasive, then that
compiler might simply be declared not supported for that release.  But
we should strive to support at least one compiler that still has some
upstream support.)

This keep-buildable effort is on an as-needed basis.  There is no
requirement to keep the buildability current at all times, and there
is no requirement to keep all platforms working at all times.
Obviously, any changes made to improve buildability should not
knowingly adversely affect other platforms.

(The above could be reconsidered if buildfarm support is available,
but I don't consider that necessary and wouldn't want to wait for it.)

There is no obligation on anyone backpatching fixes to supported
branches to also backpatch them to keep-buildable branches.  It is up
to those working on pg_dump/psql and requiring testing against old
versions to pick available fixes and apply them to keep-buildable
branches as needed.

Finally, none of this is meant to imply that there will be any
releases, packages, security support, production support, or community
support for keep-buildable branches.  This is a Git-repo-only,
developer-focusing effort.

Example under this proposal:

PG 15 supports PG 9.2
PG 14 supports PG 9.1
PG 13 supports PG 9.0
PG 12 supports PG 8.4
PG 11 supports PG 8.3
PG 10 supports PG 8.2

In practice, the effort can focus on keeping the most recent cutoff
release buildable.  So in the above example, we really only need to
keep PG >=9.2 buildable to support ongoing development.  The chances
that some needs to touch code pertaining to older versions in
backbranches is lower, so those really would need to be dealt with
very rarely.


The parent message has proposed to remove support for PG <9.0 from
master.  But I think that was chosen mainly because it was a round
number.  I suggest we pick a cutoff based on years, as I had
described, and then proceed with that patch.

  39. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-12-02T11:54:04Z

    On Thu, Dec 2, 2021 at 5:01 AM Peter Eisentraut
<peter.eisentraut@enterprisedb.com> wrote:
> * pg_dump and psql will maintain compatibility with servers at least
>    ten major releases back.
>
> * We keep old major release branches buildable as long as a new major
>    release that has support for that old release is under support.
>
> Buildable for this purpose means just enough that you can use it to
> test pg_dump and psql.  This probably includes being able to run make
> installcheck and use pg_dump and psql against the regression database.
> It does not require support for any additional build-time options that
> are not required for this purpose (e.g., new OpenSSL releases).
> Conversely, it should be buildable with default compiler options.  For
> example, if it fails to build and test cleanly unless you use -O0,
> that should be fixed.  Fixes in very-old branches should normally be
> backpatches that have stabilized in under-support branches.  Changes
> that silence compiler warnings in newer compilers are by themselves
> not considered a backpatch-worthy fix.

Sounds reasonable. It doesn't really make sense to insist that the
tools have to be compatible with releases that most developers can't
actually build.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  40. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-02T17:30:45Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> Proposal:

> * pg_dump and psql will maintain compatibility with servers at least
>    ten major releases back.
> * We keep old major release branches buildable as long as a new major
>    release that has support for that old release is under support.

> This assumes a yearly major release cadence.

If the point is to not have to count dates carefully, why does the cadence
matter?

> I say "at least" because I wouldn't go around aggressively removing
> support for old releases.  If $NEW is supposed to support 9.5 but
> there is code that says `if (version > 9.4)`, I would not s/9.4/9.5/
> that unless that code is touched for other reasons.

I can get behind something roughly like this, but I wonder if it wouldn't
be better to formulate the policy in a reactive way, i.e. when X happens
we'll do Y.  If we don't plan to proactively remove some code every year,
then it seems like the policy really is more like "when something breaks,
then we'll make an attempt to keep it working if the release is less than
ten majors back; otherwise we'll declare that release no longer
buildable."

However, this'd imply continuing to test against releases that are out of
the ten-year window but have not yet been found to be broken.  Not sure
if that's a useful expenditure of test resources or not.

> Buildable for this purpose means just enough that you can use it to
> test pg_dump and psql.  This probably includes being able to run make
> installcheck and use pg_dump and psql against the regression database.
> It does not require support for any additional build-time options that
> are not required for this purpose (e.g., new OpenSSL releases).

I agree with the idea of being conservative about what outside
dependencies we will worry about for "buildable" old versions.
(Your nearby message about Python breakage is a good example of
why we must limit that.)  But I wonder about, say, libxml or libicu,
or even if we can afford to drop all the non-plpgsql PLs.  An
example of why that seems worrisome is that it's not clear we'd
have any meaningful coverage of transforms in pg_dump with no PLs.
I don't have any immediate proposal here, but it seems like an area
that needs some thought and specific policy.

> Example under this proposal:

> PG 15 supports PG 9.2
> PG 14 supports PG 9.1
> PG 13 supports PG 9.0
> PG 12 supports PG 8.4
> PG 11 supports PG 8.3
> PG 10 supports PG 8.2

I was going to express concern about having to resurrect branches
back to 8.2, but:

> In practice, the effort can focus on keeping the most recent cutoff
> release buildable.  So in the above example, we really only need to
> keep PG >=9.2 buildable to support ongoing development.  The chances
> that some needs to touch code pertaining to older versions in
> backbranches is lower, so those really would need to be dealt with
> very rarely.

OK.  Also, when you do need to check that, there are often other ways
than rebuilding the old branch on modern platforms --- people may
well have still-executable builds laying about, even if rebuilding
from source would be problematic.

			regards, tom lane

  41. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-12-02T20:46:09Z

    On 12/2/21 12:30, Tom Lane wrote:
>
>> In practice, the effort can focus on keeping the most recent cutoff
>> release buildable.  So in the above example, we really only need to
>> keep PG >=9.2 buildable to support ongoing development.  The chances
>> that some needs to touch code pertaining to older versions in
>> backbranches is lower, so those really would need to be dealt with
>> very rarely.
> OK.  Also, when you do need to check that, there are often other ways
> than rebuilding the old branch on modern platforms --- people may
> well have still-executable builds laying about, even if rebuilding
> from source would be problematic.
>
> 			



I have a very old fedora instance where I can build every release back
to 7.2 :-) And with only slight massaging for the very old releases,
these builds run on my Fedora 34 development system. Certainly 8.2 and
up wouldn't be a problem. Currently I have only tested building without
any extra libraries/PLs, but I can look at other combinations. So, long
story short this is fairly doable at least in some environments. This
provides a good use case for the work I have been doing on backwards
compatibility of the TAP framework. I need to get back to that now that
the great module namespace adjustment has settled down.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  42. Re: pg_dump versus ancient server versions

    Andres Freund <andres@anarazel.de> — 2021-12-02T22:16:07Z

    Hi,

On 2021-12-02 11:01:47 +0100, Peter Eisentraut wrote:
> - The policy for other client-side tools (list at [0]) is less clear
>   and arguably less important.  I suggest we focus on pg_dump and psql
>   first, and then we can decide for the rest whether they want to
>   match a longer window, a shorter window, or a different policy
>   altogether (e.g., ecpg).

I think we should at least include pg_upgrade in this as well, it's pretty
closely tied to at least pg_dump.


> * pg_dump and psql will maintain compatibility with servers at least
>   ten major releases back.

Personally I think that's too long... It boils down keeping branches buildable
for ~15 years after they've been released. That strikes me as pretty far into
diminishing-returns, and steeply increasing costs, territory.

I realize it's more complicated for users, but a policy based on supporting a
certain number of out-of-support branches calculated from the newest major
version is more realistic. I'd personally go for something like newest-major -
7 (i.e. 2 extra releases), but I realize that others think it's worthwhile to
support a few more.  I think there's a considerable advantage of having one
cutoff date across all branches.

That's not to say we'd remove support for older versions from back
branches. Just that we don't ever consider them supported (or test them) once
below the cutoff.


> I use the count of major releases here instead of some number of
> years, as was previously discussed, for two reasons.  First, it makes
> computing the cutoff easier, because you are not bothered by whether
> some old release was released a few weeks before or after the
> equivalent date in the current year for the new release.  Second,
> there is no ambiguity about what happens during the lifetime of a
> major release: If major release $NEW supports major release $OLD at
> the time of $NEW's release, then that stays the same for the whole
> life of $NEW; we don't start dropping support for $OLD in $NEW.5
> because a year has passed.

Makes sense.


> * We keep old major release branches buildable as long as a new major
>   release that has support for that old release is under support.

> Buildable for this purpose means just enough that you can use it to
> test pg_dump and psql.  This probably includes being able to run make
> installcheck and use pg_dump and psql against the regression database.

I think we should explicitly limit the number of platforms we care about for
this purpose. I don't think we should even try to keep 8.2 compile on AIX or
whatnot.

Greetings,

Andres Freund

  43. Re: pg_dump versus ancient server versions

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-12-03T16:19:47Z

    On 02.12.21 18:30, Tom Lane wrote:
>> This assumes a yearly major release cadence.
> 
> If the point is to not have to count dates carefully, why does the cadence
> matter?

If we were to change the release cadence, then it would be appropriate 
to review this policy.

> I can get behind something roughly like this, but I wonder if it wouldn't
> be better to formulate the policy in a reactive way, i.e. when X happens
> we'll do Y.  If we don't plan to proactively remove some code every year,
> then it seems like the policy really is more like "when something breaks,
> then we'll make an attempt to keep it working if the release is less than
> ten majors back; otherwise we'll declare that release no longer
> buildable."

This sounds like it would give license to accidentally break support for 
old releases in the code and only fix them if someone complains.  That's 
not really what I would be aiming for.

> I agree with the idea of being conservative about what outside
> dependencies we will worry about for "buildable" old versions.
> (Your nearby message about Python breakage is a good example of
> why we must limit that.)  But I wonder about, say, libxml or libicu,
> or even if we can afford to drop all the non-plpgsql PLs.  An
> example of why that seems worrisome is that it's not clear we'd
> have any meaningful coverage of transforms in pg_dump with no PLs.
> I don't have any immediate proposal here, but it seems like an area
> that needs some thought and specific policy.

Yeah, I think questions like this will currently quickly lead to dead 
ends.  We are talking 5 years this, 10 years that here.  Everybody else 
(apart from RHEL) is talking at best in the range 3-5 years.  We will 
have to figure this out as we go.

  44. Re: pg_dump versus ancient server versions

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-12-03T16:29:41Z

    On 02.12.21 23:16, Andres Freund wrote:
> I think we should at least include pg_upgrade in this as well, it's pretty
> closely tied to at least pg_dump.

right

>> * pg_dump and psql will maintain compatibility with servers at least
>>    ten major releases back.
> 
> Personally I think that's too long... It boils down keeping branches buildable
> for ~15 years after they've been released. That strikes me as pretty far into
> diminishing-returns, and steeply increasing costs, territory.

Well, it is a lot, but it's on the order of what we have historically 
provided.

> I realize it's more complicated for users, but a policy based on supporting a
> certain number of out-of-support branches calculated from the newest major
> version is more realistic. I'd personally go for something like newest-major -
> 7 (i.e. 2 extra releases), but I realize that others think it's worthwhile to
> support a few more.  I think there's a considerable advantage of having one
> cutoff date across all branches.

I'm not sure it will be clear what this would actually mean.  Assume 
PG11 supports back to 9.4 (14-7) now, but when PG15 comes out, we drop 
9.4 support. But the PG11 code hasn't changed, and PG9.4 hasn't changed, 
so it will most likely still work.  Then we have messaging that is out 
of sync with reality.  I can see the advantage of this approach, but the 
communication around it might have to be refined.

> I think we should explicitly limit the number of platforms we care about for
> this purpose. I don't think we should even try to keep 8.2 compile on AIX or
> whatnot.

It's meant to be developer-facing, so only for platforms that developers 
use.  I think that can police itself, if we define it that way.

  45. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-03T17:10:07Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> [ policy requiring that 9.2 and up be kept buildable, as of today ]

I experimented to see what this would entail exactly.  Using
current macOS (Apple clang version 13.0.0) on M1 hardware,
I built with minimal configure options (--enable-debug --enable-cassert)
and ran the core regression tests.  I found that commit
1c0cf52b3 (Use return instead of exit() in configure) is needed
in 9.4 and before, else we don't get through configure.
That's the only fix needed to get a clean build in 9.4 and 9.3.
9.2 shows several compiler warnings, the scarier ones of which could
be cleaned up by back-patching c74d586d2 (Fix function return type
confusion).  The remainder are variable-may-be-used-uninitialized
warnings, which I think people are accustomed to ignoring in
dubious cases.  In any case, I failed to get rid of them without
back-patching 71450d7fd (Teach compiler that ereport(>=ERROR) does
not return), which seems like a bridge too far.

I also tried 9.1, but it has multiple compile-time problems:
* fails to select a spinlock implementation
* "conflicting types for 'base_yylex'"
* strange type-conflict warnings in zlib calls

So at least on this platform, there are solid technical reasons
to select 9.2 not 9.1 as the cutoff.

Obviously, we might find some other things to fix if we checked
with other compilers, or tested more than the core tests.
But this much seems quite doable, and it's probably prerequisite
for any further testing.

			regards, tom lane

  46. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-12-03T17:28:11Z

    On 12/3/21 12:10, Tom Lane wrote:
> Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
>> [ policy requiring that 9.2 and up be kept buildable, as of today ]
> I experimented to see what this would entail exactly.  Using
> current macOS (Apple clang version 13.0.0) on M1 hardware,
> I built with minimal configure options (--enable-debug --enable-cassert)
> and ran the core regression tests.  


I've mentioned my efforts on fedora previously. But like you I used a
minimal configuration. So what would be reasonable to test? I know you
mentioned building with perl and python upthread so we could possibly
test transforms. Anything else? I don't think we need to worry about all
the authentication-supporting options. XML/XSLT maybe.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  47. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-03T18:09:31Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> On 12/3/21 12:10, Tom Lane wrote:
>> I experimented to see what this would entail exactly.  Using
>> current macOS (Apple clang version 13.0.0) on M1 hardware,
>> I built with minimal configure options (--enable-debug --enable-cassert)
>> and ran the core regression tests.  

> I've mentioned my efforts on fedora previously. But like you I used a
> minimal configuration. So what would be reasonable to test? I know you
> mentioned building with perl and python upthread so we could possibly
> test transforms. Anything else? I don't think we need to worry about all
> the authentication-supporting options. XML/XSLT maybe.

Not sure.  I think we should evaluate based on
1. how integral is the option, ie how much PG code can't we test if
   we don't enable it.
2. how stable is the referenced code.

Point 2 makes me want to exclude both Python and OpenSSL, as they've
both proven to be moving API targets.  If we want to have tests for
transform modules, plperl would be sufficient for that, and perl seems
to be a lot more stable than python.  libxml is pretty morib^H^H^Hstable,
but on the other hand it seems quite noncritical for the sorts of tests
we want to run against old servers, so I'd be inclined to exclude it.

Looking through the other configure options, the only one that I find
to be a hard call is --enable-nls.  In theory this shouldn't be
critical for testing pg_dump or psql ... but you never know, and it
hasn't been a stability problem.  Every other one I think we could
ignore for these purposes.  At some point --with-icu might become
interesting, but it isn't yet relevant to any out-of-support
branches, so we can leave that call for another day.

			regards, tom lane

  48. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-03T18:30:31Z

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> writes:
> On 02.12.21 23:16, Andres Freund wrote:
>> I realize it's more complicated for users, but a policy based on supporting a
>> certain number of out-of-support branches calculated from the newest major
>> version is more realistic. I'd personally go for something like newest-major -
>> 7 (i.e. 2 extra releases), but I realize that others think it's worthwhile to
>> support a few more.  I think there's a considerable advantage of having one
>> cutoff date across all branches.

> I'm not sure it will be clear what this would actually mean.  Assume 
> PG11 supports back to 9.4 (14-7) now, but when PG15 comes out, we drop 
> 9.4 support. But the PG11 code hasn't changed, and PG9.4 hasn't changed, 
> so it will most likely still work.  Then we have messaging that is out 
> of sync with reality.  I can see the advantage of this approach, but the 
> communication around it might have to be refined.

I don't find this suggestion to be an improvement over Peter's original
formulation, for two reasons:

* I'm not convinced that it saves us any actual work; as you say, the
code doesn't stop working just because we declare it out-of-support.

* There's a real-world use-case underneath here.  If somewhere you've
discovered a decades-old server that you need to upgrade, and current
pg_dump won't dump from it, you would like it to be well-defined
which intermediate pg_dump versions you can use.  So if 10.19 can
dump from that hoary server, it would not be nice if 10.20 can't;
nor if the documentation lies to you about that based on which minor
version you happen to consult.

>> I think we should explicitly limit the number of platforms we care about for
>> this purpose. I don't think we should even try to keep 8.2 compile on AIX or
>> whatnot.

> It's meant to be developer-facing, so only for platforms that developers 
> use.  I think that can police itself, if we define it that way.

I agree that if you care about doing this sort of test on platform X,
it's up to you to patch for that.  I think Andres' concern is about
the amount of committer bandwidth that might be needed to handle
such patches submitted by non-committers.  However, based on the
experiment I just ran, I think it's not really likely to be a big deal:
there are not that many problems, and most of them just amount to
back-patching something that originally wasn't back-patched.

What's most likely to happen IMO is that committers will just start
back-patching essential portability fixes into out-of-support-but-
still-in-the-buildability-window branches, contemporaneously with
the original fix.  Yeah, that does mean more committer effort,
but only for a very small number of patches.

			regards, tom lane

  49. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-12-03T20:46:04Z

    On Fri, Dec 3, 2021 at 1:30 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> What's most likely to happen IMO is that committers will just start
> back-patching essential portability fixes into out-of-support-but-
> still-in-the-buildability-window branches, contemporaneously with
> the original fix.  Yeah, that does mean more committer effort,
> but only for a very small number of patches.

I agree. I think that's exactly what we want to have happen, and if a
given policy won't have exactly this result then the policy needs
adjusting.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  50. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-06T00:41:14Z

    I ran a new set of experiments concerning building back branches
on modern platforms, this time trying Fedora 35 (gcc 11.2.1)
on x86_64.  I widened the scope of the testing a bit by adding
"--enable-nls --with-perl" and running check-world not just the
core tests.  Salient results:

* Everything back to 9.2 passes the test, although with more
and more compile warnings the further back you go.

* 9.1 fails with "conflicting types for 'base_yylex'", much as
I saw on macOS except it's a hard error on this compiler.

* Parallel check-world is pretty unreliable before v10 (I knew
this already, actually).  But without parallelism, it's fine.

Based on these results, I think maybe we should raise our ambitions
a bit compared to Peter's original proposal.  Specifically,
I wonder if it wouldn't be wise to try to silence compile warnings
in these branches.  The argument for this is basically that if we
don't, then every time someone builds one of these branches, they
have to tediously go through the warnings and verify that
they're not important.  It won't take long for the accumulated
time-wastage from that to exceed the cost of back-patching whatever
we did to silence the warning in later branches.

Now, I'm still not interested in trying to silence
maybe-uninitialized warnings pre-9.3, mainly because of the
ereport-ERROR-doesnt-return issue.  (I saw far fewer of those
under gcc than clang, but not zero.)  We could ignore those
figuring that 9.2 will be out of scope in a year anyway, or else
teach 9.2's configure to select -Wno-maybe-uninitialized where
possible.

Likewise, getting check-world to parallelize successfully pre-v10
seems like a bridge too far.  But I would, for example, be in favor
of back-patching eb9812f27 (Make pg_upgrade's test.sh less chatty).
It's just annoying to run check-world and get that output now.

			regards, tom lane

  51. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-12-06T20:38:55Z

    On Sun, Dec 5, 2021 at 7:41 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Based on these results, I think maybe we should raise our ambitions
> a bit compared to Peter's original proposal.  Specifically,
> I wonder if it wouldn't be wise to try to silence compile warnings
> in these branches.  The argument for this is basically that if we
> don't, then every time someone builds one of these branches, they
> have to tediously go through the warnings and verify that
> they're not important.  It won't take long for the accumulated
> time-wastage from that to exceed the cost of back-patching whatever
> we did to silence the warning in later branches.

Yep. I have long been of the view, and have said before, that there is
very little harm in doing some maintenance of EOL branches. Making it
easy to test against them is a great way to improve our chances of
actually having the amount of backward-compatibility that we say we
want to have.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  52. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-06T21:19:08Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Sun, Dec 5, 2021 at 7:41 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Based on these results, I think maybe we should raise our ambitions
>> a bit compared to Peter's original proposal.  Specifically,
>> I wonder if it wouldn't be wise to try to silence compile warnings
>> in these branches.

> Yep. I have long been of the view, and have said before, that there is
> very little harm in doing some maintenance of EOL branches. Making it
> easy to test against them is a great way to improve our chances of
> actually having the amount of backward-compatibility that we say we
> want to have.

Right.  The question that's on the table is how much is the right
amount of maintenance.  I think that back-patching user-visible bug
fixes, for example, is taking things too far.  What we want is to
be able to replicate the behavior of the branch's last released
version, using whatever build tools we are currently using.  So
back-patching something like that is counterproductive, because
now the behavior is not what was released.

A minimal amount of maintenance would be "only back-patch fixes
for issues that cause failure-to-build".  The next step up is "fix
issues that cause failure-to-pass-regression-tests", and then above
that is "fix developer-facing annoyances, such as compiler warnings
or unwanted test output, as long as you aren't changing user-facing
behavior".  I now think that it'd be reasonable to include this
last group, although I'm pretty sure Peter didn't have that in mind
in his policy sketch.

			regards, tom lane

  53. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-12-07T16:33:38Z

    On Mon, Dec 6, 2021 at 4:19 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Right.  The question that's on the table is how much is the right
> amount of maintenance.  I think that back-patching user-visible bug
> fixes, for example, is taking things too far.  What we want is to
> be able to replicate the behavior of the branch's last released
> version, using whatever build tools we are currently using.  So
> back-patching something like that is counterproductive, because
> now the behavior is not what was released.
>
> A minimal amount of maintenance would be "only back-patch fixes
> for issues that cause failure-to-build".  The next step up is "fix
> issues that cause failure-to-pass-regression-tests", and then above
> that is "fix developer-facing annoyances, such as compiler warnings
> or unwanted test output, as long as you aren't changing user-facing
> behavior".  I now think that it'd be reasonable to include this
> last group, although I'm pretty sure Peter didn't have that in mind
> in his policy sketch.

Yep, that seems reasonable to me.

I guess the point about user-visible bug fixes is that, as soon as we
start doing that, we don't really want it to be hit-or-miss. We could
make a decision to back-patch all bug fixes or those of a certain
severity or whatever we like back to older branches, and then those
branches would be supported or semi-supported depending on what rule
we adopted, and we could even continue to do releases for them if we
so chose. However, it wouldn't be a great idea to back-patch a
completely arbitrary subset of our fixes into those branches, because
then it sort of gets confusing to understand what the status of that
branch is. I don't know that I'm terribly bothered by the idea that
the behavior of the branch might deviate from the last official
release, because most bug fixes are pretty minor and wouldn't really
affect testing much, but it would be a little annoying to explain to
users that those branches contain an arbitrary subset of newer fixes,
and a little hard for us to understand what is and is not there.

That being said, suppose that a new compiler version comes out and on
that new compiler version, 'make check' crashes on the older branch
due to a missing WhateverGetDatum() call that we rectified in a later,
back-patched commit. I would consider it reasonable to back-patch that
particular bug fix into an unsupported branch to make it testable,
just like we would do for a failure-to-build issue.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  54. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-07T17:46:09Z

    Robert Haas <robertmhaas@gmail.com> writes:
> I guess the point about user-visible bug fixes is that, as soon as we
> start doing that, we don't really want it to be hit-or-miss. We could
> make a decision to back-patch all bug fixes or those of a certain
> severity or whatever we like back to older branches, and then those
> branches would be supported or semi-supported depending on what rule
> we adopted, and we could even continue to do releases for them if we
> so chose. However, it wouldn't be a great idea to back-patch a
> completely arbitrary subset of our fixes into those branches, because
> then it sort of gets confusing to understand what the status of that
> branch is.

Yup, and also confusing to understand whether a given new fix should
be back-patched into the out-of-support-but-keep-buildable branches.
I want to settle on a reasonably well-defined policy for that.

I'm basically suggesting that the policy should be "back-patch the
minimal fix needed so that you can still get a clean build and clean
check-world run, using thus-and-such configure options".  (The point
of the configure options limitation being to exclude moving-target
external dependencies, such as Python.)  I think that Peter's
original suggestion could be read the same way except for the
adjective "clean".  He also said that only core regression needs
to pass not check-world; but if we're trying to test things like
pg_dump compatibility, I think we want the wider scope of what to
keep working.

			regards, tom lane

  55. Re: pg_dump versus ancient server versions

    Mark Dilger <mark.dilger@enterprisedb.com> — 2021-12-07T18:26:13Z

    
> On Dec 7, 2021, at 8:33 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> 
> However, it wouldn't be a great idea to back-patch a
> completely arbitrary subset of our fixes into those branches, because
> then it sort of gets confusing to understand what the status of that
> branch is. I don't know that I'm terribly bothered by the idea that
> the behavior of the branch might deviate from the last official
> release, because most bug fixes are pretty minor and wouldn't really
> affect testing much, but it would be a little annoying to explain to
> users that those branches contain an arbitrary subset of newer fixes,
> and a little hard for us to understand what is and is not there.

Wouldn't you be able to see what changed by comparing the last released tag for version X.Y against the RELX_Y_STABLE branch?  Something like `git diff REL8_4_22 origin/REL8_4_STABLE > buildability.patch`?

Having such a patch should make reproducing old corruption bugs easier, as you could apply the buildability.patch to the last branch that contained the bug.  If anybody did that work, would we want it committed somewhere?  REL8_4_19_BUILDABLE or such?  For patches that apply trivially, that might not be worth keeping, but if the merge is difficult, maybe sharing with the community would make sense.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  56. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-07T18:52:34Z

    Mark Dilger <mark.dilger@enterprisedb.com> writes:
> Wouldn't you be able to see what changed by comparing the last released tag for version X.Y against the RELX_Y_STABLE branch?  Something like `git diff REL8_4_22 origin/REL8_4_STABLE > buildability.patch`?

> Having such a patch should make reproducing old corruption bugs easier, as you could apply the buildability.patch to the last branch that contained the bug.  If anybody did that work, would we want it committed somewhere?  REL8_4_19_BUILDABLE or such?  For patches that apply trivially, that might not be worth keeping, but if the merge is difficult, maybe sharing with the community would make sense.

I'm not entirely following ... are you suggesting that each released minor
version needs to be kept buildable separately?  That seems like a huge
amount of extra committer effort with not much added value.  If someone
comes to me and wants to investigate a bug in a branch that's already
out-of-support, and they then say they're not running the last minor
release, I'm going to tell them to come back after updating.

It is (I suspect) true that diffing the last release against branch
tip would often yield a patch that could be used to make an older
minor release buildable again.  But when that patch doesn't work
trivially, I for one am not interested in making it work.  And
especially not interested in doing so "on spec", with no certainty
that anyone would ever need it.

			regards, tom lane

  57. Re: pg_dump versus ancient server versions

    Mark Dilger <mark.dilger@enterprisedb.com> — 2021-12-07T18:59:02Z

    
> On Dec 7, 2021, at 10:52 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> I'm not entirely following ... are you suggesting that each released minor
> version needs to be kept buildable separately?

No.  I'm just wondering if we want to share the product of such efforts if anybody (me, for instance) volunteers to do it for some subset of minor releases.  For my heap corruption checking work, I might want to be able to build a small number of old minor releases that I know had corruption bugs.

—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  58. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-12-07T19:19:36Z

    On 12/7/21 13:59, Mark Dilger wrote:
>> On Dec 7, 2021, at 10:52 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>
>> I'm not entirely following ... are you suggesting that each released minor
>> version needs to be kept buildable separately?
> No.  I'm just wondering if we want to share the product of such efforts if anybody (me, for instance) volunteers to do it for some subset of minor releases.  For my heap corruption checking work, I might want to be able to build a small number of old minor releases that I know had corruption bugs.
>

I doubt there's going to be a whole lot of changes. You should just be
able to cherry-pick them in most cases I suspect.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  59. Re: pg_dump versus ancient server versions

    Peter Eisentraut <peter.eisentraut@enterprisedb.com> — 2021-12-09T14:01:43Z

    On 06.12.21 22:19, Tom Lane wrote:
> A minimal amount of maintenance would be "only back-patch fixes
> for issues that cause failure-to-build".  The next step up is "fix
> issues that cause failure-to-pass-regression-tests", and then above
> that is "fix developer-facing annoyances, such as compiler warnings
> or unwanted test output, as long as you aren't changing user-facing
> behavior".  I now think that it'd be reasonable to include this
> last group, although I'm pretty sure Peter didn't have that in mind
> in his policy sketch.

I would be okay with that.

  60. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-09T17:50:40Z

    [ mostly for the archives' sake ]

I wrote:
> I ran a new set of experiments concerning building back branches
> on modern platforms, this time trying Fedora 35 (gcc 11.2.1)
> on x86_64.  I widened the scope of the testing a bit by adding
> "--enable-nls --with-perl" and running check-world not just the
> core tests.  Salient results:

> * 9.1 fails with "conflicting types for 'base_yylex'", much as
> I saw on macOS except it's a hard error on this compiler.

I poked a little harder at what might be needed to get 9.1 compiled
on modern platforms.  It looks like the base_yylex issue is down
to newer versions of flex doing things differently.  We fixed
that in the v10 era via 72b1e3a21 (Build backend/parser/scan.l and
interfaces/ecpg/preproc/pgc.l standalone) and 92fb64983 (Use "%option
prefix" to set API names in ecpg's lexer), which were later back-patched
as far down as 9.2.  It might not be out of the question to back-patch
those further, but the 9.2 patches don't apply cleanly to 9.1, so some
effort would be needed.

Worrisomely, I also noted warnings like

parse_coerce.c:791:67: warning: array subscript 1 is above array bounds of 'Oid[1]' {aka 'unsigned int[1]'} [-Warray-bounds]
  791 |                 Assert(nargs < 2 || procstruct->proargtypes.values[1] == INT4OID);
      |                                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~

which remind me that 9.1 lacks 8137f2c32 (Hide most variable-length fields
from Form_pg_* structs).  We did stick -fno-aggressive-loop-optimizations
into 9.1 and older branches back in 2015, but I don't have a lot of
confidence that that'd be sufficient to prevent misoptimizations in
current-vintage compilers.  Back-patching 8137f2c32 and all the follow-on
work is very clearly not something to consider, so dialing down the -O
level might be necessary if you were interested in making this go.

In short then, there is a really large gap between 9.1 and 9.2 in terms
of how hard they are to build on current toolchains.  It's kind of
fortunate that Peter proposed 9.2 rather than some earlier cutoff.
In any case, I've completely lost interest in trying to move the
keep-it-buildable cutoff to any earlier than 9.2; it doesn't look
like the effort-to-benefit ratio would be attractive at all.

			regards, tom lane

  61. Re: pg_dump versus ancient server versions

    Andrew Dunstan <andrew@dunslane.net> — 2021-12-11T15:17:17Z

    On 12/9/21 12:50, Tom Lane wrote:
>
> In short then, there is a really large gap between 9.1 and 9.2 in terms
> of how hard they are to build on current toolchains.  It's kind of
> fortunate that Peter proposed 9.2 rather than some earlier cutoff.
> In any case, I've completely lost interest in trying to move the
> keep-it-buildable cutoff to any earlier than 9.2; it doesn't look
> like the effort-to-benefit ratio would be attractive at all.
>
> 			


9.2 is how far back crake goes in testing pg_ugrade from old versions,
so that could well be a convenient stopping point. For older versions
there is still the possibility of building on older toolchains and
running on modern ones. Yes it's more cumbersome, but it does mean we
can test an awful long way back. I don't remember the last time I saw a
really old version in the wild, but I'm sure there are some out there
sitting in a cupboard humming along.

This might also be a good time to revive work on making the TAP test
framework backwards compatible via subclassing.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  62. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-12T06:24:24Z

    Andrew Dunstan <andrew@dunslane.net> writes:
> 9.2 is how far back crake goes in testing pg_ugrade from old versions,
> so that could well be a convenient stopping point. For older versions
> there is still the possibility of building on older toolchains and
> running on modern ones. Yes it's more cumbersome, but it does mean we
> can test an awful long way back.

Right.  I think the point of the current discussion is to ensure that,
if we expect new patches for pg_dump or psql to work against version-N
servers, that it's not too unpleasant for patch submitters to build
and test against version N.  There's a different discussion to be had
about what we do if we receive a bug report about compatibility with
some more-ancient-than-that version.  But that is, I hope, a far less
common scenario; so it's okay if it requires extra effort, and/or use
of setups that not everyone has handy.

Anyway, it seems like there's some consensus that 9.2 is a good
stopping place for today.  I'll push forward with
(1) back-patching as necessary to make 9.2 and up build cleanly
on the platforms I have handy;
(2) ripping out pg_dump's support for pre-9.2 servers;
(3) ripping out psql's support for pre-9.2 servers.

In a preliminary look, it did not seem that (3) would save very
much code, but it seems like we ought to do it if we're being
consistent.

A point we've not discussed is whether to drop any bits of libpq
that are only needed for such old servers.  I feel a bit more
uncomfortable about that, mainly because I'm pretty sure that
only a few lines of code would be involved, and it seems to have
more of an air of burning-the-bridges finality about it than (say)
dropping psql/describe.c support.  On the other hand, the point
about what's required to test future patches still applies.

			regards, tom lane

  63. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-13T17:23:10Z

    I wrote:
> Anyway, it seems like there's some consensus that 9.2 is a good
> stopping place for today.  I'll push forward with
> (1) back-patching as necessary to make 9.2 and up build cleanly
> on the platforms I have handy;

I've done as much as I plan to do in that direction.  As of the
respective branch tips, I see clean builds and check-world
results with minimal configure options in all branches back to 9.2
on Fedora 35 (gcc 11.2.1) and macOS Monterey (Apple clang 13.0.0).

A few notes for the archives' sake:

* As discussed, parallel check-world is unreliable before v10;
perhaps this is worth improving, but I doubt it.  I did find
that aggressive parallelism in the build process is fine.

* On some compilers, pre-v10 branches produce this warning:

scan.c: In function 'yy_try_NUL_trans':
scan.c:10189:23: warning: unused variable 'yyg' [-Wunused-variable]
     struct yyguts_t * yyg = (struct yyguts_t*)yyscanner; /* This var may be unused depending upon options. */

In principle we could back-patch 65d508fd4 to silence that,
but I think that fix is more invasive than what we want to
do in these branches.  We lived with that warning for years
before figuring out how to get rid of it, so I think we can
continue to accept it in these branches.

* 9.2's plperl fails to compile on macOS:

./plperl.h:53:10: fatal error: 'EXTERN.h' file not found
#include "EXTERN.h"

This is evidently because 9.2 predates the "sysroot" hacking
we did later (5e2217131 and many many subsequent tweaks).
I judge this not worth the trouble to fix, because the argument
for supporting --with-perl in these branches is basically that
we need a PL with transforms to test pg_dump ... but transforms
didn't come in until 9.5.  (Reviewing the commit log, I suppose
that 9.3 and 9.4 would also fail to build in some macOS
configurations, but by the same argument I see no need to work
further on those branches either.  9.5 does have all the
sysroot changes I can find in the log.)

			regards, tom lane

  64. Re: pg_dump versus ancient server versions

    Robert Haas <robertmhaas@gmail.com> — 2021-12-13T17:33:50Z

    On Mon, Dec 13, 2021 at 12:23 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I've done as much as I plan to do in that direction.  As of the
> respective branch tips, I see clean builds and check-world
> results with minimal configure options in all branches back to 9.2
> on Fedora 35 (gcc 11.2.1) and macOS Monterey (Apple clang 13.0.0).

I think this is great. Thanks for being willing to work on it.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  65. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-14T22:18:44Z

    I wrote:
> Anyway, it seems like there's some consensus that 9.2 is a good
> stopping place for today.  I'll push forward with
> (1) back-patching as necessary to make 9.2 and up build cleanly
> on the platforms I have handy;
> (2) ripping out pg_dump's support for pre-9.2 servers;
> (3) ripping out psql's support for pre-9.2 servers.

I've completed the pg_dump/pg_dumpall part of that, but while
updating the docs I started to wonder whether we shouldn't nuke
pg_dump's --no-synchronized-snapshots option.  As far as I can
make out, the remaining use case for that is to let you perform an
unsafe parallel dump from a standby server of an out-of-support
major version.  I'm not very clear why we allowed that at all,
ever, rather than saying you can't parallelize in such cases.
But for sure that remaining use case is paper thin, and leaving
the option available seems way more likely to let people shoot
themselves in the foot than to let them do anything helpful.

Barring objections, I'll remove that option in a day or two.

			regards, tom lane

  66. Re: pg_dump versus ancient server versions

    Justin Pryzby <pryzby@telsasoft.com> — 2021-12-16T04:08:07Z

    On Tue, Dec 14, 2021 at 05:18:44PM -0500, Tom Lane wrote:
> I wrote:
> > Anyway, it seems like there's some consensus that 9.2 is a good
> > stopping place for today.  I'll push forward with
> > (1) back-patching as necessary to make 9.2 and up build cleanly
> > on the platforms I have handy;
> > (2) ripping out pg_dump's support for pre-9.2 servers;
> > (3) ripping out psql's support for pre-9.2 servers.
> 
> I've completed the pg_dump/pg_dumpall part of that, but while

Is it possible to clean up pg_upgrade, too ?

-- 
Justin

  67. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-16T04:55:41Z

    Justin Pryzby <pryzby@telsasoft.com> writes:
> On Tue, Dec 14, 2021 at 05:18:44PM -0500, Tom Lane wrote:
>> I've completed the pg_dump/pg_dumpall part of that, but while

> Is it possible to clean up pg_upgrade, too ?

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=e469f0aaf3c586c8390bd65923f97d4b1683cd9f

I'm still working on psql.

			regards, tom lane

  68. Re: pg_dump versus ancient server versions

    Justin Pryzby <pryzby@telsasoft.com> — 2021-12-16T04:58:04Z

    On Wed, Dec 15, 2021 at 10:08:07PM -0600, Justin Pryzby wrote:
> Is it possible to clean up pg_upgrade, too ?

Nevermind - I found yesterday's e469f0aaf3 after git-fetch.

I think you missed a few parts though ?

src/bin/pg_upgrade/function.c
                if (GET_MAJOR_VERSION(old_cluster.major_version) <= 900)
...
                        if (GET_MAJOR_VERSION(old_cluster.major_version) <= 900 &&
                                strcmp(lib, "$libdir/plpython") == 0)

src/bin/pg_upgrade/option.c
                                 * Someday, the port number option could be removed and passed
                                 * using -o/-O, but that requires postmaster -C to be
                                 * supported on all old/new versions (added in PG 9.2).
...
        if (GET_MAJOR_VERSION(cluster->major_version) >= 901)

  69. Re: pg_dump versus ancient server versions

    Tom Lane <tgl@sss.pgh.pa.us> — 2021-12-16T05:02:54Z

    Justin Pryzby <pryzby@telsasoft.com> writes:
> I think you missed a few parts though ?

Um.  I think those are leftover from when I was intending the
cutoff to be 9.0 not 9.2.  I'll take a fresh look tomorrow.

			regards, tom lane

  70. Re: pg_dump versus ancient server versions

    Greg Stark <stark@mit.edu> — 2021-12-17T08:06:50Z

    On Fri, 22 Oct 2021 at 19:27, Robert Haas <robertmhaas@gmail.com> wrote:
>
> Another thing to think about in that regard: how likely is that
> PostgreSQL 7.4 and PostgreSQL 15 both compile and run on the same
> operating system? I suspect the answer is "not very." I seem to recall
> Greg Stark trying to compile really old versions of PostgreSQL for a
> conference talk some years ago, and he got back to a point where it
> just became impossible to make work on modern toolchains even with a
> decent amount of hackery.

That was when I compared sorting performance over time. I was able to
get Postgres to build back to the point where 64-bit architecture
support was added. From Andrew Dunstans comment later in this thread
I'm guessing that was 7.2

That was basically at the point where 64-bit architecture support was
added. It looks like the earliest date on the graphs in the talk are
2002-11-27 which matches the 7.3 release date. I think building
earlier versions would have been doable if I had built them in 32-bit
mode.

-- 
greg