Horton

Thread

Commits

  1. Add static assertions about pg_control fitting into one disk sector.

  2. doc: Remove PostgreSQL version number from xml2 deprecation notice

  1. Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-15T22:00:34Z

    The types abstime, reltime, and tinterval need to go away, or be
reimplemented, sometime well before 2038 when they will overflow.
It's not too soon to start having a plan for that, especially seeing
that it seems to take a decade or more for us to actually get rid
of anything we've deprecated.

Right offhand, I don't think there is any functionality in these
types that isn't handled as well or better by timestamptz, interval,
and tstzrange respectively.  And they're basically undocumented
except for a sort-of deprecation notice just above section 8.5.1.
So my inclination is to remove them rather than try to upgrade them
in any way.  However, we'd have to do something about:

* The legacy system views pg_shadow and pg_user have abstime columns.
Experimentation suggests that we could convert those to timestamptz(0)
and the output format wouldn't change, so maybe that's a good enough
fix there.

* contrib/spi/timetravel depends on abstime columns to represent what
would nowadays be better done as a tstzrange.  I'd have thought we
could maybe toss that example module overboard, but we just today got
a question about its usage, so I'm afraid it's not quite dead yet.
What shall we do with that?

While it's too late in the v10 cycle to do anything very meaningful
about this now, I am tempted to strengthen the deprecation notice's
wording from "might disappear" to "will disappear".  And it's not good
that the documentation of contrib/spi/timetravel contains nothing
whatever pointing out the lack of future-proof-ness of abstime.

Thoughts?

			regards, tom lane

  2. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-16T16:32:57Z

    On Sat, Jul 15, 2017 at 6:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Right offhand, I don't think there is any functionality in these
> types that isn't handled as well or better by timestamptz, interval,
> and tstzrange respectively.  And they're basically undocumented
> except for a sort-of deprecation notice just above section 8.5.1.
> So my inclination is to remove them rather than try to upgrade them
> in any way.  However, we'd have to do something about:
>
> * The legacy system views pg_shadow and pg_user have abstime columns.
> Experimentation suggests that we could convert those to timestamptz(0)
> and the output format wouldn't change, so maybe that's a good enough
> fix there.

+1 for doing that in v11.

> * contrib/spi/timetravel depends on abstime columns to represent what
> would nowadays be better done as a tstzrange.  I'd have thought we
> could maybe toss that example module overboard, but we just today got
> a question about its usage, so I'm afraid it's not quite dead yet.
> What shall we do with that?

No idea.  I think if nobody's willing to come up with a plan for this
and do the work to implement it, we should just remove the module when
we get closer to 2038.  But I don't think we have to make that
decision for at least another 5 years or so.

> While it's too late in the v10 cycle to do anything very meaningful
> about this now, I am tempted to strengthen the deprecation notice's
> wording from "might disappear" to "will disappear".

-1 for changing that; such predictions often turn out to be wrong.

> And it's not good
> that the documentation of contrib/spi/timetravel contains nothing
> whatever pointing out the lack of future-proof-ness of abstime.

+1 for fixing that, though.  Maybe we can find a wording that says in
effect "this module will need to be removed or fixed sometime not too
many years from now; if you care about keeping it around, maybe you
should volunteer to do the necessary fixing".

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  3. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-16T17:03:52Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Sat, Jul 15, 2017 at 6:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> * contrib/spi/timetravel depends on abstime columns to represent what
>> would nowadays be better done as a tstzrange.  I'd have thought we
>> could maybe toss that example module overboard, but we just today got
>> a question about its usage, so I'm afraid it's not quite dead yet.
>> What shall we do with that?

> No idea.  I think if nobody's willing to come up with a plan for this
> and do the work to implement it, we should just remove the module when
> we get closer to 2038.  But I don't think we have to make that
> decision for at least another 5 years or so.

The plan I'd tentatively suggest is to just s/abstime/timestamptz/g
in the timetravel module, something that would be unlikely to take more
than an hour's work.  If anyone was actually using it in production
they'd have to do some ALTER COLUMN TYPE changes before the trigger
would work again ... but that would get forced on them eventually
anyway.  Yup, you could turn this molehill into a mountain of work
if you wanted to have a more friendly transition, but I don't see
anyone putting in that much effort.

>> While it's too late in the v10 cycle to do anything very meaningful
>> about this now, I am tempted to strengthen the deprecation notice's
>> wording from "might disappear" to "will disappear".

> -1 for changing that; such predictions often turn out to be wrong.

Those types will definitely fail for all use-cases in 2038, and for
many use-cases significantly before that; there's no uncertainty in that
prediction.  The only way they aren't going to disappear before 2038 is
if the project is defunct first, or if somebody is sufficiently concerned
with having an easier migration path that they are willing to design and
implement one.  I don't believe any of the usual suspects are going to do
that.  But if there's somebody who would care enough to de-lurk and make
it happen, they'd be much more likely to do so soon enough if there's some
advance warning in the docs.  Or in other words, I don't want to go from
"might disappear" in vN to gone in vN+1 with no intermediate state.

			regards, tom lane

  4. Re: Something for the TODO list: deprecating abstime and friends

    Greg Stark <stark@mit.edu> — 2017-07-16T20:27:32Z

    On 15 July 2017 at 23:00, Tom Lane <tgl@sss.pgh.pa.us> wrote:

> While it's too late in the v10 cycle to do anything very meaningful
> about this now, I am tempted to strengthen the deprecation notice's
> wording from "might disappear" to "will disappear".

"Will certainly disappear at some unspecified date" is a lot less
convincing than "will disappear in 2021 in Postgres 14". The specific
year and version number is irrelevant
but picking and naming a specific one makes it a lot easier to follow
through come that date.


-- 
greg

  5. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-17T15:27:16Z

    On Sun, Jul 16, 2017 at 1:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Or in other words, I don't want to go from
> "might disappear" in vN to gone in vN+1 with no intermediate state.

I see no problem with that.  First, we remove things all the time with
no deprecation warning at all when we judge that they are dead enough,
or just unsalvageable.  Second, if we have said that something might
disappear and then it disappears, anyone who is unhappy about that is
being unreasonable.

In other words, I don't want to have a project policy that we will not
only put a deprecation notice on everything we remove, but it will
always be worded in a certain way.  If you're trying to streamline the
process of deprecating features, that's going in the exact wrong
direction.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  6. Re: Something for the TODO list: deprecating abstime and friends

    Michael Paquier <michael.paquier@gmail.com> — 2017-07-17T15:37:54Z

    On Sun, Jul 16, 2017 at 10:27 PM, Greg Stark <stark@mit.edu> wrote:
> On 15 July 2017 at 23:00, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
>> While it's too late in the v10 cycle to do anything very meaningful
>> about this now, I am tempted to strengthen the deprecation notice's
>> wording from "might disappear" to "will disappear".
>
> "Will certainly disappear at some unspecified date" is a lot less
> convincing than "will disappear in 2021 in Postgres 14". The specific
> year and version number is irrelevant
> but picking and naming a specific one makes it a lot easier to follow
> through come that date.

Exactly, having an exact deprecation policy would be nice. Of course
this depends on the feature we are talking about. pg_dump for example
supports servers older than what community still supports. But in most
cases, like in core data types, it would be rather safe to say that it
gets deprecated once all the versions supported by community share the
same state (remember for example 17d436d2 that was kept around for a
rather long time, or the handling of opaque function types for
triggers and PLs still present for 15 years).

Coming back to this thread... Removing abstime and reltime sounds like
the good answer to conclude this thread once the deprecation period
expires.
-- 
Michael

  7. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-17T16:21:13Z

    On Mon, Jul 17, 2017 at 11:37 AM, Michael Paquier
<michael.paquier@gmail.com> wrote:
> Exactly, having an exact deprecation policy would be nice. Of course
> this depends on the feature we are talking about. pg_dump for example
> supports servers older than what community still supports. But in most
> cases, like in core data types, it would be rather safe to say that it
> gets deprecated once all the versions supported by community share the
> same state (remember for example 17d436d2 that was kept around for a
> rather long time, or the handling of opaque function types for
> triggers and PLs still present for 15 years).

Yeah, but historically it never works out.  A relatively famous
example is contrib/xml2, whose documentation says:

==
>From PostgreSQL 8.3 on, there is XML-related functionality based on
the SQL/XML standard in the core server. That functionality covers XML
syntax checking and XPath queries, which is what this module does, and
more, but the API is not at all compatible. It is planned that this
module will be removed in a future version of PostgreSQL in favor of
the newer standard API, so you are encouraged to try converting your
applications. If you find that some of the functionality of this
module is not available in an adequate form with the newer API, please
explain your issue to <pgsql-hackers@postgresql.org> so that the
deficiency can be addressed.
==

But until 3163baa6d2d12c28f45fec60ab313537ea9a43a4 (February 24,
2013), it said that it would be removed in PostgreSQL 8.4 (July 1,
2009), a promise that could not be fulfilled without the use of a
serviceable time machine.  I proposed removing contrib/xml2 sometime
during the 8.4 or 9.0 cycle, IIRC, and got told "no".  While the
updated deprecation notice is less-obviously wrong, saying that
removal is "planned" is a pretty generous assessment.  It's unclear
that we've made any progress in addressing whatever the problems were
that caused the previous attempt at removal to get shot down, so it
might still be wrong: maybe xml2 is going to stick around until the
heat death of the universe.

Now, I agree that a date type which cannot represent dates after 2038
is of marginal and decreasing utility, and therefore the chances that
it will be removed are good.  Probably most other people will agree as
well.  But we could easily overlook the need to deliver on a promise
to remove it in a specific version, and the possibility that someone
will argue for keeping it as a matter of historical interest cannot be
ruled out.  In a community where anything at all can be relitigated at
the drop of a hat, making promises about what will happen in the
future is mostly wishful thinking.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  8. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-17T19:54:31Z

    > On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> The types abstime, reltime, and tinterval need to go away, or be
> reimplemented, sometime well before 2038 when they will overflow.
> It's not too soon to start having a plan for that, especially seeing
> that it seems to take a decade or more for us to actually get rid
> of anything we've deprecated.
> 
> Right offhand, I don't think there is any functionality in these
> types that isn't handled as well or better by timestamptz, interval,
> and tstzrange respectively.  

These types provide a 4-byte datatype for storing real-world second
precision timestamps, as occur in many log files.  Forcing people to
switch to timestamp or timestamptz will incur a 4 byte per row
penalty.  In my own builds, I have changed the epoch on these so
they won't wrap until sometime after 2100 C.E.  I see little point in
switching to an 8-byte millisecond precision datatype when a perfectly
good 4-byte second precision datatype already serves the purpose.

That said, I am fully aware that these are deprecated and expect you
will remove them, at which time I'll have to keep them in my tree
and politely refuse to merge in your change which removes them.

mark

  9. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-17T20:17:54Z

    > On Jul 17, 2017, at 12:54 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
> 
> 
>> On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> 
>> The types abstime, reltime, and tinterval need to go away, or be
>> reimplemented, sometime well before 2038 when they will overflow.
>> It's not too soon to start having a plan for that, especially seeing
>> that it seems to take a decade or more for us to actually get rid
>> of anything we've deprecated.
>> 
>> Right offhand, I don't think there is any functionality in these
>> types that isn't handled as well or better by timestamptz, interval,
>> and tstzrange respectively.  
> 
> These types provide a 4-byte datatype for storing real-world second
> precision timestamps, as occur in many log files.  Forcing people to
> switch to timestamp or timestamptz will incur a 4 byte per row
> penalty.  In my own builds, I have changed the epoch on these so
> they won't wrap until sometime after 2100 C.E.  I see little point in
> switching to an 8-byte millisecond precision datatype when a perfectly
> good 4-byte second precision datatype already serves the purpose.
> 
> That said, I am fully aware that these are deprecated and expect you
> will remove them, at which time I'll have to keep them in my tree
> and politely refuse to merge in your change which removes them.

Oh, and if you could be so kind, please remove them in a commit that
does nothing else.  It's much easier to skip the commit that way.

Thanks!

mark

  10. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-17T21:14:01Z

    Mark Dilger <hornschnorter@gmail.com> writes:
>> On Jul 17, 2017, at 12:54 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
> On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> The types abstime, reltime, and tinterval need to go away, or be
>>> reimplemented, sometime well before 2038 when they will overflow.

>> These types provide a 4-byte datatype for storing real-world second
>> precision timestamps, as occur in many log files.  Forcing people to
>> switch to timestamp or timestamptz will incur a 4 byte per row
>> penalty.  In my own builds, I have changed the epoch on these so
>> they won't wrap until sometime after 2100 C.E.  I see little point in
>> switching to an 8-byte millisecond precision datatype when a perfectly
>> good 4-byte second precision datatype already serves the purpose.

Well, if you or somebody is willing to do the legwork, I'd be on board
with a plan that says that every 68 years we redefine the origin of
abstime.  I imagine it could be done so that currently-stored abstime
values retain their present meaning as long as they're not too old.
For example the initial change would toss abstimes before 1970 overboard,
repurposing that range of values as being 2038-2106, but values between
1970 and 2038 still mean the same as they do today.  If anybody still
cares in circa 2085, we toss 1970-2038 overboard and move the origin
again, lather rinse repeat.

But we're already past the point where it would be time to make the
first such switch, if we're gonna do it.  So I'd like to see somebody
step up to the plate sooner not later.

I'd be inclined to toss reltime and tinterval overboard in any case.

> Oh, and if you could be so kind, please remove them in a commit that
> does nothing else.  It's much easier to skip the commit that way.

We doubtless would do that, but you're fooling yourself if you imagine
that you can maintain such a fork for long while still tracking the
community version otherwise.  Once those hand-assigned OIDs are free
they'll soon be absorbed by future feature patches, and then you'll
have enormous merge problems.

			regards, tom lane

  11. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-17T21:34:58Z

    > On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Mark Dilger <hornschnorter@gmail.com> writes:
>>> On Jul 17, 2017, at 12:54 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>> On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>> The types abstime, reltime, and tinterval need to go away, or be
>>>> reimplemented, sometime well before 2038 when they will overflow.
> 
>>> These types provide a 4-byte datatype for storing real-world second
>>> precision timestamps, as occur in many log files.  Forcing people to
>>> switch to timestamp or timestamptz will incur a 4 byte per row
>>> penalty.  In my own builds, I have changed the epoch on these so
>>> they won't wrap until sometime after 2100 C.E.  I see little point in
>>> switching to an 8-byte millisecond precision datatype when a perfectly
>>> good 4-byte second precision datatype already serves the purpose.
> 
> Well, if you or somebody is willing to do the legwork, I'd be on board
> with a plan that says that every 68 years we redefine the origin of
> abstime.  I imagine it could be done so that currently-stored abstime
> values retain their present meaning as long as they're not too old.
> For example the initial change would toss abstimes before 1970 overboard,
> repurposing that range of values as being 2038-2106, but values between
> 1970 and 2038 still mean the same as they do today.  If anybody still
> cares in circa 2085, we toss 1970-2038 overboard and move the origin
> again, lather rinse repeat.
> 
> But we're already past the point where it would be time to make the
> first such switch, if we're gonna do it.  So I'd like to see somebody
> step up to the plate sooner not later.

Assuming other members of the community would not object to such
a plan, I'd be willing to step up to that plate.  I'll wait a respectable time,
maybe until tomorrow, to allow others to speak up.

> I'd be inclined to toss reltime and tinterval overboard in any case.

Yeah, I don't use those either, preferring to cast abstime to timestamp
(or timestamptz) prior to doing any math on them.

> 
>> Oh, and if you could be so kind, please remove them in a commit that
>> does nothing else.  It's much easier to skip the commit that way.
> 
> We doubtless would do that, but you're fooling yourself if you imagine
> that you can maintain such a fork for long while still tracking the
> community version otherwise.  Once those hand-assigned OIDs are free
> they'll soon be absorbed by future feature patches, and then you'll
> have enormous merge problems.

Oh, not so much.  Knowing that they were going to be deprecated, I
already moved the Oids for these to something higher than 10000.  In
my own builds, the Oid generator does not assign Oids lower than 65536,
which leaves room for me to assign in the 10000-65535 range without
merge headaches.  It would, however, be simpler if the types did not
go away, as that would cause minor merge headaches elsewhere, such
as in the regression tests.

mark

  12. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-17T22:12:07Z

    Mark Dilger <hornschnorter@gmail.com> writes:
>> On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Well, if you or somebody is willing to do the legwork, I'd be on board
>> with a plan that says that every 68 years we redefine the origin of
>> abstime.  I imagine it could be done so that currently-stored abstime
>> values retain their present meaning as long as they're not too old.
>> For example the initial change would toss abstimes before 1970 overboard,
>> repurposing that range of values as being 2038-2106, but values between
>> 1970 and 2038 still mean the same as they do today.  If anybody still
>> cares in circa 2085, we toss 1970-2038 overboard and move the origin
>> again, lather rinse repeat.

> Assuming other members of the community would not object to such
> a plan, I'd be willing to step up to that plate.

So, thinking about how that would actually work ... the thing to do in
order to preserve existing on-disk values is to alternate between signed
and unsigned interpretations of abstimes.  That is, right now, abstime
is signed with origin 1970.  The conversion I'm arguing we should make
real soon now is to unsigned with origin 1970.  If the project lives
so long, in circa 70 years we'd switch it to signed with origin 2106.
Yadda yadda.

Now, this should mostly work conveniently, except that we have
+/-infinity (NOEND_ABSTIME/NOSTART_ABSTIME) to deal with.  Those
are nicely positioned at the ends of the signed range so that
abstime_cmp_internal() doesn't need to treat them specially.
In an unsigned world they'd be smack in the middle of the range.
We could certainly teach abstime_cmp_internal to special-case them
and deliver logically correct results, but there's a bigger problem:
those will correspond to two seconds in January 2038 that will need
to be representable when the time comes.  Maybe we can make that
work by defining the values past 2038 as being two seconds less than
you might otherwise expect, but I bet it's gonna be messy.  It might
be saner to just desupport +/-infinity for abstime.

The same goes for INVALID_ABSTIME, which doesn't have any clear
use-case that I know of, and is certainly not documented.

			regards, tom lane

  13. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-17T22:39:44Z

    > On Jul 17, 2017, at 3:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Mark Dilger <hornschnorter@gmail.com> writes:
>>> On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Well, if you or somebody is willing to do the legwork, I'd be on board
>>> with a plan that says that every 68 years we redefine the origin of
>>> abstime.  I imagine it could be done so that currently-stored abstime
>>> values retain their present meaning as long as they're not too old.
>>> For example the initial change would toss abstimes before 1970 overboard,
>>> repurposing that range of values as being 2038-2106, but values between
>>> 1970 and 2038 still mean the same as they do today.  If anybody still
>>> cares in circa 2085, we toss 1970-2038 overboard and move the origin
>>> again, lather rinse repeat.
> 
>> Assuming other members of the community would not object to such
>> a plan, I'd be willing to step up to that plate.
> 
> So, thinking about how that would actually work ... the thing to do in
> order to preserve existing on-disk values is to alternate between signed
> and unsigned interpretations of abstimes.  That is, right now, abstime
> is signed with origin 1970.  The conversion I'm arguing we should make
> real soon now is to unsigned with origin 1970.  If the project lives
> so long, in circa 70 years we'd switch it to signed with origin 2106.
> Yadda yadda.

Right, I already had the same idea.  That's not how I am doing it in my
fork currently, but that's what you would want to do to support any values
already stored somewhere.

> Now, this should mostly work conveniently, except that we have
> +/-infinity (NOEND_ABSTIME/NOSTART_ABSTIME) to deal with.  Those
> are nicely positioned at the ends of the signed range so that
> abstime_cmp_internal() doesn't need to treat them specially.
> In an unsigned world they'd be smack in the middle of the range.
> We could certainly teach abstime_cmp_internal to special-case them
> and deliver logically correct results, but there's a bigger problem:
> those will correspond to two seconds in January 2038 that will need
> to be representable when the time comes.  Maybe we can make that
> work by defining the values past 2038 as being two seconds less than
> you might otherwise expect, but I bet it's gonna be messy.  It might
> be saner to just desupport +/-infinity for abstime.

I don't use those values, so it is no matter to me if we desupport them.  It
seems a bit pointless, though, because we still have to handle legacy
values that we encounter.  I assume some folks will have those values in
their tables when they upgrade.

> The same goes for INVALID_ABSTIME, which doesn't have any clear
> use-case that I know of, and is certainly not documented.

Likewise, I don't know how to desupport this while still supporting legacy
tables.

mark

  14. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-17T22:56:20Z

    Mark Dilger <hornschnorter@gmail.com> writes:
> On Jul 17, 2017, at 3:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Now, this should mostly work conveniently, except that we have
>> +/-infinity (NOEND_ABSTIME/NOSTART_ABSTIME) to deal with ... It might
>> be saner to just desupport +/-infinity for abstime.

> I don't use those values, so it is no matter to me if we desupport them.  It
> seems a bit pointless, though, because we still have to handle legacy
> values that we encounter.  I assume some folks will have those values in
> their tables when they upgrade.

Well, some folks will also have pre-1970 dates in their tables, no?
We're just blowing those off.  They'll print out as some post-2038
date or other, and too bad.

Basically, the direction this is going in is that abstime will become
an officially supported type, but its range of supported values is "not
too many decades either way from now".  If you are using it to store
very old dates then You're Doing It Wrong, and eventually you'll get
bitten.  Given that contract, I don't see a place for +/-infinity.

			regards, tom lane

  15. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-17T22:59:55Z

    > On Jul 17, 2017, at 3:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Mark Dilger <hornschnorter@gmail.com> writes:
>> On Jul 17, 2017, at 3:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Now, this should mostly work conveniently, except that we have
>>> +/-infinity (NOEND_ABSTIME/NOSTART_ABSTIME) to deal with ... It might
>>> be saner to just desupport +/-infinity for abstime.
> 
>> I don't use those values, so it is no matter to me if we desupport them.  It
>> seems a bit pointless, though, because we still have to handle legacy
>> values that we encounter.  I assume some folks will have those values in
>> their tables when they upgrade.
> 
> Well, some folks will also have pre-1970 dates in their tables, no?
> We're just blowing those off.  They'll print out as some post-2038
> date or other, and too bad.
> 
> Basically, the direction this is going in is that abstime will become
> an officially supported type, but its range of supported values is "not
> too many decades either way from now".  If you are using it to store
> very old dates then You're Doing It Wrong, and eventually you'll get
> bitten.  Given that contract, I don't see a place for +/-infinity.

Works for me.

mark

  16. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-18T00:15:47Z

    > On Jul 17, 2017, at 3:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Mark Dilger <hornschnorter@gmail.com> writes:
>>> On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Well, if you or somebody is willing to do the legwork, I'd be on board
>>> with a plan that says that every 68 years we redefine the origin of
>>> abstime.  I imagine it could be done so that currently-stored abstime
>>> values retain their present meaning as long as they're not too old.
>>> For example the initial change would toss abstimes before 1970 overboard,
>>> repurposing that range of values as being 2038-2106, but values between
>>> 1970 and 2038 still mean the same as they do today.  If anybody still
>>> cares in circa 2085, we toss 1970-2038 overboard and move the origin
>>> again, lather rinse repeat.
> 
>> Assuming other members of the community would not object to such
>> a plan, I'd be willing to step up to that plate.
> 
> So, thinking about how that would actually work ... the thing to do in
> order to preserve existing on-disk values is to alternate between signed
> and unsigned interpretations of abstimes.  That is, right now, abstime
> is signed with origin 1970.  The conversion I'm arguing we should make
> real soon now is to unsigned with origin 1970.  If the project lives
> so long, in circa 70 years we'd switch it to signed with origin 2106.
> Yadda yadda.
> 
> Now, this should mostly work conveniently, except that we have
> +/-infinity (NOEND_ABSTIME/NOSTART_ABSTIME) to deal with.  Those
> are nicely positioned at the ends of the signed range so that
> abstime_cmp_internal() doesn't need to treat them specially.
> In an unsigned world they'd be smack in the middle of the range.
> We could certainly teach abstime_cmp_internal to special-case them
> and deliver logically correct results, but there's a bigger problem:
> those will correspond to two seconds in January 2038 that will need
> to be representable when the time comes.  Maybe we can make that
> work by defining the values past 2038 as being two seconds less than
> you might otherwise expect, but I bet it's gonna be messy.  It might
> be saner to just desupport +/-infinity for abstime.
> 
> The same goes for INVALID_ABSTIME, which doesn't have any clear
> use-case that I know of, and is certainly not documented.

I use the abstime type in some catalog tables, and if I allowed those
fields to have something equivalent to NULL, which I do not, I would need
INVALID_ABSTIME, since NULL is not allowed in the constant header of a
catalog table.

I wonder if old versions of postgres had catalog tables with abstime used
in this way?  Other than that, I can't think of a reason to use INVALID_ABSTIME.

mark

  17. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-19T04:13:10Z

    > On Jul 17, 2017, at 2:34 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
> 
>> 
>> On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> 
>> Mark Dilger <hornschnorter@gmail.com> writes:
>>>> On Jul 17, 2017, at 12:54 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>>> On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>>> The types abstime, reltime, and tinterval need to go away, or be
>>>>> reimplemented, sometime well before 2038 when they will overflow.
>> 
>>>> These types provide a 4-byte datatype for storing real-world second
>>>> precision timestamps, as occur in many log files.  Forcing people to
>>>> switch to timestamp or timestamptz will incur a 4 byte per row
>>>> penalty.  In my own builds, I have changed the epoch on these so
>>>> they won't wrap until sometime after 2100 C.E.  I see little point in
>>>> switching to an 8-byte millisecond precision datatype when a perfectly
>>>> good 4-byte second precision datatype already serves the purpose.
>> 
>> Well, if you or somebody is willing to do the legwork, I'd be on board
>> with a plan that says that every 68 years we redefine the origin of
>> abstime.  I imagine it could be done so that currently-stored abstime
>> values retain their present meaning as long as they're not too old.
>> For example the initial change would toss abstimes before 1970 overboard,
>> repurposing that range of values as being 2038-2106, but values between
>> 1970 and 2038 still mean the same as they do today.  If anybody still
>> cares in circa 2085, we toss 1970-2038 overboard and move the origin
>> again, lather rinse repeat.
>> 
>> But we're already past the point where it would be time to make the
>> first such switch, if we're gonna do it.  So I'd like to see somebody
>> step up to the plate sooner not later.
> 
> Assuming other members of the community would not object to such
> a plan, I'd be willing to step up to that plate.  I'll wait a respectable time,
> maybe until tomorrow, to allow others to speak up.

There was not much conversation about this, so I went ahead with what
I think makes a logical first step.  The attached patch removes the tinterval
datatype from the sources.

I intend to remove reltime next, and then make the changes to abstime in
a third patch.

mark

  18. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-19T05:44:02Z

    > On Jul 18, 2017, at 9:13 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
> 
>> 
>> On Jul 17, 2017, at 2:34 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>> 
>>> 
>>> On Jul 17, 2017, at 2:14 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> 
>>> Mark Dilger <hornschnorter@gmail.com> writes:
>>>>> On Jul 17, 2017, at 12:54 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>>>> On Jul 15, 2017, at 3:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>>>> The types abstime, reltime, and tinterval need to go away, or be
>>>>>> reimplemented, sometime well before 2038 when they will overflow.
>>> 
>>>>> These types provide a 4-byte datatype for storing real-world second
>>>>> precision timestamps, as occur in many log files.  Forcing people to
>>>>> switch to timestamp or timestamptz will incur a 4 byte per row
>>>>> penalty.  In my own builds, I have changed the epoch on these so
>>>>> they won't wrap until sometime after 2100 C.E.  I see little point in
>>>>> switching to an 8-byte millisecond precision datatype when a perfectly
>>>>> good 4-byte second precision datatype already serves the purpose.
>>> 
>>> Well, if you or somebody is willing to do the legwork, I'd be on board
>>> with a plan that says that every 68 years we redefine the origin of
>>> abstime.  I imagine it could be done so that currently-stored abstime
>>> values retain their present meaning as long as they're not too old.
>>> For example the initial change would toss abstimes before 1970 overboard,
>>> repurposing that range of values as being 2038-2106, but values between
>>> 1970 and 2038 still mean the same as they do today.  If anybody still
>>> cares in circa 2085, we toss 1970-2038 overboard and move the origin
>>> again, lather rinse repeat.
>>> 
>>> But we're already past the point where it would be time to make the
>>> first such switch, if we're gonna do it.  So I'd like to see somebody
>>> step up to the plate sooner not later.
>> 
>> Assuming other members of the community would not object to such
>> a plan, I'd be willing to step up to that plate.  I'll wait a respectable time,
>> maybe until tomorrow, to allow others to speak up.
> 
> There was not much conversation about this, so I went ahead with what
> I think makes a logical first step.  The attached patch removes the tinterval
> datatype from the sources.
> 
> I intend to remove reltime next, and then make the changes to abstime in
> a third patch.

As predicted, this second patch (which should be applied *after* the prior
tinterval_abatement patch) removes the reltime datatype from the sources.

mark

  19. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T13:35:49Z

    Mark Dilger <hornschnorter@gmail.com> writes:
>> On Jul 18, 2017, at 9:13 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>> There was not much conversation about this, so I went ahead with what
>> I think makes a logical first step.  The attached patch removes the tinterval
>> datatype from the sources.

> As predicted, this second patch (which should be applied *after* the prior
> tinterval_abatement patch) removes the reltime datatype from the sources.

Seems like a good plan.  Please be sure to add these patches to the next
commitfest, so we remember them when the time comes.

			regards, tom lane

  20. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-19T16:53:34Z

    On Mon, Jul 17, 2017 at 6:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> So, thinking about how that would actually work ... the thing to do in
> order to preserve existing on-disk values is to alternate between signed
> and unsigned interpretations of abstimes.  That is, right now, abstime
> is signed with origin 1970.  The conversion I'm arguing we should make
> real soon now is to unsigned with origin 1970.  If the project lives
> so long, in circa 70 years we'd switch it to signed with origin 2106.
> Yadda yadda.

Doesn't this plan amount to breaking pg_upgrade compatibility and
hoping that nobody notice?  Existing values will be silently
reinterpreted according to the new rules.  If we had two actually
separate types and let people convert columns from the old type to the
new type with just a validation scan, that would be engineering at the
level of quality that I've come to associate with this project.  If
this type is so marginal that we don't want to do that kind of work,
then I think we should just rip it out; that doesn't preclude someone
maintaining it in their own fork, or even adding it back as a new type
in a contrib module with a #define for the base year.  Silently
changing the interpretation of the same data in the core code, though,
seems both far too clever and not clever enough.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  21. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T17:12:02Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Mon, Jul 17, 2017 at 6:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> So, thinking about how that would actually work ... the thing to do in
>> order to preserve existing on-disk values is to alternate between signed
>> and unsigned interpretations of abstimes.  That is, right now, abstime
>> is signed with origin 1970.  The conversion I'm arguing we should make
>> real soon now is to unsigned with origin 1970.  If the project lives
>> so long, in circa 70 years we'd switch it to signed with origin 2106.
>> Yadda yadda.

> Doesn't this plan amount to breaking pg_upgrade compatibility and
> hoping that nobody notice?

Well, what we'd need to do is document that the type is only meant to be
used to store dates within say +/- 30 years from current time.  As long
as people adhere to that use-case, the proposal would work conveniently
long into the future ...

> If we had two actually
> separate types and let people convert columns from the old type to the
> new type with just a validation scan, that would be engineering at the
> level of quality that I've come to associate with this project.

... whereas that would be seriously INconvenient.  It's not just the
ALTER TABLE pushups, which presumably would be something you could do
and forget.  It's that the new type name would be something you'd have
to change your applications to know about, and then you (or more likely
your successor) would have to do it over again decades later.

I'd definitely be on board with just dropping the type altogether despite
Mark's concern.  But I am not sure that the way you are proposing would
please anybody except pedants.

			regards, tom lane

  22. Re: Something for the TODO list: deprecating abstime and friends

    David Fetter <david@fetter.org> — 2017-07-19T17:15:33Z

    On Wed, Jul 19, 2017 at 01:12:02PM -0400, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > On Mon, Jul 17, 2017 at 6:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> So, thinking about how that would actually work ... the thing to do in
> >> order to preserve existing on-disk values is to alternate between signed
> >> and unsigned interpretations of abstimes.  That is, right now, abstime
> >> is signed with origin 1970.  The conversion I'm arguing we should make
> >> real soon now is to unsigned with origin 1970.  If the project lives
> >> so long, in circa 70 years we'd switch it to signed with origin 2106.
> >> Yadda yadda.
> 
> > Doesn't this plan amount to breaking pg_upgrade compatibility and
> > hoping that nobody notice?
> 
> Well, what we'd need to do is document that the type is only meant to be
> used to store dates within say +/- 30 years from current time.  As long
> as people adhere to that use-case, the proposal would work conveniently
> long into the future ...
> 
> > If we had two actually
> > separate types and let people convert columns from the old type to the
> > new type with just a validation scan, that would be engineering at the
> > level of quality that I've come to associate with this project.
> 
> ... whereas that would be seriously INconvenient.  It's not just the
> ALTER TABLE pushups, which presumably would be something you could do
> and forget.  It's that the new type name would be something you'd have
> to change your applications to know about, and then you (or more likely
> your successor) would have to do it over again decades later.
> 
> I'd definitely be on board with just dropping the type altogether despite
> Mark's concern.  But I am not sure that the way you are proposing would
> please anybody except pedants.

+1 for just dropping the types, preferably modifying the contrib
extensions that depend on it, less preferably, dropping those, too.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter      XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

  23. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-19T17:23:18Z

    On Wed, Jul 19, 2017 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Doesn't this plan amount to breaking pg_upgrade compatibility and
>> hoping that nobody notice?
>
> Well, what we'd need to do is document that the type is only meant to be
> used to store dates within say +/- 30 years from current time.  As long
> as people adhere to that use-case, the proposal would work conveniently
> long into the future ...

Typically, when you try to store an out-of-range value in PostgreSQL,
you get an ERROR, and that's one of the selling points of PostgreSQL.
PostgreSQL users regularly beat up other projects for, say, allowing
0000-00-00 to be considered a valid date, or any similar perceived
laxity in enforcing data consistency.  I don't like the idea that we
can just deviate from that principle whenever adhering to it is too
much work.

> I'd definitely be on board with just dropping the type altogether despite
> Mark's concern.

Then I vote for that option.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  24. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T17:35:59Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Jul 19, 2017 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I'd definitely be on board with just dropping the type altogether despite
>> Mark's concern.

> Then I vote for that option.

BTW, another possible compromise is to move abstime into a contrib
module; we've always accepted that contrib modules can be held to a
lower standard than core features.  I'm not volunteering to do the
work for that, but it's worth contemplating.

Alternatively, we could turn the origin point for abstime into a
pg_control field, and regard changing it as a reason for a database
not being pg_upgrade'able unless it lacks any abstime columns.

			regards, tom lane

  25. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-19T18:18:22Z

    On Wed, Jul 19, 2017 at 1:35 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> On Wed, Jul 19, 2017 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> I'd definitely be on board with just dropping the type altogether despite
>>> Mark's concern.
>
>> Then I vote for that option.
>
> BTW, another possible compromise is to move abstime into a contrib
> module; we've always accepted that contrib modules can be held to a
> lower standard than core features.  I'm not volunteering to do the
> work for that, but it's worth contemplating.

I would be OK with that, provided the documentation calls out the hazard.

> Alternatively, we could turn the origin point for abstime into
> pg_control field, and regard changing it as a reason for a database
> not being pg_upgrade'able unless it lacks any abstime columns.

I would be OK with that, too, but is there any danger that we're going
to grow pg_control to a size where reads and writes can no longer be
assumed atomic, if we keep adding things?

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  26. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T18:36:21Z

    Robert Haas <robertmhaas@gmail.com> writes:
> On Wed, Jul 19, 2017 at 1:35 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Alternatively, we could turn the origin point for abstime into
>> pg_control field, and regard changing it as a reason for a database
>> not being pg_upgrade'able unless it lacks any abstime columns.

> I would be OK with that, too, but is there any danger that we're going
> to grow pg_control to a size where reads and writes can no longer be
> assumed atomic, if we keep adding things?

Hm.  Currently sizeof(struct ControlFileData) = 296, at least on my
machine.  Letting it grow past 512 would be problematic.  It's hard
to see getting to that any time soon, though; we don't add fields
there often.

Note that I'm not seriously pushing for this solution.  I'm just trying
to make sure that we've considered all the reasonable options.

			regards, tom lane

  27. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T18:56:31Z

    I wrote:
> Hm.  Currently sizeof(struct ControlFileData) = 296, at least on my
> machine.  Letting it grow past 512 would be problematic.  It's hard
> to see getting to that any time soon, though; we don't add fields
> there often.

I wonder if it'd be worth the trouble to stick something like this into
xlog.c:

	/*
	 * For reliability's sake, it's critical that pg_control updates
	 * be atomic writes.  That generally means the active data can't
	 * be more than one disk sector, which is 512 bytes on common
	 * hardware.  Be very careful about raising this limit.
	 */
	 StaticAssertStmt(sizeof(ControlFileData) <= 512,
	                  "pg_control is too large for atomic disk writes");


			regards, tom lane

  28. Re: Something for the TODO list: deprecating abstime and friends

    Michael Paquier <michael.paquier@gmail.com> — 2017-07-19T19:04:09Z

    On Wed, Jul 19, 2017 at 8:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I wonder if it'd be worth the trouble to stick something like this into
> xlog.c:
>
>         /*
>          * For reliability's sake, it's critical that pg_control updates
>          * be atomic writes.  That generally means the active data can't
>          * be more than one disk sector, which is 512 bytes on common
>          * hardware.  Be very careful about raising this limit.
>          */
>          StaticAssertStmt(sizeof(ControlFileData) <= 512,
>                           "pg_control is too large for atomic disk writes");

+1. Even if it just gets triggered in 20 years by some hacker, that's
a good reminder about assumptions behind the update logic.
-- 
Michael

  29. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-19T19:26:43Z

    > On Jul 19, 2017, at 10:23 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> 
> On Wed, Jul 19, 2017 at 1:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Doesn't this plan amount to breaking pg_upgrade compatibility and
>>> hoping that nobody notice?
>> 
>> Well, what we'd need to do is document that the type is only meant to be
>> used to store dates within say +/- 30 years from current time.  As long
>> as people adhere to that use-case, the proposal would work conveniently
>> long into the future ...
> 
> Typically, when you try to store an out-of-range value in PostgreSQL,
> you get an ERROR, and that's one of the selling points of PostgreSQL.
> PostgreSQL users regularly beat up other projects for, say, allowing
> 0000-00-00 to be considered a valid date, or any similar perceived
> laxity in enforcing data consistency.  I don't like the idea that we
> can just deviate from that principle whenever adhering to it is too
> much work.

I don't see the relevance of this statement.  I am not planning to allow
abstime data that is outside the range of the new epoch.  Attempts to
cast strings like '1962-07-07 01:02:03' to abstime would draw an
exception with a suitably informative message.

Now, the objection to having on-disk data automagically change meaning
is concerning, and I was particularly unsatisfied with the idea that
NOSTART_ABSTIME and NOEND_ABSTIME would suddenly be
reinterpreted as seconds in the year 2068, which is why I made mention
of it upthread.  I was less concerned with dates prior to 1970 being
reinterpreted, though it is hard to justify why that bothers me less.

>> I'd definitely be on board with just dropping the type altogether despite
>> Mark's concern.
> 
> Then I vote for that option.

I was somewhat surprised when Tom was onboard with the idea of keeping
abstime around (for my benefit).  My original post was in response to his
statement that, right offhand, he couldn't think of any use for abstime that
wasn't handled better by timestamptz (paraphrase), and so I said that
improving storage efficiency was such a use.  I maintain that position.  The
abstime type is a good and useful type, and we will lose that use when we
discard it.  Those who feel otherwise might like to also argue for dropping
float4 because float8 does all the same stuff better.

mark

  30. Re: Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-07-19T19:37:09Z

    > On Jul 19, 2017, at 9:53 AM, Robert Haas <robertmhaas@gmail.com> wrote:
> 
> On Mon, Jul 17, 2017 at 6:12 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> So, thinking about how that would actually work ... the thing to do in
>> order to preserve existing on-disk values is to alternate between signed
>> and unsigned interpretations of abstimes.  That is, right now, abstime
>> is signed with origin 1970.  The conversion I'm arguing we should make
>> real soon now is to unsigned with origin 1970.  If the project lives
>> so long, in circa 70 years we'd switch it to signed with origin 2106.
>> Yadda yadda.
> 
> Doesn't this plan amount to breaking pg_upgrade compatibility and
> hoping that nobody notice?  Existing values will be silently
> reinterpreted according to the new rules.  If we had two actually
> separate types and let people convert columns from the old type to the
> new type with just a validation scan, that would be engineering at the
> level of quality that I've come to associate with this project.  

This is what I have done in my fork.  I repurposed the type as "secondstamp"
since I think of it as a timestamp down to second precision (truncating
fractional seconds.)  I changed the Oids assigned to the catalog entries
associated with the type, and considered myself somewhat immune to
the project dropping the abstime type, which the documentation warned
would happen eventually.


> If
> this type is so marginal that we don't want to do that kind of work,
> then I think we should just rip it out; that doesn't preclude someone
> maintaining it in their own fork, or even adding it back as a new type
> in a contrib module with a #define for the base year.  Silently
> changing the interpretation of the same data in the core code, though,
> seems both far too clever and not clever enough.

I would be happy to contribute the "secondstamp" type as part of a patch
that removes the abstime type.  I can add a catalog table that holds the
epoch, and add documentation for the type stating that every time the
epoch changes, their data will be automatically reinterpreted, and as such
they should only use the datatype if that is ok. 

Under this plan, users with abstime columns who upgrade to postgres 11
will not have an easy time, because the type will be removed.  But that is
the same and no worse than what they would get if we just remove the
abstime type in postgres 11 without any replacement. 

I'm not implementing any of this yet, as I expect further objections.

Thoughts?

mark

  31. Re: Something for the TODO list: deprecating abstime and friends

    Robert Haas <robertmhaas@gmail.com> — 2017-07-19T19:45:06Z

    On Wed, Jul 19, 2017 at 3:26 PM, Mark Dilger <hornschnorter@gmail.com> wrote:
>> Typically, when you try to store an out-of-range value in PostgreSQL,
>> you get an ERROR, and that's one of the selling points of PostgreSQL.
>> PostgreSQL users regularly beat up other projects for, say, allowing
>> 0000-00-00 to be considered a valid date, or any similar perceived
>> laxity in enforcing data consistency.  I don't like the idea that we
>> can just deviate from that principle whenever adhering to it is too
>> much work.
>
> I don't see the relevance of this statement.  I am not planning to allow
> abstime data that is outside the range of the new epoch.  Attempts to
> cast strings like '1962-07-07 01:02:03' to abstime would draw an
> exception with a suitably informative message.
>
> Now, the objection to having on-disk data automagically change meaning
> is concerning, ...

I see those things as very similar issues, but YMMV.

> ... and I was particularly unsatisfied with the idea that
> NOSTART_ABSTIME and NOEND_ABSTIME would suddenly be
> reinterpreted as seconds in the year 2068, which is why I made mention
> of it upthread.  I was less concerned with dates prior to 1970 being
> reinterpreted, though it is hard to justify why that bothers me less.

I think any change in the meaning of bytes already on disk is a
concern.  It wouldn't bother me to say "these are second-class types,
so if you have columns of this type, pg_upgrade will fail and tell you
that you can't upgrade a database with those columns from a catversion
< X to a catversion > X".  But having it silently work and silently
corrupt your data doesn't seem OK to me.

> Those who feel otherwise might like to also argue for dropping
> float4 because float8 does all the same stuff better.

I don't think that's the same thing, but given my notorious hatred of
floating point arithmetic, it would also bother me less than you might
think.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  32. Re: Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-07-19T20:19:10Z

    Michael Paquier <michael.paquier@gmail.com> writes:
> On Wed, Jul 19, 2017 at 8:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I wonder if it'd be worth the trouble to stick something like this into
>> xlog.c:
>> StaticAssertStmt(sizeof(ControlFileData) <= 512,
>> "pg_control is too large for atomic disk writes");

> +1. Even if it just gets triggered in 20 years by some hacker, that's
> a good reminder about assumptions behind the update logic.

Done.  I found the size checks had metastasized into pg_resetwal as well,
and probably should have been in pg_rewind since it also rewrites
pg_control, so it ended up a slightly larger patch than I first thought.
Seems cleaner than before though.

			regards, tom lane

  33. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    Andres Freund <andres@anarazel.de> — 2017-12-13T08:05:06Z

    Hi,

On 2017-07-15 18:00:34 -0400, Tom Lane wrote:
> The types abstime, reltime, and tinterval need to go away, or be
> reimplemented, sometime well before 2038 when they will overflow.
> It's not too soon to start having a plan for that, especially seeing
> that it seems to take a decade or more for us to actually get rid
> of anything we've deprecated.

> Right offhand, I don't think there is any functionality in these
> types that isn't handled as well or better by timestamptz, interval,
> and tstzrange respectively.  And they're basically undocumented
> except for a sort-of deprecation notice just above section 8.5.1.
> So my inclination is to remove them rather than try to upgrade them
> in any way.

Hah, I'd completely missed this thread. But very recently arrived at the
the same conclusion, albeit for different reasons. Largely because the
code isn't very rigid about dealing with overflows.  I really don't want
to invest time in polishing up code that, as you say, doesn't really
have much of a usecase.


> However, we'd have to do something about:
> 
> * The legacy system views pg_shadow and pg_user have abstime columns.
> Experimentation suggests that we could convert those to timestamptz(0)
> and the output format wouldn't change, so maybe that's a good enough
> fix there.

Yea, I think that's reasonable. I think it'd also be defensible to go to
a pure timestamptz rather than timestamptz(0). That's going to
negatively effect very few users...


> * contrib/spi/timetravel depends on abstime columns to represent what
> would nowadays be better done as a tstzrange.  I'd have thought we
> could maybe toss that example module overboard, but we just today got
> a question about its usage, so I'm afraid it's not quite dead yet.
> What shall we do with that?

Looking at the code I'd be pretty strongly inclined to scrap it.


Before I'd discovered this thread, I'd started to write up a
patch. Attached. It's clearly not fully done. Questions I'd while
hacking things up:
- what to do with contrib/spi/timetravel - I'd just removed it from the
  relevant Makefile for now.
- nabstime.c currently implements timeofday() which imo is a pretty
  weird function. I'd be quite inclined to remove it at the same time as
  this.

Greetings,

Andres Freund

  34. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    Andres Freund <andres@anarazel.de> — 2017-12-13T08:07:29Z

    Hi,

On 2017-07-17 12:54:31 -0700, Mark Dilger wrote:
> These types provide a 4-byte datatype for storing real-world second
> precision timestamps, as occur in many log files.

These seem to be getting less common IME, most products have higher
resolution these days.  If this were nicely written, maintainable, and
documented code my position would differ, but it really isn't.


> That said, I am fully aware that these are deprecated and expect you
> will remove them, at which time I'll have to keep them in my tree
> and politely refuse to merge in your change which removes them.

It'd be way less work to package abstime as an extension if you want to
do so.

Greetings,

Andres Freund

  35. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    Mark Dilger <hornschnorter@gmail.com> — 2017-12-13T15:15:52Z

    > On Dec 13, 2017, at 12:07 AM, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2017-07-17 12:54:31 -0700, Mark Dilger wrote:
>> These types provide a 4-byte datatype for storing real-world second
>> precision timestamps, as occur in many log files.
> 
> These seem to be getting less common IME, most products have higher
> resolution these days.  If this were nicely written, maintainable, and
> documented code my position would differ, but it really isn't.
> 
> 
>> That said, I am fully aware that these are deprecated and expect you
>> will remove them, at which time I'll have to keep them in my tree
>> and politely refuse to merge in your change which removes them.
> 
> It'd be way less work to package abstime as an extension if you want to
> do so.

After proposing to submit a patch for the secondstamp datatype (which
I mentioned upthread), Robert objected to the idea of data on disk changing
meaning, which was a part of the idea that Tom seemed to be willing to
accept.  Since I couldn't get both Tom and Robert on board with any
particular design, I silently withdrew from the development of any such
patch.

This has happened on several proposals I have made on this list over
the years.  Unless there is fairly unanimous support from the committers,
I don't bother following through with a proposal, given the improbability
of it getting accepted.

I would happily finish and submit that prior proposal if there were general
agreement that it is a good design.  I have no interest in making abstime
into an extension, however.

mark

  36. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    David Fetter <david@fetter.org> — 2017-12-17T22:24:19Z

    On Wed, Dec 13, 2017 at 12:05:06AM -0800, Andres Freund wrote:
> On 2017-07-15 18:00:34 -0400, Tom Lane wrote:
> > * contrib/spi/timetravel depends on abstime columns to represent what
> > would nowadays be better done as a tstzrange.  I'd have thought we
> > could maybe toss that example module overboard, but we just today got
> > a question about its usage, so I'm afraid it's not quite dead yet.
> > What shall we do with that?
> 
> Looking at the code I'd be pretty strongly inclined to scrap it.
> 
> 
> Before I'd discovered this thread, I'd started to write up a
> patch. Attached. It's clearly not fully done. Questions I'd while
> hacking things up:
> - what to do with contrib/spi/timetravel - I'd just removed it from the
>   relevant Makefile for now.

Porting it to tstzrange seems friendlier, however:

- Does that look like significant work to do this port?
- How about the work involved with upgrading existing installations?

> - nabstime.c currently implements timeofday() which imo is a pretty
>   weird function. I'd be quite inclined to remove it at the same time as
>   this.

+1 for binning it.

Best,
David.
-- 
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

  37. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    Andres Freund <andres@anarazel.de> — 2018-09-28T22:32:40Z

    Hi,

On 2017-12-13 00:05:06 -0800, Andres Freund wrote:
> > * contrib/spi/timetravel depends on abstime columns to represent what
> > would nowadays be better done as a tstzrange.  I'd have thought we
> > could maybe toss that example module overboard, but we just today got
> > a question about its usage, so I'm afraid it's not quite dead yet.
> > What shall we do with that?
> 
> Looking at the code I'd be pretty strongly inclined to scrap it.
> 
> 
> Before I'd discovered this thread, I'd started to write up a
> patch. Attached. It's clearly not fully done. Questions I'd while
> hacking things up:
> - what to do with contrib/spi/timetravel - I'd just removed it from the
>   relevant Makefile for now.
> - nabstime.c currently implements timeofday() which imo is a pretty
>   weird function. I'd be quite inclined to remove it at the same time as
>   this.

Here's a refreshed version of this patch.  First patch removes
contrib/spi/timetravel, second patch removes abstime, reltime, tinterval
together with timeofday().

I think we should just go ahead and commit something like this soon.

- Andres

  38. Re: [HACKERS] Something for the TODO list: deprecating abstime and friends

    Tom Lane <tgl@sss.pgh.pa.us> — 2018-09-29T04:40:58Z

    Andres Freund <andres@anarazel.de> writes:
> Here's a refreshed version of this patch.  First patch removes
> contrib/spi/timetravel, second patch removes abstime, reltime, tinterval
> together with timeofday().

I'd kind of like to keep timeofday(); it's the only simple way to
get a time display that includes "native" timezone info.  For
instance, I get

regression=# select timeofday();
              timeofday              
-------------------------------------
 Sat Sep 29 00:37:35.490977 2018 EDT
(1 row)

I think every other option would show me "-04" not "EDT".

+1 for removing the rest of that, though.  Unless somebody is
motivated to recast contrib/spi/timetravel with timestamptz
as the datetime type?

			regards, tom lane