Horton

Thread

Commits

  1. Use zic's new "-b slim" option to generate smaller timezone files.

  1. New "-b slim" option in 2019b zic: should we turn that on?

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-07-17T22:42:07Z

    I just finished updating our timezone code to match IANA release
2019b.  There's an interesting new switch in zic: if you say
"-b slim", it generates zone data files that have only 64-bit
data (not the 32-bit plus 64-bit data that it's been emitting
for years), and it drops other space-wasting hacks that are needed
only for backwards compatibility with old timezone libraries.

That is not us, so I wonder whether we shouldn't turn on that switch.

I did a quick comparison of the file sizes, and indeed there's
a noticeable per-file savings, eg

$ ls -l timezone.fat/America/New_York timezone.slim/America/New_York 
-rw-r--r--. 3 postgres postgres 3536 Jul 17 18:08 timezone.fat/America/New_York
-rw-r--r--. 3 postgres postgres 1744 Jul 17 18:07 timezone.slim/America/New_York

$ ls -l timezone.fat/Europe/Paris timezone.slim/Europe/Paris  
-rw-r--r--. 1 postgres postgres 2962 Jul 17 18:08 timezone.fat/Europe/Paris
-rw-r--r--. 1 postgres postgres 1105 Jul 17 18:07 timezone.slim/Europe/Paris

Now, since the files are pretty much all under 4K, that translates
to exactly no disk space savings on my ext4 filesystem :-(

$ du -hs timezone.fat timezone.slim
1.6M    timezone.fat
1.6M    timezone.slim

But other filesystems that are smarter about small files would
probably benefit.  Also, there's a significant difference in
the size of a compressed tarball:

-rw-rw-r--.  1 postgres postgres 148501 Jul 17 18:09 timezone.fat.tgz
-rw-rw-r--.  1 postgres postgres  80511 Jul 17 18:09 timezone.slim.tgz

not that that really helps us, because we don't include these
generated files in our tarballs.

Despite the marginal payoff, I'm strongly tempted to enable this
switch.  The only reason I can think of not to do it is if somebody
is using a Postgres installation's share/timezone tree as tzdata
for some other program with not-up-to-date timezone library code.
But who would that be?

A possible compromise is to turn it on only in HEAD, though I'd
rather keep all the branches working the same as far as the
timezone code goes.

Thoughts?

			regards, tom lane

  2. Re: New "-b slim" option in 2019b zic: should we turn that on?

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-05T21:43:49Z

    On 7/17/19 6:42 PM, Tom Lane wrote:
>
> Despite the marginal payoff, I'm strongly tempted to enable this
> switch.  The only reason I can think of not to do it is if somebody
> is using a Postgres installation's share/timezone tree as tzdata
> for some other program with not-up-to-date timezone library code.
> But who would that be?
>
> A possible compromise is to turn it on only in HEAD, though I'd
> rather keep all the branches working the same as far as the
> timezone code goes.
>


I've just run into an issue with this (commit a1207910968). The makefile
now assumes that zic has this switch. But I was attempting to get around
an issue on msys2 by using its zic, (ZIC=/usr/bin/zic configure ...). It
crashes on the floor because it doesn't know about "-b slim". I think we
probably need a way to turn this off.


cheers

andrew

-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  3. Re: New "-b slim" option in 2019b zic: should we turn that on?

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-05T22:33:13Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> I've just run into an issue with this (commit a1207910968). The makefile
> now assumes that zic has this switch. But I was attempting to get around
> an issue on msys2 by using its zic, (ZIC=/usr/bin/zic configure ...). It
> crashes on the floor because it doesn't know about "-b slim". I think we
> probably need a way to turn this off.

I had contemplated injecting the -b switch via

 # any custom options you might want to pass to zic while installing data files
-ZIC_OPTIONS =
+ZIC_OPTIONS = -b slim

which would allow overriding it by defining the ZIC_OPTIONS macro.
Does that seem appropriate?  I didn't do it because I worried about
interference with existing uses of ZIC_OPTIONS ... but who knows
whether there are any.

BTW, building with old versions of zic is not guaranteed to work anyway.
They do tend to wait a year or two before they start to use new zic
features in the timezone data files, but they don't wait indefinitely.

			regards, tom lane

  4. Re: New "-b slim" option in 2019b zic: should we turn that on?

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-06T02:22:01Z

    On 10/5/19 6:33 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> I've just run into an issue with this (commit a1207910968). The makefile
>> now assumes that zic has this switch. But I was attempting to get around
>> an issue on msys2 by using its zic, (ZIC=/usr/bin/zic configure ...). It
>> crashes on the floor because it doesn't know about "-b slim". I think we
>> probably need a way to turn this off.
> I had contemplated injecting the -b switch via
>
>  # any custom options you might want to pass to zic while installing data files
> -ZIC_OPTIONS =
> +ZIC_OPTIONS = -b slim
>
> which would allow overriding it by defining the ZIC_OPTIONS macro.
> Does that seem appropriate?  I didn't do it because I worried about
> interference with existing uses of ZIC_OPTIONS ... but who knows
> whether there are any.


I don't think that's going to work very well with a buildfarm member,
where there's no convenient way to set it.


But it turns out there are bigger problems with what I'm doing, anyway,
so let's leave sleeping dogs lie for now.


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  5. Re: New "-b slim" option in 2019b zic: should we turn that on?

    Tom Lane <tgl@sss.pgh.pa.us> — 2019-10-06T02:33:19Z

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
> On 10/5/19 6:33 PM, Tom Lane wrote:
>> I had contemplated injecting the -b switch via
>> -ZIC_OPTIONS =
>> +ZIC_OPTIONS = -b slim

> I don't think that's going to work very well with a buildfarm member,
> where there's no convenient way to set it.

Can't you set that from build_env?

			regards, tom lane

  6. Re: New "-b slim" option in 2019b zic: should we turn that on?

    Andrew Dunstan <andrew.dunstan@2ndquadrant.com> — 2019-10-06T15:17:31Z

    On 10/5/19 10:33 PM, Tom Lane wrote:
> Andrew Dunstan <andrew.dunstan@2ndquadrant.com> writes:
>> On 10/5/19 6:33 PM, Tom Lane wrote:
>>> I had contemplated injecting the -b switch via
>>> -ZIC_OPTIONS =
>>> +ZIC_OPTIONS = -b slim
>> I don't think that's going to work very well with a buildfarm member,
>> where there's no convenient way to set it.
> Can't you set that from build_env?
>
> 			


No, build_env sets the environment, not makefile variables, and
configure doesn't fill in ZIC_OPTIONS, unlike what it does with ZIC.


Anyway, it turns out that avoiding the issue I was having here just
postpones the problem for a few seconds, so while we should probably do
something here it's not urgent from my POV.


cheers


andrew


-- 
Andrew Dunstan                https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services