Horton

Re: Large writable variables

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Thomas Munro <thomas.munro@enterprisedb.com>
Date: 2018-10-17T18:34:06Z
Lists: pgsql-hackers 
On 2018-10-16 19:48:42 -0400, Tom Lane wrote:
> BTW, I looked around for .o files with large BSS numbers, and came across
> 
> $ size src/interfaces/ecpg/ecpglib/prepare.o
>    text    data     bss     dec     hex filename
>    4023       4 1048576 1052603  100fbb src/interfaces/ecpg/ecpglib/prepare.o

Btw, I think one can fairly argue that large .data vars have a
higher cost than .bss. That's not to say we shouldn't fix .bss ones ;)

I've this awful shell script to figure out the sizes:

for file in $(find . -type f -name '*.o'); do objdump -j .data -t $file|tail -n +5|grep -v '^$'|awk -v "file=$file" '{print $4, $5, $6, file}';done|sort -nr|less

.data 0000000000001180 datetktbl ./src/interfaces/ecpg/pgtypeslib/dt_common.o
.data 0000000000000c28 ibmkanji ./src/backend/utils/mb/conversion_procs/euc_jp_and_sjis/euc_jp_and_sjis.o
.data 00000000000003f0 deltatktbl ./src/interfaces/ecpg/pgtypeslib/dt_common.o
.data 0000000000000100 sqlca_init ./src/interfaces/ecpg/ecpglib/misc.o
.data 0000000000000100 sqlca_init ./src/interfaces/ecpg/compatlib/informix.o
.data 0000000000000068 day_tab ./src/interfaces/ecpg/pgtypeslib/dt_common.o
.data 0000000000000030 ecpg_query ./src/interfaces/ecpg/preproc/preproc.o
.data 0000000000000030 ecpg_no_indicator ./src/interfaces/ecpg/preproc/preproc.o
.data 0000000000000030 descriptor_type.5770 ./src/interfaces/ecpg/preproc/descriptor.o
.data 000000000000002b ssl_nomem ./src/interfaces/libpq/fe-secure-openssl.o
.data 000000000000002a PLy_subtransaction_doc ./src/pl/plpython/plpy_subxactobject.o
.data 0000000000000023 PLy_cursor_doc ./src/pl/plpython/plpy_cursorobject.o
.data 000000000000001e PLy_result_doc ./src/pl/plpython/plpy_resultobject.o
.data 0000000000000018 PLy_plan_doc ./src/pl/plpython/plpy_planobject.o
.data 0000000000000010 base_yylloc ./src/interfaces/ecpg/preproc/preproc.o

Should also work for other sections.


Hm, there's a lot of ecpg in this :(

> That megabyte is from a statically allocated statement cache array.
> Seems a bit unfriendly to users of ecpglib, given that many apps
> would never use the statement cache (AFAICT you have to explicitly
> ask for auto-prepare mode to get to that code).
> 
> Doesn't look hard to fix though.

Thanks for tackling that.

Greetings,

Andres Freund

Commits

  1. Apply unconstify() in more places

  2. Improve unconstify() documentation

  3. Drop const cast from dlsym() calls

  4. Const-ify a few more large static tables.

  5. Improve tzparse's handling of TZDEFRULES ("posixrules") zone data.

  6. Avoid statically allocating statement cache in ecpglib/prepare.c.

  7. Reorder FmgrBuiltin members, saving 25% in size.

  8. Add macro to cast away const without allowing changes to underlying type.

  9. Mark constantly allocated dest receiver as const.

  10. Avoid statically allocating formatting.c's format string caches.

  11. Correct constness of system attributes in heap.c & prerequisites.

  12. Avoid statically allocating gmtsub()'s timezone workspace.

  13. Correct constness of a few variables.

  14. Move the replication lag tracker into heap memory.