Horton

Increase of maintenance_work_mem limit in 64-bit Windows

Vladlen Popolitov <v.popolitov@postgrespro.ru>

View thread View on postgr.es
From: Пополитов Владлен <v.popolitov@postgrespro.ru>
To: pgsql-hackers@postgresql.org
Date: 2024-09-19T13:55:42Z
Lists: pgsql-hackers

Attachments


Hello!
Currently PostgreSQL built on 64-bit Windows has 2Gb limit for
GUC variables due to sizeof(long)==4 used by Windows compilers.
Technically 64-bit addressing for maintenance_work_mem is possible,
but code base historically uses variables and constants of type "long",
when process maintenance_work_mem value.
Modern vector indexes like pgvector or pgvectorscale require as much
as possible maintenance_work_mem, 2 Gb limit  dramatically decrease
build index performace making impossible to build indexes for large
datasets.
​​​ The proposed patch fixes all appearences of "long" variables and constants
that can affect maintenance_work_mem (hash index, vacuum, planner only
affected, gin, gist, brin, bloom, btree indexes process value
correctly).
Constant MAX_SIZE_T_KILOBYTES added as upper limit for GUC variables
that depend on size_t only (currently only maintenance_work_mem).
Other GUC variables could use this constant after fixing "long" type
dependence.
This patch tested on
a) Windows 10 64-bit AMD64, compiled by msvc-19.37.32822
b) linux gcc (Debian 12.2.0-14) AMD64
All tests are passed.

Best regards

Vladlen Popolitov
postgrespro.com

 

Commits

  1. Get rid of our dependency on type "long" for memory size calculations.

  2. Use "ssize_t" not "long" in max_stack_depth-related code.

  3. Avoid integer overflow while testing wal_skip_threshold condition.