Re: Making type Datum be 8 bytes everywhere
Ranier Vilela <ranier.vf@gmail.com>
From: Ranier Vilela <ranier.vf@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Tomas Vondra <tomas@vondra.me>, Peter Eisentraut <peter@eisentraut.org>, Joe Conway <mail@joeconway.com>, Michael Paquier <michael@paquier.xyz>, Andres Freund <andres@anarazel.de>, pgsql-hackers@lists.postgresql.org, Robert Haas <robertmhaas@gmail.com>
Date: 2025-09-11T12:43:35Z
Lists: pgsql-hackers
Em qua., 10 de set. de 2025 às 17:35, Tom Lane <tgl@sss.pgh.pa.us> escreveu: > Tomas Vondra <tomas@vondra.me> writes: > > While testing a different patch, I tried running with address sanitizer > > on rpi5, running the 32-bit OS (which AFAIK is 64-bit kernel and 32-bit > > user space). With that, stats_ext regression tests fail like this: > > > extended_stats.c:1082:27: runtime error: store to misaligned address > > 0x036671dc for type 'Datum', which requires 8 byte alignment > > 0x036671dc: note: pointer points here > > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 7e > > 7f 08 00 00 00 7f 7f 7f 7f > > ^ > > > This happens because build_sorted_items() does palloc(), and then > > accesses the pointer as array of structs, with a Datum field. And it > > apparently expects the pointer to be a multiple of 8 bytes. Isn't that a > > bit strange, with 32-bit user space? The pointer is indeed a multiple of > > 4B, so maybe the expected alignment is wrong? > > I think build_sorted_items is plainly at fault here, where it does > > /* Compute the total amount of memory we need (both items and values). > */ > len = data->numrows * sizeof(SortItem) + nvalues * (sizeof(Datum) + > sizeof(bool)); > > /* Allocate the memory and split it into the pieces. */ > ptr = palloc0(len); > > /* items to sort */ > items = (SortItem *) ptr; > ptr += data->numrows * sizeof(SortItem); > > /* values and null flags */ > values = (Datum *) ptr; > ptr += nvalues * sizeof(Datum); > > This is silently assuming that sizeof(SortItem) is a multiple of > alignof(Datum), which on a 32-bit-pointer platform is not true > any longer. We ought to MAXALIGN the two occurrences of > data->numrows * sizeof(SortItem). > We possibly have two more instances? 1. Function ndistinct_for_combination (src/backend/statistics/mvdistinct.c) - items = (SortItem *) palloc(numrows * sizeof(SortItem)); + items = (SortItem *) palloc(MAXALIGN(numrows * sizeof(SortItem))); 2. Function build_distinct_groups (src/backend/statistics/mcv.c) - SortItem *groups = (SortItem *) palloc(ngroups * sizeof(SortItem)); + SortItem *groups = (SortItem *) palloc(MAXALIGN(ngroups * sizeof(SortItem))); best regards, Ranier Vilela
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Avoid faulty alignment of Datums in build_sorted_items().
- 09036dc71c68 19 (unreleased) landed
-
Grab the low-hanging fruit from forcing USE_FLOAT8_BYVAL to true.
- ee54046601de 19 (unreleased) landed
-
Grab the low-hanging fruit from forcing sizeof(Datum) to 8.
- 6aebedc38497 19 (unreleased) landed
-
Make type Datum be 8 bytes wide everywhere.
- 2a600a93c7be 19 (unreleased) landed
-
Mop-up for Datum conversion cleanups.
- 665c3dbba497 19 (unreleased) landed