Thread

BUG #18341: SIGBUS with > 4GB DSM and 'dynamic_shared_memory_type = mmap'

The Post Office <noreply@postgresql.org> — 2024-02-13T17:32:26Z

The following bug has been logged on the website:

Bug reference:      18341
Logged by:          Heikki Linnakangas
Email address:      hlinnaka@iki.fi
PostgreSQL version: 16.2
Operating system:   Linux
Description:        

There's a bug in the mmap DSM implementation: If you create a DSM segment
larger than 4 GB, the file that's created is too small. As a result, at
least on Linux, you get a SIGBUS when you try to use it.

I bumped into this with pgvector v0.6.0, which creates a one giant DSM
segment with size roughly equal to `maintenance_work_mem`, when you create
an HNSW index:

postgres=# SET maintenance_work_mem='5 GB';
SET
postgres=# create index on items_small using hnsw (embedding
vector_cosine_ops) WITH (m=16, ef_construction=100);
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.

The culprit is this 'uint32' local variable that's used to store the size:

diff --git a/src/backend/storage/ipc/dsm_impl.c
b/src/backend/storage/ipc/dsm_impl.c
index 03aa47a104..8dd669e0ce 100644
--- a/src/backend/storage/ipc/dsm_impl.c
+++ b/src/backend/storage/ipc/dsm_impl.c
@@ -873,7 +873,7 @@ dsm_impl_mmap(dsm_op op, dsm_handle handle, Size
request_size,
 		 * transferring data to the kernel.
 		 */
 		char	   *zbuffer = (char *) palloc0(ZBUFFER_SIZE);
-		uint32		remaining = request_size;
+		Size		remaining = request_size;
 		bool		success = true;
 
 		/*

Barring objections, I'll commit and backpatch that shortly.

- Heikki