DSA failed to allocate memory
Dongming Liu <ldming101@gmail.com>
From: Dongming Liu <ldming101@gmail.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-24T09:58:44Z
Lists: pgsql-hackers
Attachments
- 0001-add-dsa-test-module.patch (application/octet-stream) patch 0001
Hi,
I am using Dynamic shared memory areas(DSA) to manage some variable
length shared memory, I've found that in some cases allocation fails even
though there are enough contiguous pages.
The steps to reproduce are as follows:
1. create a dsa area with a 1MB DSM segment
2. set its size limit to 1MB
3. allocate 4KB memory until fails
4. free all allocated memory in step 3
5. repeat step 3 and step 4
When I first run the step 3, there is 240 4KB memory allocated successfully.
But when I free all and allocate again, no memory can be allocated even
though there are 252 contiguous pages. IMO, this should not be expected
to happen, right?
The call stack is as follows:
#0 get_best_segment (area=0x200cc70, npages=16) at dsa.c:1972
#1 0x0000000000b46b36 in ensure_active_superblock (area=0x200cc70,
pool=0x7fa7b51f46f0,
size_class=33) at dsa.c:1666
#2 0x0000000000b46555 in alloc_object (area=0x200cc70, size_class=33)
at dsa.c:1460
#3 0x0000000000b44f05 in dsa_allocate_extended (area=0x200cc70,
size=4096, flags=2) at dsa.c:795
I read the relevant code and found that get_best_segment re-bin the segment
to segment index 4 when first run the step 3. But when free all and run the
step 3 again, get_best_segment search from the first bin that *might*
have enough
contiguous pages, it is calculated by contiguous_pages_to_segment_bin(),
for a superblock with 16 pages, contiguous_pages_to_segment_bin is 5.
So the second time, get_best_segment search bin from segment index 5 to 16,
but the suitable segment has been re-bin to 4 that we do not check.
Finally, the get_best_segment return NULL and dsa_allocate_extended return
a invalid dsa pointer.
Maybe we can use one of the following methods to fix it:
1. re-bin segment to suitable segment index when called dsa_free
2. get_best_segment search all bins
I wrote a simple test code that is attached to reproduce it.
Ant thoughts?
--
Best Regards
Dongming(https://www.aliyun.com/)
Commits
-
Re-bin segment when memory pages are freed.
- 1605623ec686 11.21 landed
- 74ad9b0d1e7c 12.16 landed
- a0003572f214 13.12 landed
- b7ec66731dd8 14.9 landed
- 9ffb10f183f1 15.4 landed
- af8f9ec66bd1 16.0 landed
- 03f80daac8cf 17.0 landed
-
De-support floating-point timestamps.
- b6aa17e0ae36 10.0 cited