Re: Use generation context to speed up tuplesorts

Julien Rouhaud <rjuju123@gmail.com>

From: Julien Rouhaud <rjuju123@gmail.com>
To: Ronan Dunklau <ronan.dunklau@aiven.io>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, pgsql-hackers@lists.postgresql.org, Andres Freund <andres@anarazel.de>, Tomas Vondra <tv@fuzzy.cz>, David Rowley <dgrowleyml@gmail.com>
Date: 2022-01-18T06:45:39Z
Lists: pgsql-hackers
Hi,

On Fri, Jan 07, 2022 at 12:03:55PM +0100, Ronan Dunklau wrote:
> 
> (Sorry for trying to merge back the discussion on the two sides of the thread)
> 
> In  https://www.postgresql.org/message-id/4776839.iZASKD2KPV%40aivenronan, I 
> expressed the idea of being able to tune glibc's malloc behaviour. 
> 
> I implemented that (patch 0001) to provide a new hook which is called on 
> backend startup, and anytime we set work_mem. This hook is # defined depending 
> on the malloc implementation: currently a default, no-op implementation is 
> provided as well as a glibc's malloc implementation.

This patch apparently misses something to have the no-op on windows:

https://cirrus-ci.com/task/4978247640285184
[03:03:01.676] Build FAILED.
[03:03:01.676]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(19,15): warning C4013: 'mallinfo2' undefined; assuming extern returning int [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(18,19): error C2079: 'm' uses undefined struct 'mallinfo2' [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(32,1): error C2224: left of '.arena' must have struct/union type [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(32,1): error C2224: left of '.hblkhd' must have struct/union type [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(32,1): error C2224: left of '.uordblks' must have struct/union type [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(32,1): error C2224: left of '.fordblks' must have struct/union type [c:\cirrus\malloc_stats.vcxproj]
[03:03:01.676] c:\cirrus\contrib\malloc_stats\malloc_stats.c(32,1): error C2224: left of '.keepcost' must have struct/union type [c:\cirrus\malloc_stats.vcxproj]

I'm also a bit confused about which patch(es) should actually be reviewed in
that thread.  My understanding is that the original patch might not be relevant
anymore but I might be wrong.  The CF entry should probably be updated to
clarify that, with an annotation/ title change / author update or something.

In the meantime I will switch the entry to Waiting on Author.



Commits

  1. Use Generation memory contexts to store tuples in sorts

  2. Adjust tuplesort API to have bitwise option flags

  3. Improve the generation memory allocator