Re: Vacuum: allow usage of more than 1GB of work mem

Claudio Freire <klaussfreire@gmail.com>

From: Claudio Freire <klaussfreire@gmail.com>
To: Daniel Gustafsson <daniel@yesql.se>
Cc: Andres Freund <andres@anarazel.de>, Masahiko Sawada <sawada.mshk@gmail.com>, Anastasia Lubennikova <a.lubennikova@postgrespro.ru>, Anastasia Lubennikova <lubennikovaav@gmail.com>, PostgreSQL-Dev <pgsql-hackers@postgresql.org>
Date: 2017-10-02T14:02:52Z
Lists: pgsql-hackers

Attachments

On Sun, Oct 1, 2017 at 8:36 PM, Daniel Gustafsson <daniel@yesql.se> wrote:
>> On 18 Aug 2017, at 13:39, Claudio Freire <klaussfreire@gmail.com> wrote:
>>
>> On Fri, Apr 7, 2017 at 10:51 PM, Claudio Freire <klaussfreire@gmail.com> wrote:
>>> Indeed they do, and that's what motivated this patch. But I'd need
>>> TB-sized tables to set up something like that. I don't have the
>>> hardware or time available to do that (vacuum on bloated TB-sized
>>> tables can take days in my experience). Scale 4000 is as big as I can
>>> get without running out of space for the tests in my test hardware.
>>>
>>> If anybody else has the ability, I'd be thankful if they did test it
>>> under those conditions, but I cannot. I think Anastasia's test is
>>> closer to such a test, that's probably why it shows a bigger
>>> improvement in total elapsed time.
>>>
>>> Our production database could possibly be used, but it can take about
>>> a week to clone it, upgrade it (it's 9.5 currently), and run the
>>> relevant vacuum.
>>
>> It looks like I won't be able to do that test with a production
>> snapshot anytime soon.
>>
>> Getting approval for the budget required to do that looks like it's
>> going to take far longer than I thought.
>>
>> Regardless of that, I think the patch can move forward. I'm still
>> planning to do the test at some point, but this patch shouldn't block
>> on it.
>
> This patch has been marked Ready for committer after review, but wasn’t
> committed in the current commitfest so it will be moved to the next.  Since it
> no longer applies cleanly, it’s being reset to Waiting for author though.
>
> cheers ./daniel

Rebased version of the patches attached

Commits

  1. Prefetch blocks during lazy vacuum's truncation scan

  2. Explain unaccounted for space in pgstattuple.