Re: Vacuum: allow usage of more than 1GB of work mem

Claudio Freire <klaussfreire@gmail.com>

From: Claudio Freire <klaussfreire@gmail.com>
To: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Cc: Anastasia Lubennikova <lubennikovaav@gmail.com>, PostgreSQL-Dev <pgsql-hackers@postgresql.org>
Date: 2017-01-19T11:31:54Z
Lists: pgsql-hackers
On Thu, Jan 19, 2017 at 6:33 AM, Anastasia Lubennikova
<a.lubennikova@postgrespro.ru> wrote:
> 28.12.2016 23:43, Claudio Freire:
>
> Attached v4 patches with the requested fixes.
>
>
> Sorry for being late, but the tests took a lot of time.

I know. Takes me several days to run my test scripts once.

> create table t1 as select i, md5(random()::text) from
> generate_series(0,400000000) as i;
> create index md5_idx ON  t1(md5);
> update t1 set md5 = md5((random() * (100 + 500))::text);
> vacuum;
>
> Patched vacuum used 2.9Gb of memory and vacuumed the index in one pass,
> while for old version it took three passes (1GB+1GB+0.9GB).
> Vacuum duration results:
>
> vanilla:
> LOG: duration: 4359006.327 ms  statement: vacuum verbose t1;
> patched:
> LOG: duration: 3076827.378 ms  statement: vacuum verbose t1;
>
> We can see 30% vacuum speedup. I should note that this case can be
> considered
> as favorable to vanilla vacuum: the table is not that big, it has just one
> index
> and disk used is a fast fusionIO. We can expect even more gain on slower
> disks.
>
> Thank you again for the patch. Hope to see it in 10.0.

Cool. Thanks for the review and the tests.


Commits

  1. Prefetch blocks during lazy vacuum's truncation scan

  2. Explain unaccounted for space in pgstattuple.