Re: [HACKERS] Small improvement to compactify_tuples

Юрий Соколов <funny.falcon@gmail.com>

From: Юрий Соколов <funny.falcon@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Claudio Freire <klaussfreire@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Sokolov Yura <funny.falcon@postgrespro.ru>, Alvaro Herrera <alvherre@2ndquadrant.com>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL-Dev <pgsql-hackers@postgresql.org>
Date: 2017-11-28T07:46:21Z
Lists: pgsql-hackers
2017-11-08 23:44 GMT+03:00 Юрий Соколов <funny.falcon@gmail.com>:
>
> 2017-11-08 20:02 GMT+03:00 Tom Lane <tgl@sss.pgh.pa.us>:
> >
> > Claudio Freire <klaussfreire@gmail.com> writes:
> > > What's perhaps not clear is whether there are better ideas. Like
> > > rebuilding the page as Tom proposes, which doesn't seem like a bad
> > > idea. Bucket sort already is O(bytes), just as memcopy, only it has a
> > > lower constant factor (it's bytes/256 in the original patch), which
> > > might make copying the whole page an extra time lose against bucket
> > > sort in a few cases.
> >
> > > Deciding that last point does need more benchmarking. That doesn't
> > > mean the other improvements can't be pursued in the meanwhile, right?
> >
> > Well, I doubt we're going to end up committing more than one of these
> > ideas.  The question is which way is best.  If people are willing to
> > put in the work to test all of them, let's do it.
> >
> > BTW, it strikes me that in considering the rebuild-the-page approach,
> > we should not have blinders on and just measure the speed of
> > PageRepairFragmentation.  Rather, we should take a look at what happens
> > subsequently given a physically-ordered set of tuples.  I can recall
> > Andres or someone moaning awhile ago about lack of locality of access in
> > index page searches --- maybe applying that approach while vacuuming
> > indexes will help?
> >
> >                         regards, tom lane
>
> I'd like to add qsort_template.h as Claudio suggested, ie in a way close
to
> simplehash.h. With such template header, there will be no need in
> qsort_tuple_gen.pl .

Attached patched replaces gen_qsort_tuple.pl with qsort_template.h - generic
qsort template header.
Some tests do not specify exact order (ie their output depends on order of
equal elements). Such tests output wes fixed.

I didn't apply this qsort to compactify_tuples yet. Will do soon.

With regards,
Sokolov Yura aka funny_falcon.

Commits

  1. Avoid looping through line pointers twice in PageRepairFragmentation().

  2. Reduce pinning and buffer content locking for btree scans.

  3. Speed up in-memory tuplesorting.