Horton

Re: POC: converting Lists into arrays

Andrew Gierth <andrew@tao11.riddles.org.uk>

View thread View on postgr.es
From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: David Rowley <david.rowley@2ndquadrant.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-02-28T05:42:13Z
Lists: pgsql-hackers 
>>>>> "David" == David Rowley <david.rowley@2ndquadrant.com> writes:

 David> I went and had a few adventures with this patch to see if I
 David> could figure out why the small ~1% regression exists.

Just changing the number of instructions (even in a completely unrelated
place that's not called during the test) can generate performance
variations of this size, even when there's no real difference.

To get a reliable measurement of timing changes less than around 3%,
what you have to do is this: pick some irrelevant function and add
something like an asm directive that inserts a variable number of NOPs,
and do a series of test runs with different values.

See http://tinyurl.com/op9qg8a for an example of the kind of variation
that one can get; this plot records timing runs where each different
padding size was tested 3 times (non-consecutively, so you can see how
repeatable the test result is for each size), each timing is actually
the average of the last 10 of 11 consecutive runs of the test.

To establish a 1% performance benefit or regression you need to show
that there's still a difference _AFTER_ taking this kind of
spooky-action-at-a-distance into account. For example, in the test shown
at the link, if a substantive change to the code moved the upper and
lower bounds of the output from (6091,6289) to (6030,6236) then one
would be justified in claiming it as a 1% improvement.

Such is the reality of modern CPUs.

-- 
Andrew (irc:RhodiumToad)

Commits

  1. Remove EState.es_range_table_array.

  2. Rationalize use of list_concat + list_copy combinations.

  3. Cosmetic improvements in setup of planner's per-RTE arrays.

  4. Make better use of the new List implementation in a couple of places

  5. Fix sepgsql test results for commit d97b714a2.

  6. Avoid using lcons and list_delete_first where it's easy to do so.

  7. Remove lappend_cell...() family of List functions.

  8. Clean up some ad-hoc code for sorting and de-duplicating Lists.

  9. Redesign the API for list sorting (list_qsort becomes list_sort).

  10. Remove dead code.

  11. Represent Lists as expansible arrays, not chains of cons-cells.

  12. Standardize some more loops that chase down parallel lists.

  13. Reimplement the linked list data structure used throughout the backend.