Re: Minmax indexes

Jeff Janes <jeff.janes@gmail.com>

From: Jeff Janes <jeff.janes@gmail.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Erik Rijkers <er@xs4all.nl>, Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2013-11-15T20:06:05Z
Lists: pgsql-hackers

Attachments

On Fri, Nov 8, 2013 at 12:11 PM, Alvaro Herrera <alvherre@2ndquadrant.com>wrote:

> Erik Rijkers wrote:
> > On Thu, September 26, 2013 00:34, Erik Rijkers wrote:
> > > On Wed, September 25, 2013 22:34, Alvaro Herrera wrote:
> > >
> > >> [minmax-5.patch]
> > >
> > > I have the impression it's not quite working correctly.
>
> Here's a version 7 of the patch, which fixes these bugs and adds
> opclasses for a bunch more types (timestamp, timestamptz, date, time,
> timetz), courtesy of Martín Marqués.  It's also been rebased to apply
> cleanly on top of today's master branch.
>
> I have also added a selectivity function, but I'm not positive that it's
> very useful yet.
>



I tested it with attached script, but broke out of the "for" loop after 5
iterations (when it had 300,000,005 rows inserted)

Then I did an analyze, and got an error message below:

jjanes=# analyze;
ERROR:  could not truncate file "base/16384/16388_vm" to 488 blocks: it's
only 82 blocks now

16388 is the index's relfilenode.

Here is the backtrace upon entry to the truncate that is going to fail:

#0  mdtruncate (reln=0x23c91b0, forknum=VISIBILITYMAP_FORKNUM, nblocks=488)
at md.c:858
#1  0x000000000048eb4a in mmRevmapTruncate (rmAccess=0x26ad878,
heapNumBlocks=1327434) at mmrevmap.c:360
#2  0x000000000048d37a in mmvacuumcleanup (fcinfo=<value optimized out>) at
minmax.c:1264
#3  0x000000000072dcef in FunctionCall2Coll (flinfo=<value optimized out>,
collation=<value optimized out>, arg1=<value optimized out>,
    arg2=<value optimized out>) at fmgr.c:1323
#4  0x000000000048c1e5 in index_vacuum_cleanup (info=<value optimized out>,
stats=0x0) at indexam.c:715
#5  0x000000000052a7ce in do_analyze_rel (onerel=0x7f59798589e8,
vacstmt=0x23b0bd8, acquirefunc=0x5298d0 <acquire_sample_rows>,
relpages=1327434,
    inh=0 '\000', elevel=13) at analyze.c:634
#6  0x000000000052b320 in analyze_rel (relid=<value optimized out>,
vacstmt=0x23b0bd8, bstrategy=<value optimized out>) at analyze.c:267
#7  0x000000000057cba7 in vacuum (vacstmt=0x23b0bd8, relid=<value optimized
out>, do_toast=1 '\001', bstrategy=<value optimized out>,
    for_wraparound=0 '\000', isTopLevel=<value optimized out>) at
vacuum.c:249
#8  0x0000000000663177 in standard_ProcessUtility (parsetree=0x23b0bd8,
queryString=<value optimized out>, context=<value optimized out>,
params=0x0,
    dest=<value optimized out>, completionTag=<value optimized out>) at
utility.c:682
#9  0x00007f598290b791 in pgss_ProcessUtility (parsetree=0x23b0bd8,
queryString=0x23b0220 "analyze \n;", context=PROCESS_UTILITY_TOPLEVEL,
params=0x0,
    dest=0x23b0f18, completionTag=0x7fffd3442f30 "") at
pg_stat_statements.c:825
#10 0x000000000065fcf7 in PortalRunUtility (portal=0x24195e0,
utilityStmt=0x23b0bd8, isTopLevel=1 '\001', dest=0x23b0f18,
completionTag=0x7fffd3442f30 "")
    at pquery.c:1187
#11 0x0000000000660c6d in PortalRunMulti (portal=0x24195e0, isTopLevel=1
'\001', dest=0x23b0f18, altdest=0x23b0f18, completionTag=0x7fffd3442f30 "")
    at pquery.c:1318
#12 0x0000000000661323 in PortalRun (portal=0x24195e0,
count=9223372036854775807, isTopLevel=1 '\001', dest=0x23b0f18,
altdest=0x23b0f18,
    completionTag=0x7fffd3442f30 "") at pquery.c:816
#13 0x000000000065dbb4 in exec_simple_query (query_string=0x23b0220
"analyze \n;") at postgres.c:1048
#14 0x000000000065f259 in PostgresMain (argc=<value optimized out>,
argv=<value optimized out>, dbname=0x2347be8 "jjanes", username=<value
optimized out>)
    at postgres.c:3992
#15 0x000000000061b7d0 in BackendRun (argc=<value optimized out>,
argv=<value optimized out>) at postmaster.c:4085
#16 BackendStartup (argc=<value optimized out>, argv=<value optimized out>)
at postmaster.c:3774
#17 ServerLoop (argc=<value optimized out>, argv=<value optimized out>) at
postmaster.c:1585
#18 PostmasterMain (argc=<value optimized out>, argv=<value optimized out>)
at postmaster.c:1240
#19 0x00000000005b5e90 in main (argc=3, argv=0x2346cd0) at main.c:196


Cheers,

Jeff

Commits

  1. Refactor per-page logic common to all redo routines to a new function.

  2. Reduce use of heavyweight locking inside hash AM.

  3. Scan the buffer pool just once, not once per fork, during relation drop.

  4. Major patch from Thomas Lockhart <Thomas.G.Lockhart@jpl.nasa.gov>