Horton

Thread

Commits

  1. amcheck: Harden tests against concurrent autovacuums.

  2. Add amcheck extension to contrib.

  1. pgsql: Add amcheck extension to contrib.

    Andres Freund <andres@anarazel.de> — 2017-03-10T00:40:56Z

    Add amcheck extension to contrib.

This is the beginning of a collection of SQL-callable functions to
verify the integrity of data files.  For now it only contains code to
verify B-Tree indexes.

This adds two SQL-callable functions, validating B-Tree consistency to
a varying degree.  Check the, extensive, docs for details.

The goal is to later extend the coverage of the module to further
access methods, possibly including the heap.  Once checks for
additional access methods exist, we'll likely add some "dispatch"
functions that cover multiple access methods.

Author: Peter Geoghegan, editorialized by Andres Freund
Reviewed-By: Andres Freund, Tomas Vondra, Thomas Munro,
   Anastasia Lubennikova, Robert Haas, Amit Langote
Discussion: CAM3SWZQzLMhMwmBqjzK+pRKXrNUZ4w90wYMUWfkeV8mZ3Debvw@mail.gmail.com

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/3717dc149ecf44b8be95350a68605ba7299474fd

Modified Files
--------------
contrib/Makefile                         |    1 +
contrib/amcheck/Makefile                 |   21 +
contrib/amcheck/amcheck--1.0.sql         |   24 +
contrib/amcheck/amcheck.control          |    5 +
contrib/amcheck/expected/check.out       |    1 +
contrib/amcheck/expected/check_btree.out |   90 +++
contrib/amcheck/sql/check.sql            |    1 +
contrib/amcheck/sql/check_btree.sql      |   59 ++
contrib/amcheck/verify_nbtree.c          | 1249 ++++++++++++++++++++++++++++++
doc/src/sgml/amcheck.sgml                |  273 +++++++
doc/src/sgml/contrib.sgml                |    1 +
doc/src/sgml/filelist.sgml               |    1 +
src/tools/pgindent/typedefs.list         |    2 +
13 files changed, 1728 insertions(+)

  2. Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

    Tom Lane <tgl@sss.pgh.pa.us> — 2017-03-13T19:45:01Z

    Andres Freund <andres@anarazel.de> writes:
> Add amcheck extension to contrib.

axolotl just failed on this:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=axolotl&dt=2017-03-13%2017%3A49%3A24

***************
*** 78,86 ****
  
  -- make sure we don't have any leftover locks
  SELECT * FROM pg_locks WHERE relation IN ('bttest_a_idx'::regclass, 'bttest_b_idx'::regclass);
!  locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid | mode | granted | fastpath 
! ----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-----+------+---------+----------
! (0 rows)
  
  COMMIT;
  -- cleanup
--- 78,87 ----
  
  -- make sure we don't have any leftover locks
  SELECT * FROM pg_locks WHERE relation IN ('bttest_a_idx'::regclass, 'bttest_b_idx'::regclass);
!  locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction |  pid  |      mode       | granted | fastpath 
! ----------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-------+-----------------+---------+----------
!  relation |    57562 |    57573 |      |       |            |               |         |       |          | 4/29               | 20342 | AccessShareLock | t       | t
! (1 row)
  
  COMMIT;
  -- cleanup


I could be wrong, but the most obvious explanation for this failure is
that autovacuum had a lock on the table or index when we looked.
Even if that isn't why axolotl failed in this particular case, I think
it's dead certain that we will see such failures from time to time
if this test script isn't tightened up.  IIUC what the test is trying
to look for, I think adding "AND pid = pg_backend_pid()" to this query
would be an appropriate fix.

			regards, tom lane

  3. Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

    Andres Freund <andres@anarazel.de> — 2017-03-13T21:09:39Z

    Hi,

On 2017-03-13 15:45:01 -0400, Tom Lane wrote:
> I could be wrong, but the most obvious explanation for this failure is
> that autovacuum had a lock on the table or index when we looked.
> Even if that isn't why axolotl failed in this particular case, I think
> it's dead certain that we will see such failures from time to time
> if this test script isn't tightened up.  IIUC what the test is trying
> to look for, I think adding "AND pid = pg_backend_pid()" to this query
> would be an appropriate fix.

Yes, that sounds reasonable.  Will do in a bit.  Thanks for noticing.

- Andres

  4. Re: [COMMITTERS] pgsql: Add amcheck extension to contrib.

    Andres Freund <andres@anarazel.de> — 2017-03-14T20:09:17Z

    On 2017-03-13 14:09:39 -0700, Andres Freund wrote:
> Hi,
> 
> On 2017-03-13 15:45:01 -0400, Tom Lane wrote:
> > I could be wrong, but the most obvious explanation for this failure is
> > that autovacuum had a lock on the table or index when we looked.
> > Even if that isn't why axolotl failed in this particular case, I think
> > it's dead certain that we will see such failures from time to time
> > if this test script isn't tightened up.  IIUC what the test is trying
> > to look for, I think adding "AND pid = pg_backend_pid()" to this query
> > would be an appropriate fix.
> 
> Yes, that sounds reasonable.  Will do in a bit.  Thanks for noticing.

Pushed.

- Andres