Re: [CORE] Restore-reliability mode

On 6/4/15 11:28 PM, Michael Paquier wrote:
<list of things to test>
* More configuration variations; ./configure, initdb options, and *.conf
* More edge-case testing. (ie: what happens to each varlena as it 
approaches 1GB? 1B tables test. Etc.)
* More race-condition testing, like the tool Peter used heavily during 
ON CONFLICT development (written by Jeff Janes?)
* More non-SQL testing. For example, the logic in HeapTupleSatisfies* is 
quite complicated yet there's no tests dedicated to ensuring the logic 
is correct because it'd be extremely difficult (if not impossible) to 
construct those tests at a SQL level. Testing them with direct test 
calls to HeapTupleSatisfies* wouldn't be difficult, but we have no 
machinery to do C level testing.

>> Is pg_tap a reasonable starting point for this sort of testing?
> IMO, using the TAP machinery would be a good base for that. What lacks
> is a basic set of perl routines that one can easily use to set of test
> scenarios.

I think Stephen was referring specifically to pgTap (http://pgtap.org/).

Isn't our TAP framework just different output from pg_regress? Is there 
documentation on our TAP stuff?

>> >How would a test that would've caught the multixact issues look?
> I have not followed closely those discussions, not sure about that.

I've thought about this and unfortunately I think this may be a scenario 
that's just too complex to completely protect against with a test. What 
might help though is having better testing of edge cases (such as MXID 
wrap) and then combining that with other forms of testing, such as 
pg_upgrade and streaming rep. testing. Test things like "What happens if 
we pg_upgrade a cluster that's in danger of wraparound?"
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Data in Trouble? Get it in Treble! http://BlueTreble.com