Re: File truncation within PostgresNode::issues_sql_like() wrong on Windows

On 4/14/21 4:13 AM, Michael Paquier wrote:
> Hi all,
>
> As fairywren has proved a couple of days ago, it is not really a good
> idea to rely on a file truncation to check for patterns in the logs of
> the backend:
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fairywren&dt=2021-04-07%2013%3A29%3A28
>
> Visibly, a logic based on the log file truncation fails on Windows
> because of the concurrent access of the backend that outputs its logs
> there.  In PostgresNode.pm, connect_ok() and connect_access() enforce
> a rotation of the log file before restarting the server on Windows to
> make sure that a given step does not find logs generated by a previous
> test, but that's not the case of issues_sql_like().  Looking at the
> existing tests using this routine (src/bin/scripts/), I have found on
> test in 090_reindexdb.pl that could lead to a false positive.  The
> test is marked in the patch attached, just for awareness.
>
> Would there be any objections to change this routine so as we avoid
> the file truncation on Windows?  The patch attached achieves that.
>
> Any thoughts?


That seems rather heavy-handed. The buildfarm's approach is a bit
different. Essentially it seeks to the previous position of the log file
before reading contents. Here is its equivalent of slurp_file:


    use Fcntl qw(:seek);
    sub file_lines
    {
        my $filename = shift;
        my $filepos  = shift;
        my $handle;
        open($handle, '<', $filename) || croak "opening $filename: $!";
        seek($handle, $filepos, SEEK_SET) if $filepos;
        my @lines = <$handle>;
        close $handle;
        return @lines;
    }



A client wanting what's done in PostgresNode would do something like:


    my $logpos  = -s $logfile;
    do_some_stuff();
    my @lines = file_lines($logfile, $logpos);


This has the benefit of working the same on all platforms, and no
truncation, rotation, or restart is required.



cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com