Re: CI and test improvements

Hi,

On 2024-06-18 08:36:57 -0500, Justin Pryzby wrote:
> On Fri, Jun 14, 2024 at 08:34:37AM -0700, Andres Freund wrote:
> > Hm. There actually already is the mingw ccache installed. The images for mingw and msvc used to be separate (for startup performance when using containers), but we just merged them.  So it might be easiest to just explicitly use the ccache from there
> 
> > (also an explicit path might be faster).
> 
> I don't think the path search is significant; it's fast so long as
> there's no choco stub involved.

Comparatively it's definitely small, but I've seen it make a difference on
windows.


> > Could you check if that's fast?
> 
> Yes, it is.

Cool.


> > If not, we can just install the binaries distributed by the project, it's just more work to keep up2date that way. 
> 
> I guess you mean a separate line to copy choco's ccache.exe somewhere.

I was thinking of alternatively just using the binaries upstream
distributes. But the mingw way seems easier.

Perhaps we should add an environment variable pointing to ccache to the image,
or such?


>    build_script: |
>      vcvarsall x64
> -    ninja -C build
> +    ninja -C build |tee build.txt
> +    REM Since pipes lose the exit status of the preceding command, rerun the compilation
> +    REM without the pipe, exiting now if it fails, to avoid trying to run checks
> +    ninja -C build > nul

Perhaps we could do something like
        (ninja -C build && touch success) | tee build.txt
        cat success
?


> +    CCACHE_MAXSIZE: "500M"

Does it have to be this big to work?


>    configure_script: |
>      vcvarsall x64
> -    meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Dinjection_points=true -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" build
> +    meson setup build --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Dinjection_points=true -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" -Dc_args="/Z7"

A comment explaining why /Z7 is necessary would be good.



> From 3a399c6350ed2f341425431f184e382c3f46d981 Mon Sep 17 00:00:00 2001
> From: Justin Pryzby <pryzbyj@telsasoft.com>
> Date: Sat, 26 Feb 2022 19:39:10 -0600
> Subject: [PATCH 4/7] WIP: cirrus: upload changed html docs as artifacts
> 
> This could be done on the client side (cfbot).  One advantage of doing
> it here is that fewer docs are uploaded - many patches won't upload docs
> at all.
> 
> https://www.postgresql.org/message-id/flat/20220409021853.GP24419@telsasoft.com
> https://www.postgresql.org/message-id/CAB8KJ=i4qmEuopQ+PCSMBzGd4O-Xv0FCnC+q1x7hN9hsdvkBug@mail.gmail.com
> 
> https://cirrus-ci.com/task/5396696388599808

I still think that this runs the risk of increasing space utilization and thus
increase frequency of caches/artifacts being purged.



> +  # The commit that this branch is rebased on.  There's no easy way to find this.

I don't think that's true, IIRC I've complained about it before. We can do
something like:

major_num=$(grep PG_MAJORVERSION_NUM build/src/include/pg_config.h|awk '{print $3}');
echo major: $major_num;
if git rev-parse --quiet --verify origin/REL_${major_num}_STABLE > /dev/null ; then
    basebranch=origin/REL_${major_num}_STABLE;
else
    basebranch=origin/master;
fi;
echo base branch: $basebranch
base_commit=$(git merge-base HEAD $basebranch)



Greetings,

Andres Freund