ci: CCache churns through available space too quickly

Hi,

I noticed that a handfull of CI runs already lead to exceeding the available
cache space.  One can pay for more cache space, but I think the problem is
more that what we currently do doesn't work well.

With cirrus-ci all branches shared one cache, but that's not the case with
github actions. Except for being able to read caches from the default branch
(master in our case), other branches have completely separate cache
namespaces.  That's probably the right call, safety wise, but makes our ccache
approach .. not great.

We should only upload a new cache when the ccache cache hit ratio of the
existing cache entry has gotten low.


We also chose the cache key unfortunately, so that if a branch name started
with the name of the default branch, followed by a -, we'd always end up using
the main branches cache.


The attached patch fixes these, and a few other problems.  See commit message
for details.  With it I see a lot less cache churn and therefore also a higher
hit rate once one has more than 2-3 branches.


I'm not entirely happy with the amount of per job repetition this has. While
staying within the confines of a single .yml file, I couldn't find a better
way to deal with that.  We could move a fair bit of that complexity into a
separate file, using so called "composite actions".  But that's a bit of
additional github actions specific stuff that one would be exposed to, so I'm
not sure we should go that way?

It would result in having only two references to ccache in each job (one
before the build, one after). Each of those could encapsulate a bunch of steps
defined in another file.


Thoughts?

Greetings,

Andres Freund