Thread

Re: Improving tracking/processing of buildfarm test failures

Alexander Lakhin <exclusion@gmail.com> — 2025-12-01T20:00:01Z

Hello hackers,

Please take a look at the November report on buildfarm failures:
# SELECT br, COUNT(*) FROM failures WHERE dt >= '2025-11-01' AND
  dt < '2025-12-01' GROUP BY br;
REL_13_STABLE: 8
REL_14_STABLE: 10
REL_15_STABLE: 4
REL_16_STABLE: 11
REL_17_STABLE: 14
REL_18_STABLE: 22
master: 104
-- Total: 173
(Counting test failures only, excluding indent-check, Configure, Build
errors.)

# SELECT COUNT(*) FROM (SELECT DISTINCT issue_link FROM failures WHERE
  dt >= '2025-11-01' AND dt < '2025-12-01');
21

# SELECT issue_link, COUNT(*) FROM failures WHERE dt >= '2025-11-01' AND
  dt < '2025-12-01' GROUP BY issue_link ORDER BY 2 DESC LIMIT 6;
https://www.postgresql.org/message-id/OSCPR01MB14966FE0BFB6C212298BFFEDEF5D1A%40OSCPR01MB14966.jpnprd01.prod.outlook.com 
: 21
-- Fixed

https://www.postgresql.org/message-id/2874644f-6431-41f4-abe2-99e5ab052606%40gmail.com: 15
-- Hurd

https://www.postgresql.org/message-id/aPAl27_urHSODwRN%40paquier.xyz: 10

https://www.postgresql.org/message-id/CA+hUKGL0bikWSC2XW-zUgFWNVEpD_gEWXndi2PE5tWqmApkpZQ@mail.gmail.com : 9

https://www.postgresql.org/message-id/202511261817.fyixgtt3hqdr%40alvherre.pgsql : 7
-- Fixed

https://www.postgresql.org/message-id/CAApHDvqALm%2Bk7FyfdQdCw1yF_8HojvR61YRrNhwRQPE%3DzSmnQA%40mail.gmail.com : 6
-- Fixed

# SELECT COUNT(*) FROM failures WHERE dt >= '2025-11-01' AND
  dt < '2025-12-01' AND issue_link IS NULL; -- Unsorted/unhelpful failures
73

Short-lived failures:
49

Best regards,
Alexander