Cylc is a workflow engine, allowing anyone to run a custom schedule of inter-dependent tasks. Its general-purpose open-source nature has led to a technically diverse user base, from scientists & other RSEs to HPC analysts and operational forecasting staff, distributed across research centres worldwide. Development has been driven by the requirements of users; it has been a ongoing challenge to cater for the unique automation needs emerging from such varied contexts, for example building a system that scales both down, to transcend mere cron jobs, and up, to complex workflows of thousands of tasks. Hosting Cylc publicly on GitHub has been vital, enabling input, including code contributions, from all factions, and centralising communication between our international team working under inverse time zones. We invest much of our time providing training and support in the effective use of Cylc and notably at the Met Office our users have spun-up their own community group for effective and maintainable Cylc workflow design, adding the balancing act of directing, but not dictating, best practice. Hear us share our wisdom from the pursuit of reproducible workflows for all.
