We are well-used to long lead times in particle physics. Detectors that are commissioned often rely on technology that was in an R&D phase decades before and now, as the CERN community, we now plan for new accelerators that will still be operational at the end of this century. Computing has not usually had to have quite the same degree of advanced planning - generally software solutions are more flexible and even the commodity hardware platforms that are used for our offline computing are not well-defined decades in advance. However, that certainly does not mean that computing and software isn’t planned and, in fact, technology changes are requiring us to make significant strategic decisions that will impact the future of the field now.
It also is clear that HEP is not alone in this, with large new projects from other sciences coming soon that will have very significant computing demands. In Bologna in 2023 the ECFA, NuPPEC (nuclear physics) and APPEC (astroparticle physics) started to look at their computing needs for the coming decade under the JENA umbrella. The three communities aim to understand how we can scale up coherently in the years to come.
For the challenges to come, each community identified some key issues. In particle physics, the computing model for the LHC, WLCG, is well established and grows to encompass additional experiments. However, the needs for HL-LHC, from 2029, are still extremely challenging to manage, despite impressive advances in software performance. In nuclear physics, the FAIR facility will greatly increase data rates and computing needs, but there is also an issue to effectively support many smaller experiments and to find common solutions for that. Then, in astroparticle physics, the challenge is to generalise access and give scientists data access from many observatories in the era of high alert rates and multi-messenger astronomy.
All of these challenges must be addressed in an evolving landscape of European computing. The ESCAPE project has done good work to provide a general toolkit of data management, consisting of common components such as Rucio, FTS, CVMFS, which first saw light in HEP. Adopting FAIR (findable, accessible, interoperable, and reusable) principles requires technical support and cultural evolution, so we need to reward and, eventually, require such practices for our data lifecycles. Europe also invests heavily into high-performance computing, through the EuroHPC project. These HPC centres heavily favour GPU computing. This is no accident, given the evolution towards more and more parallel processing as Moore’s Law continues to hold (for now!), but where the processing power of a single core improves only marginally. That processing model is not traditionally used in ENA areas, and it is hard to adapt to without significant investment. This heterogeneous landscape for software looks inevitable, and the challenge is to adapt our codes sustainably, given the complex landscape of different vendor offerings. HEP already has many success stories here (LHCb, CMS and ALICE all use GPUs in Run-3), but do we have the support and resources to continue this progress?
Then, for Machine Learning and Artificial Intelligence, which are almost ubiquitous in analysis, do we understand the true resource needs and implications arising from the shift towards ML? And can we define the interfaces physicists will need to run ML workloads? This will again require working with our HPC and WLCG partners and ensuring that the needs of physicists are well satisfied across facilities.
The JENA initiative has launched 5 working groups now that will grapple with these details - stressing both our common needs and our important differences. Reporting by the next JENA Symposium in 2025 this will help the community, and the funding agencies, to direct resources in a coherent way to ensure that software and computing remain a success story for all of our communities.