CERN Accelerating science

CernVM 2019 Workshop

by Jakob Blomer (CERN)

From 3rd June to 5th June, the 2019 CernVM Workshop took place. As about every 18 months, the CernVM workshop brings together software developers from EP-SFT, computing experts from the LHC experiments, WLCG system operators, and invited speakers from the IT industry. More than 40 participants discussed the state and future directions of the CernVM File System (CernVM-FS) and the CernVM Virtual Appliance. Frédéric Hemmer, head of CERN's IT department, kindly provided the workshop welcome note, which underlines CernVM’s strong collaboration between EP-SFT and IT-SM.

Started as a four years R&D project in 2008, the CernVM software ecosystem became a mission critical component for the LHC experiments. It takes care, at LHC scale, of delivering our scientific applications and platforms to the worldwide distributed computing infrastructure and has since been adopted across HEP and beyond, such as by the LIGO and EUCLID collaborations.

For the future development, three themes emerged during the workshop. First, the need for seamless integration of the CernVM-FS technology with industry container ecosystems - a topic already raised in previous events and workshops - was confirmed, with a focus on the technologies Kubernetes, Docker, and Singularity. HEP users will need the application isolation and orchestration capabilities offered by industry standard tools combined with the efficient distribution capabilities of CernVM-FS to effectively deploy complex containerized applications. Second, various previous efforts on accessing supercomputers at HPC centers beared fruit. Targeted developments made it possible to access scientific software on many HPC systems. Thereby the CernVM specific part in harnessing supercomputers is getting under control, in what generally remains a challenging task on many other fronts. Third, there is a continuous need for faster and more convenient ways to publish data in CernVM-FS. This is most acute for the software development teams that currently prepare the experiment frameworks for run 3 upgrades and HL-LHC.

In order to get a fresh look onto the technology landscape, the CernVM workshop has a tradition of inviting speakers from industry and academia on hot computing topics. This year, we were glad to hear from Harris Hancock, software engineer at the U.S. CDN provider Cloudflare, on the state of the art in edge computing and serverless computing. Dorian Krause, head of the HPC systems division at the Jülich Supercomputing Centre, presented the roadmap of future supercomputing hardware. Michel Bauer from the U.S. start-up Sylabs gave insights into the Singularity container engine and sketched out a path for integrating CernVM-FS technology. Jesse Williamson, software developer with the Ubuntu creator Canonical and former CernVM-FS contributor, provided a critical review of application areas for Ceph-FS and CernVM-FS. Prof. Douglas Thain from the University of Notre Dame (Indiana) shared his research on building and managing scalable scientific applications. From the Linux technology company Red Hat, Giuseppe Scrivano completed the technology outlook session with the latest developments on overlay file systems and Linux kernel container mechanics.

Group photo of the invited speakers of the CernVM workshop during their visit to the CMS experiment at P5.

All the invited speakers where given a guided tour to the CMS experiment, and several stayed at CERN for the entire week for further discussions with LHC computing experts. The CernVM team is very grateful to all participants for the very valuable contributions and the fruitful workshop atmosphere! The provided feedback is now due for incorporation in the medium-term development plan.