Overview

R&D 100 2018 winner logo

What is Charliecloud?

Charliecloud provides user-defined software stacks (UDSS) for high-performance computing (HPC) centers. This “bring your own software stack” functionality addresses needs such as:

  • software dependencies that are numerous, complex, unusual, differently configured, or simply newer/older than what the center provides;

  • build-time requirements unavailable within the center, such as relatively unfettered internet access;

  • validated software stacks and configuration to meet the standards of a particular field of inquiry;

  • portability of environments between resources, including workstations and other test and development system not managed by the center;

  • consistent environments, even archivally so, that can be easily, reliably, and verifiably reproduced in the future; and/or

  • usability and comprehensibility.

How does it work?

Charliecloud uses Linux user namespaces to run containers with no privileged operations or daemons and minimal configuration changes on center resources. This simple approach avoids most security risks while maintaining access to the performance and functionality already on offer.

Container images can be built using Docker or anything else that can generate a standard Linux filesystem tree.

How do I learn more?

Who is responsible?

Contributors:

How can I participate?

Use our GitHub page: https://github.com/hpc/charliecloud

Bug reports and feature requests should be filed as “Issues”. Questions, comments, support requests, and everything else should use our “Discussions”. Don’t worry if you put something in the wrong place; we’ll be more than happy to help regardless.

We also have a mailing list for announcements: https://groups.io/g/charliecloud

Patches are much appreciated on the software itself as well as documentation. Optionally, please include in your first patch a credit for yourself in the list above.

We are friendly and welcoming of diversity on all dimensions.

Technical publications

If Charliecloud helped your research, or it was useful to you in any other context where bibliographic citations are appropriate, please cite the following open-access paper:

Reid Priedhorsky and Tim Randles. “Charliecloud: Unprivileged containers for user-defined software stacks in HPC”, 2017. In Proc. Supercomputing. DOI: 10.1145/3126908.3126925.

Note: This paper contains out-of-date number for the size of Charliecloud’s code. Please instead use the current number in the FAQ.

Other publications:

  • We compare the performance of three HPC-specific container technologies against bare metal, finding no concerns about performance degradation.

    Alfred Torrez, Tim Randles, and Reid Priedhorsky. “HPC container runtimes have minimal or no performance impact”, 2019. In Proc. CANOPIE HPC Workshop @ SC. DOI: 10.1109/CANOPIE-HPC49598.2019.00010.

  • A demonstration of how low-privilege containers solve increasing demand for software flexibility.

    Reid Priedhorsky, R. Shane Canon, Timothy Randles, and Andrew J. Younge. “Minimizing privilege for building HPC containers”, 2021. In Proc. Supercomputing. DOI: 10.6084/m9.figshare.14396099.

  • Charliecloud’s build cache performs competitively with the standard many-layered union filesystem approach and has structural advantages including a better diff format, lower cache overhead, and better file de-duplication.

    Reid Priedhorsky, Jordan Ogas, Claude H. (Rusty) Davis IV, Z. Noah Hounshel, Ashlyn Lee, Benjamin Stormer, and R. Shane Goff. “Charliecloud’s layer-free, Git-based container build cache”, 2023. In Proc. Supercomputing. DOI: 10.1145/3624062.3624585.