What is OpenHPC?

What is OpenHPC?

Community-based effort provides tools, documentation, and more to accelerate innovation and broaden access to HPC.

What is OpenHPC?
Image by : 

opensource.com

Get the newsletter

Join the 85,000 open source advocates who receive our giveaway alerts and article roundups.

High performance computing (HPC)—the aggregation of computers into clusters to increase computing speed and power—relies heavily on the software that connects and manages the various nodes in the cluster. Linux is the dominant HPC operating system, and many HPC sites expand upon the operating system's capabilities with different scientific applications, libraries, and other tools.

As HPC began developing, that there was considerable duplication and redundancy among the HPC sites compiling HPC software became apparent, and sometimes dependencies between the different software components made installations cumbersome. The OpenHPC project was created in response to these issues. OpenHPC is a community-based effort to solve common tasks in HPC environments by providing documentation and building blocks that can be combined by HPC sites according to their needs. 

The project's mission and vision are:

Mission: To provide a reference collection of open source HPC software components and best practices, lowering barriers to deployment, advancement, and use of modern HPC methods and tools.

Vision: OpenHPC components and best practices will enable and accelerate innovation and discoveries by broadening access to state-of-the-art, open source HPC methods and tools in a consistent environment, supported by a collaborative, worldwide community of HPC users, developers, researchers, administrators, and vendors.

OpenHPC history

OpenHPC started at International Supercomputing 2015 (ISC'15) with a birds of a feather (BoF) discussion on the merits of a community effort towards a HPC software repository and management framework. At Supercomputing 2015 (SC'15), OpenHPC 1.0 was released and the Linux Foundation announced the Open HPC Collaborative Project's technical leadership, founding members, and formal governance structure.

Since then, the growing number of OpenHPC project members have collaborated on regular updates of OpenHPC's software packages and documentation, with v1.1.1 released in June 2016 at ISC'16. The newest, v1.3.2, was released in September 2017.

What is OpenHPC?

OpenHPC provides a collection of prebuilt software components common in HPC environments, such as provisioning tools, resource managers, I/O libraries, development tools, and a variety of scientific libraries. It supports different Linux distributions and architectures and provides a package repository for CentOS 7 and SUSE Linux Enterprise Server (SLES) 12.

One key tenet of the project is that no one is ever required to install the entire OpenHPC stack; rather, users can pick and choose the components that are relevant to their HPC needs.

OpenHPC v1.2 and later include builds for x86_64 and aarch64 (tech preview); v.1.3.2 adds CentOS 7.3 and SLES 12 SP2 builds. In addition to the provided software packages, OpenHPC includes installation guides for different provisioning systems (Warewulf, xCAT) as well as different resource managers (Slurm, PBS Professional).

The software and documentation are maintained by the OpenHPC project members and external collaborators in a Git repository hosted at GitHub. To ensure that the software and documentation works as described, OpenHPC has a continuous integration system that verifies the software using bare metal installs and integration test suites.

OpenHPC takes into consideration that HPC sites often rely on multiple compilers that may have partially compatible or even incompatible Application Binary Interfaces (ABIs) and Message Passing Interfaces (MPIs). With the adoption of hierarchical build configurations, OpenHPC delivers unique builds of software packages for each desired compiler/MPI permutation. Software packages that have specific compiler and MPI dependencies have those added to the package name in the following manner:

package-<compiler>-<mpi>-ohpc

By using environment modules to load the corresponding compiler and MPI family, the environment variables will point to the user's compiler and MPI choices.

Supported compiler families are different versions of the GNU Compiler Collection or the Intel compiler, and supported MPI families are Open MPI, MPICH, MVAPICH2, and Intel MPI. Using those options, the resulting package name could be:

fftw-gnu7-openmpi-ohpc

OpenHPC software packages

OpenHPC divides its software packages into different functional areas, and each includes multiple components:

  • Provisioning: Warewulf, xCAT
  • Resource management: Slurm, Munge, PBS Professional
  • Runtimes: OpenMP, OCR, Singularity
  • Numerical/scientific libraries: GSL, FFTW, Metis, PETSc, Trilinos, Hypre, SuperLU, Mumps, OpenBLAS, Scalapack
  • I/O libraries: HDF5, NETCDF, Adios
  • Compilers: GCC, LLVM
  • MPI families: MVAPICH2, Open MPI, MPICH
  • Performance tools: PAPI, IMB, mpiP, pdtoolkit, TAU, Scalasca, ScoreP, SIONLib

The next release (1.3.3) is planned for November 2017. As with every new release, new packages will be added and existing packages will be updated. Latest version can always be obtained from openhpc.community/downloads/.

Additional resources

OpenHPC is a collaborative, community effort that is open for contributions. If you would like to learn more about OpenHPC and how you can contribute, please visit the following resources:

About the author

Adrian Reber - Adrian is a Senior Software Engineer at Red Hat and is migrating processes at least since 2010. He started to migrate processes in a high performance computing environment and at some point he migrated so many processes that he got a PhD for that and since he joined Red Hat he started to migrate containers. Occasionally he still migrates single processes and is still interested in high performance computing topics.