Linux Containers

Practical Guidelines for Scientific Computing Applications



Technologies based on linux containers have become very popular among software developers. The main reason behind this success is the flexibility containers offer when it comes to deploy software.

This section contains basic information on the concept on containers, and tools to support the usage of Linux containers in scientific computing infrastructures.


On Linux Containers

Some links to appear (TBC)


Using containers on Scientific Multiuser Environments

At present the usage of containers is very limited in multiuser environments such as High Performance Computing mainframes, Linux Clusters or Grid infrastructures.

The main limitation is that execution of containers using Docker or LXC requires having root access privileges in the host system. A tool that supports the execution of containers in user space is udocker.

Detailed information on udocker, howtos & downloading page is in our github repository here.

A practical guide containing the instructions to work with a real scientific application in a Linux cluster is to be found in our subsection dedicated to udocker.


Using a Scientific Computing Linux container on Mac OS

Mac OS does present many problems for scientific code developers. One needs to rely on the installation of complex packages (complex both, to install and to maintain) such as Fink or Darwin Libraries,... in order to have compilers or mathematical libraries available. Unfortunately such adapted libraries, often, do not comply with standards either.

Many problems have been detected, related with the usage of system reserved variables on GNU compilers, wrong compilation messages, or violations of the MPI standard in the MPI libraries available for Mac OS.

Thus the ideal situation would be having the well-known and standard Linux scientific utilities available under Mac OS.

Containers technology offers an elegant solution to this problem via the package "Docker for Mac", which is based on xhyve, a tool to run virtual machines on user space, based on a Hypervisor from BSD. But you do not need to know any of this. Just follow the instructions.

How to deploy a Centos7-based container on your Mac

This solution works under the following conditions:

If you meet all three conditions, follow these steps:

  1. Install Docker for Mac from this link
  2. Start the Application by clicking on the new Docker Whale icon
  3. The icon will appear on the top bar of your Mac like this:

    Installing Docker for Mac

  4. Open a new terminal and download an suitable container
  5. Note: if you are a member of the Mastercode collaboration follow this link from this point on for concrete instructions to download and run the Mastercode container.

    A generic scientific computing container is docker.io/icampos/centosscicomp, which is available in the docker hub repository.

    You may download it with the following command on the terminal:

    docker pull iscampos/centosscicomp

    Downloading container

    It will use about 1GB of your disk space.

    This container is based on Centos7, and has been enhanced with scientific computing tools (compilers, Math libraries, texlive, etc...). In the Dockerfile you can see what is actually installed.

    For convenience there is a dummy user ("developer") created in the container.

  6. Launch the container mounting your home directory on /mnt
  7. docker run -ti --user developer -w /home/developer -v /Users/yourusername:/mnt iscampos/centosscicomp /bin/bash

  8. You should then enter in the container with a bash shell, directly into /home/developer (-w option). You will find your $HOME directory in /mnt.

    Running container

    Type

    $ cat /etc/centos-release

    and convince yourself that you are now on Linux.

    Typing

    $ ls /mnt

    you will see your home directory.

    To close the container session, just type

    $ exit

    on the terminal.

  9. Remarks (FAQ):
  10. In the typical situation you will be working on your local files, and use the Linux software of the container for those task for which Mac OS software is not suitable: compiling with GNU compilers, running codes, etc...

    In this respect:

Last updated 28th November 2016