Skip to content

Compiling FHI-aims (Parallel Version)

In case of errors during the build process

If you encounter any unexpected behavior/errors during the CMake build, please consider deleting the content of your current build directory and restarting the CMake build from scratch. This has helped in most cases.

Prerequisites

To start compiling FHI-aims, you need the following setup:

  1. A working MPI Fortran compiler (e.g. mpiifort (intel), mpif90 (gnu))
  2. A working C compiler (e.g. icc (intel), gcc (gnu))
  3. A working C++ compiler (e.g. icpc (intel), g++ (gnu))
  4. BLAS, LAPACK, MPI (MPI 3 standard), ScaLAPACK, BLACS libraries (please read note below)
  5. A version of CMake (>= v3.1) and Make

Note to libraries (point 3 above): If you are installing FHI-aims on a cluster, all these libraries might already be installed. If you have Intel on your cluster installed, the libraries are collected in the impi and mkl modules. If you are installing FHI-aims on your local machine, you usually have to install them on your own before.

For a straightforward way to obtain all the prerequisites inside a conda environment, see this FHI-aims wiki page.

Minimal Example

Firstly, we'll introduce the basic flags needed to specify building FHI-aims. The following can serve as a template for your FHI-aims build.

Important: Do not blindly copy-and-paste the lines. Some of them will explicitly need your customization!

1. Create a Build Folder

We will first create a build folder. Please go to the FHI-aims root directory (that is, FHIaims when you have cloned it from the GitLab, or, fhi-aims.<version-number> when you have downloaded from the aimsclub). Here, create a build folder and navigate into it:

mkdir build; cd build

2. Create the initial.cmake File

Please create a new file called initial.cmake in this build folder. Populate it with content similar to this:

###############
# Basic Flags
###############
set(CMAKE_Fortran_COMPILER <your_fortran_compiler> CACHE STRING "" FORCE)
set(CMAKE_Fortran_FLAGS "<your_compiler_flags>" CACHE STRING "" FORCE)
set(Fortran_MIN_FLAGS "<your_compiler_min_flags>" CACHE STRING "" FORCE)
set(LIB_PATHS "<your_list_to_library_paths>" CACHE STRING "" FORCE)
set(LIBS "<your_list_of_libraries>" CACHE STRING "" FORCE)

###########
# C Flags 
###########
set(CMAKE_C_COMPILER <your_c_compiler> CACHE STRING "" FORCE)
set(CMAKE_C_FLAGS "<your_c_compiler_flags>" CACHE STRING "" FORCE)
set(CMAKE_CXX_COMPILER <your_c++_compiler> CACHE STRING "" FORCE)
set(CMAKE_CXX_FLAGS "<your_c++_compiler_flags>" CACHE STRING "" FORCE)

This file provides the minimal configuration. Remeber, this is just a template! You need to substitute the content marked with <your_something> with keywords specific to your platform and setup (please also remove the < and > symbols). In the table below, you'll find two examples of commonly-used setups: intel refers to a setup, where the mkl and impi libraries are installed (e.g. on a cluster). gnu refers to a gnu compiler setup (gfortran and gcc) together with OpenMPI (so you have mpif90 and mpicc available).

From our experience, Intel's Fortran compiler and MPI library produce significantly faster code and execution on Intel-based platforms. If code execution speed is crucial (as it usually is), we strongly recommend testing the speed of execution - the difference can be large. Mixing compilers, such as using Intel Fortran together with gcc instead of icc, is possible and sometimes even preferable.

Two common compiler setups
CMake Flag Explanation intel gnu
CMAKE_Fortran_COMPILER Fortran Compiler mpiifort mpif90
CMAKE_Fortran_FLAGS Fortran Compiler Flags -O3 -fp-model precise -O3 -ffree-line-length-none
Fortran_MIN_FLAGS Minimal Fortran Compiler Flags for non-optimized source files -O0 -fp-model precise -O0 -ffree-line-length-none
LIB_PATHS List of paths to your libraries $ENV{MKLROOT}/lib/intel64 ~/.local/lib
LIBS List of libraries to include from, which can be found in the LIB_PATHS mkl_intel_lp64 mkl_sequential mkl_core mkl_blacs_intelmpi_lp64 mkl_scalapack_lp64 reflapack scalapack refblas tmg
CMAKE_C_COMPILER C compiler icc gcc
CMAKE_CXX_COMPILER C++ compiler icpc g++
CMAKE_C_FLAGS C compiler flags -O3 -ip -fp-model precise -std=gnu99 -O3
CMAKE_CXX_FLAGS C++ compiler flags -O3 -ip -fp-model precise -O3

For a comprehensive list of all available CMake variables within FHI-aims, we refer to the FHI-aims manual, accessible at fhi-aims.org, or within the FHIaims/doc folder.

Usage of environment variables

As you can see from the intel example in the LIB_PATHS row, you are also able to use environment variables via $ENV{VARIABLE}.

The FORCE flag in the set() command

The FORCE flag instructs CMake to override existing entries. We recommend it as default. Without it, CMake will not change entries, once they have been initialized. This means in particular if you change some of the variables in the initial.cmake file and re-run cmake -C initial.cmake .. in the same folder, CMake still will use the previously cached variables. This can lead to some confusion. If you have not set FORCE and you want to change some of the variables, you have a few options:

  1. Use the CMake CLI (cmake -DKeyword=Value ..) to change the cached variables.
  2. Use ccmake to edit the cached variables.
  3. Use a new directory.
  4. Delete the whole directory (except the initial_cache.cmake file).
FHI-aims for high-performance computing

In addition to the flags presented in the minimal example, you should definitely choose the appropriate ELPA2 kernel for your computer, which often leads to a non-trivial speedup. Detailed instructions on how to choose this kernel, are given in the section ELPA Kernels.

Creating custom FHI-aims binary names

By default, FHI-aims builds binaries named "aims.\<version>.scalapack.mpi.x" or similar, where \<version> is the current version stamp, usually a date. We recommend to keep it that way since this will avoid any accidental overwriting of older binaries with newer ones (and making it much more difficult to reproduce earlier results with the exact same code binary as before). However, if you wish to change the binary name, for instance to aims.x, simply add this variable to initial.cmake:

set(TARGET_NAME aims.x CACHE STRING "")

3. Initialize the CMake Cache

After you have customized your initial.cmake file in the build directory, you can let CMake initialize the cache variables. To do that, input the following in the terminal:

cmake -C initial.cmake ..

Please use the capital C in -C and the trailing .. should point to the FHI-aims root directory (usually FHIaims).

If everything was initialized correctly, your terminal should display output similar to the following:

Example CMake output
loading initial cache file initial_cache.cmake
-- FHI-aims version: 230905
-- The Fortran compiler identification is Intel 20.2.2.20210228
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mpi/2021.2.0/bin/mpiifort - skipped
-- Checking whether /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mpi/2021.2.0/bin/mpiifort supports Fortran 90
-- Checking whether /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mpi/2021.2.0/bin/mpiifort supports Fortran 90 - yes
-- The C compiler identification is Intel 20.2.2.20210228
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /mpcdf/soft/SLE_15/packages/x86_64/intel/2021.2.0/bin/icc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is Intel 20.2.2.20210228
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /mpcdf/soft/SLE_15/packages/x86_64/intel/2021.2.0/bin/icpc - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mkl/latest/lib/intel64/libmkl_intel_lp64.so
-- Found /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mkl/latest/lib/intel64/libmkl_sequential.so
-- Found /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mkl/latest/lib/intel64/libmkl_core.so
-- Found /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mkl/latest/lib/intel64/libmkl_blacs_intelmpi_lp64.so
-- Found /mpcdf/soft/SLE_15/packages/x86_64/intel_oneapi/2021.2/mkl/latest/lib/intel64/libmkl_scalapack_lp64.so
-- Found Git: /usr/bin/git (found version "2.35.3")
-- Update Submodules
-- Enabling internal ELPA
-- Using ELPA generic kernel
-- Enabling internal libOMM
-- Enabling internal NTPoly
-- Setting option BUILD_TESTING: OFF
-- Setting (unspecified) option BUILD_FPIC: ON
-- Setting (unspecified) option NAMESPACE_INSTALL_INCLUDEDIR: /
-- Setting (unspecified) option ENABLE_GENERIC: OFF
-- Setting option ENABLE_FORTRAN: ON
-- Setting (unspecified) option ENABLE_PYTHON: OFF
-- Setting (unspecified) option DISABLE_VXC: OFF
-- Setting (unspecified) option DISABLE_FXC: OFF
-- Setting (unspecified) option DISABLE_KXC: ON
-- Setting (unspecified) option DISABLE_LXC: ON
-- Setting (unspecified) option DISABLE_FHC: OFF
-- Performing Test standard_math_library_linked_to_automatically
-- Performing Test standard_math_library_linked_to_automatically - Success
Test program linked without flags
-- Performing Test HAVE_CBRT
-- Performing Test HAVE_CBRT - Success
-- Version: Full 6.1.0
-- SO Version: Full 13:0:1 Major 12
-- Configuring done
-- Generating done
-- Build files have been written to: ~/git_aims/FHIaims/build

4. Start the Make Build

After successfully initializing the CMake cache, CMake has produced a Makefile. You can now proceed with the actual build process using:

make -j 4

In the above line, we use 4 tasks to build FHI-aims, however, you could use less or more tasks depending on your platform. We recommend using the actual available numbers of cores on your machine for a fast compilation.

After make has successfully finished the build process, you should find the executable within the build folder.

Recompiling with CMake

If you want to recompile after you made some changes to the code, simply type make -j <numberOfTasks>.

Warning

If you need to change compiler flags or other CMake variables make sure you initially have set set(<...> FORCE) for each variable. If not, delete the whole content of the build directory to start fresh.

Quick Summary

  1. Create build folder: mkdir build; cd build
  2. Create initial.cmake file and fill with content
  3. Initialize CMake cache: cmake -C initial.cmake ../.
  4. Start the make build: make -j <N>, with number of cores <N>

initial.cmake files for specific platforms and builds

A collection of working initial.cmake for specific platforms can be found on the FHI-aims wiki page: Known compiler options

Moreover, in the FHI-aims repository itself, you can find sample initial.cmake files for various builds. Navigate to the FHIaims/cmake/toolchains directory and explore the available files.