Tutorial: FHI-aims for Large Scale Simulations
This tutorial introduces helpful concepts and techniques for large scale simulations with FHI-aims. By "large scale", we mean simulations that either cover large numbers of atoms or electrons in the structure, or simulations that require a large, complex supercomputer for execution due to runtime and memory demands.
FHI-aims has a long history of enabling large, massively parallel simulations of complex systems on supercomputers. For sufficiently large problems, the code can use ten thousands of processor cores efficiently. In density-functional theory based simulations, the simulations sizes that we can reach in typical production simulations are of the order of several thousands of atoms. In fact, more would be possible, except that "thousands of atoms" appears to be a practical limit for system sizes for which electronic structure calculations generally make sense.
The real challenge of large-scale simulations often is not FHI-aims itself. As noted above, large simulations can be carried out on large supercomputers. In our experience, it is often the complexity of such computers that poses significant challenges (command line operation, operating system environment, queueing system, available compilers and libraries, etc.)
Much of this tutorial therefore focuses on explaining how to ensure that FHI-aims functions correctly on such a computer, and with acceptable performance.
Objectives of This Tutorial
This tutorial introduces strategies for carrying out large-scale simulations with FHI-aims on supercomputers. Specifically, we address:
- The environment in which FHI-aims is expected to run
- Standard benchmarks that demonstrate that the code operates correctly
- Memory efficiency
- Keywords that can be helpful (or essential) in large-scale simulations
Prerequisites
-
A sufficient understanding about the basics of running FHI-aims is required. Please review our tutorial Basics of Running FHI-aims tutorial if you have not yet done so and/or if you are unfamiliar with the code.
-
A sufficiently large computer. The exercises shown here do not lend themselves to execution on a laptop or in a virtual machine. If you do not have a sufficiently large computer available yet, please follow through the tutorials based on the solutions provided. For many, this tutorial may become truly relevant once they set up FHI-aims on a new supercomputer of your choice.
-
The FHI-aims code distribution must be present on your machine.
The complete set of input and output files of this tutorial are available at https://gitlab.com/FHI-aims-club/tutorials/scaling-in-fhi-aims, which can be easily obtained by cloning this repository onto your own computer.
Useful links
The following links contain useful information and tools regarding this tutorial and FHI-aims in general:
- The present gitlab repository, which contains all the documents and simulation data for this tutorial: https://gitlab.com/FHI-aims-club/tutorials/scaling-in-fhi-aims
- The CLIMS gitlab repository, which contains useful utilities that can be used alongside with FHI-aims to facilitate input files preparation and/or output data analysis, in particular CLIMS: https://gitlab.com/FHI-aims-club/utilities
- FHI-aims browser-based Graphical Interface for Materials Simulation (GIMS): https://gims.ms1p.org
Outline
- Introduction
- Part 1: Preparing the computer environment
- Part 2: Small benchmark: Ac-Ala19-LysH+
- Part 3: Large benchmark: Graphene on SiC(0001)
- Summary