Skip to content

Quick Start

Jump to bottom
Bob Carpenter edited this page Jul 1, 2020 · 7 revisions

Using the Stan Math Library

To use the Stan math library include exactly one of the following header files.

include contents also includes
stan/math/prim.hpp primitives n/a
stan/math/rev.hpp reverse-mode autodiff prim.hpp
stan/math/fwd.hpp forward-mode autodiff prim.hpp
stan/math/mix.hpp mixed-mode autodiff prim.hpp, fwd.hpp, rev.hpp

These top-level header files ensure that all necessary traits files are included in the appropriate order. For more detail, see

Multi-core processing with MPI

The message passing interface (MPI) allows the exchange of messages between different processes. We can use MPI to parallelize the computation of a single log probability computation by using multiple processes. The target audience for MPI are those with large computer clusters. For users looking for parallel computation on a single computer, please turn to a threading based approach, which is easier to use and provides similar performance gains.

Supported platforms

Stan supports MPI on Mac OS X and Linux. Windows is not supported, though we have heard that some people have successfully used it.

Requirements for MPI

A base MPI installation must be installed on the system. See the instructions from boost.mpi to verify that there is a working MPI system. For Mac OS X and Linux, any MPI installation that works with boost.mpi is supported. The two major open-source base MPI implementations are mpich and openMPI. The Math library is tested with these two implementations while others supported by boost.mpi may work as well. The base MPI installation provides the command line tools

  • mpicxx: The recommended compiler command to use when building any MPI application.
  • mpirun: Wrapper binary used to start a MPI enabled binary on a given machine.

Please ask your system administrator for details on how to compile, execute, and submit MPI applications on a cluster.

Installation on Mac OS X

Install mpich from MacPorts or homebrew.

Installation on Linux

The package distribution system on your version of linux should have pre-built mpich (or openmpi) available. In addition to that, you must have the following packages installed (Ubuntu package names listed): python-dev, libxml2-dev, libxslt-dev, and you may be required to add the following to your make/local: LDLIBS+=-lpthread.

Boost configuration and build

Stan builds it's own boost.mpi and boost.serialization libraries and installs these into its library subfolder. If the operating system provides these Boost libraries and it's required to use them, there is additional configuration that needs to be done (through make/local) to use that installation. The boost libraries are built using the boost build system. Boost build will attempt to auto-detect the MPI installation specifics on your system and the toolset to use. Should Boost's auto-detect fail or a specific configuration be required, then users can configure the boost build system through the configuration file stan-math/lib/boost_1.xx.x/user_config.jam manually as needed.

Recommended compiler tools

We strongly recommend to use the mpicxx command to build any program using MPI within Math. While it is possible to change the compiler used with these commands (openMPI has a -cxx= option, for example), this can only be done with great caution. The complication is that during compilation of the base MPI libraries the exact bit representation of each type is analyzed and strong deviations due to compiler changes may lead to unexpected behavior. In case of compiler mismatch between the base MPI libraries and boost.mpi (and Math) changes in the compiler ABI can lead to unexpected segfaults. Therefore, we recommend to use the mpicxx as compiler and do not recommend to deviate from the compiler used to build MPI. Often this means to use the system default compiler which may be rather old and not ideal for Stan. In such cases a more modern gcc (if gcc is the system compiler) can be considered as long as no ABI changes are known.

Setting up the Math library with MPI

Stan uses the boost.mpi library to interface with the installed MPI implementation. boost.mpi is built automatically by the Math library when the Math library is configured for MPI. To configure MPI for the Math library,

  1. Ensure that a base MPI installation is available and accessible on the system. See Requirements.
  2. Open a text file called make/local; if it does not exist, create one.
  3. Add these lines to the make/local file:
STAN_MPI=true
CXX=mpicxx
TBB_CXX_TYPE=gcc
  1. Optional: instead of using CXX=mpicxx, the user can specify the compiler with the proper compiler and linker options needed to build an MPI enabled binary (the command mpicxx -show displays for mpich what is executed / openmpi uses mpicxx -show-me), but please read the note on compilers above.
  2. Clean all binaries. After changing configuration through make/local, all the tests should be rebuilt. Please type:
make clean-all

Once the Math library is configured for MPI, the tests will be built with MPI. Note that the boost.mpi and boost.serialization library are build and linked against dynamically.

Home | Users | Developers

[Home]

Clone this wiki locally