Difference between revisions of "BGQ"

From Mpich
Jump to: navigation, search
(Blue Gene/Q build instructions)
Line 9: Line 9:
  
 
The configure is simpler when specifying the $PATH, however the generated mpi compile scripts such as ''${prefix}/bin/mpicc'' will also not contain the path information to the cross compiler. This means users of ''mpicc'' must also have the cross compiler in their $PATH for the compile script to work.
 
The configure is simpler when specifying the $PATH, however the generated mpi compile scripts such as ''${prefix}/bin/mpicc'' will also not contain the path information to the cross compiler. This means users of ''mpicc'' must also have the cross compiler in their $PATH for the compile script to work.
 +
 +
If one wishes to build MPICH with the XL compilers, use the environment variable approach.
 +
 
=== Required ===
 
=== Required ===
 
Specify the bgq cross compile and pamid device
 
Specify the bgq cross compile and pamid device

Revision as of 14:16, 10 July 2013

This page describes how to build mpich from the master branch of the mpich.git repository on git.mpich.org.

Blue Gene/Q build instructions

The bgq toolchain must be in the $PATH before configure, otherwise the gnu cross-compiler will not be found.

export PATH=$PATH:/bgsys/drivers/V1R2M0/ppc64/gnu-linux/bin

The alternative is to provide the location of all compiler binaries using environment variables. For example,

CC=/bgsys/drivers/V1R2M0/ppc64/gnu-linux/bin/powerpc64-bgq-linux-gcc

The configure is simpler when specifying the $PATH, however the generated mpi compile scripts such as ${prefix}/bin/mpicc will also not contain the path information to the cross compiler. This means users of mpicc must also have the cross compiler in their $PATH for the compile script to work.

If one wishes to build MPICH with the XL compilers, use the environment variable approach.

Required

Specify the bgq cross compile and pamid device

--host=powerpc64-bgq-linux
--with-device=pamid

Customize the ROMIO file system.

--with-file-system=bg+bglockless

Optional

Customize the required bgq system software libraries

The latest installed bgq system software is used by default. The location of the bgq system software can also be specified with the configure option below or the BGQ_INSTALL_DIR environment variable.

--with-bgq-install-dir=/bgsys/drivers/V1R2M0/ppc64

A pami installation outside of the bgq system software directory may be specified using the --with-pami configure option(s). For example:

--with-pami=/bgsys/drivers/V1R2M0/ppc64/comm/sys
--with-pami-include=/bgsys/drivers/V1R2M0/ppc64/comm/sys/include
--with-pami-lib=/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib

Customize the bgq cross compile settings

A different cross compile settings file for bgq pamid can be specified using the --with-cross-file configure option. Below is the configure option that specifies what is the default cross file for a bgq pamid configuration.

--with-cross-file=src/mpid/pamid/cross/bgq8

Disable rpath

When shared libraries are installed it is recommended to also disable the "wrapper rpath" configure option in order to take advantage of a shared library load optimization on the bgq io nodes.

--disable-wrapper-rpath

When a million processes each individually read from the filesystem the performance of the shared library load will be poor. The io node shared library optimization is a way to "stage" shared libraries on a bgq io node ramfs directory that is, in the absence of rpath information, searched first by the bgq loader. Any rpath information will be searched before this io node ramfs location and will result in a query all the way down to the filesystem.

Shared libraries can be added to the io node ramfs directory by packaging the libraries into a *.tar.gz file and copying that file into the /bgsys/linux/bgfs directory.

Enable common "no debug" and "performance" options

The xl.ndebug and xl.legacy.ndebug mpich versions installed with the bgq system software use the following options to eliminate debug and other error checks that would cause performance degradations.

--enable-fast=nochkmsg,notiming,O3
--with-assert-level=0
--disable-error-messages
--disable-debuginfo

Enable fine grain locking

The gcc, xl, and xl.ndebug mpich versions installed with the bgq system software use the following options to enable fine grain locking and synchronous progress mode.

--enable-thread-cs=per-object
--with-atomic-primitives
--enable-handle-allocation=tls
--enable-refcount=lock-free
--disable-predefined-refcount