[ Home | Overview | Publications ] • [ Examples ] • [ Software/Downloads | Documentation/Questions ] • [ People | Acks ]
[Contents]
To reduce our maintenance efforts, we have opted to distribute HPCToolkit primarily in source form. The following prerequisites apply to our default source code distribution.
HPCToolkit can be installed on the following platforms:
Platform (Supported) | Notes |
---|---|
Linux + x86_64 | |
Linux + x86 (32-bit) | |
Linux + ppc64 | |
Cray XT/XE/XK/XC (CNL + x86_64) | Compute Node Linux ≥ 2.1 |
IBM Blue Gene/Q (BG/Q kernel + ppc64) | |
IBM Blue Gene/P (BG/P kernel + ppc32) | BG/P kernel ≥ V1R3M0 |
Platform (Legacy and Not Supported) | |
Linux + IA64 | testing (uses libunwind) |
Linux + MIPS-le (64 and n32) | for SiCortex clusters; little endian MIPS (no longer tested) |
Compiler: HPCToolkit requires GNU GCC version 4.4 or later. Some vendor compilers that support GNU extensions may also work, but we recommend building with GNU gcc/g++. HPCToolkit also requires GNU make, preferably version 3.81 or later.
To use hpcviewer and hpctraceviewer (HPCToolkit's graphical interface) on MacOS, Windows, or Linux, a Java runtime environment (JRE) ≥ 1.6 is required. Use the Oracle or IBM version; we have had problems with OpenJDK.
To use hpcviewer and hpctraceviewer on Linux, GTK+ is also required.
Note: Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections.
Highly recommended: To collect measurements based on hardware performance monitoring unit (PMU) information, PAPI ≥ 4.1 is required. Without PAPI, HPCToolkit can only collect measurements based on an operating system timer. (Exception: On IBM Blue Gene/P, PAPI is not required. On this system, HPCToolkit uses IBM's native Universal Performance Counters directly.)
There are currently three ways to support PAPI on Linux-based OSs:
Use a kernel with perf_events (see also: this), which was introduced with kernel 2.6.32 (née PCL in kernel 2.6.31) and which is now becoming part of standard distributions. With perf_events, use PAPI ≥ 4.1.2.
Apply the perfctr kernel patch (x86_64-based systems only). One benefit of perfctr is that it includes out-of-the-box support for many Red Hat kernels. As of this writing, perfctr supports vanilla kernels through 2.6.32.
Apply the perfmon2 kernel patch. As of this writing, perfmon2 supports vanilla kernels through 2.6.30. (Unfortunately, development has stopped because of perf_events.)
Minor requirement: To use hpcsummary (a relatively minor tool), Python 2.x is required.
Minor requirement: To build HPCToolkit's man pages, a variant of Perl 5 is required. We have successfully used versions 5.6.x - 5.10.x.
[Contents]
Download HPCToolkit components as follows:
Download hpctoolkit from GitHub.
git clone https://github.com/HPCToolkit/hpctoolkit.git
Download hpctoolkit-externals from GitHub.
git clone https://github.com/HPCToolkit/hpctoolkit-externals.git
Think of hpctoolkit-externals as a "package manager" that automatically configures and builds support libraries that hpctoolkit needs. It contains several external libraries. (Some of these libraries, like libelf and libxml2 are commonly found on Linux systems, but not always. Others, like OpenAnalysis and SymtabAPI are almost never available. Finally, a few packages, like GNU binutils have been heavily patched.) In some cases it may be possible to use versions of these packages that are already installed on your system. However, we do not support such configurations.
Download binary releases of hpcviewer and hpctraceviewer for the platform on which your are building HPCToolkit.
Note: It is perfectly reasonable to install hpcviewer and hpctraceviewer in multiple locations. Since HPCToolkit's databases are portable, we often install hpcviewer and hpctraceviewer on local laptops/desktops to avoid sending display traffic over relatively slow network connections from systems where we collect measurement data.
[Contents]
The build and install follows autotools conventions (./configure && make && make install
).
In each case, more options are available by executing ./configure --help
.
See the README file for more details.
The typical build is as follows.
Please note that a list of examples below provides configure
commands for selected platforms.
HPCToolkit's configure command will automatically check the availability of the Linux perf_events interface on the build system.
The test includes checking the existence of /usr/include/linux/perf_event.h
and /proc/sys/kernel/perf_event_paranoid
files. If the two files do not exist, it assumes the system does not support the perf_events interface.
--with-perfmon=<libpfm4_install_directory>
to HPCToolkit's configure
command.
Build and install HPCToolkit's Externals, which typically does not require any special options; we recommend using the GCC compilers:
cd hpctoolkit-externals
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>]
[CXX=<c++-compiler>] \
[--prefix=<hpctoolkit-externals-install>
] make install
make clean
Externals is not persistent as HPCToolkit will copy everything it needs from the installation.
Thus, the installation path <hpctoolkit-externals-install>
should be a local path and not something like /usr
.
If --prefix
is not supplied it will automatically be set to ./<Autoconf-host>
.
To specify a C and C++ compiler other than gcc
and g++
, use CC
and CXX
.
Please be sure the Externals build completes before configuring HPCToolkit. (Note: After building Externals once, HPCToolkit can be configured and built several times using the same Externals installation.)
Build and install HPCToolkit using commands similar to the following; we recommend using the GCC compilers:
cd hpctoolkit
mkdir BUILD && cd BUILD
../configure [CC=<c-compiler>]
[CXX=<c++-compiler>]
[MPICXX=<mpi-c++-compiler>] \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals-install> \
[--with-papi=<papi-install>]
make install
The above commands will build HPCToolkit and install it into <hpctoolkit-install>
.
To specify a C and C++ compiler other than gcc
and g++
, use CC
and CXX
.
To enable MPI support, (1) ensure a working mpicxx
or mpiCC
is in your shell's path; or (2) use MPICXX
.
(Naturally, CXX
and MPICXX
should be compatible; typically MPICXX
is a wrapper for CXX
.)
To enable PAPI-based measurements, use the --with-papi
option; it takes as an argument the path of your PAPI installation, <papi-install>
.
Because the HPCToolkit installation is self-contained, it may be renamed or moved to another location. All HPCToolkit Externals source code, build files and installations may be removed; all HPCToolkit source code and build files may be removed.
Install hpcviewer and hpctraceviewer .
cd <hpcviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
cd <hpctraceviewer> && ./install [ -j <path-to-java> ] <hpctoolkit-install>
<hpcviewer>
and <hpctraceviewer>
are the directories created by unpacking the respective tarballs; and <hpctoolkit-install>
is the above HPCToolkit installation directory.The list below gives example HPCToolkit Externals and HPCToolkit configure
commands (respectively) for selected platforms.
N.B.: These commands are designed to replace the respective commands in the above build instructions.
Blue Gene/Q:
+mpiwrapper-gcc
' softenv package (for the MPICXX
compiler) and check that the back-end powerpc64-bgq-linux
compilers are on your PATH.
Use the '--enable-bgq
' option for HPCToolkit externals
to build for the Blue Gene/Q compute nodes.
soft add +mpiwrapper-gcc
PATH="/bgsys/drivers/ppcfloor/gnu-linux/bin/:$PATH"
../configure --prefix=`pwd`/../powerpc64-linux \
--enable-bgq
../configure \
MPICXX=/bgsys/drivers/ppcfloor/comm/gcc/bin/mpicxx \
HPCPROFMPI_LT_LDFLAGS="-all-static" \
--prefix=<install-path> \
--with-externals=<hpctoolkit-externals>/powerpc64-linux \
--with-papi=/soft/perftools/papi
--enable-bgq
' flag adds options specific
to the Blue Gene/Q compute nodes and suppresses the warning.
If you want to build hpcrun
to run on the front-end nodes,
then use the option '--disable-bgq
'.
MPICXX
and PAPI may be different on your system.Blue Gene/P:
../configure --prefix=`pwd`/../powerpc32-linux
../configure \
HPC_LT_LDFLAGS="-all-static -L../../lib/stubs-gcc_s" \
HPCPROFMPI_LT_LDFLAGS="-L/bgsys/drivers/ppcfloor/gnu-linux/powerpc-bgp-linux/lib -lstdc++" \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/powerpc32-linux
Cray XT/XE/XK/XC:
module swap PrgEnv-pgi PrgEnv-gnu
../configure --prefix=`pwd`/../x86_64-linux
../configure \
MPICXX="CC" \
HPCPROFMPI_LT_LDFLAGS="-all-static" \
--prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/x86_64-linux \
--with-papi=/opt/cray/papi/4.3.0.1/perf_events/no-cuda
--enable-xop
'.
The AMD Interlagos CPUs (6200 series) use these instructions.
Currently, Jaguar at ORNL uses these CPUs, but Hopper at NERSC does not.
Intel MIC (early and experimental):
The solution is to build all of externals and hpctoolkit twice:
once for the main cores and once for the MIC cores.
Install the builds into two separate install trees (prefixes).
Both externals and hpctoolkit support VPATH builds, so you only
need to checkout one copy of each source tree.
Let's say that you use BUILD-x86_64
and BUILD-k1om
for the build directories and install-x86_64
and
install-k1om
for the install directories.
cd BUILD-x86_64
../configure --prefix=/path/to/externals/install-x86_64 \
--enable-mic
PATH="/usr/linux-k1om-4.7/bin/:$PATH"
cd BUILD-k1om
../configure --prefix=/path/to/externals/install-k1om \
--host=x86_64-k1om-linux \
CC=x86_64-k1om-linux-gcc \
CXX=x86_64-k1om-linux-g++
cd BUILD-x86_64
../configure --prefix=/path/to/hpctoolkit/install-x86_64 \
--with-externals=/path/to/externals/install-x86_64 \
--with-papi=/path/to/papi-install
PATH="/usr/linux-k1om-4.7/bin/:$PATH"
cd BUILD-k1om
../configure --prefix=/path/to/hpctoolkit/install-k1om \
--with-externals=/path/to/externals/install-k1om \
--host=x86_64-k1om-linux \
CC=x86_64-k1om-linux-gcc \
CXX=x86_64-k1om-linux-g++
--with-papi=/path/to/papi-k1om
to the hpctoolkit configure
for MIC.
Make sure that this version of PAPI is built for the MIC cores.
hpcprof-mpi
is untested for the MIC cores.
Be sure to use the option --enable-mic
in the
externals's build for the main cores.
This option selects a version of XED that runs on the main (x86_64)
cores but understands the MIC (k1om) instructions.
(This option is ignored for the k1om build.)
Without this option, hpcstruct
will crash on MIC binaries.
Unfortunately, this version of XED is new and currently available
only as a shared library, thus making it incompatible with
hpclink
and the static case (why the default is no).
If you need to run static binaries on Stampede with
hpclink
, then build a second version of hpctoolkit
for the main cores without this option.
You can build a restricted set of tools with the
--enable-back-end
option for hpctoolkit (experimental).
This option builds only the tools, libraries and prerequisites needed for
running hpcrun
and hpclink
and collecting data
on the back-end compute nodes.
In that case, run hpcstruct
and hpcprof
on the main cores.
On Stampede, you can launch a mixed-mode job (symmetric computing) with
ibrun.symm
and give separate command lines for the main
and MIC cores.
ibrun.symm -c "command line for main cores" \
-m "command line for MIC cores"
Linux-x86 (32-bit) on Linux-x86_64:
../configure CC="gcc -m32" CXX="g++ -m32" \
--host=i686-linux --prefix=`pwd`/../i686-linux \
--without-libunwind
../configure CC="gcc -m32" CXX="g++ -m32" MPICXX="mpicxx -m32" \
--host=i686-linux --prefix=<hpctoolkit-install> \
--with-externals=<hpctoolkit-externals>/i686-linux \
--with-papi=<papi-install> \
--disable-hpcrun-static
[Contents]
The following options may be useful in exceptional circumstances.
To enable/disable HPCToolkit features, the following options may be used.
Option | Meaning |
---|---|
--enable-mpi |
enable/disable build of hpcprof-mpi [default: yes] |
--with-upc=PATH |
path to IBM BlueGene/P UPC installation |
--with-upc-lib=LIBRARY |
path to IBM BlueGene/P UPC library(s), absolute or relative to WITH_UPC/runtime/SPI |
--enable-develop |
enable/disable build for development (sane debugging) [default: no] |
To request that HPCToolkit Externals use an alternate user-supplied installation for a given package, the following options may be used.
Option | Meaning |
---|---|
--with-binutils=PATH |
path to binutils install directory |
--with-libdwarf=PATH |
path to libdwarf install directory |
--with-libelf=PATH |
path to libelf install directory |
--with-libmonitor=PATH |
path to libmonitor install directory |
--with-libunwind=PATH |
path to libunwind install directory |
--with-libxml2=PATH |
path to libxml2 install directory |
--with-old-monitor=PATH |
path to old monitor install directory |
--with-open-analysis=PATH |
path to open analysis install directory |
--with-symtabAPI=PATH |
path to symtabAPI install directory |
--with-xed2=PATH |
path to xed2 install directory |
--with-xerces=PATH |
path to xerces/c install directory |
In all cases, the default is for HPCToolkit externals build the package itself, depending on platform.
To explicitly request that a package x
not be built, use --without-x
.
[Page last updated: 2017/01/20]