Getting Started with Intel Trace Analyzer
The Intel Trace Analyzer was formerly known as Vampir. This tool lets you analyze the performance and communication patterns of your MPI application.
Configuring User Environment
In order to use the Intel Trace Analyzer, you must enable X11 forwarding to your local display (ssh -X). The ITA is installed on CITA's only Itanium-2 server: lobster6. Before you log in, make the following changes to your ~/.cshrc:
setenv PATH '/opt/intel_mpi_10/bin:'$PATH
setenv PATH '/opt/intel/ITA-IA64-LIN-AS21-PRODUCT.4.0.3.1/bin/:'$PATH
setenv I_MPI_DEVICE 'ssm'
source '/opt/intel/ict/1.0/ictvars.csh'
setenv KMP_MONITOR_STACKSIZE 262144
This assumes you are using the default cita .cshrc, with the useroptional.csh template.
Modifying Compilation
In order for the trace analyzer to work properly, you must compile your mpi binaries to include the Intel Trace Collector library. By default we only have this installed to work with Intel MPI (set up above). It has different compiler wrappers than LAM, so one should add or change the following in a standard makefile:
F90=mpiifort
#I believe it actually uses g77 for mpif77 instead of ifort
F77=mpif77
CC=mpicc
CXX=mpicxx
#path to Intel Trace Collector
ITCLIB=/opt/intel/ITC-IA64-LIN-IMPI-PRODUCT.5.0.0.1/lib
# libraries to link -- these are in addition (and should come before) any your application needs
LDLIBS= -lVT -lvtunwind -ldwarf -lelf -lpthread -lm
#library paths called during linking
LDPATH= -L$(ITCLIB)
If you are using other libraries that require the mpi implementation at build time (fftw,scalapack) you will need to recompile them as well, using the above.
Generating the Trace
Once your code compiles, you should run it with mpdrun instead of mpirun. It operates similarly, but requires one to boot a team of mpd daemons instead of lamd daemons. This is done using an mpd file, 'mpd.hosts', containing an entry for the hostname of each node that will be a member of your mpi network. This only needs one entry for now,
lobster6, and can be executed using:
mpdboot
Once it is started, run your code with:
mpdrun -np $NUMBER_PROCESSES $PROGRAM_NAME
And it should run, slower than normal, and produce a number of files in the pwd. The trace file is $PROGRAM_NAME.stf. To look at it in the trace analyzer, run:
traceanalyzer $PROGRAM_NAME.stf
You should now see 2 windows, one is a menu and the other is a plot of your program behaviour. You are now ready to start analyzing! As well as the default trace collection indicated above, there is also the ability to insert hooks in your source code that access the Trace Collector API to do more precise analysis. For more information, see the following documentation.
Important Documentation:
Updated 2005.01.14