Getting Started with Intel Trace Analyzer
The Intel Trace Analyzer was formerly known as Vampir. This tool lets you analyze the performance and communication patterns of your MPI application.
Configuring User Environment
In order to use the Intel Trace Analyzer, you must enable X11 forwarding to your local display (ssh -X). The ITA is installed on CITA's only Itanium-2 server: lobster6. Before you log in, make the following changes to your ~/.cshrc:
setenv PATH '/opt/intel_mpi_10/bin:'$PATH
setenv PATH '/opt/intel/ITA-IA64-LIN-AS21-PRODUCT.18.104.22.168/bin/:'$PATH
setenv I_MPI_DEVICE 'ssm'
setenv KMP_MONITOR_STACKSIZE 262144
This assumes you are using the default cita .cshrc, with the useroptional.csh template.
In order for the trace analyzer to work properly, you must compile your mpi binaries to include the Intel Trace Collector library. By default we only have this installed to work with Intel MPI (set up above). It has different compiler wrappers than LAM, so one should add or change the following in a standard makefile:
If you are using other libraries that require the mpi implementation at build time (fftw,scalapack) you will need to recompile them as well, using the above.
#I believe it actually uses g77 for mpif77 instead of ifort
#path to Intel Trace Collector
# libraries to link -- these are in addition (and should come before) any your application needs
LDLIBS= -lVT -lvtunwind -ldwarf -lelf -lpthread -lm
#library paths called during linking
Generating the Trace
Once your code compiles, you should run it with mpdrun instead of mpirun. It operates similarly, but requires one to boot a team of mpd daemons instead of lamd daemons. This is done using an mpd file, 'mpd.hosts', containing an entry for the hostname of each node that will be a member of your mpi network. This only needs one entry for now,
lobster6, and can be executed using:
Once it is started, run your code with:
mpdrun -np $NUMBER_PROCESSES $PROGRAM_NAME
And it should run, slower than normal, and produce a number of files in the pwd. The trace file is $PROGRAM_NAME.stf. To look at it in the trace analyzer, run:
You should now see 2 windows, one is a menu and the other is a plot of your program behaviour. You are now ready to start analyzing! As well as the default trace collection indicated above, there is also the ability to insert hooks in your source code that access the Trace Collector API to do more precise analysis. For more information, see the following documentation.