This page lists major freely-available astrophysics codes that are known to run on McKenzie. Please email Jonathan if you have instructions for getting a code up and running that isn't already listed here.

Unless otherwise stated, these instructions assume that you're using HDF 1.6.4, LAM MPI, and the Intel 8.1 compilers: that is, that you have something like

module add hdf/1.6.4
module add intel/8.1
module add mpi/lam7.1.2b24-intel

[ FLASH | Gadget-2 | Zeus-MP V2 ]

FLASH

FLASH is a general purpose astrophysical hydrodynamics AMR code out of the University of Chicago.

Getting and compiling the code

To get a copy of the code, go to the FLASH code request site and request a copy; it may take several days.

When you have your copy, un-tar it, and in the same directory as the FLASH2.tar.gz, download and untar mckenzie.tgz. Then, when you use

setup

-site=mckenzie

setup sedov -auto -2d -maxblocks=5000 -site=mckenzie

Relevant web pages

• FLASH homepage
• FLASH code support
• FLASH cosmology

Relevant Papers

• FLASH 1.6 Code paper
• FLASH user's guide
• Validation and Verification tests of the FLASH code

Performance

Above are the results for a simple (but non-trivial) constant-work-per-processor test problem including hydrodynamics and AMR that comes with FLASH, run on McKenzie. FLASH hydrodynamics and AMR scales extremely well on Mckenzie on out to at least 160 procs (the largest number easy to get through the queue), with any sub-linear scaling within the ~2% variablity noise. IO and selfgravity remain issues.

Gadget-2

Gadget-2 is a N-body/SPH code written by Volker Springel at the MPA.

Getting and compiling the code

The code can be downloaded directly from the Gadget website. To run on McKenzie, you just have to point to the correct directories for the libraries. You'll also have to make sure you have

module add fftw/2.1.5-intel8

The easiest way to point to the correct libraries is to edit the Makefile you use (note; different problems use different Makefiles! Makefiles are found in the parameterfiles directory) to include a setup for McKenzie. That is, find the group of lines that specify different SYSTYPEs, add the line

SYSTYPE="McKenzie"

ifeq ($(SYSTYPE),"McKenzie")
CC       =  mpicc
OPTIMIZE =  -O3 -Wall
GSL_INCL =  -I/usr/include/gsl/
GSL_LIBS =  -L/usr/lib/  
FFTW_INCL=  -I${MCKENZIE_FFTW_INC_PATH}
FFTW_LIBS=  -L${MCKENZIE_FFTW_LIB_PATH}
MPICHLIB =
HDF5INCL =  -I${MCKENZIE_HDF_INC_PATH}
HDF5LIB  =  -L${MCKENZIE_HDF_LIB_PATH} -lhdf5 -lz
endif

Warning: some of the Makefiles in the parameterfiles directory are out of date; in particular, they refer to a compiler option -DISOTHERMAL which should instead be -DISOTHERM_EQS . I believe that the Makefile in the source code directory has all of the options correctly defined.

Relevant web pages

• The Gadget webpage

Relevant Papers

• The Gadget-2 code paper.
• The Gadget-1 code paper.

Performance

Above are the results for running the cluster simulation, a 276k particle run that comes with Gadget-2, out to about 16 timesteps (from time 0.04166 to 0.0534094) on varying numbers of processors (out to 32 nodes) on McKenzie. The solid line is linear scaling. Note that this uses the PM (FFT) gravity, and that at 64 processors, this problem has only 4320 particles per processor.

Zeus-MP v2

Zeus MP 2 is an update of the venerable ZEUS code, maintained out of UCSD.

Getting and compiling the code

Zeus can be downloaded directly from the Zeus webpage.

As of Zeus-MP v. 2.01, Zeus includes a Makefile expressly for McKenzie, meaning that compiling is just a matter of typing:

gmake -r -f Makefile.cita.mckenzie

/home/ljdursi/hdf4/

Relevant web pages

• The Zeus-2 website
• Another person's version of an updated Zeus-MP

Relevant Papers

• The Zeus-MP-2 code paper.