Sunday, March 7, 2010

Parallel compilation of Wien2k on Intel Itanium architecture

Here I briefly describe how to compile Wien2k on the supercomputer with Intel Itanium processors. I suppose that you have read the Wien2k UserGuide and know how to compile the serial version of the program.

What do we need:
Intel Fortran Compiler (ifort), I use version 11.0
Gcc or Intel C/C++ compiler (icc)
Intel CMKL libraries (I have used 10.1.1.019 version)
FFTW 2.1.5 (not 3.0!!) compiled with mpi support
MPI (I use MPICH2)

Compiling serial version
For serial version the compilation options given to siteconfig are the following:

System type: K Linux (Intel ifort 11.x compiler + mkl )
Fortran compiler: ifort
C compiler: cc
Compiler options, BLAS and LAPACK:
O Compiler options: -FR -mp1 -w -prec_div -pc80 -pad -align -DINTEL_VML -traceback
L Linker Flags: $(FOPT) -L/opt/intel/cmkl/10.1.1.019/lib/64 -lguide -pthread
P Preprocessor flags '-DParallel'
R R_LIB (LAPACK+BLAS): $(FOPT) -L/opt/intel/mkl/10.1.1.019/lib/64 -lmkl_lapack -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lguide -pthread

Note: replace the path to your own one, and mind that
lib/64 libraries are needed for Itanium. If you have Intel Xeon or similar processor, use lib/emt64 libraries instead.

Compiling FFTW
This is needed only for the parallel compilation. Download FFTW 2.1.5, unpack it and use the following commands for the compilation:

export CC=icc
export F77=ifort
./configure --enable-mpi --enable-threads
make

Copy the libraries from fftw-2.1.5/fftw/.libs and mpi/.libs somewhere to /opt/fftw or leave them there (as I did).

Compiling in parallel
I have mpich2 installed in /opt/mpich2, so I add /opt/mpich2/bin into the path:

export PATH="$PATH:/opt/mpich2/bin"

Check if mpif90 command works:

$mpif90 -v
mpif90 for 1.0.8p1
Version 11.0

The compilation options will be the following:
Compiler: mpif90

RP RP_LIB(SCALAPACK+PBLAS): -L/opt/intel/mkl/10.1.1.0.19/lib/64 -lmkl_intel_lp64 -lmkl_scalapack_lp64 -lmkl_blacs_lp64 -lmkl_sequential -lmkl -L/opt/mpich2/lib -lmpich -L/home/hartree-fock/fftw-2.1.5/mpi/.libs -lfftw_mpi -L/home/hartree-fock/fftw-2.1.5/fftw/.libs -lfftw
FP FPOPT(par.comp.options): $(FOPT) -I/opt/mpich2/include
MP MPIRUN commando : mpirun -np _NP_ -machinefile _HOSTS_ _EXEC_

That's it! Don't forget to add the necessary lines to .bashrc (LD_LIBRARY_PATH etc).