Home     Research     Code Library     Publications     Presentations     Teaching

I am mainly interested in the development and implementation of computational strategies for applications in biology, medicine and datascience. I am especially intrigued by mathematical modeling and the practical aspects of numerics such as algorithmic design, numerical optimization and specialized implementations. I am also generally interested in programming and high performance computing.

Contents

Search

Search my page

Personal Information

CV

View my English CV


CV

View my German CV


Linkedin

Visit my Linkedin profile

Research Interests

Computational Working Environment

Python

Python


R

R


Matlab

MATLAB


C

C

Software

Most of my codes are written in Python (visit my code library). NumPy and SciPy turned out to be eminently suitable for numerical image processing. In conjunction with matplotlib these packages provide an outstanding environment for numerical simulation as well as visualization. Binary packages for NumPy, SciPy and matplotlib are available for all platforms. However, for best performance you should probably build NumPy and SciPy yourself. In this case the Section below might be helpful.

Building NumPy and SciPy

The following instructions explain how to set up and build NumPy and Scipy on a Linux System. This is of course not the only tutorial explaining this procedure. However, I found the official notes a little sparse in some (in my opinion) crucial places. This is especially the case for UMFPACK, SciPy’s default solver for sparse linear systems. UMFPACK is part of Tim Davis’ SuiteSparse and widely used in many software packages (e.g. MATLAB’s “backslash” operator relies heavily on it). However, SciPy does not require UMFPACK to be installed on your system (it will fall back to SuperLU instead which is built auto-magically together with SciPy). If you do not care/want/need to set up UMFPACK for the use in SciPy, jump directly to Step 2. Otherwise let’s start with Step 1 (Note that it is assumed that you already have BLAS and LAPACK installed on your system)

  1. In contrast to many other packages SciPy requires to be linked against a shared library libumfpack.so. However, SuiteSparse does only build static libraries by default. Unfortunately tweaking SuiteSparse to build shared libraries libumfpack.so and libamd.so (both needed by SciPy) turned out to be painstakingly cumbersome. Thus it may be easiest to just replace the original Makefiles with my tuned versions and then run the build process as explained in the following.

    1. Download SuiteSparse from here. Extract the archive and move the generated directory to a convenient location. Let’s call this folder $SuiteSparseDIR in the following.
    2. UMFPACK can use METIS for fast matrix reordering. If you do not care/want/need to set up METIS just skip this step. Otherwise download METIS-4.0.1 from here. Unpack the archive and move the metis-4.0-directory to $SuiteSparseDIR. Rename metis-Makefile.in to Makefile.in and place it in $SuiteSparseDIR/metis-4.0/. Fire up a terminal, cd to $SuiteSparseDIR/metis-4.0/ and type make. If all goes well proceed to the next step.
    3. Download SuiteSparse_config.mk and edit the lines where the variables INSTALL_LIB and INSTALL_INCLUDE are defined to your liking. Then put the file in $SuiteSparseDIR/SuiteSparse_config/.
    4. Rename GNUmakefile_umfpackLib to GNUmakefile and move it to $SuiteSparseDIR/UMFPACK/Lib/. Similarly rename Makefile_umf to Makefile and put it in $SuiteSparseDIR/UMFPACK/.
    5. Now the same for AMD: rename GNUmakefile_amdLib to GNUmakefile and move it to $SuiteSparseDIR/AMD/Lib/. Similarly rename Makefile_amd to Makefile and put it in $SuiteSparseDIR/AMD/.
    6. Now cd to $SuiteSparseDIR and type make library followed by make install (add sudo if INSTALL_LIB and INSTALL_INCLUDE are not in your home-folder). Note that these steps assume that you did not change the directory structure of SuiteSparse - if you moved any of AMD, CAMD, CHOLMOD etc. additional modifications are necessary (more details here [1]).
  2. Download NumPy and SciPy and extract both archives. Edit the file numpy-x.y.z/site.cfg as follows. Make sure that the install location of your SuiteSparse, i.e. INSTALL_LIB and INSTALL_INCLUDE, appears in the [DEFAULT] section (if you do not care/want/need to use UMFPACK ignore this). Thus if, for instance, SuiteSparse resides in /usr/local/lib and /usr/local/include it should be something like

    [DEFAULT]
    library_dirs = /usr/lib:/usr/lib64:/usr/local/lib
    include_dirs = /usr/include:/usr/local/include
    

    Further, make sure to specify your BLAS/LAPACK/ATLAS installations

    [blas_opt]
    libraries = ptf77blas, ptcblas, atlas
    
    [lapack_opt]
    libraries = lapack, ptf77blas, ptcblas, atlas
    

    Finally make sure that your site.cfg has a section like

    [amd]
    amd_libs = amd
    
    [umfpack]
    umfpack_libs = umfpack
    
  3. Since NumPy makes heavy use of BLAS it is crucial that the same FORTRAN compiler that built BLAS is used to build NumPy (see this note in the official documentation). If your shared BLAS is /usr/lib64/libblas.so then type ldd /usr/lib64/libblas.so in a terminal. If libgfortran.so appears somewhere in the output then your BLAS was most probably built with gfortran. Thus to build NumPy cd to numpy-x.y.z and type

    python setup.py build --fcompiler=gnu95
    

    Conversely, if you find libg2c.so in ldd ‘s output then g77 has been used to build your BLAS. Hence type

    python setup.py build --fcompiler=gnu
    

    Finally the command

    python setup.py install --prefix=/path/to/somewhere
    

    installs NumPy in /path/to/somewhere (sudo will be necessary again if /path/to/somewhere is not in your home directory).

  4. If you have installed NumPy in a non-standard location you have to tell Python where to find it before you can use it. Assume you have installed NumPy in the location /path/to/somewhere using Python 2.7 on a 64 bit system. Then type in a bash shell

    export PYTHONPATH='/path/to/somewhere/lib64/python-2.7/site-packages':$PYTHONPATH
    

    to update your Python path (csh users might want to use setenv instead). Then fire up Python and type

    import numpy
    numpy.__file__
    

    This should result in the output

    '/path/to/somewhere/lib64/python-2.7/site-packages/numpy/__init__.pyc'
    

    Thus the just installed NumPy version has been imported. Now you may want to check the output of numpy.show_config() to make sure that all libraries have been detected correctly. Finally numpy.test() (which needs nose) tests the installation.

  5. After the successful installation of NumPy building SciPy is quite straight forward. Copy site.cfg from numpy-x.y.z to scipy-x.y.z. Make sure that your $PYTHONPATH points to the just installed NumPy (explained above). Depending on your findings before build SciPy using

    python setup.py build --fcompiler=XXX
    

    and install it with

    python setup.py install --prefix=/path/to/somewhere
    
  6. Test the installation with

    import scipy
    scipy.__file__
    

    which should similarly give

    '/path/to/somewhere/lib64/python-2.7/site-packages/scipy/__init__.pyc'
    

    Further, check scipy.show_config() to make sure that SciPy uses the right libraries. Due to a not so smart ImportError - silencing in SciPy (compare e.g. this post) it is not too easy to see if UMFPACK is actually used by SciPy after all. Even if scipy.show_config() proudly announces that it found libumfpack.so using linalg.spsolve may produce surprising results. Thus you might want to run testumfpack.py to check UMFPACK’s functionality. If you encounter troubles try to

    cd /path/to/somewhere/lib64/python-2.7/site-packages/scipy/sparse/linalg/dsolve/umfpack/
    

    and import _umfpack. This should work without raising an ImportError. Finally you can test SciPy’s functionality by typing scipy.test() (which also needs nose)

  7. If you are tired of updating your $PYTHONPATH in order to be able to use your homebrewed NumPy/SciPY each time you start a new shell you may want to include the line

    export PYTHONPATH='/path/to/somewhere/lib64/python-2.7/site-packages':$PYTHONPATH
    

    in your .bashrc-file (for bash users).

If you have any comments/questions/criticism please contact me so that I can improve this tutorial.

[1]All Makefiles assume that AMD, CAMD, CHOLMOD etc. are found in their respective standard locations. If you have moved these static libraries, you should let the GNUmakefiles know by explicitly telling the .so build command where to find them. For instance, if CHOLMOD is installed in the non-standard location /where/ever, the build command should be changed from -lcholmod to -L /where/ever -lcholmod. Non-standard installations of metis can be handled analogously. Thanks to Stefan Becuwe from the HPC core facility CalcUA at the University of Antwerp for pointing this out to me!