virtualenv with numpy and scipy on Mac OSX
Getting numpy, scipy and associated libraries and their corresponding C/C++ libraries installed correctly on virtualenv (with the help of virtualenv wrapper), Mac OSX can be a little troublesome.
Here’s a step-by-step as a reference for my colleagues and fellow pythonista peers…
1. Get our virtual environment up-and-running
$ mkvirtualenv -p python2.7 --distribute python-for-scientists Running virtualenv with interpreter /opt/local/bin/python2.7 New python executable in python-for-scientists/bin/python Installing distribute..........................................................................................................................................................................................................done. Installing pip................done. virtualenvwrapper.user_scripts creating /Users/calvin/.virtualenvs/python-for-scientists/bin/predeactivate virtualenvwrapper.user_scripts creating /Users/calvin/.virtualenvs/python-for-scientists/bin/postdeactivate virtualenvwrapper.user_scripts creating /Users/calvin/.virtualenvs/python-for-scientists/bin/preactivate virtualenvwrapper.user_scripts creating /Users/calvin/.virtualenvs/python-for-scientists/bin/postactivate virtualenvwrapper.user_scripts creating /Users/calvin/.virtualenvs/python-for-scientists/bin/get_env_details
2. Personal Convention: working directory for project-specific python files
As a personal convention, I like to synchronize my project’s root directory name with my virtual environment name. In this case, “python-for-scientists”.
$ mkdir -p ~/work/python-for-scientists $ cd ~/work/python-for-scientists
And the fun begins…
3. NumPy: pip install numpy
To install numpy in our python virtual environment, it’s a simple as running `pip install numpy`.
But if this is your first time getting your feet wet with coding, be aware that on Mac OSX, you will need to install XCode and associated SDKs/compilers provided by https://developer.apple.com/. Typically, once we have gotten that installed, the clang and clang++ compilers will be available on our system. Checking…
(python-for-scientists)220.127.116.11.client.dyn.strong-sf15.reliablehosting.com ttys004 Fri Nov 02 15:18:57 |~/work/python-for-scientists| calvin$ which clang++ /usr/bin/clang++ (python-for-scientists)18.104.22.168.client.dyn.strong-sf15.reliablehosting.com ttys004 Fri Nov 02 15:19:16 |~/work/python-for-scientists| calvin$ clang --version Apple clang version 4.0 (tags/Apple/clang-421.0.60) (based on LLVM 3.1svn) Target: x86_64-apple-darwin11.4.2 Thread model: posix (python-for-scientists)22.214.171.124.client.dyn.strong-sf15.reliablehosting.com ttys004 Fri Nov 02 15:19:19 |~/work/python-for-scientists| calvin$ clang++ --version Apple clang version 4.0 (tags/Apple/clang-421.0.60) (based on LLVM 3.1svn) Target: x86_64-apple-darwin11.4.2 Thread model: posix
Now, numpy can be downloaded and installed as expected:-
calvin$ pip install numpy
If numpy and all its c modules get downloaded and compiled correctly, we should see something similar at the end.
clang: numpy/random/mtrand/randomkit.c clang: numpy/random/mtrand/mtrand.c /usr/bin/clang -bundle -undefined dynamic_lookup -isysroot / -L/opt/local/lib build/temp.macosx-10.7-x86_64-2.7/numpy/random/mtrand/mtrand.o build/temp.macosx-10.7-x86_64-2.7/numpy/random/mtrand/randomkit.o build/temp.macosx-10.7-x86_64-2.7/numpy/random/mtrand/initarray.o build/temp.macosx-10.7-x86_64-2.7/numpy/random/mtrand/distributions.o -Lbuild/temp.macosx-10.7-x86_64-2.7 -o build/lib.macosx-10.7-x86_64-2.7/numpy/random/mtrand.so Creating build/scripts.macosx-10.7-x86_64-2.7/f2py adding 'build/scripts.macosx-10.7-x86_64-2.7/f2py' to scripts changing mode of build/scripts.macosx-10.7-x86_64-2.7/f2py from 644 to 755 changing mode of /Users/calvin/.virtualenvs/python-for-scientists/bin/f2py to 755 Successfully installed numpy Cleaning up...
Notice also that numpy include f2py, which is an excellent program for scientists or engineers that would like to integrate their python program with existing fortran based programs.
Checking that we have installed numpy correctly:-
calvin$ python -c "import numpy; print numpy.version.version" 1.6.2
4. SciPy on Mac OSX
That was easy. Here comes the complication. Getting SciPy to install correctly on Mac OSX is a little more troublesome. We will need a proper fortran compiler installed first before pip install scipy would work.
This is the link where I got my gfortran compiler - http://gcc.gnu.org/wiki/GFortranBinaries#MacOS
Download the appropriate gfortran binary for your Mac OSX version. You can use this if you are running Mac OSX Lion – http://quatramaran.ens.fr/~coudert/gfortran/gfortran-4.6.2-x86_64-Lion.dmg.
I am in an adventurous mood, so I decided to go for the high performance computing (HPC) for Mac OSX version instead. That version can be found on http://hpc.sourceforge.net/, specifically here - http://prdownloads.sourceforge.net/hpc/gfortran-lion.tar.gz?download.
Once you have gotten your gfortran compiler installed correctly, you can compile your Fortran programs too, if that matters to you as an engineer/scientist. :-)
(python-for-scientists)126.96.36.199.client.dyn.strong-sf15.reliablehosting.com ttys004 Fri Nov 02 15:38:45 |~/work/python-for-scientists| calvin$ gfortran --version GNU Fortran (GCC) 4.8.0 20120603 (experimental) Copyright (C) 2012 Free Software Foundation, Inc. GNU Fortran comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Fortran under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING
With f2py from NumPy mentioned above, we now have the power of Fortran and Python in our hands!
Moving on, now that we have got gfortran sorted, we want to get SciPy installed. So let’s get cracking…
calvin$ pip install scipy ... ... /usr/bin/clang -bundle -undefined dynamic_lookup -isysroot / -L/opt/local/lib build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/nd_image.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_filters.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_fourier.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_interpolation.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_measure.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_morphology.o build/temp.macosx-10.7-x86_64-2.7/scipy/ndimage/src/ni_support.o -Lbuild/temp.macosx-10.7-x86_64-2.7 -o build/lib.macosx-10.7-x86_64-2.7/scipy/ndimage/_nd_image.so warning: no files found matching 'f2py.py' no previously-included directories found matching 'scipy/special/tests/data/boost' Successfully installed scipy Cleaning up...
If all goes well, we should see the “successfully installed scipy” message at the end of it all.
Now that we have gotten this out of the way, we can check
calvin$ python -c "import scipy; print scipy.version.version" 0.11.0
5. What’s Next?
So this is pretty much there is to it for our basic set-up to begin powerful numerical analysis and exploratory data science with NumPy and SciPy.
In my next post, I will run through iPython install (which is trivial but with a small readline hiccup though) and Qt4 integration with iPython (which is a little messy for virtualenv in Mac OSX) giving us a Qt4-based GUI iPython shell that comes with wonderful functionalities like rendering embedded charts and visualization inline in iPython.