When using Linux devs and data scientists often end up using the default Python version included in the package repositories. This can lead you to having to wait for a long time to try out Python’s new features! The following post describes how to compile and install an extra Python version without interfering with the system Python and creating a virtual environment to use the new Python version.

Prerequisites

  • Ubuntu, tested with 18.04,
    • will probably work on other versions, let me know if it doesn’t!
  • Basic understanding of Python virtual environments

Installing your new version of Python

Install the required Ubuntu packages for the procedure.

sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev \
    libnss3-dev libssl-dev libreadline-dev libffi-dev wget

Download and extract the version of Python you would like to install. For example here I want to install version 3.7.3. You can try directly altering the URL if you would like a different version or download it from the full list here.

cd /tmp
wget https://www.python.org/ftp/python/3.7.2/Python-3.7.3.tar.xz
tar -xf Python-3.7.3.tar.xz

Setup the configuration for the compilation process. Here you should pass the path to where you would like to install your version of Python, my example path is /home/gns/python373. This is preferable as it does not interfere with the OS Python installation and so shouldn’t have any side effects.

cd Python-3.7.3
./configure --enable-optimizations --with-lto --prefix=/home/${USER}/python373

Compile and install to your chosen directory. I picked a prefix within my home directory and so do not need sudo for make install line. The 4 here says to use 4 cores for compiling, you should change this to the appropriate number for your machine. This process takes a few minutes, so you may want to wander off for a bit!

time make -j 4
make install

Now you can create a virtual environment and specify your newly compiled Python binary as the Python version to use. Don’t forget to substitute the prefix path with the one you provided above. While working within the virtual environment python and pip install commands will all be tied to the new Python version.

mkvirtualenv --python=/home/${USER}/python373/bin/python3 awesomeenv

Bare in mind that wheels for packages may not have been uploaded to PyPi for your version of Python, especially if it is very new, so pip installs may require compiling from source (and so could take longer).

Have fun!