How to set up virtual environments for Python on MacOS

How to set up virtual environments for Python on MacOS

Save yourself a lot of confusion by managing your virtual environments with pyenv and virtualwrapper.

A python with a package.
Image credits : 
Jason van Gumster. CC BY-SA 4.0
x

Subscribe now

Get the highlights in your inbox every week.

If you're a Python developer and a MacOS user, one of your first tasks upon getting a new computer is to set up your Python development environment. Here is the best way to do it (although we have written about other ways to manage Python environments on MacOS).

Preparation

First, open a terminal and enter xcode-select --install at its cold, uncaring prompt. Click to confirm, and you'll be all set with a basic development environment. This step is required on MacOS to set up local development utilities, including "many commonly used tools, utilities, and compilers, including make, GCC, clang, perl, svn, git, size, strip, strings, libtool, cpp, what, and many other useful commands that are usually found in default Linux installations," according to OS X Daily.

Next, install Homebrew by executing the following Ruby script from the internet:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

If you, like me, have trust issues with arbitrarily running scripts from the internet, click on the script above and take a longer look to see what it does.

Once this is done, congratulations, you have an excellent package management tool in Homebrew. Naively, you might think that you next brew install python or something. No, haha. Homebrew will give you a version of Python, but the version you get will be out of your control if you let the tool manage your environment for you. You want pyenv, "a tool for simple Python version management," that can be installed on many operating systems. Run:

$ brew install pyenv

You want pyenv to run every time you open your prompt, so include the following in your configuration files (by default on MacOS, this is .bash_profile in your home directory):

$ cd ~/
$ echo 'eval "$(pyenv init -)"' >> .bash_profile

By adding this line, every new terminal will initiate pyenv to manage the PATH environment variable in your terminal and insert the version of Python you want to run (as opposed to the first one that shows up in the environment. For more information, read "How to set your $PATH variable in Linux.") Open a new terminal for the updated .bash_profile to take effect.

Before installing your favorite version of Python, you'll want to install a couple of helpful tools:

$  brew install zlib sqlite

The zlib compression algorithm and the SQLite database are dependencies for pyenv and often cause build problems when not configured correctly. Add these exports to your current terminal window to ensure the installation completes:

$ export LDFLAGS="-L/usr/local/opt/zlib/lib -L/usr/local/opt/sqlite/lib"
$ export CPPFLAGS="-I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include"

Now that the preliminaries are done, it's time to install a version of Python that is fit for a modern person in the modern age:

$ pyenv install 3.7.3

Go have a cup of coffee. From beans you hand-roast. After you pick them. What I'm saying here is it's going to take some time.

Adding virtual environments

Once it's finished, it's time to make your virtual environments pleasant to use. Without this next step, you will effectively be sharing one Python development environment for every project you work on. Using virtual environments to isolate dependency management on a per-project basis will give us more certainty and reproducibility than Python offers out of the box. For these reasons, install virtualenvwrapper into the Python environment:

$ pyenv global 3.7.3
# Be sure to keep the $() syntax in this command so it can evaluate
$ $(pyenv which python3) -m pip install virtualenvwrapper

Open your .bash_profile again and add the following to be sure it works each time you open a new terminal:

# We want to regularly go to our virtual environment directory
$ echo 'export WORKON_HOME=~/.virtualenvs' >> .bash_profile
# If in a given virtual environment, make a virtual environment directory
# If one does not already exist
$ echo 'mkdir -p $WORKON_HOME' >> .bash_profile
# Activate the new virtual environment by calling this script
# Note that $USER will substitute for your current user
$ echo '. ~/.pyenv/versions/3.7.3/bin/virtualenvwrapper.sh' >> .bash_profile

Close the terminal and open a new one (or run exec /bin/bash -l to refresh the current terminal session), and you'll see virtualenvwrapper initializing the environment:

$ exec /bin/bash -l
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/premkproject
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/postmkproject
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/initialize
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/premkvirtualenv
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/postmkvirtualenv
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/prermvirtualenv
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/postrmvirtualenv
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/predeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/postdeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/preactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/postactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/get_env_details

From now on, all your work should be in a virtual environment, allowing you to use temporary environments to play around with development safely. With this toolchain, you can set up multiple projects and switch between them, depending upon what you're working on at that moment:

$ mkvirtualenv test1
Using base prefix '/Users/moshe/.pyenv/versions/3.7.3'
New python executable in /Users/moshe/.virtualenvs/test1/bin/python3
Also creating executable in /Users/moshe/.virtualenvs/test1/bin/python
Installing setuptools, pip, wheel...
done.
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test1/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test1/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test1/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test1/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test1/bin/get_env_details
(test1)$ mkvirtualenv test2
Using base prefix '/Users/moshe/.pyenv/versions/3.7.3'
New python executable in /Users/moshe/.virtualenvs/test2/bin/python3
Also creating executable in /Users/moshe/.virtualenvs/test2/bin/python
Installing setuptools, pip, wheel...
done.
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test2/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test2/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test2/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test2/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/moshe/.virtualenvs/test2/bin/get_env_details
(test2)$ ls $WORKON_HOME
get_env_details         postmkvirtualenv        premkvirtualenv
initialize              postrmvirtualenv        prermvirtualenv
postactivate            preactivate             test1
postdeactivate          predeactivate           test2
postmkproject           premkproject
(test2)$ workon test1
(test1)$

The deactivate command exits you from the current environment.

Recommended practices

You may already set up your long-term projects in a directory like ~/src. When you start working on a new project, go into this directory, add a subdirectory for the project, then use the power of Bash interpretation to name the virtual environment based on your directory name. For example, for a project named "pyfun":

$ mkdir -p ~/src/pyfun && cd ~/src/pyfun
$ mkvirtualenv $(basename $(pwd))
# we will see the environment initialize
(pyfun)$ workon
pyfun
test1
test2
(pyfun)$ deactivate
$

Whenever you want to work on this project, go back to that directory and reconnect to the virtual environment by entering:

$ cd ~/src/pyfun
(pyfun)$ workon .

Since initializing a virtual environment means taking a point-in-time copy of your Python version and the modules that are loaded, you will occasionally want to refresh the project's virtual environment, as dependencies can change dramatically. You can do this safely by deleting the virtual environment because the source code will remain unscathed:

$ cd ~/src/pyfun
$ rmvirtualenv $(basename $(pwd))
$ mkvirtualenv $(basename $(pwd))

This method of managing virtual environments with pyenv and virtualwrapper will save you from uncertainty about which version of Python you are running as you develop code locally. This is the simplest way to avoid confusion—especially when you're working with a larger team.

If you are just beginning to configure your Python environment, read up on how to use Python 3 on MacOS. Do you have other beginner or intermediate Python questions? Leave a comment and we will consider them for the next article.

Python in a coffee cup.

JupyterLab, the successor to Jupyter Notebook, feels like playing video games with the cheat codes enabled.
Team checklist

Learn more about solving common Python problems in our series covering seven PyPI libraries.

Topics

About the author

Moshe sitting down, head slightly to the side. His t-shirt has Guardians of the Galaxy silhoutes against a background of sound visualization bars.
Moshe Zadka - Moshe has been involved in the Linux community since 1998, helping in Linux "installation parties". He has been programming Python since 1999, and has contributed to the core Python interpreter. Moshe has been a DevOps/SRE since before those terms existed, caring deeply about software reliability, build reproducibility and other such things. He has worked in companies as small as three people and as big as tens of thousands -- usually some place around where software meets system administration...

About the author

I'm happiest at a microphone
Matthew Broberg - Matt is an advocate for open source software and currently the Technical Editor of Opensource.com. He specializes in designing technology communities that develop products and content in a way that tells a powerful story. Matt was an EMC storage expert, VMware vExpert, and former fan of other proprietary technologies. He now focuses on open source and DevRel adoption. He is a serial podcaster, best known for the Geek Whisperers podcast, co-built ...