The right and wrong way to set Python 3 as default on a Mac

There are several ways to get started with Python 3 on macOS, but one way is better than the others.
844 readers like this.
Python programming language logo with question marks

Opensource.com

I've been dipping my toe back into Python development as I get ready to head to PyCon US. (If you're headed there as well and want to share your Python story, let me know!) When I installed a module to tinker around with, I got a reminder that I needed to install Python 3 soon.

$ pip install todoist-python 
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.

So, I did what any of us would do and googled around looking for a guide to update my development environment, which runs on Mac (the macOS operating system, formerly known as OS X). To my surprise, I found only a handful of StackOverflow posts, and they pointed me to partial solutions. Here's the full story of how to set up your environment without breaking anything built into the macOS operating system.

1. Install pyenv

Moshe Zadka cautions that doing this wrong could result in an unreliable idea of which Python is running that depends too closely on shells loading aliases. I knew Moshe was familiar with Python, but what I didn't know is that he is an author of many Python tutorials as well as an upcoming book on Python development on macOS. He helped 40 colleagues develop Python safely and consistently on macOS systems following one core principle:

"The basic premise of all Python development is to never use the system Python. You do not want the Mac OS X 'default Python' to be 'python3.' You want to never care about default Python."

How do we stop caring about the default? Moshe recommends using pyenv to manage Python environments (for a deeper dive on configuring pyenv, see this article). This tool manages multiple versions of Python and is described as "simple, unobtrusive, and follows the Unix tradition of single-purpose tools that do one thing well."

While other installation options are available, the easiest way to get started is with Homebrew:

$ brew install pyenv 
?  /usr/local/Cellar/pyenv/1.2.10: 634 files, 2.4MB

2. Install Python

Now let's install the latest Python version (3.7.3 as of this writing):

$ pyenv install 3.7.3
python-build: use openssl 1.0 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.3.tar.xz...
-> https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
Installing Python-3.7.3...
## further output not included ##

3. Set your global default

Now that Python 3 is installed through pyenv, we want to set it as our global default version for pyenv environments:

$ pyenv global 3.7.3
# and verify it worked 
$ pyenv version
3.7.3 (set by /Users/mbbroberg/.pyenv/version)

The power of pyenv comes from its control over our shell's path. In order for it to work correctly, we need to add the following to our configuration file (.zshrc for me, possibly .bash_profile for you):

$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init -)"\nfi' >> ~/.zshrc

After that command, our dotfile (.zshrc for zsh or .bash_profile for Bash) should include these lines:

if command -v pyenv 1>/dev/null 2>&1; then
  eval "$(pyenv init -)"
fi

Now we know for certain that we're using Python 3.7.3 and pip will update alongside it without any manual aliasing between versions. Using Moshe's recommendation to use a version manager (pyenv) enables us to easily accept future upgrades without getting confused about which Python we are running at a given time.

Success

As you get comfortable with this workflow, you can use pyenv to manage multiple versions of Python. It's also essential, for dependency management, to use virtual environments. I mention how to use the built in venv library in the article, and Moshe recommends virtualenvwrapper for managing virtual environments.

Understanding Python runtimes

Now that you have your Python versions fixed, it's safe to explore why this problem confuses so many people.

The version of Python that ships with macOS is well out of date from what Python recommends using for development. Pondering Python runtimes can be comically challenging at times, as noted by XKCD.

Python environment webcomic by xkcd

Many users have dozens of Python interpreters on their computer already, but have no idea how to manage them effectively. Too often, people just download the latest Python release, move it to their path, and call it a day (or use brew install python3, which would do something similar). This can cause breakages down the line in frustrating ways that can be difficult to troubleshoot.

What NOT to do

My first idea on how to make Python 3 the default Python on my system was to move the old version and add the new one:

# what I thought would work 
# first, I'll find my python binary 
$ which python 
/usr/bin/python 
# next, I'll move it to an unused name 
$ sudo mv /usr/bin/python /usr/bin/python2 
# lastly, I'll move the new binary to the previous path 
$ sudo mv $PATHTOBINARY/python3 /usr/bin/python 

The pattern followed what /usr/bin/ usually does between major releases of Python, but I quickly learned it was the wrong move:

$ sudo mv /usr/bin/python /usr/bin/python2 
mv: rename /usr/bin/python to /usr/bin/python2: Operation not permitted 

Thankfully, macOS protected me from breaking something I don't fully understand. Further research proves this is exactly what we shouldn't do.

Another thing not to try

Now that we know what not to do, let's look at what we could do. There are a couple options when we think about common installation patterns for applications on macOS.

Use Python 3 as the macOS default

Python's website has a macOS Python 3 installer we can download and use. If we use the package installation, a python3 fill will be available in /usr/local/bin/.

Aliasing is a must since the Python binary stored in /usr/bin/ can't be changed. What's nice about an alias is that it's specific to our command-line shell. Since I use zsh by default, I put the following into the .zshrc file:

$ echo "alias python=/usr/local/bin/python3.7" >> ~/.zshrc 

If you are using the default Bash shell, you can append this same text to your .bashrc:

$ echo "alias python=/usr/local/bin/python3.7" >> ~/.bashrc 

This strategy works, but it isn't ideal for making future updates to Python. It means we have to remember to check the website and download the new files since Python doesn't include a command-line way to update.

Have Homebrew manage Python 3

The Homebrew project provides a free and open source package manager for macOS that many people rely on. It gives Apple users a power similar to apt-get or yum. If you are a Homebrew user, you may already have Python installed. To quickly check, run:

$ brew list | grep python 
python 

If Python shows up under the command, it's installed. What version is it? Let's check:

$ brew info python 
python: stable 3.7.3 (bottled), HEAD 
Interpreted, interactive, object-oriented programming language 
https://www.python.org/ 
/usr/local/Cellar/python/3.7.2_1 (8,437 files, 118MB) * 
## further output not included ##

Okay, great! The Homebrew maintainers have updated the default Python bottle to point to the latest release. Since the Homebrew maintainers are more dependable at updating the release than most of us, we can use Homebrew's version of Python 3 with the following command:

$ brew update && brew upgrade python 

Now we want to point our alias (from above) to the copy of Python that Homebrew manages:

# If you added the previous alias, use a text editor to update the line to the following 
alias python=/usr/local/bin/python3 

To make sure the path above points to where Homebrew installed Python in our environment, we can run brew info python and look for the path information.

This method, of using Homebrew to manage our Python environment, is a good starting place, and it made sense to me at the time.

What if we still need Python 2?

It makes sense for anyone new to Python to begin with Python 3. But those of us who still need Python 2—for example, to contribute to a Python project that's only available in Python 2—can continue to use the default macOS Python binary available in /usr/bin/python:

$ /usr/bin/python 
>>> print("This runtime still works!") 
This runtime still works! 

Homebrew is so wonderful, it even offers a different formula for Python 2:

# If you need Homebrew's Python 2.7 run 
$ brew install python@2 

At any time, we can remove the aliases from our shell's configuration file to go back to using the default copy of Python on the system.

Don't forget to update pip to pip3!

The pip command is the default package manager specifically for Python packages. Although we changed our default Python command to be version 3, we have to alias our pip command separately if it's on the previous version. First, we need to check what version we're on:

# Note that this is a capital V (not lowercase) 
$ pip -V 
pip 19.0.3 from /Library/Python/2.7/site-packages/pip-19.0.3-py2.7.egg/pip (python 2.7)

To ensure we're installing packages compatible with our new version of Python, we'll use another alias to point to the compatible version of pip. Since we're using Homebrew as our package manager in this situation, we know it installed pip3 when we installed Python 3. The default path should be the same as Python 3, but we can confirm this by asking the shell to find it:

$ which pip3 
/usr/local/bin/pip3 

Now that we know the location, we will add it to our shell configuration file, as we did before:

$ echo "alias pip=/usr/local/bin/pip3" >> ~/.zshrc  
# or for Bash 
$ echo "alias pip=/usr/local/bin/pip3" >> ~/.bashrc

Last, we can confirm that running pip points to pip3 by opening a new shell or by resetting our current shell and seeing what we point to:

# This command reloads the current shell without exiting the session 
# Alternatively, exit the shell and start a new one
$ exec $0
# Now we can look to see where pip points us 
$ which pip 
pip: aliased to /usr/local/bin/pip3

We can avoid using Homebrew to update pip, but that requires a much longer tutorial from the Python documentation.

Do it right from the start

If you are just getting started with Python development on a macOS, do the necessary configurations to make sure you're using the right version of Python from the start. Installing Python 3, with or without Homebrew, and using alias will let you start coding, but it's not a good strategy for the long run. Using pyenv as a simple version management solution to get you off to a good start.


This article was originally published in May 2019 and has been updated by the editor.

Tags
I'm happiest at a microphone
Matt was an EMC storage expert, VMware vExpert, and former fan of other proprietary technologies. He now focuses on open source and DevRel adoption.
Moshe sitting down, head slightly to the side. His t-shirt has Guardians of the Galaxy silhoutes against a background of sound visualization bars.
Moshe has been involved in the Linux community since 1998, helping in Linux "installation parties". He has been programming Python since 1999, and has contributed to the core Python interpreter. Moshe has been a DevOps/SRE since before those terms existed, caring deeply about software reliability, build reproducibility and other such things.

Contributors

13 Comments

Great article. Thanks for sharing.

Ran into an issue right away with zlib. For anyone with the same problem error, you can find the solution at pyenv's github issues page. Issue number 1219

Just wanted to say thanks for the tutorial. I'm just now getting my feet wet with python coding and want to get started with the right foot forward!

Thanks for spending time and write this tutorial. I just want to add see if anyone is having this minor issue. Just right before "exec $SHELL" I have to quit iTerm2 completely in order to get the 3.7.4 reported. I think I can do source ~/.zshrc. I didn't try.

Thanks for letting me know Jay. I believe exec should be enough to instantiate a new shell in the current session, but source is sure to reload the information from the file.

In reply to by Jay (not verified)

I see what went wrong now. `$SHELL` is set to the default shell, which may not be what you're currently using. I've adjusted it to `exec $0` so $0 detects the current shell, not the default shell. Thanks again or noting this Jay!

In reply to by mbbroberg

If I put #!/usr/bin/python on a script, will the python 3.7 managed by pyenv
be used?

very nice work thanks a lot

Moshe was right. Please don't EVER use the system Python for development. Not even with virtual environments. You want to be able to switch versions at will.

Use "python -m pip" instead of running "pip" or "pip3" - that way the pip and python versions always match, whichever you currently have selected as "python".

I'd recommend you look at Conda as well - it's especially handy for people who aren't familiar with system administration.

Extra space before the eval keyword when using pyenv to control shell path. Make sure to remove it, or you will see "eval command not found". Make sure you also append the output to the correct rc file :).

I did all the steps as mentioned and they showed everything with python 3.7, except when in the end I do a python -V, it still shows the version as 2.7. Any idea how I could override the python version?

Hey Sauav! When you run `which python` does it show a shim pathway or a direct pathway to /usr/bin/python? If it's the latter, make sure you have the `eval "$(pyenv init -)"` statement in your .bashrc or .zshrc depending on your shell.

In reply to by Saurav Ganguli (not verified)

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.