I met a lot of wonderful people and learned a great deal about how the Python community works at PyCon 2019 in Cleveland. But my main technical takeaway has to do with the magic of JupyterLab, which is described as a web-based user interface for Jupyter Notebook, an application "that allows you to create and share documents that contain live code, equations, visualizations, and narrative text."
JupyterLab is the god-mode of programming Python.
That's my takeaway so far from #pycon2019.— Matt @ Home ? (@mbbroberg) May 2, 2019
I felt like a wizard when working on Python code in JupyterLab, long before I felt as confident developing data science-related Python from the command line. Here is a little bit about that magic and how you can get hands-on with it.
Project Jupyter, Jupyter Notebook, JupyterLab, and Python
Before PyCon, I'd heard of Jupyter Notebook, but I never quite understood how it relates to Python. Let's start at the top.
Project Jupyter is the umbrella organization overseeing the design of several interactive and highly visual software development interfaces that allow for code to be executed in a visual way.
Project Jupyter is a non-profit, open source project, born out of the IPython Project in 2014 as it evolved to support interactive data science and scientific computing across all programming languages. Jupyter will always be 100% open source software, free for all to use and released under the liberal terms of the modified BSD license.
The power of Jupyter projects comes in the form of kernels, which act as a “computational engine” to execute code contained in a document. The original kernel is for Python, called IPython, though there are many more available. As of today, 128 kernels are listed on the project wiki for everything from Ansible to Fortran.
The more visible part of the Jupyter project comes in the form of its user interfaces (UIs) where developers can visually program in any language supported by a kernel. The most well-known UI under the project's umbrella is Jupyter Notebook, where users develop software in a notebook.
The notebook extends the console-based approach to interactive computing in a qualitatively new direction, providing a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results.
Jupyter Notebook is considered to be the organization's flagship project, and it's had a massive impact on code visualization since it began back in 2011. More recently, JupyterLab launched as a more modular design for the future of Jupyter UIs.
Both Jupyter Notebook and JupyterLab allow for Python development in a more visual way and are powerful ways to edit code.
Why JupyterLab instead of Jupyter Notebook?
Jupyter Notebook's long history comes with some cost to flexibility.
According to Jupyter's blog, the project's background dating from 2011 makes it "difficult to customize and extend." That made sense as I read more about how all of these tools and standards developed over a long period. Way back in 2001, IPython, a Python-specific notebook standard, was developed by Fernando Perez. IPython maintains a standard method of writing notebooks in the Python language, and in recent years, Jupyter projects became the place to render them. The Jupyter Notebook README gives a summary:
Jupyter notebook is a language-agnostic HTML notebook application for Project Jupyter. In 2015, Jupyter notebook was released as a part of The Big Split™ of the IPython codebase. IPython 3 was the last major monolithic release containing both language-agnostic code, such as the IPython notebook, and language-specific code, such as the IPython kernel for Python. As computing spans across many languages, Project Jupyter will continue to develop the language-agnostic Jupyter notebook in this repo and, with the help of the community, develop language-specific kernels which are found in their own discrete repos. [The Big Split™ announcement] [Jupyter Ascending blog post]
In 2018, the JupyterLab project announced it was ready for users. Its decision to start recently, when there is more certainty around standardization and how to provide a high-performance notebook experience, makes sense to me. It also solved for Jupyter Notebook's extension challenges by building on top of an extension system that will get away from the challenges faced in extending Notebook.
What does this all have to do with programming in Python? Python is the wildly popular programming language that's growing more popular for data science analysis. Any IPython file (.ipynb) can run in Jupyter projects for an incredible development experience.
| Project | Description |
|---|---|
| Project Jupyter | "Project Jupyter exists to develop open source software, open standards, and services for interactive computing across dozens of programming languages." |
| Jupyter Notebook | "The notebook extends the console-based approach to interactive computing in a qualitatively new direction." |
| JupyterLab | "JupyterLab is the next-generation web-based user interface for Project Jupyter." |
| IPython | "IPython provides a rich architecture for interactive computing." |
If all these project names are getting mixed up in your head, remember these points:
- Project Jupyter provides standards to visualize programming languages.
- Jupyter Notebook is a highly popular visualization software.
- JupyterLab is a modern redesign of Jupyter Notebook.
- IPython provides tooling to run Python code in these projects.
According to the JupyterLab documentation, it's slated to eventually replace Jupyter Notebook.
Try it with a tutorial
Now that you know more about the terminology and history, try this "IPython and Jupyter in Depth" tutorial presented at PyCon. I found it to be a phenomenal introduction to JupyterLab.
After installing it (I used Anaconda to manage the installation), navigate to "1 - Beyond Plain Python.ipynb" in JupyterLab.
Magic is magical
JupyterLab has particularly powerful functionality that begins with the % symbol. Each of the many functions you can run makes JupyterLab the most flexible development environment I have ever seen.
While you're still in the notebook loaded above (zoomed in here from the screenshot above), press Option+Return (or Alt+Enter) to make a new cell:
Then use a bang/exclamation point (!)—my favorite bit of magic—to send a command to a terminal session in the current directory. For instance, running !ls lists out local files:
This can become much more powerful as you mix Python with your magic; for example, you can pull your local files into a Python variable:
While Python has a powerful os module that allows you to interact with the development environment, it's fun to have the option of using Bash syntax to quickly pull in the data you want while keeping it in a Python object type, like a list of strings.
The takeaways
If you have ever wanted to use Jupyter Notebook to explore Python, I hope this article helps you understand how the ecosystem is evolving and that you decide to jump into JupyterLab right away. If you're new to Python and new to the command line, JupyterLab will make you feel like you have superpowers from the very beginning. I have never had so much fun learning through a UI as I did with this tutorial. Said another way, JupyterLab feels like playing video games with the cheat codes enabled. You can do seemingly anything!
If you're more familiar with a Linux-based command line interface, be sure to use the bang operator a whole bunch while you're getting started. Writing !ls -la will call to your default terminal emulator and have you running commands to make sure you understand where you are and what you're doing.
I highly recommend exploring more of the magic available in JupyterLab while having fun with this tutorial.
3 Comments