8 great Python libraries for side projects

These Python libraries make it easy to scratch that personal project itch.
307 readers like this.
Hands on a keyboard with a Python book

WOCinTech Chat. Modified by Opensource.com. CC BY-SA 4.0

We have a saying in the Python/Django world: We came for the language and stayed for the community. That is true for most of us, but something else that has kept us in the Python world is how easy it is to have an idea and quickly work through it over lunch or in a few hours at night.

This month we're diving into Python libraries we love to use to quickly scratch those side-project or lunchtime itches.

To save data in a database on the fly: Dataset

Dataset is our go-to library when we quickly want to collect data and save it into a database before we know what our final database tables will look like. Dataset has a simple, yet powerful API that makes it easy to put data in and sort it out later.

Dataset is built on top of SQLAlchemy, so extending it will feel familiar. The underlying database models are a breeze to import into Django using Django's built-in inspectdb management command. This makes working with existing databases pretty painless.

To scrape data from web pages: Beautiful Soup

Beautiful Soup (BS4 as of this writing) makes extracting information out of HTML pages easy. It's our go-to anytime we need to turn unstructured or loosely structured HTML into structured data. It's also great for working with XML data that might otherwise not be readable.

To work with HTTP content: Requests

Requests is arguably one of the gold standard libraries for working with HTTP content. Anytime we need to consume an HTML page or even an API, Requests has us covered. It's also very well documented.

To write command-line utilities: Click

When we need to write a native Python script, Click is our favorite library for writing command-line utilities. The API is straightforward, well thought out, and there are only a few patterns to remember. The docs are great, which makes looking up advanced features easy.

To name things: Python Slugify

As we all know, naming things is hard. Python Slugify is a useful library for turning a title or description into a unique(ish) identifier. If you are working on a web project and you want to use SEO-friendly URLs, Python Slugify makes this easier.

To work with plugins: Pluggy

Pluggy is relatively new, but it's also one of the best and easiest ways to add a plugin system to your existing application. If you have ever worked with pytest, you have used pluggy without knowing it.

To convert CSV files into APIs: Datasette

Datasette, not to be confused with Dataset, is an amazing tool for easily turning CSV files into full-featured read-only REST JSON APIs. Datasette has tons of features, including charting and geo (for creating interactive maps), and it's easy to deploy via a container or third-party web host.

To handle environment variables and more: Envparse

If you need to parse environment variables because you don't want to save API keys, database credentials, or other sensitive information in your source code, then envparse is one of your best bets. Envparse handles environment variables, ENV files, variable types, and even pre- and post-processors (in case you want to ensure that a variable is always upper or lower case, for instance). 

Do you have a favorite Python library for side projects that's not on this list? Please share it in the comments.

User profile image.
Lacey Williams Henschel is a software engineer with REVSYS and part of the organizing team for DjangoCon US. In the past, she's chaired DjangoCon US, organized several Django Girls workshops, taught courses for Treehouse, and written about accessibility at tech events.
User profile image.
Jeff Triplett is an open source developer and works as a software engineer, consultant, and partner at REVSYS. He is a Board of Director at The Python Software Foundation. He occasionally blogs on his website.

1 Comment

"Do you have a favorite Python library for side projects that's not on this list? Please share it in the comments."

PySimpleGUI is a good way for beginners to get started in creating a GUI for their code. Perhaps a project could be to add a GUI to the front-end of a favorite command line program.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.