By relaxing its rather constrictive membership process, The Python Software Foundation is starting to uncoil. And Nick Coghlan, Provisioning Architect in Red Hat Engineering Operations, couldn't be happier.
Coghlan recently earned a seat on the Foundation's Board of Directors, where he'll help oversee the open source programming language's governing body. He assumes his position at a critical juncture for Python. Many high-profile Linux distributions are beginning to offer Python 3 by default, encouraging developers to transition to the latest version as Python 2 support dwindles. And the PSF is altering its membership structure—making joining a much more straightforward matter for developers all over the world.
Coghlan welcomes the group's pivot toward greater inclusivity. A Python user since 2002, when he was working at Boeing Australia's High Frequency Modernisation Project, Coghlan joined the PSF in 2005. But at that time, he said, the Foundation's membership process didn't completely embody the values that initially drew him to Python.
"I first joined the PSF shortly after becoming a CPython core developer," Coghlan said. "At that time, PSF membership was still 'by invitation only.'"
But that's changing, he said, as the Foundation opens up.
"The PSF has transitioned to an open membership model that better aligns with our commitment to growing the global Python community," Coghlan said.
As he settles into his new role, Coghlan explained to us the challenges currently facing Python and its community—and laid out his plans for the future.
What is the Python Software Foundation, and what are the responsibilities of its Board of Directors?
The Python Software Foundation is a US-based non-profit, and our mission is "to promote, protect, and advance the Python programming language, and to support and facilitate the growth of a diverse and international community of Python programmers."
The role of the board is to act as the elected representatives of the membership in deciding how best to pursue that mission. At this point, our major focus is to work through the transition from the previous closed membership model (where new PSF members had to be invited to join by existing members) to the current model, where membership is open to anyone that wishes to join. While the core change to the membership model has already happened, there are several associated changes around the way working groups are managed and contributions are recognised that still need to be put into place.
From a day-to-day perspective, the board is also directly responsible for reviewing most grant proposals submitted to the PSF, as well as looking for additional opportunities to improve the support we provide to the broader Python community.
What do you hope to accomplish in your time on the board?
One of the ongoing challenges of any open source project is getting people involved in maintaining the support infrastructure for the project, rather than just working on the project itself. This kind of work plays a big part in defining the "user experience" of a community, but is also often underappreciated and unexciting, so it can be hard to get volunteers passionate about it. I think there's an opportunity for the PSF to contribute to improving several aspects of that experience for the core development community, although (as with many things) there are various practical considerations that make that easier said than done.
In more general terms, I'm really excited about the shift to an open membership model, and helping to put those changes into practice. For a long time, the PSF has been struggling to reconcile the mismatch between the relatively closed membership model of the organisation itself and the emerging commitment to openness and diversity in the broader Python community, and these changes mean the Foundation's structure now better reflects the community's ideals. With board members from four countries (Australia, India, Germany and the US), we're also in a good position to start forging even stronger links between the various parts of the global Python community.
What makes Python such a special and important project and community?
For me, the most fascinating thing about Python is the sheer breadth of the domains it competes in. In the projects I worked on at Boeing, Python became our "go to" glue language for getting different parts of a complex system to play nicely together, as well for writing simulation tools for testing environments. Linux distributions tend to use it in a similar fashion. In the scientific space it goes head to head with the likes of MATLAB for numeric computing, and R for statistical analysis. It was the original implementation language for YouTube, and the language of choice for OpenStack components, yet still simple enough to be chosen as the preferred programming language for the Raspberry Pi and One Laptop Per Child educational programs. With the likes of Maya and Blender using it as their embedded scripting engine, animation studios love it because animators can learn to handle tasks that previously had to be handled by the studios' development teams.
That diversity of use cases can make things fraught at times, especially in core development where the competing interests can often collide, but it's also a tremendous strength.
What are the greatest immediate challenges facing Python?
Managing the impact of the OpenStack leviathan on the core development team is a big one. At the moment, we have tremendous amounts of corporate funding being poured into the Linux kernel and into OpenStack, but next to nothing going into the CPython runtime that provides the link between them. That particular challenge is also a tremendous opportunity, however, as funnelling even a fraction of the resources going into OpenStack into full time contributions to CPython could make some significant inroads into the ever increasing number of open bug reports and requests for enhancement at bugs.python.org.
We're also playing catch-up in the cross-platform dependency management space, as Python's original software distribution tools were born in an era where "into open source" and "uses Linux" were close to synonymous. Modernising that infrastructure to meet the expectations of a generation that grew up with open source and widespread Internet access isn't easy, but it's also essential as most of the easy language-level usability improvements are behind us, and we need to make it easier for Python users to access the many high quality tools that exist in the Python ecosystem beyond CPython, the standard library, and python.org.
Related to that, the discoverability of the broader Python ecosystem is a significant problem in general. Unlike newer, more centralised language communities that are often focused around a particular problem domain or sponsoring organisation, Python is old enough to have become a loose, decentralised sprawl with a wide variety of powerful options available to users. Yet, it isn't as simple as just listing all the things folks might be interested in somewhere on python.org—that can easily overwhelm users with too many choices when they don't know enough to choose sensibly between them.
How do we direct users lamenting the lack of static typing to analysis tools like pylint which can similarly help detect structural defects in code? How do we direct folks seeking faster code execution to JIT compilation with PyPy or Numba, or ahead of time compilation with Cython? How do we direct novice web developers to an all-inclusive framework like Django, while pointing out more flexible frameworks like Flask and Pyramid as alternatives more experienced developers may wish to explore? How do we highlight the many excellent Integrated Development Environments developers may wish to consider, without overwhelming them with choice? And that's without even getting into the scientific side of things, or the merits of IPython and IPython notebooks over using the minimalist default environments that are provided with the CPython runtime.
Managing the long term support commitments for Python 2 that are part and parcel of the Python 3 transition is also fairly high up the list. Long term maintenance is one of the dreariest tasks in the open source world, and, at four years, the support provided for Python 2.7 has already been exceptional for an almost purely volunteer development effort. I think we actually have a pretty good precedent for addressing this problem already set by the handling of long term support kernels in the Linux space: commercial redistributors collaborating upstream to minimise the need to duplicate each others' work. We've already had some success on that front, with Microsoft's Visual Studio team now contributing developer time to handle the creation of the binary installers for Windows, and we'll continue to make the point to downstream redistributors that the rate of volunteer provided patches to the upstream Python 2.7 branch is already slowing down, so they may need to make additional investments in upstream maintenance to continue meeting their own customer support obligations.
What can we expect from the Python Foundation—and from the Python programming language—in the future?
From the Foundation side, the aim is to increase transparency around the Foundation's activities and to increase participation in the Foundation itself. For a long time, many members of the community have perceived the Foundation as an external entity, only for the elite few that were lucky enough to receive an invitation to participate, and even members who had been invited would often ask "Well, what's different now that I'm a member?" and not get a particularly clear answer. With the transition to an open membership model, I hope to see the Foundation grow to a point where it is clearly operated by the community, for the community, with participation open to anyone that is interested in being involved.
For the programming language, a comment I occasionally make is that the Python Packaging Authority are having a greater impact on the future of Python right now than python-dev itself. The reason I say that is because python-dev does much of its significant work on timescales of years; when we add new features to solve problems we're having today, we know it will often be two, three, four or more years before we can rely on having them consistently available.
The Python Packaging Authority, by contrast, releases a new version of pip every six months or so, and pip is our chosen tool for making the wide array of Python software on the Python Package Index more readily available to all Python users. And unlike a lot of language specific packaging tools, which often appear to focus on the Software-as-a-Service use case to the exclusion of all others, we're attempting to explicitly take the needs of system integrators and Python redistributors into account when designing the overall system. Part of that's due to backwards compatibility concerns, but the other key aspect is about lowering the barriers to entry for the rest of the Python ecosystem into the many and varied redistribution channels that are enjoyed by CPython itself.
If we're successful in our aims, then it will become possible to convert almost any PyPI package into a policy compliant downstream package automatically, and in those cases where it isn't, the tooling should support tweaking the installation process appropriately without needed to duplicate existing metadata from the upstream package.
That said, there are still some exciting changes happening in the language and CPython runtime itself. For the web and network programming community, the asyncio module added in Python 3.4 finally brings full fledged asynchronous IO features to the Python standard library. An increased focus on security and better access to underlying operating system features has brought support for isolated mode (where Python 3 ignores almost all environmental settings), support for openat and other directory descriptor based commands, an improved hashing algorithm, access to operating system SSL certificate stores, a variety of of other SSL improvements, and more configurable behaviour of the multiprocessing module, include the ability to completely avoid using the fork system call.
Debugging support has been much improved, with the introduction of enumerations to the standard library making socket module constants more self-documenting, enhanced introspection support for C extension modules and builtins, and facilities for printing tracebacks on segmentation faults and tracing memory allocations. Common programming operations are now much simpler, thanks to the inclusion of the pathlib, statistics and ipaddress modules in the standard library. A variety of longstanding issues around object and interpreter cleanup have also been addressed.
Looking forward, the one confirmed change for Python 3.5 so far means that Python will gain a dedicated infix operator for matrix multiplication, the culmination of more than a decade of discussion amongst the scientific Python community of their needs to improve the readability of scientific and mathematical code. The coming year will also see major Linux distributions like Ubuntu and Fedora continue their migration towards only including Python 3 on their installation media, which may result in some adjustments to the way Python 3 interacts with operating system boundaries on Linux systems.