My signature hobby project these days is a computerized instrument cluster for my car, which happens to be a DeLorean. But, whenever I show it to someone, I usually have to give them a while to marvel at the car before they even notice that there's a computer screen in the dashboard. There's a similar problem when I start describing the software; programmers immediately get hung up on "Why Perl???" when they learn that the real-time OpenGL rendering of instrument data is all coded in Perl. So, any discussion of my project usually starts with the history of the DeLorean or a discussion of the merits of Perl vs. other, more-likely tools.
The author and the Delorean. Photo by Michael Conrad.
I started the project in 2010 with the concept to integrate a computer in the dashboard to act as a personal assistant, but it quickly became a project about replacing the stock instrument cluster with something software-rendered. Based on the level of processing I wanted (I dream big) and the size of screen I wanted, I decided against the usual high-end microcontrollers people might use and instead went with a full Linux PC and desktop monitor with low-end microcontroller to read the analog measurements from the car. I was doing OpenGL and C++ at work at the time, so that was my first pick for software. I could write multiple articles about hardware selection, but I'll try to stay focused on the software for this one. (You can find more of that story on my website, nrdvana.net.)
Demo mode. Photo by Michael Conrad.
After several years of effort, it became apparent that C++ is not a good fit for my large-scale personal projects. Although C++ yields great performance and low resource usage, the biggest resource shortage I had was time and "mental state." Sometimes I would be away from the project for an entire month, and when I finally had a single day of free time to work on it, I spent it trying to remember where I left off. The worst aspect was that I usually couldn't finish refactoring my design in a single session, so when I came back to it weeks later, I wasn't catching all the places where the design change had broken the code. Also, while C++ is generally better than C for catching bugs, I would still end up with occasional memory corruption that could eat up hours of debugging time. There's also just a lot of development overhead to write the logging and debugging routines needed to diagnose a real-time, multi-threaded application.
Meanwhile, my day job had shifted to working on Perl. I didn't seek Perl on my own; it was just sort of thrust my way along with urgent projects. However, within a few months I was intrigued by its possibilities, and now it's my favorite language.
Enter Perl
In 2014, I took the plunge and rewrote the instrument cluster software in Perl. After years of trudging along with C++ I was able to get a working prototype (of the software, at least) within a few months, and move to completing the hardware and microcontroller in 2015.
My little Perl success story is primarily about agility. I'm not really a buzzword fan or the kind of guy who reads books about methodologies, but "agile" definitely means something to me now. I feel like Perl hits a magic sweet spot of providing enough structure to build a correct, performant program, while being minimal and generic enough to plug things together with ease, and even offering enough syntax features to express complex operations in terse but readable code. (If you aren't familiar with Perl's capabilities, see my companion article "Perl from a Systems Programmer Perspective," which elaborates on how Perl can be suited for systems work.)
Dash view. Photo by Michael Conrad.
The main, ongoing benefit is the ability to make ad-hoc changes. Because I don't have a lot of time to plan out the full requirements of my objects, it has been a great boost to productivity to just toss in an additional few attributes on unsuspecting objects, or quickly sort through a list of objects based on criteria that would require awkward reflection code in Java or C++. If I decide I like the change, I go back and rewrite it with properly declared attributes and interfaces. I've found I can author a new graphic widget, complete with animations, in less than an hour.
Toolchain
One of the real killers for the C++ version of my project was keeping all the binary-level code in sync. The various components (rendering, message bus, logic core, microcontroller firmware, control tools, debug tools) were all sharing binary data structures, and keeping the dependencies straight in the makefile was a headache. I'm personally sour toward the automake family of tools, so whenever I needed to do something odd (like compile the microcontroller code using avr-gcc), I would risk getting frustrated and detouring into a new grand scheme to create a replacement for autotools (certainly a thing I don't need to waste time on).
During my change to Perl, I converted the microcontroller to show up as a Linux serial device and changed the protocol to strings of short text notation. (The messages are actually smaller than the binary packet structs I had been using before.) This let me debug it with a simple socat on /dev/ttyS0. It also simplified the daemon that talks to the microcontroller. The C++ version was written with two threads, since I was using libusb, and its easiest mode of operation has a blocking read method. The Perl version simply opens a stream to the character device and reads lines of text.
I made a similar change to the host-side communication and had the daemon generate lines of JSON instead of binary packets. Since it is so incredibly easy to implement this in Perl with libraries like AnyEvent, I ditched the "message bus" idea entirely and just had each program create its own Unix socket, to which other programs can connect as needed. Debugging a single thread is much less painful, and there wasn't even much debugging to do anyway, because AnyEvent does most of the work for me.
With everything passed around as JSON, there are no longer any message structs to worry about. None of my Perl programs requires a make process anymore, so the only piece of the project that still has a makefile is the microcontroller firmware, and it is simple enough that I just wrote it out by hand.
Performance
Processing low-level math directly with Perl can be slow, but the best way to use Perl where performance counts is to glue together C libraries. Perl has an extension system called XS to help you bind C code to Perl functions, but even better, there's a CPAN repository module called Inline, which lets you paste C or C++ (and others) directly into a Perl module, and it compiles the first time the module is loaded. (But, yes, I pre-compile them before building the firmware image for the car.)
Thanks to Inline, I can move code back and forth from Perl to C as needed without messing around with library versions. I was able to bring over some of my C++ classes directly into the new Perl version of the instrument cluster. I was also able to wrap the C++ objects of the FreeType for OpenGL (FTGL) library, which is an important piece I didn't want to have to re-invent.
The CPU usage of the system was about 15% with the C++ implementation. With Perl it's about 40%. Almost all of that is the rendering code, so if I need to I can always push more of it back into C++. But, I could also just upgrade the computer, and 40% isn't even a problem because I'm maintaining a full 60 frames per second (and I'm running a 6.4-watt processor).
Broader horizons
Perl's CPAN public package repository is especially large, documented, tested, and stable compared to other languages. Naturally this depends on the individual authors (and there are plenty of counter-examples), but I've been impressed with the pervasive culture of test coverage and helpful documentation. Installing and using new Perl modules is also ridiculously easy. Not only do I avoid the toolchain efforts of C/C++, I get the advantage of Perl authors who have already overcome conflicting thread models or event loops or logging systems to give me a plugin experience.
With everything written in Perl, I can just grab anything I like off CPAN. For instance, I could have the car send me emails or text messages, host a web app for controlling features via phone, write Excel files of fuel mileage, and so on. I haven't started on these features yet, but it feels nice that the barriers are gone.
Contributing back
In a decade of doing C++, I never once released a library for public consumption. A lot of it is due to the extreme awkwardness of autotools, and the fact that just creating a system-installed C++ library is a royal pain even without packaging it up properly for distribution.
Perl makes module authoring and testing and documentation extremely easy. It is so easy that I wrote test cases and documentation for my Math-InterpolationCompiler for my own benefit, and then published them on CPAN because, "why not?" I also became maintainer of X11-Xlib and greatly expanded its API, and then wrote X11-GLX so that I could finally have all my OpenGL setup code in proper order. (This was also part of my attempt to make the instrument renderer into a compositing window manager, which turned out to be much harder than I expected.) Currently, I'm working on making my maps/navigation database a CPAN module as well.
But why not...
"But, why not Language X" you say, with "Python" a common value for X. Well, for one, I know a lot more Perl than Python. I'm using a lot of deep and advanced Perl features, so picking up Python would be another large learning curve. I'm also partial to Perl's toolchain, especially elements like prove and perldoc. I suspect it's possible to do it all in Python as well, but I have no compelling reason to switch. For any other language X... well no other language can match the wealth of packages that Perl or Python offer, so I'm less inclined to experiment with them. I could mix languages, since my project is comprised of multiple processes, but having everything in the same language means I can more easily share code between programs.
"Why not Android?" is another common question. Indeed, a tablet is a much more embeddable device than a whole PC, and it comes with access to mapping apps. The obvious first problem is, I'd be back on Java and lose most of my prized agility. Second, I'm not aware of any way to merge the graphics of separate apps (such as using Google Maps as a texture within the dashboard), although there might be one. And third, I've been working on a feature to take video feeds and tie them directly into the graphics as textures. I don't know of any tablets that could capture video from external sources in real time at a low enough latency, much less directly into a graphics texture buffer. Linux desktop software is much more open to this sort of deep mangling, so I'll probably continue with it.
On the whole, I'm just happy I've finished enough that I can drive my DeLorean.
3 Comments