Ask Hackaday: What’s Your Favourite Build Tool? Can Make Ever Be Usurped?
What do you do whilst your code’s compiling? Pull up Hackaday? Check Elon Musk’s net worth? Research the price of a faster PC? Or do you wonder what’s taking so long, and decide to switch out your build system?
Clamber aboard for some musings on Makefiles, monopolies, and the magic of Ninja. I want to hear what you use to build your software. Should we still be using make in 2021? Jump into the fray in the comments.
What is a Build Tool Anyway?
Let’s say you’ve written your C++ program, compiled it with g++ or clang++ or your compiler flavor of the week, and reveled in the magic of software. Life is good. Once you’ve built out your program a bit more, adding code from other files and libraries, these g++ commands are starting to get a bit long, as you’re having to link together a lot of different stuff. Also, you’re having to recompile every file every time, even though you might only have made a small change to one of them.
People realised fairly early on that this sucked, and that we can do better. They started to make automated software that could track compilation dependencies, track which bits of code were tweaked since the last build, and combine this knowledge to automatically optimise what gets compiled – ensuring your computer does the minimum amount of work possible.
Enter: GNU Make
Yet another product of the famous Bell Labs, make was written by [Stuart Feldman] in response to the frustration of a co-worker who wasted a morning debugging an executable that was accidentally not being updated with changes. Make solves the problems I mentioned above – it tracks dependencies between sources and outputs, and runs complex compilation commands for you. For numerous decades, make has remained utterly ubiquitous, and for good reason: Makefiles are incredibly versatile and can be used for everything from web development to low level embedded systems. In fact, we’ve already written in detail about how to use make for development on AVR or ARM micros. Make isn’t limited to code either, you can use it to track dependencies and changes for any files – automated image/audio processing pipelines anyone?
But, well, it turns out writing Makefiles isn’t actually fun. You’re not adding features to your project, you’re just coercing your computer into running code that’s already written. Many people (the author included) try to spend as little of their life on this planet as possible with a Makefile open in their editor, often preferring to “borrow” other’s working templates and be done with it.
The problem is, once projects get bigger, Makefiles grow too. For a while we got along with this – after all, writing a super complex Makefile that no-one else understands does make you feel powerful and smart. But eventually, people came up with an idea: what if we could have some software generate Makefiles for us?
CMake, Meson, Autotools et al
Yes, there are a sizeable number of projects concerned only with generating config files purely to be fed into other software. Sounds dumb right? But when you remember that people have different computers, it actually makes a lot of sense. Tools like CMake allow you to write one high-level project description, then will automatically generate config files for whatever build platforms you want to use down the line – such as Makefiles or Visual Studio solutions. For this reason, a very large number of open source projects use CMake or similar tools, since you can slot in a build system of your choice for the last step – everyone’s happy.
Except, it’s really quite hard to tell if everyone is happy or not. As we know, when people selflessly spend time writing and maintaining good quality open source software, others are very kind to them online, do not complain, and do not write passive-aggressive blog posts about 27 reasons why they’re never using it again. Just kidding!
What I’m getting at here is that it’s hard to judge popular opinion on software that’s ubiquitous, because regardless of quality, beyond a critical mass there will always be pitchfork mobs and alternatives. Make first appeared in 1976, and still captures the lion’s share of many projects today. The ultimate question: is it still around because it’s good software, or just because of inertia?
Either way, today its biggest competitor – a drop-in replacement – is Ninja.
Ninja
Ninja was created by [Evan Martin] at Google, when he was working on Chrome. It’s now also used to build Android, and by most developers working on LLVM. Ninja aims to be faster than make at incremental builds: re-compiling after changing only a small part of the codebase. As Evan wrote, reducing iteration time by only a few seconds can make a huge difference to not only the efficiency of the programmer, but also their mood. The initial motivation for the project was that re-building Chrome when all targets were already up to date (a no-op build) took around ten seconds. Using Ninja, it takes under a second.
Ninja makes extensive use of parallelization, and aims to be light and fast. But so does every other build tool that’s ever cropped up – why is Ninja any different? According to Evan, it’s because it didn’t succumb to the temptation of writing a new build tool that did everything — for example replacing both CMake and Make — but instead replaces only Make.
This means that it’s designed to have its input files generated by a higher-level build system (not manually written), so integrates easily with the backend of CMake and others.
In fact, whilst it’s possible to handwrite your own .ninja files, it’s advised against. Ninja’s own documentation states that “In contrast [to Make], Ninja has almost no features; just those necessary to get builds correct. […] Ninja by itself is unlikely to be useful for most projects.”
Above you can see the differences in incremental build times in a medium-sized project. The two Ninja-based systems are clear winners. Note that for building the entire codebase from scratch, Ninja will not be any faster than other tools – there are no shortcuts to crunching 1s and 0s.
In his reflections on the success and failure of Ninja, Evan writes that:
“The irony of this aspect of Ninja’s design is that there is nothing preventing anyone else from doing this. Xcode or Visual Studio’s build systems (for example) could just as well do the same thing: do a bunch of work up front, then snapshot the result for quick reexecution. I think the reason so few succeed at this is that it’s just too tempting to mix the layers.”
It’s undeniable that this approach has been successful, with more and more projects using Ninja over time. Put simply, if you’re already using CMake, I can’t see many reasons why you wouldn’t use Ninja instead of make in 2021. But I want to know what you think.
Over to you
It’s impossible to write about all the build tools around today. So I want to hear from you. Have you switched from make to Ninja? Do you swear by Autotools, Buck or something else? Will make ever go away? Will there ever be a tool that can eclipse them all? Let me know below.
Post a Comment