by Adam Brett

Introduction to Make

Make is an awesome build tool that is seeing a bit of a resurgence. It's a powerful tool, but can be complicated to get started with. There are no good guides around that help you learn make in a simple and straight forward way, add to that that most are heavily focused on C code, and it quickly gets too much.

Make is not limited to C, and most examples of Makefiles I've come across are complicated and difficult to understand. It doesn't have to be that way.

Getting Started With Make

The first thing to understand about Make is that it was designed for C. Out of the box, it will do its best to compile a C file with no configuration.

Drop this into a file called hello_world.c somewhere:

#include<stdio.h>

int main() {
    printf("Hello World\n");
    return 0;
}

Now open up a terminal, and from the same directory, type:

make hello_world

As long as you have a C compiler installed, you should now see a file called hello_world in the current directory. Run ./hello_world and you should see the familiar output.

Make Makes Files

Running make is always in the format make <target>, where <target> is usually the file you want to create. It doesn't always have to be this way, but that's how make is designed to be used.

From the same directory, if you were to run make hello_world again, you should see:

make: 'hello_world' is up to date.

This is because make will track changes to files, and only re-run a target if the source file is newer than the file you want to build. This is a great time saver when you have a long list of dependencies, and only one has changed, as make will only re-compile those that need changing.

Edit hello_world.c, and change Hello World! to Goodbye World, then run make hello_world again, this time, it should compile, and running ./hello_world should now print Goodbye World.

Makefiles

Most of us don't write C on a day-to-day basis, so instead, we need to tell make how to work with our language of choice, or do the things we want it to do. To do that, we need a Makefile.

A Makefile is a list of targets, denoted by a colon, followed by a list of shell commands to run, which must be indented by a single tab. It looks like this:

test:
    echo 'Hello World'

Create a new file in your current directory, and call it Makefile. Add the above, and then run make test. You should see the command make has run, followed by its output:

echo 'Hello World'
Hello World

Phony Targets

What we've just done goes against how make was designed. Remember, targets should be files we want to create, not arbritrary names, and we didn't create a file called test, instead we just used it to echo a message.

This can cause problems for make. If you were to have a file or directory that was called test, make wouldn't run this target. Let's see that in action:

touch test
make test

You should see a familiar message:

make: 'test' is up to date.

In these instances, what you want to do is list the target as phony, which is way of telling make: "this doesn't really generate a file". Makefiles have a special target for this, called .PHONY, and it's just a list of phony targets. Update your Makefile to match the below:

test:
    echo 'Hello World'

.PHONY: test

Now run make test again and you should now see Hello World, even though we have a file called test.

Something Useful

Let's put this to some use, and see a real example by using make to install some npm modules. Create a new file called package.json with an empty json object:

echo {} > package.json

Now grab a simple module:

npm install --save lodash

The contents of your package.json should now look like this:

{
  "dependencies": {
    "lodash": "^4.16.4"
  }
}

Now open up your Makefile and replace it with the following:

node_modules:
    npm install

We haven't listed this as phony, because this actually creates a directory called node_modules. Let's run our new target:

$ make node_modules
make: 'node_modules' is up to date.

That's to be expected, kindof, as node_modules already exists. Let's trash that directory and try again:

rm -rf node_modules/
make node_modules

You should see the familiar npm install output, and our node_modules directory should be re-created.

Running make node_modules for a third time should yield the same make: 'node_modules' is up to date. message.

Dependent Files

Let's try something a little different. Manually edit your package.json and add a new dependency. Update it to match the following:

{
  "dependencies": {
    "lodash": "^4.16.4",
    "request": "^2.75.0"
  }
}

This new dependency is now missing, so we want to run make to go get it. Drop back to your terminal and run make node_modules.

At this point, make should tell you that node_modules is still up-to-date, but we know it isn't. This is because make doesn't know about the relationship between node_modules and package.json. We need to tell it. To do that, list the dependencies on the same line as the target name, it's that simple:

node_modules: package.json
    npm install

After updating your Makefile to match the above example, run make node_modules again, it should run, and install the request library. Running it again after this should tell you that node_modules is up-to-date. This is the behaviour we're looking for. If we change package.json, we need to update node_modules. In simple forms, think of it like this:

create-this-file: when-this-file-is-newer
    by-doing-this

If you have multiple file dependencies, simply list them on the same line, separated by a space.

Dependent Targets

Imagine a scenario where you don't keep package.json in source control, and instead you generate it by some other means. You probably want a target in your Makefile to generate it, but then your node_modules target doesn't have a physical file to depend on.

This is ok. Make can infinitely nest targets as dependencies of each other, and will run them in order. This is where make starts to become really powerful. This example is contrived, but please bear with me.

Rename package.json to package.example, and remove node_modules:

mv package.json package.example
rm -rf node_modules

We're now back to a pretty blank state, but if we were to run make node_modules, it would fail:

make: *** No rule to make target 'package.json', needed by 'node_modules'.  Stop.

Let's add a rule to generate package.json from our package.example file:

package.json: package.example
    cat package.example > package.json

node_modules: package.json
    npm install

Our package.json should be updated when package.example changes, so I've set that as a dependency. I've also used cat instead of cp, because we want it to work even if package.json already exists. We could have written it as:

package.json: package.example
    rm -f package.json
    cp package.example package.json

But the cat example is more terse, so that's what I went for. We wouldn't simply mv package.example package.json, because then package.example would be missing as the package.json dependency on subsequent runs.

With this change make knows how to generate the package.json for our node_modules target, let's run make node_modules and see what happens:

cat package.example > package.json
npm install
npm WARN package.json @ No description
npm WARN package.json @ No repository field.
npm WARN package.json @ No README data
npm WARN package.json @ No license field.
[email protected] node_modules/request
[email protected] node_modules/lodash

Nice! Let's try make package.json and make node_modules to see what happens:

make: 'package.json' is up to date.
make: 'node_modules' is up to date.

Excellent. If we were to change package.example, make would know to re-generate package.json before updating node_modules. We could also call make package.json directly to see those changes without affecting node_modules, which would then be updated on a subsequent make node_modules, which would know not to bother re-generating package.json.

Compiling & Transpiling

Alarming as it may be, transpiling ES6 code to ES5 has become the norm for modern JavaScript, and is something make is really good at. Let's see how we can handle that. Create a src directory, and in there, put some ES6 code:

mkdir src
touch src/index.js

Now open up index.js in your editor, and add:

import _ from 'lodash';

const fibonacci = _.memoize(function(n) {
  return n < 2 ? n : fibonacci(n - 1) + fibonacci(n - 2);
});

console.log(fibonacci(process.argv[2]));

If you try to run this with a recent version of node, it should fail on the import:

$ node src/index.js 10

src/index.js:1
(function (exports, require, module, __filename, __dirname) { import _ from 'lodash';
                                                              ^^^^^^

SyntaxError: Unexpected reserved word
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:373:25)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:139:18)
    at node.js:974:3

Let's grab babel and setup some transpiling. Open package.example in your editor and replace request with babel:

{
  "dependencies": {
    "lodash": "^4.16.4"
  },
  "devDependencies": {
    "babel-cli": "^6.0.0",
    "babel-preset-es2015": "^6.0.0"
  }
}

Next, let's update our dependencies:

make node_modules

You should now have access to babel in ./node_modules/.bin/babel. Run ./node_modules/.bin/babel --version to double check:

6.16.0 (babel-core 6.17.0)

This just proves our process from earlier is working. Babel requires a config file to load the ES2015 plugin, so let's add that:

echo '{
  "presets": ["es2015"]
}' > .babelrc

Now let's add a build target to our Makefile which will output our ES5 source in a build directory, to keep in line with what make expects:

package.json: package.example
    cat package.example > package.json

node_modules: package.json
    npm install

build: node_modules src
    ./node_modules/.bin/babel src -d build

build won't work without the node_modulesand src directories, and we will probably want to regenerate our ES5 source if either of them change, so we list those as our dependencies. Let's run make build and see what happens:

./node_modules/.bin/babel src -d build
src/index.js -> build/index.js

Cool. Let's see what happens when we try to run it:

$ node build/index.js 10
55

And it works! Let's run make build again and see what happens:

./node_modules/.bin/babel src -d build
src/index.js -> build/index.js

It's created it again, even though nothing has changed! That's not what we want at all. This is probably because we're depending on directories, and not files. Let's update our Makefile to reference src/index.js directly and see what happens:

build/index.js: node_modules src/index.js
    ./node_modules/.bin/babel src/index.js -d build/index.js

You can run it with make build/index.js:

make: 'build/index.js' is up to date.

That's better, but we don't want to be doing that for all of our ES6 files, there could be hundreds of them.

Fortunately, Make has you covered.

Pattern Targets

Pattern targets are exactly the same as normal targets, except they allow you to treat all files that match a pattern in the same way, rather than specifying a specific file. For us this is really useful, as we can re-write our rule like so:

build/%.js: src/%.js node_modules
    ./node_modules/.bin/babel $< -o $@

The only slightly dodgy things there are the $@ variable, which matches the target name, so make build/index.js would be build/index.js, and the $< variable, but that basically means "the first dependency that matches", so if we were to run make build/index.js that would be src/index.js, and if we ran make build/foo.js it would be src/foo.js.

Ordering of dependencies is important. Normally, it will be the execution order of the dependencies, but with pattern targets, the first dependency is used as the matching dependency, so if we were to put node_modules before src/%.js, then $< would always contain node_modules, no matter what file we tried to build. That's not what we want.

Now we have that setup, run make build/index.js and we'll see what happens:

make: 'build/index.js' is up to date.

Looks promising, let's remove it and make sure it works:

rm build/index.js
make build/index.js

And your transpiled file should be back in build/index.js.

Pulling it Together

Whilst pattern targets help us avoid having hundreds of rules in our Makefile, it would be really annoying to have to type make build/index.js or make build/foo.js ad infinitum. Fortunately, make has our back here too.

First, let's duplicate src/index.js to src/foo.js so that we have a couple of files to build:

cp src/index.js src/foo.js

We can now add a build target that builds them both:

build: build/index.js build/foo.js

Here, all we're doing is telling make that it needs to build both of our files as dependencies of the build target. This allows us to build our entire project with make build. Try it out:

rm -f build/*
make build

You should see both files get built and placed in the build directory. If you were to remove a single file, and run make build again, it should only build that file:

rm -rf build/index.js
make build

And you should see build/index.js has been re-created.

Variables

Having to keep a list of all of our files as dependencies of the build target isn't much better than what we had before. There's a good chance we'll miss one eventually, and it will get messy to debug. Instead, it's better to store this in a variable that's dynamically populated. Update your Makefile to match the following:

SOURCE_FILES := $(wildcard src/*.js)
BUILD_FILES := $(patsubst src/%.js, build/%.js, ${SOURCE_FILES})

package.json: package.example
    cat package.example > package.json

node_modules: package.json
    npm install

build: ${BUILD_FILES}

build/%.js: src/%.js node_modules
    ./node_modules/.bin/babel $< -o $@

Here, we've added two variables to the top of the file. They don't have to be there, but that's kindof a make convention.

Make has a number of ways of assigning variables, this one, using the := symbol, is the simplest form of assignment, which simply means "store this as the value". The other types are a little more complex, and we don't need them here, so I will cover them in another post.

On the right hand side of the assignment, we have a make function call. This is denoted by the $(). The first word inside the dollar-brackets is the name of the function we're calling. It's then followed by a comma separated list of arguments for the function. There is no comma after the function name.

Make doesn't have many built in functions, the two we're using here are wildcard and patsubst. You can probably guess what they do by their names. wildcard will take a wildcard file path and return a space separated list of all files from the filesystem that matches that name, here that's a list of all of our .js files in the src/ directory.

The second, patsubst, will match the first argument, and replace it with the second, in the string provided as the third argument. We can use the % character as a simple wildcard match and replace.

The result is that SOURCE_FILES contains the string src/foo.js src/index.js, and BUILD_FILES contains the string build/foo.js build/index.js.

Take a look around line 11 of our Makefile, at our build target. Here, we're just making the dependency our BUILD_FILES variable (which is wrapped dollar-braces ${} to "echo" it), which, as I said above evaluates to the string build/foo.js build/index.js, so those two targets become our dependencies. Any new files we add will get picked up and added to the variables, and therefore added as dependencies to our build target. Go ahead and try it out, see what happens.

Wrapping Up

The astute amoung you would have noticed that we're invoking babel for every single file for our build target when we could transpile the entire directory in one go - it would be faster, and is likely what I would do in the real world. You lose the ability to only build what's changed with that target, but the trade-off is minimal in this instance. I did it the way I did here for the sake of example.

The second thing you may have noticed, if you're playing around a little, is that there's no way to make the wildcard function recursive. For that, you need another built-in function: shell. The shell function allows you to call out to the shell below and capture the resulting output. If you want to collect a list of files recursively, the best way is a combination of shell and find, like so:

SOURCE_FILES := $(shell find src/ -type f -name '*.js')

Conclusion

I hope this post has given you a good grounding in how to use make, why things are the way they are, and how it should be used for the greatest effect. There is far more to make, and especially to writing good Makefiles than what I've outlined here, but this foundation should allow you to start using make, and proceed to my next post, Makefile Variables.

For exclusive content, including screen-casts, videos, and early beta access to my projects, subscribe to my email list below.


I love discussion, but not blog comments. If you want to comment on what's written above, head over to twitter.