Building web applications with Make
Make is a build automation tool originally designed to compile source code into executable programs. However, because it is such a proven, flexible and versatile tool, it is also a great fit for other applications. I have been using Make to automate build tasks for web applications for a few years now with great satisfaction, and in this article I will show some of its interesting features and some makefile snippets I commonly use in the projects I work on.
As the complexity of a software project grows, it becomes more and more cumbersome to perform all these steps by hand. Not only do they need to be documented so everyone does it the same way, but they may also change over time as the project evolves. Forgetting a step or executing a wrong command can cause unexpected results. To speed up the build process, reduce the risk of human errors and allow for changes over time, the build process should be automated and the automation scripts should be committed as part of the project's source code.
Introduction to Make
Make has been created in 1976 and has since become a widespread tool. It is actually a family of tools, as there are a couple of different implementations of Make, such as BSD Make, GNU Make and Microsoft nmake, each with their own enhancements and features. In this article I will focus on GNU Make, the standard implementation of Make on Linux and OS X.
Make uses makefiles to understand the relationships between source files, output
files, and the commands that are required to create these output files. Make
will look for a file called
Makefile in the current directory (unless another
filename is specified).
Based on the information provided in the makefile, Make will automatically determine whether an output file (called target) is out of date and needs to be rebuilt by comparing its file modification time with that of its source files (prerequisites). If the target doesn't exist yet, it will always be built.
When you run Make, you can provide an optional list of target names that should be built:
make [TARGET ...]
If you don't specify a target, Make will build the default target (more about this later).
The syntax of a basic rule looks like this:
target [target ...]: [prerequisite ...] recipe
A target is usually the name of an output file that can be created by executing the recipe. A rule can have multiple targets, which can be useful when multiple files can be created with the same (or similar) command.
A prerequisite is a file that is used as input to create a target. As I noted earlier, Make compares the timestamps of the target and its prerequisites to determine whether the target is up to date or should be rebuilt. A target can have zero, one or many prerequisites.
A recipe is the command that Make should execute to build the target. A recipe can also consist of multiple commands, which may be spread over multiple lines. Note that each recipe line should be indented with a tab character.
Installing dependencies using a package manager
A common build task for modern web applications is to install third-party
dependencies using a package manager, such as Composer or NPM.
My first (naive) approach to install dependencies using Make was to have an
install target in my Makefile to execute the install commands of the different
package managers. I would run
make install whenever I checked out a project
for the first time or after switching branches. The
install target would look
install: composer install npm install
Because a file or directory called
install would never be created, this was
effectively a phony target
(although I didn't know that term by then) and Make would rerun these commands
every time I ran
make install, even if there was nothing to install or update.
I also had to always remember running this before any other Make targets,
because none of my other targets had
install as their prerequisite.
My first improvement was to split up the
install target and create targets
named after the directories containing dependencies (
etc.), with the dependency and lock files as prerequisites:
vendor: composer.json composer.lock composer install node_modules: package.json package-lock.json npm install
This worked nicely because Make would now compare the timestamp of the package directory with that of the dependency files, and only execute the recipe when the directory didn't exist or was out of date with one of the dependency files. I also started to add these targets as prerequisites to other rules that depended on (up-to-date) packages in order to run.
Later, I realised that I could drop the file listing the requirements
package.json) and only use the lock file as the
prerequisite. The idea is that the lock file will change whenever a dependency
is installed or updated. The file listing the requirements often also contains
other metadata, which can be changed without requiring an update of the
installed packages. So, my rules now look like this:
vendor: composer.lock composer install node_modules: package-lock.json npm install
I'm currently experimenting to further improve these rules by replacing them with pattern rules that match with any file within the directory, so I can use them as prerequisites for targets that depend on a specific file from a package. I'm not really sure about this move yet, but I will update this article if it happens to work out (or not).
As a final remark: if a prerequisite is missing and there is no rule to create
it, Make will fail. So for projects (mainly libraries) where I add the lock
files to the
.gitignore, I often use Make's
to treat the lock file as an "optional prerequisite":
vendor: $(wildcard composer.lock) composer install
Note that this optional prerequisite is not really a Make concept. It's just a way to have a prerequisite that will not let Make fail when it doesn't exist.
Using a default target
The documentation of GNU Make lists a number of conventions for writing
Makefiles for GNU programs.
Not all of them can be applied to the context of web application development,
but I've adopted some that make sense for web applications.
One of these is to have every makefile contain an
which the documentation describes as:
Compile the entire program. This should be the default target. (...)
Being the default target refers to the fact that Make, when called without
arguments, will build the first target that appears in the makefile. By adopting
this convention I can enter any project, run
make all), and I have
a working application! Depending on the specific requirements of the project I
add prerequisites to the
all target so that dependencies are installed, assets
are compiled, etc:
.PHONY: all all: vendor public/app.js public/app.css
.PHONY: all tells Make that
all should be treated as a
meaning a target that is not really the name of a file. This ensures that Make
will execute the recipe, even if a file or directory with the same name happens
to exist already.
Compiling Sass or LESS files
When I work on a fairly simple project that requires some Sass but not a
full-blown Webpack setup, I prefer to run the
sass executable directly.
Compiling Sass files (or LESS for that matter) is a good use case for Make
because of the clear relationship
between source (Sass) and output (CSS) files, which allows you to take the most
advantage of Make's out of date calculation:
app.css: app.scss sass app.scss app.css
Now when you run
make app.css, Make will only compile the Sass file when it
is newer than the compiled CSS.
To avoid repetition and make this rule more flexible, we can use automatic variables that contain the names of the target and the prerequisite:
app.css: app.scss sass $< $@
$< contains the name of the first prerequisite (in this case
$@ contains the name of the target, which in this case is
In real-life applications my Sass files often import other Sass files, which can either be my own (partials) or maintained by a third party (for example, the Bootstrap Sass files). To ensure that when these are updated my output CSS is rebuilt, I list them as prerequisites as well:
app.css: sass/app.scss sass/_*.scss node_modules sass $< $@
Note that this is why I'm using the
$< variable (which contains only the first
prerequisite): I want Make to rebuild
app.css if any of the prerequisites
are newer than the target, but I only want the first prerequisite
sass/app.scss) to be passed as an argument to the
The last examples assumed that there is only a single CSS file to be built. However, in some cases I want to create a generic rule that can be used to build multiple targets. In these cases pattern rules are quite useful:
%.css: sass/%.scss sass/_*.scss node_modules sass $< $@
A pattern rule is a rule where the target contains the
% character. The
character can match any nonempty string. If a prerequisite also contains the
character, it will be replaced with the part of the target that matched the
character (called the stem). For example, when calling
make print.css with
the above rule, its first prerequisite will be
Last but not least,
sass accepts a number of options which can be used to
change the generated CSS. For instance, when building for production I might
want to minify the generated CSS, while during development I would rather want
expanded CSS and source mapping.
By creating a variable
SASSFLAGS and using that in the recipe, I have a recipe
that can be reused but produces different output for development and production:
SASSFLAGS = --style expanded %.css: sass/%.scss sass/_*.scss node_modules sass $(SASSFLAGS) $< $@
This variable can be overridden when building for production, in order to get
minified output. This can be done by either setting the variable on the command
line when calling Make (i.e.
make SASSFLAGS='--style compressed
--sourcemap=none' app.css) or by setting a target-specific variable (more about
that later). The only issue I haven't really been able to solve yet (without
building assets for production in a separate directory or giving them different
filenames) is that when the output file already exists, Make will not re-run the
Creating favicons from source images
asset build system, I sometimes just want to generate a set of favicons from a
source image. To do so, I use
convert (part of
ImageMagick) to resize the input image into
16x16, 32x32 and 48x48 pixel PNG images which are padded with a background color
to make them square, which are then combined into a
favicon.ico file using
icotool (part of the icoutils package):
favicons = public/favicon.ico public/favicon-16x16.png public/favicon-32x32.png public/apple-touch-icon.png all: public/app.js public/app.css $(favicons) public: mkdir -p $@ public/favicon-%.png: images/portrait.png | public convert $< -resize $* -background white -gravity center -extent $* $@ public/apple-touch-icon.png: images/portrait.png | public convert $< -resize 180x180 -background white -gravity center -extent 180x180 $@ public/favicon.ico: public/favicon-16x16.png public/favicon-32x32.png public/favicon-48x48.png | public icotool --create --output=$@ $^
At the top of the makefile I create a variable
favicons which holds the names
of the favicon files I need to generate, so I can use that as the prerequisites
for another target (such as
I then have the various rules for creating the PNG and ICO files.
In these rules I use two automatic variables
I haven't discussed yet:
$*, which contains the stem from my pattern rule (the part of the target
that matches with the
% character), and
$^ which contains the names of all
prerequisites (with spaces between them).
public prerequisite is an
(denoted by the pipe symbol
| between the normal prerequisites and order-only
prerequisites). This means that the
build directory will be created if it
doesn't already exist, but the files inside it don't have to be recreated
whenever the timestamp of the directory changes.
Of course this example can be improved to generate even more favicon files to support higher resolution icons that are used by different browsers and operating systems. The FAQ from RealFaviconGenerator.net contains a good list of recommended favicon files.
Compiling frontend assets with Webpack
When I do work on a project that uses Webpack or a similar tool to build frontend assets, I still like to add these tools to my Makefile instead of running them directly. This helps integrating them in the rest of the application's build process (so I can build the whole application with only one command) while at the same time making sure that they will only run when one of the source files has actually changed.
assets = public/main.css public/main.js public/favicon.ico $(assets): sass/* src/*.js images/* node_modules node_modules/.bin/webpack
assets variable conveniently contains all assets that need to be built for
this project, and can be used as prerequisite for other targets such as the
Building a deployable tar archive
In order to create a build artifact that can be deployed to a remote server,
I again use one of the standard targets
from GNU's Makefile conventions:
make dist should create a
tarball containing all the files required to run the application in production.
I then use Ansible to copy this archive to the server, extract it into a new
release directory, switch symlinks, etc.
sources = config/ public/ src/ templates/ version = $(shell git describe --tags --dirty --always) build_name = my-app-$(version) build_dir = build/my-app-$(version) # (other targets) .PHONY: dist dist: SASSFLAGS = --style compressed --sourcemap=none dist: all -rm -rf $(build_dir) mkdir -p $(build_dir) ln --symbolic --relative $(sources) composer.json composer.lock $(build_dir) composer install --working-dir=$(build_dir) --no-dev rm $(build_dir)/composer.json $(build_dir)/composer.lock tar --create --dereference --gzip --file=build/$(build_name).tar.gz --directory=build $(build_name) -rm -rf $(build_dir)
At the top of the Makefile, I set a couple of variables which can be reused
throughout the file.
sources contains a list of source files and/or
version runs a Git command to describe the current state of the
workspace. Especially when using tags this is very useful, as it will return the
most recent tag and some other information to identify this build.
contains a name for this build based on the name of the project and the value of
build_dir contains the name of a temporary directory which will
be archived to produce the build artifact. This allows to filter development
dependencies out of the build artifact without affecting the current working
Remember how I created a
SASSFLAGS variable in order to pass different options
to Sass depending on whether I was building for development or production? Using
a target-specific variable value,
I can override the value specifically for the
dist target, and this value is
even passed to rules that are prerequisites of
dist. This way when I
run any other target the global value of
SASSFLAGS applies, but when I run
make dist the value is overwritten with one that results in compressed CSS
without source maps.
The recipe itself contains the following steps:
- The temporary directory for this build is removed (if it exists already) and
re-created. Note that the dash in front of the
rmcommand tells Make to ignore any errors, so Make will not fail if the directory does not exist.
- Symbolic links are created to the files and directories listed in the
- Composer is used to install the production dependencies inside the build
directory. Afterwards, the
composer.lockfiles are removed.
- A gzip-compressed tarball is created containing the build directory and its
--dereferenceflag is used to follow the symlinks we made earlier, instead of having actual symlinks in the archive).
- The temporary directory is cleaned up.
To prevent the the contents of the build directory and build artifacts from
accidentally being committed to Git, the
build directory is included in the
.gitignore file of the project.
Running automated tests
Although this normally doesn't result in files being created, I find it very
convenient to also include test targets in my Makefiles. For most projects I
test target to run the full test suite, which has more fine-grained
targets as its prerequisites. This way I can easily run the whole suite with
make test or run a smaller part of the test suite, for instance
.PHONY: test lint unit-tests acceptance-tests coding-standards security-tests test: lint unit-tests acceptance-tests coding-standards security-tests lint: vendor vendor/bin/parallel-lint $(sources) bin/console lint:twig templates/ vendor/bin/phpstan analyse --level=7 $(sources) unit-tests: vendor vendor/bin/phpunit acceptance-tests: vendor vendor/bin/behat -v coding-standards: vendor vendor/bin/phpcs -p --colors security-tests: vendor vendor/bin/security-checker security:check npm audit
Of course all these targets are phony targets. Note that for every target I list the required prerequisites, so I can always run the tests with a single command even if I just did a fresh checkout of the code.
A last convention I've adopted from GNU's Makefile conventions is to always
clean target in my Makefiles. Although the targets in the Makefile
should always contain the right prerequisites to ensure that it will be rebuilt
when it gets out of date, it can sometimes just be very convenient to remove all
the files that were built by the Makefile and start with a clean slate.
.PHONY: clean clean: rm -rf build node_modules vendor $(assets)
The Makefile conventions also list other cleanup targets
maintainer-clean), but I haven't
really found a use case for these yet in my projects so I haven't implemented