Sunday, January 17, 2016

Introduce erlang.mk

As Java doesn't have neither Erlang has a standard tool with which we can handle dependencies, compile the application modules and generates Erlang modules. In Java the Maven and Ivy are for dependency handling, the Erlang world also has a palette from which we can choose if we are talking about dependency handling.

Later in my projects I used rebar which was good, but as I wanted to extend my build with specific steps (generating sys.config from a template, or starting riak during testing) I found it difficult to solve those problems with rebar. Then I found erlang.mk. At first I found it overly complex, but as I made more and more project builds with erlang.mk I liked it. I liked it because of its extensibility. In this post I like to show how erlang.mk works and how one can extend the build lifecycle.

How erlang.mk works

Basically erlang.mk is a big parametric Makefile. At first we need to provide the parameter values and then we can include erlang.mk. Let us create an empty directory and download erlang.mk bootstrap file.

curl https://raw.githubusercontent.com/ninenines/erlang.mk/master/erlang.mk -O
And create a Makefile.
PROJECT = webshop
PROJECT_DESCRIPTION = A webshop application

DEPS = cowboy jsx ejson lager riakc

dep_ejson = git https://github.com/jonasrichard/ejson.git

include erlang.mk

In a typical erlang.mk makefile there will be a project identification and description, and also the DEPS variable will contain a list of dependencies we require. Dependencies are project names, so a question can pop into our mind, how does erlang.mk know what is the url of jsx or cowboy. erlang.mk contains the name, repo url of typical Erlang dependencies, so they are in the erlang.mk file. However ejson is not such a module therefore I need to specify the repo url by specifying the dep_ejson variable.

After running make, erlang.mk will be downloaded and then all the dependencies are fetched and compiled. With make help you can see the targets erlang.mk defines by default. With make deps erlang.mk gets the dependencies from various source repositories. If you add a new dependency you need to fetch that by make deps, if they are not fetched automatically. make app compiles the dependencies and the application itself. With make test we can run unit and common tests. make rel builds the Erlang release and what we can run with the generated start script. Or we can run the project with make run.

As always with environment variables we can specify parameters to the targets, like if we want to run unit tests with cover enabled run COVER=1 make test cover-report. In that way we can specify ERLC_OPTS which is the command line switches of the Erlang compiler. Also, we can write a build environment sensitive makefile, so with ENV=dev make run we can run the application in development environment.

Build lifecycle

Simple makefiles can be debugged with make -sn which prints to the stderr what make will do. Try this with erlang.mk, it will result in a tons of messages. It is because erlang.mk creates variables which contain Erlang code snippets, which will be evaluated by erl -eval. It is a fair way of defining custom build targets by writing Erlang codes. So it is way easier is to open the source of erlang.mk which shows you what will be done. At first this many-thousand-line makefile can be intimidating but after a bit of analyzation you can see that it starts with the general part, then there are embedded parts (like unit testing, cover, dialyzer), then it contains some thousand line of knows dependencies, then the 3rd party plugins come.


In makefile one can write something like

.PHONY: compile eunit

compile:
    erlc src/*.erl

eunit: compile
    erlc test/*.erl -o test
    erl -pz ebin test -eval "run the eunit tests :)"

Which is nice if the parameters are correctly specified. Compile target compiles the source, eunit target depends on compile and it also compiles tests and run them. The only problem is that if we define such a framework, nobody can make hooks which can be execute before or after compilation or running unit tests.

In most cases erlang.mk uses double-colon rules. There may be more double-colon rules with the same name. In that case all rules will be run in the order of their occurence. App target is such a double-colon target, so one can hook commands before and after running app target. If we write a rule before including erlang.mk that rule will occur earlier that the rules in erlang.mk, so it will be execute before the erlang.mk app rule, and vice versa. So one can filter sources before compilation and make an archive file then.

app::
    filter source files, replace things in them

include erlang.mk

app::
    tar vxfz beams.tar.gz ebin/

The only problem with this solution is that in Makefile the first target will be the default target. And we overwrote the default target of erlang.mk which was all and now it is app (erlang.mk had all:: deps app rel as a first target). So we need to write a .DEFAULT_GOAL: all somewhere in the Makefile to set the default target back. Why is it a problem? Because if we are overriding the rel target, the rel target will be the default. And if somebody uses our project as a dependency, when erlang.mk builds dependencies it will go in each deps/project directory and executes a make command without specifying the target. In our case the rel (or app) will be the default target which may or may not work.

But back to the business let us build a release.

Specify relx.config

In default erlang.mk uses relx to build releases. Relx makes Erlang release creation very simple, at least it simplifies the first steps very much. To run relx we need to have a relx.config file which describes what relx needs to do. Here we have a simple relx.config

{release, {webshop_release, "0.0.1"}, [
    webshop
]}.
{extended_start_script, true}.

relx relies on application app file (in this case ebin/webshop.app) content, namely on the application tuple which contains the dependencies. So in relx.config we don't need to specify the dependencies of our applications, only our applications. And relx will traverse the app files and collect all the applications required. So after run make rel the release is created in the _rel directory.

How to go on?

We are just scratching the surface of what we can do with erlang.mk. There are a lot of plugins which can execute not-so-popular tasks, and we also can define 3rd party plugins for erlang.mk. Anyway in this post I described the basic idea how erlang.mk can be used, advanced topic can be understood after understanding this introduction. The main takeaway is, if you have question about erlang.mk, always use the source.

Share:

0 comments :

Post a Comment

Richard Jonas. Powered by Blogger.

About me

My name is Richárd Jónás, live in Budapest, Hungary. In this blog I want to share my coding experiences in Erlang, Elixir and other languages I use. Some topics are simpler ones but you can use them as a reference. I also present some of my thoughts about developing distributed systems.