Boot, the Fancy Clojure Build Framework

Boot is an alternative to Leiningen that provides the same functionality. Leiningen’s more popular (as of the summer of 2015), but I personally like to work with Boot because it’s easier to extend. This appendix explains Boot’s underlying concepts and guides you through writing your first Boot tasks. If you’re interested in using Boot to build projects right this second, check out its GitHub README (https://github.com/boot-clj/boot/) and its wiki (https://github.com/boot-clj/boot/wiki/).

Note As of this writing, Boot has limited support for Windows. The Boot team welcomes contributions!

Boot’s Abstractions

Created by Micha Niskin and Alan Dipert, Boot is a fun and powerful addition to the Clojure tooling landscape. On the surface, it’s a convenient way to build Clojure applications and run Clojure tasks from the command line. Dig a little deeper and you’ll see that Boot is like the Lisped-up lovechild of Git and Unix in that it provides abstractions that make it more pleasant to write code that exists at the intersection of your operating system and your application.

Unix provides abstractions that we’re all familiar with to the point where we take them for granted. (Would it kill you to take your computer out to a nice restaurant once in a while?) The process abstraction lets you reason about programs as isolated units of logic that can be easily composed into a stream-processing pipeline through the STDIN and STDOUT file descriptors. These abstractions make certain kinds of operations, like text processing, very straightforward.

Similarly, Boot provides abstractions that make it easy to compose independent operations into the kinds of complex, coordinated operations that build tools end up doing, like converting ClojureScript into JavaScript. Boot’s task abstraction lets you easily define units of logic that communicate through filesets. The fileset abstraction keeps track of the evolving build context and provides a well-defined, reliable method of task coordination.

That’s a lot of high-level description, which hopefully has hooked your attention. But I would be ashamed to leave you with a plateful of metaphors. Oh no, dear reader, that was only the appetizer. Throughout the rest of this appendix, you’ll learn how to build your own Boot tasks. Along the way, you’ll discover that build tools can actually have a conceptual foundation.

Tasks

Like make, rake, grunt, and other build tools of yore, Boot lets you define tasks. Tasks are named operations that take command line options dispatched by some intermediary program (make, rake, Boot).

Boot provides the dispatching program, boot, and a Clojure library that makes it easy for you to define named operations and their command line options with the deftask macro. To see what all the fuss is about, let’s create your first task. Normally, programming tutorials encourage you to write code that prints “Hello World,” but I like my examples to have real-world utility, so your task is to print “My pants are on fire!” This information is objectively more useful. First, install Boot; then create a new directory named boot-walkthrough, navigate to that directory, create a file named build.boot, and write this:

(deftask fire
  "Prints 'My pants are on fire!'"
  []
  (println "My pants are on fire!"))

Now run this task from the command line with boot fire; you should see the message you wrote printed to your terminal. This task demonstrates two out of the three task components: the task is named (fire), and it’s dispatched by boot. This is super cool. You’ve essentially created a Clojure shell script, stand-alone Clojure code that you can run from the command line with ease. No project.clj, directory structure, or namespaces needed!

Let’s extend the example to demonstrate how you’d write command line options:

(deftask fire
  "Announces that something is on fire"
  [t thing     THING str  "The thing that's on fire"
   p pluralize       bool "Whether to pluralize"]
  (let [verb (if pluralize "are" "is")]
    (println "My" thing verb "on fire!")))

Try running the task like so:

boot fire -t heart
# => My heart is on fire!

boot fire -t logs -p
# => My logs are on fire!

In the first instance, either you’re newly in love or you need to be rushed to the emergency room. In the second, you are a Boy Scout awkwardly expressing your excitement over meeting the requirements for a merit badge. In both instances, you were able to easily specify options for the task.

This refinement of the fire task introduced two command line options, thing and pluralize. Both options are defined using a domain-specific language (DSL). DSLs are their own topic, but briefly, the term refers to mini-languages that you can use within a larger program to write compact, expressive code for narrowly defined domains (like defining options).

In the option thing, t specifies its short name, and thing specifies its long name. THING is a bit complicated, and I’ll get to it in a second. str specifies the option’s type, and Boot uses that to validate the argument and convert it. "The thing that's on fire" is the documentation for the option. You can view a task’s documentation in the terminal with boot task-name -h:

boot fire -h
# Announces that something is on fire
#
# Options:
#   -h, --help         Print this help info.
#   -t, --thing THING  Set the thing that's on fire to THING.
#   -p, --pluralize    Whether to pluralize

Pretty groovy! Boot makes it very easy to write code that’s meant to be invoked from the command line.

Now, let’s look at THING. THING is an optarg, and it indicates that this option expects an argument. You don’t have to include an optarg when you’re defining an option (notice that the pluralize option has no optarg). The optarg doesn’t have to correspond to the full name of the option; you could replace THING with BILLY_JOEL or whatever you want and the task would work the same. You can also designate complex options using the optarg. (Visit https://github.com/boot-clj/boot/wiki/Task-Options-DSL#complex-options for Boot’s documentation on the subject.) Basically, complex options allow you to specify that option arguments should be treated as maps, sets, vectors, or even nested collections. It’s pretty powerful.

Boot provides you with all the tools you could ask for to build command line interfaces with Clojure. And you’ve only just started learning about it!

The REPL

Boot comes with a number of useful built-in tasks, including a REPL task. Run boot repl to fire up that puppy. The Boot REPL is similar to Leiningen’s in that it handles loading your project code so you can play around with it. You might not think this applies to the project you’ve been writing because you’ve only written tasks, but you can actually run tasks in the REPL (I’ve omitted the boot.user=> prompt). You can specify options using a string:

(fire "-t" "NBA Jam guy")
; My NBA Jam guy is on fire!
; => nil

Notice that the option’s value comes right after the option.

You can also specify an option using a keyword:

(fire :thing "NBA Jam guy")
; My NBA Jam guy is on fire!
; => nil

You can also combine options:

(fire "-p" "-t" "NBA Jam guys")
; My NBA Jam guys are on fire!
; => nil

(fire :pluralize true :thing "NBA Jam guys")
; My NBA Jam guys are on fire!
; => nil

And of course, you can use deftask in the REPL as well—it’s just Clojure, after all. The takeaway is that Boot lets you interact with your tasks as Clojure functions, because that’s what they are.

Composition and Coordination

If what you’ve seen so far was all that Boot had to offer, it’d be a pretty swell tool, but it wouldn’t be very different from other build tools. One feature that sets Boot apart is how it lets you compose tasks. For comparison’s sake, here’s an example Rake invocation (Rake is the premier Ruby build tool):

rake db:create d{:tag :a, :attrs {:href "db:seed"}, :content ["b:migra"]}te db:seed

This code will create a database, run migrations on it, and populate it with seed data when run in a Rails project. However, worth noting is that Rake doesn’t provide any way for these tasks to communicate with each other. Specifying multiple tasks is just a convenience, saving you from having to run rake db:create; rake db:migrate; rake db:seed. If you want to access the result of Task A within Task B, the build tool doesn’t help you; you have to manage that coordination yourself. Usually, you’ll do this by shoving the result of Task A into a special place on the filesystem and then making sure Task B reads that special place. This looks like programming with mutable, global variables, and it’s just as brittle.

Handlers and Middleware

Boot addresses this task communication problem by treating tasks as middleware factories. If you’re familiar with Ring, Boot’s tasks work very similarly, so feel free to skip to “Tasks Are Middleware Factories” on page 287. If you’re not familiar with the concept of middleware, allow me to explain! Middleware refers to a set of conventions that programmers adhere to so they can flexibly create domain-specific function pipelines. That’s pretty dense, so let’s un-dense it. I’ll discuss the flexible part in this section and cover domain-specific in “Filesets” on page 288.

To understand how the middleware approach differs from run-of-the-mill function composition, here’s an example of composing everyday functions:

(def strinc (comp str inc))
(strinc 3)
; => "4"

There’s nothing interesting about this function composition. In fact, this function composition is so unremarkable that it strains my abilities as a writer to actually say anything about it. There are two functions, each does its own thing, and now they’ve been composed into one. Whoop-dee-doo!

Middleware introduces an extra step to function composition, giving you more flexibility in defining your function pipeline. Suppose, in the preceding example, that you wanted to return "I don't like the number X" for arbitrary numbers but return a string-ified number for everything else. Here’s how you could do that:

(defn whiney-str
  [rejects]
  {:pre [(set? rejects)]}
  (fn [x]
    (if (rejects x)
      (str "I don't like " x)
      (str x))))

(def whiney-strinc (comp (whiney-str #{2}) inc))
(whiney-strinc 1)
; => "I don't like 2"

Now let’s take it one step further. What if you want to decide whether or not to call inc in the first place? Listing B-1 shows how you could do that:

(defn whiney-middleware
  [next-handler rejects]
  {:pre [(set? rejects)]}
  (fn [x]
➊     (if (= x 1)
        "I'm not going to bother doing anything to that"
        (let [y (next-handler x)]
          (if (rejects y)
            (str "I don't like " y)
            (str y))))))

(def whiney-strinc (whiney-middleware inc #{2}))
(whiney-strinc 1)
; => "I'm not going to bother doing anything to that"

1. B-1. The middleware approach to function composition lets you introduce choice

Here, instead of using comp to create your function pipeline, you pass the next function in the pipeline as the first argument to the middleware function. In this case, you’re passing inc as the first argument to whiney-middleware as next-handler. whiney-middleware then returns an anonymous function that closes over inc and has the ability to choose whether to call it or not. You can see this choice at ➊.

We say that a middleware takes a handler as its first argument and returns a handler. In this example, whiney-middleware takes a handler as its first argument, inc, and it returns another handler, the anonymous function with x as its only argument. Middleware can also take extra arguments, like rejects, that act as configuration. The result is that the handler returned by the middleware can behave more flexibly (thanks to configuration), and it has more control over the function pipeline (because it can choose whether or not to call the next handler).

Tasks Are Middleware Factories

Boot takes this pattern of making function composition more flexible one step further by separating middleware configuration from handler creation. First, you create a function that takes n configuration arguments. This is the middleware factory, and it returns a middleware function. The middleware function expects one argument, the next handler, and it returns a handler, just like in the preceding example. Here’s a whiney middleware factory:

(defn whiney-middleware-factory
  [rejects]
  {:pre [(set? rejects)]}
  (fn [handler]
    (fn [x]
      (if (= x 1)
        "I'm not going to bother doing anything to that"
        (let [y (handler x)]
          (if (rejects y)
            (str "I don't like " y " :'(")
            (str y)))))))

(def whiney-strinc ((whiney-middleware-factory #{3}) inc))

As you can see, this code is nearly identical to Listing B-1. The change is that the topmost function, whiney-middleware-factory, now only accepts one argument, rejects. It returns an anonymous function, the middleware, which expects one argument, a handler. The rest of the code is the same.

In Boot, tasks can act as middleware factories. To show this, let’s split the fire task into two tasks: what and fire (see Listing B-2). what lets you specify an object and whether it’s plural, and fire announces that it’s on fire. This is great modular software engineering because it allows you to add other tasks, like gnomes, to announce that a thing is being overrun with gnomes, which is just as objectively useful. (As an exercise, try creating the gnome task. It should compose with the what task, just as fire does.)

(deftask what
  "Specify a thing"
  [t thing     THING str  "An object"
   p pluralize       bool "Whether to pluralize"]
  (fn middleware [next-handler]
➊     (fn handler [fileset]
      (next-handler (merge fileset {:thing thing :pluralize pluralize})))))

(deftask fire
  "Announce a thing is on fire"
  []
  (fn middleware [next-handler]
➋     (fn handler [fileset]
      (let [verb (if (:pluralize fileset) "are" "is")]
        (println "My" (:thing fileset) verb "on fire!")
        fileset))))

1. The full code for composable Boot tasks that announce something’s on fire

Here’s how you’d run this on the command line:

boot what -t "pants" -p – fire

And here’s how you’d run it in the REPL:

(boot (what :thing "pants" :pluralize true) (fire))

Wait a minute, what’s that boot call doing there? And what’s with fileset at ➊ and ➋? In Micha’s words, “The boot macro takes care of setup and cleanup (creating the initial fileset, stopping servers started by tasks, things like that). Tasks are functions, so you can call them directly, but if they use the fileset, they will fail unless you call them via the boot macro.” Let’s take a closer look at filesets.

Filesets

Earlier I mentioned that middleware is for creating domain-specific function pipelines. All that means is that each handler expects to receive domain-specific data and returns domain-specific data. With Ring, for example, each handler expects to receive a request map representing the HTTP request, which might look something like this:

{:server-port 80
 :request-method :get
 :scheme :http}

Each handler can choose to modify this request map in some way before passing it on to the next handler, say, by adding a :params key with a nice Clojure map of all query string and POST parameters. Ring handlers return a response map, which consists of the keys :status, :headers, and :body, and once again each handler can transform this data in some way before returning it to its parent handler.

In Boot, each handler receives and returns a fileset. The fileset abstraction lets you treat files on your filesystem as immutable data, and this is a great innovation for build tools because building projects is so file-centric. For example, your project might need to place temporary, intermediary files on the filesystem. Usually, with most build tools, these files get placed in some specially named place, say, project/target/tmp. The problem with this is that project/target/tmp is effectively a global variable, and other tasks can accidentally muck it up.

Boot’s fileset abstraction solves this problem by adding a layer of indirection on top of the filesystem. Let’s say Task A creates File X and tells the fileset to store it. Behind the scenes, the fileset stores the file in an anonymous, temporary directory. The fileset then gets passed to Task B, and Task B modifies File X and asks the fileset to store the result. Behind the scenes, a new file, File Y, is created and stored, but File X remains untouched. In Task B, an updated fileset is returned. This is the equivalent of doing assoc-in with a map: Task A can still access the original fileset and the files it references.

And you didn’t even use any of this cool file management functionality in the what and fire tasks in Listing B-2! Nevertheless, when Boot composes tasks, it expects handlers to receive and return fileset records. Therefore, to convey your data across tasks, you sneakily added it to the fileset record using (merge fileset {:thing thing :pluralize pluralize}).

Although that covers the basic concept of a middleware factory, you’ll need to learn a bit more to take full advantage of filesets. The mechanics of working with filesets are all explained in the fileset wiki (https://github.com/boot-clj/boot/wiki/Filesets). In the meantime, I hope this information gave you a good conceptual overview!

Next Steps

The point of this appendix was to explain the concepts behind Boot. However, Boot also has a bunch of other functions, like set-env! and task-options!, that make your programming life easier when you’re actually using it. It offers amazing magical features, like providing classpath isolation so you can run multiple projects using one JVM, and letting you add new dependencies to your project without having to restart your REPL. If Boot tickles your fancy, check out its README for more information on real-world usage. Also, its wiki provides top-notch documentation.