In this final chapter, we cover some tangential topics that you might already be familiar with, but perhaps not from a Clojure perspective. Our discussion will start with domain-specific languages (DSLs) and the unique way that Clojure applications are built from a layering of unique application-specific DSLs. Next, you’re unlikely to be ignorant of the general push toward a test-driven development (TDD) philosophy, with a special focus on unit testing. We’ll explore why Clojure is especially conducive to unit testing and why it’s often unnecessary. Next, whether you agree with the cult of design patterns or not, it’s inarguable that patterns have changed the way that object-oriented software is designed and developed. The classical design patterns are often invisible, or at times outright nonexistent in Clojure code, which we’ll discuss in this chapter. As we’ll then show, error handling in Clojure flows in two directions: from inner functions to outer via exceptions, and from outer functions in via dynamic bindings. Finally, we’ll explore how having the entire language at your disposal can help to change the way that your debugging occurs. We hope that by the time you’ve finished this chapter, you’ll agree—Clojure changes the way you think about programming.
Lisp is not the right language for any particular problem. Rather, Lisp encourages one to attack a new problem by implementing new languages tailored to that problem.
“Lisp: A Language for Stratified Design” (Abelson 1988)
In chapter 8, we explored the notion of a domain-specific language for describing domains. This meta-circularity, while playful, was meant to make a subtle point: Clojure blurs, and often obliterates, the line between DSL and API. When a language is built from the same data structures that the language itself manipulates, it’s known as homoiconic (Mooers 1965). When a programming language is homoiconic, it’s simple to mold the language into a form that bridges the gap between the problem and solution domains. When designing DSLs in Clojure, it’s important to determine when the existing language facilities will suffice (Raymond 2003) and when it’s appropriate to create one from whole cloth (Ghosh 2010). In this section we’ll do both and provide a little discussion about each.
The declarative language SQL is among the most widespread DSLs in use today. In section 1.2 we showed a simple Clojure DSL, which provided a simple subset of the SELECT statement that created a representational SQL string. Though that particular example was meant to be instructive, Clojure provides a comprehensive library for relational algebra, on which SQL is based (Date 2009). Imagine a dataset of the following:
(def artists
#{{:artist "Burial" :genre-id 1}
{:artist "Magma" :genre-id 2}
{:artist "Can" :genre-id 3}
{:artist "Faust" :genre-id 3}
{:artist "Ikonika" :genre-id 1}
{:artist "Grouper"}})
(def genres
#{{:genre-id 1 :genre-name "Dubstep"}
{:genre-id 2 :genre-name "Zeuhl"}
{:genre-id 3 :genre-name "Prog"}
{:genre-id 4 :genre-name "Drone"}})
You can try Clojure’s relational functions by entering the examples shown in the following listing.
(require '[clojure.set :as ra])
(def ALL identity)
(ra/select ALL genres)
;=> #{{:genre-id 4, :genre-name "Drone"}
{:genre-id 3, :genre-name "Prog"}
{:genre-id 2, :genre-name "Zeuhl"}
{:genre-id 1, :genre-name "Dubstep"}}
(ra/select #(#{1 3} (:genre-id %)) genres)
;=> #{{:genre-id 3, :genre-name "Prog"}
{:genre-id 1, :genre-name "Dubstep"}}
(take 2 (ra/select ALL (ra/join artists genres)))
;=> #{{:artist "Burial", :genre-id 1, :genre-name "Dubstep"}
{:artist "Magma", :genre-id 2, :genre-name "Zeuhl"}}
The relational functions in clojure.set are a perfect example of the way that Clojure blurs the line between API and DSL. No macro tricks are involved, but through the process of functional composition, the library provides a highly expressive syntax matching closely (Abiteboul 1995) that of SQL itself. Though you might be tempted to create a custom query language for your own application(s), there are times when the relational functions are exactly what you need. Your time might be better spent solving actual problems, one of which we’ll cover in the following section.
Many applications deal in measurements of differing units. For example, it’s widely known that the U.S. works almost exclusively in English units of measure, whereas most of the rest of the planet works in SI, or metric units. To convert[1] from one to the other isn’t an arduous task and can be handled easily with a set of functions of this general form:
1 A spectacular general-purpose JVM language named Frink excels at conversions of many different units. We highly advocate exploring Frink at your next available opportunity: http://futureboy.us/frinkdocs/.
(defn meters->feet [m] (* m 3.28083989501312)) (defn meters->miles [m] (* m 0.000621)) (meters->feet 1609.344) ;=> 5279.9999999999945 (meters->miles 1609.344) ;=> 0.999402624
This approach will certainly work if only a few functions define the extent of your conversion needs. But if your applications are like ours, then you probably need to convert to and from differing units of measure of many different magnitudes. You may also need to convert back and forth between units of time, dimension, orientation, and a host of others. Therefore it’d be nice to be able to write a specification of unit conversions (Hoyte 2008) as a Clojure DSL and use its results as a low-level layer for high-layer application specifics. This is precisely the nature of Lisp development in general—each level in an application provides the primitive abstractions for the levels above it.
In this section, we’re going to create a small specification and then convert it into a Clojure DSL using a technique coined by Rainer Joswig as “putting parentheses around the specification.”
An ideal representation for a unit-conversion specification language would be simple:
Our base unit of distance is the meter. There are 1,000 meters in a kilometer. There are 100 centimeters in a meter. There are 10 millimeters in a centimeter. There are 3.28083 feet in a meter. And finally, there are 5,280 feet in a mile.
Of course, to make sense of free text is a huge task in any language, so it behooves us to change it so that it’s easier to reason about programmatically, but not so much that it’s cumbersome for someone attempting to describe unit conversions. As a first pass, we’ll try to group the most obvious parts using some Clojure syntactical elements:
(Our base unit of distance is the :meter [There are 1000 :meters in a :kilometer] [There are 100 :centimeters in a :meter] [There are 10 :millimeters in a :centimeter] [There are 3.28083 :feet in a :meter] [There are 5280 :feet in a :mile])
This specification is starting to look a little like Clojure code, but it would still be difficult to parse this into a usable form. Likewise, it’ll be difficult for the person writing the specification to use the correct terms, avoid spelling mistakes, properly punctuate, and so forth. In a word, this form is still not useful. It’d be ideal if we could make this into a form that’s still recognizable to both Clojure and a conversion expert. We’ll try one more time:
(define unit of distance
{:m 1,
:km 1000,
:cm 1/100,
:mm [1/10 of a :cm],
:ft 0.3048,
:mile [is 5280 :ft]})
This almost looks like Clojure source code, except for a few minor details. We’ve changed the measure of feet from an “in a” relationship to a relative one with regard to the meter base unit. Also, a vector indicates the use of a different relative unit, keeping the DSL regular in its meaning between one conversion and the next and providing a way to describe intermediate relative units of measure. Those definitions look like a map, so we should write a utility function that takes a unit and a map like the preceding one and returns the number of units it takes to compose the base unit.
The function relative-units goes through the map units looking up units and multiplying their compositional values. When it finds an indirect specification (such as millimeters defined in terms of centimeters), it traverse the chain of indirect references multiplying the factors along the way, as shown:
(relative-units :m {:m 1 :cm 100 :mm [10 :cm]})
;=> 1
(relative-units :cm {:m 1 :cm 100 :mm [10 :cm]})
;=> 100
(relative-units :mm {:m 1 :cm 100 :mm [10 :cm]})
;=> 1000
We changed the unit conversions map to remove the natural language phrase “in a,” because English isn’t good for a DSL. Natural language often lacks the precision that a simple yet regular form has. Now that we have the auxiliary function created, we’d like to create a macro to interpret the unit specification as shown:
(defunits-of distance :m :km 1000 :cm 1/100 :mm [1/10 :cm] :ft 0.3048 :mile [5280 :ft])
This is a simplification versus the original verbal form of the conversion specification. This final form is indubitably more conducive to parsing, yet doesn’t appreciably sacrifice readability. The implementation of the defunits-of macro is presented in the following listing.
The macro defunits-of is different than any macro that you’ve seen thus far, but it’s typical for macros that expand into another macro definition. In this book you’ve yet to see a macro that builds another macro and uses multiple levels of nested[2] syntax-quotes. You won’t likely see macros of this complexity often, but in this case we use nested syntax-quotes so that we can feed structures from the inner layers of the nested macros to the outer layers, processing each fully before proceeding. At this point, we can now run a call to the defunits-of macro with the simplified metric to English units conversion specification to define a new macro named unit-of-distance:
2 We talked briefly about making sense out of nested syntax-quotes in section 8.1. However, you’re not likely to need them very often.
(unit-of-distance 1 :m) ;=> 1 (unit-of-distance 1 :mm) ;=> 1/1000 (unit-of-distance 1 :ft) ;=> 0.3048 (unit-of-distance 1 :mile) ;=> 1609.344
Perfect! Everything is relative to the base unit :m, just as we’d like (read as “how many meters are in a _”). The generated macro unit-of-distance allows you to work in your given system of measures relative to a standard system without loss of precision or the need for a bevy of awkward conversion functions. To calculate the distance a home run hit by the Orioles’ Matt Wieters travels in Canada is a simple call to (unit-of-distance 441 :ft) away. The expansion of the distance specification given as (defunits-of distance :m ...) looks approximately like the following:
(defmacro unit-of-distance [G__43 G__44]
`(* ~G__43
(case ~G__44
:mile 1609.344
:km 1000
:cm 1/100
:m 1
:mm 1/1000
:ft 0.3048)))
The defunits-of macro is an interpreter of the unit-conversion DSL, which generates another macro unit-of-distance that performs a straightforward lookup of relative unit values. Amazingly, the expansion given by (macroexpand '(unit-of-distance 1:cm)) is that of a simple multiplication (* 1 1/100). This is an awe-inspiring revelation. What we’ve managed to achieve is to fuse the notion of compilation and evaluation by writing a relative units of measure “mini-language” that’s interpreted into a simple multiplication at compile time!
This is nothing new; Lisp programmers have known about this technique for decades, but it never ceases to amaze. There’s one downside to our implementation—it allows for circular conversion specifications (seconds defined in terms of minutes, which are then defined in terms of seconds), but this can be identified and handled in relative-units if you’re so inclined.
DSLs and control structures implemented as macros in Common Lisp tend to be written in a style more conducive to macro writers. But Clojure macros such as defunitof, cond, and case are idiomatic in their minimalism; their component parts are paired and meant to be grouped through proper spacing. Clojure macro writers should understand that the proliferation and placement of parentheses are legitimate concerns for some, and as a result you should strive to reduce the number whenever possible. Why would you explicitly group your expressions when their groupings are only a call to partition away?
If a project elicits a sense of being lost, then start from the bottom up.
DSLs are an important part of a Clojure programmer’s toolset and stem from a long Lisp tradition. When Paul Graham talks about “bottom-up programming” in his perennial work On Lisp, this is what he’s referring to. In Clojure, it’s common practice to start by defining and implementing a low-level language specifically for the levels above. Creating complex software systems is hard, but using this approach, you can build the complicated parts out of smaller, simpler pieces.
Clojure changes the way that you think.
Object-oriented programs can be highly complicated beasts to test properly, thanks to mutating state coupled with the need to test across class hierarchies. Programs are a vast tapestry of interweaving execution paths, and to test each path comprehensively is difficult, if not impossible. In the face of unrestrained mutation, the execution paths are overlaid with mutation paths, further adding to the chaos. Conversely, Clojure programs tend to be compositions of pure functions with isolated pools of mutation. The result of this approach helps to foster an environment conducive to unit testing. But though the layers of an application are composed of numerous functions, each individually and compositionally tested, the layers themselves and the wiring between them must also be tested.
Test-driven development (Beck 2002) has conquered the software world, and at its core it preaches that test development should drive the architecture of the overall application. Unfortunately, this approach isn’t likely to bear fruit in your Clojure programs. Instead, Clojure provides the foundation for a contracts-based program specification that’s more amenable for writing correct programs. But before we discuss contracts, we’ll touch on the ways Clojure facilitates one part of TDD, unit testing.
We don’t want to disparage test-driven development, because its goals are virtuous and testing in general is essential. Because Clojure programs are organized using namespaces, and they are themselves aggregations of functions, often pure, the act of devising a unit-test suite at the namespace boundary is often mechanical in its directness. From a larger perspective, devising comprehensive test strategies is the subject of numerous volumes and therefore outside of the scope of this book; but there are a few Clojure-specific techniques that we wish to discuss.
Stubbing (Fowler 2007) is the act of supplying an imitation implementation of a function for testing purposes. One mechanism that can perform this stubbing is the with-redefs macro implemented in the following listing. Though this exact macro will likely be included in future versions of Clojure, it’s not in Clojure 1.2, so a definition is provided.
(defn with-redefs-fn [binding-map func & args]
(let [root-bind (fn [m]
(doseq [[a-var a-val] m] (.bindRoot a-var a-val)))
old-vals (zipmap (keys binding-map)
(map deref (keys binding-map)))]
(try
(root-bind binding-map)
(apply func args)
(finally
(root-bind old-vals)))))
(defmacro with-redefs [bindings & body]
`(with-redefs-fn ~(zipmap (map #(list `var %) (take-nth 2 bindings))
(take-nth 2 (next bindings)))
(fn [] ~@body)))
The function rss-children from section 11.6 parses a Twitter RSS2 feed, returning a sequence of the top-level feed elements. Testing functions that rely on rss-children is futile against live Twitter feeds, so a stubbed implementation returning a known sequence would be more prudent, as shown next.
The tweetless-rss-children function returns a sequence of some canned data. Therefore, when testing the count-rss2-children function we temporarily change the value of rss-children so that it resolves to tweetless-rss-children instead. This change is made at the root of the rss-children Var and so is visible to all threads. As long as all the test calls to it are made before control leaves the with-redefs form, the stub will be invoked every time. Because tweet-occurrences doesn’t return until it collects results from all the futures it creates, it will use the redef given by with-redefs:
(with-redefs [rss-children tweetless-rss-children] (tweet-occurrences "dummy" "test-url")) ;=> 0
Another option that is sometimes suggested is to use binding in place of with-redefs. This would push a thread-local binding for rss-children, which might seem attractive in that it could allow other threads to bind the same Var to a different stub function, potentially for simultaneously running different tests. But because tweetoccurrences uses futures, the other threads will be calling rss-children and will see the root binding rather than the stub,[3] causing an error:
3 Alpha versions for Clojure 1.3 handle binding’s interaction with future and Agent send differently, passing dynamic bindings through to code executed in these other thread contexts. But because these are not the only kinds of threads that can be spawned, with-redefs (which may be included in Clojure 1.3) is still recommended for mocking out functions during tests.
(binding [rss-children tweetless-rss-children] (tweet-occurrences "dummy" "test-url")) ; java.util.concurrent.ExecutionException: ; java.io.FileNotFoundException: test-url
When the root binding of rss-children runs, it tries to actually load “test-url” and fails, instead of calling our stub and succeeding. The with-redefs macro is a better solution for mocking.
Clojure ships with a testing library in the clojure.test namespace used to create test suites that can further serve as partial system specifications. We won’t provide a comprehensive survey of the clojure.test functionality, but you should get a feel for how it works. Unit-test specifications in Clojure are declarative in nature, as shown next.
(require '[clojure.test :as test])
(test/deftest feed-tests
(with-redefs [rss-children tweetless-rss-children]
(test/testing "RSS2 Child Counting"
(test/is (= 1000 (count-rss2-children "dummy"))))
(test/testing "Twitter Occurrence Counting"
(test/is (= 0 (count-tweet-text-task "#clojure" ""))))))
(defn test-ns-hook []
(feed-tests))
Clojure’s test library provides a DSL for describing unit test cases. If you’ll notice, we added a failing test to the RSS2 Child Counting test so that when run, the test will fail as expected:
(test/run-tests 'user)
; Testing user
;
; FAIL in (feed-tests) (NO_SOURCE_FILE:101)
; RSS2 Child Counting
; expected: (= 1000 (count-rss2-children "dummy"))
; actual: (not (= 1000 1))
;
; Ran 1 tests containing 2 assertions.
; 1 failures, 0 errors.
;=> {:type :summary, :test 1, :pass 1, :fail 1, :error 0}
Though tests are a good way to find some errors, they make few guarantees that the system works properly. The ideal approach is the design and implementation of a framework corresponding closely with the domain of the application itself. This framework would ideally take the literal form of a domain DSL built incrementally through an interaction with domain experts and must come before testing begins. No amount of testing can substitute for thoroughly thinking through the fundamental design details. That’s not to say that the domain DSLs should be fully realized from the start; instead, the form of the DSL and its constituent parts should be reflective of the actual domain. In our experience, there are no languages comparable to Clojure for this kind of domain modeling, save for perhaps Haskell, Factor, and Scala. Having said that, the domain isn’t simply defined by the shape of its language; it also includes its expectations, which we’ll discuss presently.
Test-driven development is in many ways a heuristic affair. People tend to only test the error conditions and expectations that they can conceptualize. Surely there’s no such thing as an exhaustive test suite, but in many cases test suites tend toward a local maxima. There’s a better way to define semantic expectations within your application: using Clojure pre- and postconditions.
In section 7.1, we explored Clojure’s pre- and postcondition facility. Function constraint specification is a conceptually simple model for declaring the expectations for any given function. Function constraints can cover the full range of expected conditions imposed on the function’s inputs, its outputs, and their relative natures. The beauty of specifying constraints is that they can augment a testing regimen with the application of random values. The reason this works is that you can effectively throw out the values that fail the preconditions and instead focus on the values that cause error in the postconditions. We’ll try this approach for a simple function to square a number:
(def sqr (partial
(contract sqr-contract
[n]
(require (number? n))
(ensure (pos? %)))
#(* % %)))
[(sqr 10) (sqr -9)]
;=> [100 81]
The contract for sqr states simply: require a number and ensure that its return is positive. Now we can create a simple test driver[4] that throws many random values at it to see if it breaks:
4 For the sake of highlighting this technique, we’ve simplified our test driver. Testing a limited range of input values might not be an appropriate approach in all circumstances.
(doseq [n (range Short/MIN_VALUE Short/MAX_VALUE)]
(try
(sqr n)
(catch AssertionError e
(println "Error on input" n)
(throw e))))
; Error on input 0
;=> java.lang.AssertionError: Assert failed: (pos? %)
Even when adhering to the tenets of the preconditions, we’ve uncovered an error in the sqr function at the postcondition end. Postconditions should be viewed as the guarantee of the return value given that the preconditions are met. The reason for the postcondition error is that the function’s contract doesn’t specify that the number n should be nonzero. By adding a check for zero (not= 0 n) in the preconditions, we can guarantee that the sqr function acts as expected. To perform this same verification using unit testing is trivial in this case, but what if the edge condition wasn’t as obvious? In such a case, it’s probable that the error might not be caught until it’s too late. Of course, there’s no guarantee that your contracts are comprehensive, but that’s why domain expertise is often critical when defining them.
Function constraints aren’t code. They take the form of code, but that fact is only a matter of representation. Instead, constraints should be viewed as a specification language describing expectations and result assurances. On the other hand, unit tests are code, and code has bugs. Contracts, on the other hand, are essential semantic coupling.
Another potential advantage of contracts over tests is that in some cases, tests can be generated from the contracts themselves. Also, pre- and postconditions are amenable to being expressed as an overall description of the system itself, which can thus be fed into a rule base for query and verification. Both of these cases are outside of the scope of this book, but you shouldn’t be surprised if they make their way into future versions of Clojure. There’s tremendous potential in Clojure’s pre- and postconditions. Though they’re currently low-level constructs, they can be used to express full-blown design by contract facilities for your own applications.
Clojure changes the way that you think.
Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
Greenspun’s Tenth Rule
The book Design Patterns: Elements of Reusable Object-Oriented Software (Gamma et al 1995) was a seminal work of software design and development. You’d be hard pressed to find a software programmer in this day and age who’s not familiar with this work. The book describes 24 software best practices encountered throughout the course of experience in developing software projects of varying sizes.
Design patterns have obtained a bad reputation in some circles, whereas in others they’re considered indispensable. From our perspective, design patterns are a way to express software best practices in a language-neutral way. But where patterns fall short is that they don’t represent pure abstraction. Instead, design patterns have come to be viewed as goals in and of themselves, which is likely the source of the antagonism aimed at them. The ability to think in abstractions is an invaluable skill for a software programmer to strengthen. In this section, we’ll attempt to dissuade you from viewing Clojure features as design patterns (Norvig 1998) and instead as an inherent nameless quality.
Most if not all of the patterns listed in the book are applicable to functional programming languages in general, and to Clojure in particular. But at its most pragmatic, the patterns described are aimed at patching deficiencies in popular object-oriented programming languages. This practical view of design patterns isn’t directly relevant to Clojure, because in many ways the patterns are ever-present and are first-class citizens of the language itself. We won’t provide a comprehensive survey of the ways that Clojure implements or eliminates popular design patterns but will provide enough to make our point.
Clojure’s add-watch and remove-watch functions provide the underpinnings of an observer (publisher/subscriber) capability based on reference types. We can illustrate this through the implementation of the simple defformula macro shown in listing 13.7.
By using watchers on references, you can use defformula to provide an abstract value that changes when any of its parts change. A more traditional Lisp approach is to provide predefined hooks (Glickstein 1997) that are called at certain times within the execution cycle. In addition, using proxy or gen-class to extend java.util.Observable is the most straightforward way to wire into existing source code using the Observer pattern.
Algorithm strategies selected at runtime are common practice in Clojure, and there are a number of ways to implement them. One such way is via continuation-passing style, as we explored in section 7.3. A more general solution is to pass the desired function as an argument to a higher-order function, such as you’d see in the ubiquitous map, reduce, and filter functions. Further, we’ll provide a case of dynamic error functions in the next section illustrating how Clojure’s multimethods are a more powerful substitute for the classic strategy pattern.
The Visitor pattern is designed to describe a way to decouple operations on a structure from the structure itself. Even casual observers will see the parallel to Clojure’s multimethods, protocols, types, proxies, and reify features.
The Abstract Factory pattern is used to describe a way to create related objects without having to name explicit types at the point of creation. Clojure’s types avoid the creation of explicit hierarchies (although ad hoc hierarchies can be created, as seen in section 9.2). Therefore, in Clojure this particular usage scenario is relegated to use within Java interoperability contexts. But the use of factory functions to abstract the call to the constructors of types and records is idiomatic and in fact actively promoted. The reasons for a Clojure-style factory are to simplify the importing requirements of a type or record, and also to add additional project-specific functionality to the constructor (keyword arguments, default values, and so on).
The Interpreter pattern is in every way Greenspun’s Tenth Rule formalized. Many projects of sufficient size can be well served by the inclusion of a specialized grammar describing parts of the system itself. Clojure macros make the matter of creating specialized grammars a first-class member of the language.
The creation of complex structures from representation is central to Clojure programming, although it’s viewed differently from a similar object-oriented approach—the Builder pattern. In section 8.4, we used a simple data representation as the input to Clojure’s clojure.xml/emit function to produce an analogous XML representation. If you preferred a different output representation, then you could write another conversion function. If you preferred finer control over the constituent parts, then you could write functions or multimethods for each and specialize at runtime.
The use of Clojure namespaces, as seen in section 9.1, is the most obvious way to provide a simplified façade for a more complex API. You can also use the varying levels of encapsulation (as outlined in section 2.4) for more localized façades.
Iteration in Clojure is defined through an adherence to the seq protocol, as outlined in section 5.1 and later elaborated on in sections 9.3 about types and protocols.
Though not a classical pattern in the Design Patterns sense, dependency injection has become a de facto pattern for object-oriented languages that don’t allow overridable class constructors. This condition requires that separate factory methods and/or classes create concrete instances conforming to a given interface. In forsaking the ability to define classes, Clojure completely avoids the problem that DI solves. Instead, Clojure’s closest analogue to this “pattern” is the use of functions returning closures that are specialized based on the original arguments. Likewise, you could use partial application and composition similarly.
We could go further with this survey, but to do so would belabor the point: most of what are known as design patterns are either invisible or trivial to implement in Clojure. But what about the Prototype pattern, you ask? We implemented the UDP in section 9.2. Decorators or chain of responsibility? Why not use a macro that returns a function built from a list of forms spliced into the -> or ->> macro? Proxies would likely be implemented as closures and so would commands. The list goes on and on, and in the end you must face the inevitable—Clojure changes the way that you think.
Our goal throughout this book was to show the proper way to write Clojure code, with mostly deferral and hand-waving regarding error handling and debugging. In this section, we’ll cover these topics with what you might view as a unique twist, depending on your programming background.
As we showed in figure 10.7, there are two directions for handling errors. The first, and likely most familiar, refers to the passive handling of exceptions bubbling outward from inner functions. But built on Clojure’s dynamic Var binding is a more active mode of error handling, where handlers are pushed into inner functions. In section 11.10, we mentioned that the binding form is used to create thread-local bindings, but its utility isn’t limited to this use case. In its purest form, dynamic scope is a structured form of a side effect (Steele 1978). You can use it to push Vars down a call stack from the outer layers of a function nesting into the inner layers, a technique that we’ll demonstrate next.
In section 8.4, we built a simple tree structure for a domain model where each node was of this form:
{:tag <node form>, :attrs {}, :content [<nodes>]}
As it turns out, the traversal of a tree built from such nodes is straightforward using mundane recursion, as shown:
(defn traverse [node f]
(when node
(f node)
(doseq [child (:content node)]
(traverse child f))))
For each node in the tree, the function f is called with the node itself, and then each of the node’s children is traversed in turn. Observe how traverse works for a single root node:
(traverse {:tag :flower :attrs {:name "Tanpopo"} :content []}
println)
; {:tag :flower, :attrs {:name Tanpopo}, :content []}
But it’s much more interesting if we traverse trees larger than a single node. Therefore, we can build a quick tree from an XML representation using Clojure’s clojure.
xml/parse function:
(use '[clojure.xml :as xml])
(def DB
(-> "<zoo>
<pongo>
<animal>orangutan</animal>
</pongo>
<panthera>
<animal>Spot</animal>
<animal>lion</animal>
<animal>Lopshire</animal>
</panthera>
</zoo>"
.getBytes
(java.io.ByteArrayInputStream.)
xml/parse))
The DB Var contains an animal listing for a small zoo. Note that two of the animals listed have the elements Spot and Lopshire; both are seemingly out of order for a zoo. Therefore, we can write a function to handle these nefarious intruders.
The multimethod visit can be used as the input function to the traverse function and will only trigger when a node with the :tag attribute of :animal is encountered. When the method triggered on :animal is executed, the node :content is destructured and checked against the offending Spot and Lopshire values. When found, the devious node is then passed along to an error handler handle-weird-animal for reporting.[5] By default, the error handler throws an exception. This model of error handling is the inside-out model of exceptions. But handling errors in this way stops the processing:
5 The metadata {:dynamic true} attached to handle-weird-animal isn’t really used in Clojure 1.2, but it may be required in future versions of Clojure starting with 1.3 to allow the dynamic binding we’re about to demonstrate.
(traverse DB visit) ; orangutan ; java.lang.Exception: Spot must be 'dealt with'
We’ve managed to identify Spot, but the equally repugnant Lopshire escapes our grasp. It’d be nice to instead use a different version of handle-weird-animal that allows us to both identify and deal with every such weird creature. We could pass handle-weird-animal along as an argument to be used as an error continuation,[6] but that pollutes the argument list of every function along the way. Likewise, we could inject catch blocks at a point further down the call chain, say within visit, but we might not be able to change the source, and if we could it makes for a more insidious pollution. Instead, using a dynamic binding is a perfect solution, because it allows us to attach specific error handlers at any depth in the stack according to their appropriate context:
6 See section 7.3 for more information on continuation-passing style.
(defmulti handle-weird (fn [{[name] :content}] name))
(defmethod handle-weird "Spot" [_]
(println "Transporting Spot to the circus."))
(defmethod handle-weird "Lopshire" [_]
(println "Signing Lopshire to a book deal."))
(binding [handle-weird-animal handle-weird]
(traverse DB visit))
; orangutan
; Transporting Spot to the circus.
; lion
; Signing Lopshire to a book deal.
As you might expect, this approach works across threads to allow for thread-specific handlers:
(def _ (future
(binding [handle-weird-animal #(println (:content %))]
(traverse DB visit))))
; orangutan
; [Spot]
; lion
; [Lopshire]
What we’ve outlined here is a simplistic model for a grander error-handling scheme. Using dynamic scope via binding is the preferred way to handle recoverable errors in a context-sensitive manner.
The natural progression of debugging techniques as discovered by a newcomer to Clojure follows a fairly standard progression:
| 1 | (println) |
| 2 | A macro to make (println) inclusion simpler |
| 3 | Some variation on debugging as discussed in this section |
| 4 | IDEs, monitoring, and profiling tools |
Many Clojure programmers stay at step 1, because it’s simple to understand and also highly useful, but there are better ways. After all, you’re dealing with Clojure—a highly dynamic programming environment. Observe the following function:
(defn div [n d] (int (/ n d)))
The function div simply divides two numbers and returns an integer value. You can break div in a number of ways, but the most obvious is to call it with zero as the denominator: (div 10 0). Such an example would likely not give you cause for concern should it fail, because the conditions under which it fails are fairly limited, well known, and easily identified. But not all errors are this simple, and the use of println is fairly limited. Instead, a better tool would likely be a generic breakpoint[7] that could be inserted at will and used to provide a debug console for the current valid execution context. Imagine it would work as follows:
7 The code in this section is based on debug-repl created by the amazing George Jahad, extended by Alex Osborne, and integrated into Swank-Clojure by Hugo Duncan.
(defn div [n d] (break) (int (/ n d))) (div 10 0) debug=>
At this prompt, you can query the current lexical environment, experiment with different code, and then resume the previous execution as before. As it turns out, such a tool is within your grasp.
We hope that by the end of this section, you’ll understand that Lisps in general, and Clojure in particular, provide an environment where the whole of the language truly is “always available” (Graham 1993). First of all, an interesting fact to note is that the Clojure REPL is available and extensible via the Clojure REPL itself, via the clojure.main/repl function. By accessing the REPL implementation directly, you can customize it as you see fit for application-specific tasks.
Typing (clojure.main/repl) at the REPL seemingly does nothing, but rest assured you’ve started a sub-REPL. What use is this? To start, the repl function takes a number of named parameters, each used to customize the launched REPL in different ways. We’ll utilize three such hooks—:prompt, :eval, and :read—to fulfill a breakpoint functionality.
The repl function’s :read hook takes a function of two arguments: the first corresponding to a desired display prompt, and the second to a desired exit form. We want the debug console to provide convenience functions—we’d like it to show all of the available lexical bindings and also to resume execution. It also needs to be able to read valid Clojure forms, but because that’s too complex a task, we’ll instead farm that functionality out to Clojure’s default REPL reader.
(defn readr [prompt exit-code]
(let [input (clojure.main/repl-read prompt exit-code)]
(if (= input ::tl)
exit-code
input)))
We can start testing the reader immediately:
(readr #(print "invisible=> ") ::exit) [1 2 3] ;=> [1 2 3] (readr #(print "invisible=> ") ::exit) ::tl ;=> :user/exit
The prompt that we specified was of course not printed, and typing ::tl at the prompt did nothing because the readr function isn’t yet provided to the repl as its :read hook. But before we do that, we need to provide a function for the :eval hook. Needless to say, this is a more complex task.
In order to evaluate things in context, we first need a function cab to garner the bindings in the current context. Fortunately for us, Clojure macros provide an implicit argument &env that’s a map of the local bindings available at macro-expansion time. We can then extract from &env the values associated with the bindings and zip them up with their names into a map for the local context, as shown next.
(defmacro local-context []
(let [symbols (keys &env)]
(zipmap (map (fn [sym] `(quote ~sym)) symbols) symbols)))
(local-context)
;=> {}
(let [a 1, b 2, c 3]
(let [b 200]
(local-context)))
;=> {a 1, b 200, c 3}
The local-context macro provides a map to the most immediate lexical bindings, which is what we want. But what we really want to do is to provide a way to evaluate expressions with this contextual bindings map. Wouldn’t you know it, the contextual-eval function from section 8.1 fits the bill. So now that we have the bulk of the implementation complete, we’ll now hook into the repl function to provide a breakpoint facility.
The hard parts are done, so to wire them into a usable debugging console is relatively easy, as shown next.
(defmacro break []
`(clojure.main/repl
:prompt #(print "debug=> ")
:read readr
:eval (partial contextual-eval (local-context))))
Using this macro, we can now debug the original div function:
(defn div [n d] (break) (int (/ n d))) (div 10 0) debug=>
Querying locals to find the “problem” is simple:
debug=> n
;=> 10
debug=> d
;=> 0
debug=> (local-context)
;=> {div #<user$div__155 user$div__155@51e67ac>, n 10, d 0}
debug=> ::tl
; java.lang.ArithmeticException: Divide by zero
So there’s the problem! We passed in a zero as the denominator. We should fix that.
What would be the point if you couldn’t set multiple breakpoints? Fortunately, you can, as we show in the following listing.
(defn keys-apply [f ks m]
(break)
(let [only (select-keys m ks)]
(break)
(zipmap (keys only) (map f (vals only)))))
(keys-apply inc [:a :b] {:a 1, :b 2, :c 3})
debug=> only
; java.lang.Exception: Unable to resolve symbol: only in this context
debug=> ks
;=> [:a :b]
debug=> m
;=> {:a 1, :b 2, :c 3}
debug=> ::tl
debug=> only
;=> {:b 2, :a 1}
debug=> ::tl
;=> {:a 2, :b 3}
And finally, you can use breakpoints within the body of a macro (in its expansion, not its logic), as shown next.
(defmacro awhen [expr & body]
(break)
`(let [~'it ~expr]
(if ~'it
(do (break) ~@body))))
(awhen [1 2 3] (it 2))
debug=> it
; java.lang.Exception: Unable to resolve symbol: it in this context
debug=> expr
;=> [1 2 3]
debug=> body
;=> ((it 2))
debug=> ::tl
debug=> it
;=> [1 2 3]
debug=> (it 1)
;=> 2
debug=> ::tl
;=> 3
There’s much room for improvement, but we believe that the point has been made. Having access to the underpinnings of the language allows you to create a powerful debugging environment with little code. We’ve run out of ideas by now, so we’ll say our credo only once more, and we hope by now you believe us.
Clojure changes the way that you think.
This book possess many lacunae, but it’s this way by design. In many cases, we’ve skipped approaches to solving problems via a certain route to avoid presenting nonidiomatic code. In many examples, we’ve left exposed wiring. For example, the defcontract macro requires that you partially apply the contract to the function under constraint instead of providing a comprehensive contract overlay façade. It was our goal to leave wiring exposed because exposed wiring can be explored, tampered with, and ultimately enhanced—which we hope you’ll find the motivation to do. We’ve worked hard to provide a vast array of relevant references should you choose to further enhance your understanding of the workings and motivations for Clojure. But it’s likely that we’ve missed some excellent resources, and we hope that you instead are able to uncover them in time. Finally, this wasn’t a survey of Clojure, and many of the functions available to you weren’t used in this book. We provide some pointers in the resource list, but there’s no way that we could do justice to the libraries and applications mentioned and those unmentioned. We implore you to look deeper into the functionality of not only Clojure, but the rich ecology of libraries and applications that have sprung up in its relatively short life span.
Thank you for taking the time to read this book; we hope it was as much a pleasure to read as it was for us to write. Likewise, we hope that you’ll continue your journey with Clojure. Should you choose to diverge from this path, then we hope that some of what you’ve learned has helped you to view the art of programming in a new light. Clojure is an opinionated language, but it and most of its community believe that these opinions can work to enhance the overall state of affairs in our software industry. The onus is on us to make our software robust, performant, and extensible. We believe that the path toward these goals lies with Clojure.
Do you?
—FOGUS AND HOUSER 2010