Daniel Compton

The personal blog of Daniel Compton - Projects

Clojure Reader Conditionals by Example

One of the most interesting features of the upcoming Clojure 1.7 release (and ClojureScript 0.0-3196) is Reader Conditionals. These are designed to allow different variants of Clojure to share common logic, while also writing platform specific code in the same file.

Previously this was solved by cljx which processed a Clojure file with a .cljx extension and output multiple platform specific files. These were then read as normal by the Clojure Reader. This worked well, but required another piece of tooling to run, and it wasn’t able to be used in Clojure Core projects.

Reader Conditionals are an answer from the Clojure Core team to that problem. They are similar in spirit and appearance to cljx, however they are integrated into the Clojure Compiler, and don’t require any extra tooling beyond Clojure 1.7 or greater. That bears repeating again, there is no extra tooling required. To use Reader Conditionals, all you need is for your file to have a .cljc extension and to use Clojure 1.7 or ClojureScript 0.0-3196 or higher. Reader Conditionals also have the advantage that they are just data, and can be manipulated like ordinary Clojure data.

There are two types of reader conditionals, standard and splicing. The standard conditional reader behaves similarly to a traditional cond. The syntax for usage is #? and looks like:

#?(:clj  (Clojure expression)
   :cljs (ClojureScript expression)
   :clr  (Clojure CLR expression))

The syntax for a splicing conditional read is #?@. It is used to splice lists into the containing form. So the Clojure reader would read this:

(defn build-list []
  (list #?@(:clj  [5 6 7 8]
            :cljs [1 2 3 4])))

as this:

(defn build-list []
  (list 5 6 7 8))

One important thing to note is that in Clojure 1.7 a splicing conditional reader cannot be used to splice in multiple top level forms (tracked in CLJ-1706). In concrete terms, this means you can’t do this:

;; Don't do this!, will throw an error
#?@(:clj 
    [(defn clj-fn1 [] :abc)
     (defn clj-fn2 [] :cde)])
;; CompilerException java.lang.RuntimeException: Reader conditional splicing not allowed at the top level.

Instead you’d need to do wrap each function individually:

#?(:clj (defn clj-fn1 [] :abc))
#?(:clj (defn clj-fn2 [] :cde))

or use a do to wrap all of the top level functions:

#?(:clj
    (do (defn clj-fn1 [] :abc)
        (defn clj-fn2 [] :cde)))

Which one you choose would probably depend on aesthetics, and what your plans for porting these functions to more Clojure platforms were. Let’s go through some examples of places you might want to use these new reader conditionals.

Host interop

Host interop is one of the biggest pain points that cljx and reader conditionals both solve. You may have a Clojure file that is almost pure Clojure, but needs to call out to the host environment for one function. This is a classic example:

(defn str->int [s]
  #?(:clj  (java.lang.Integer/parseInt s)
     :cljs (js/parseInt s)))

Namespaces

Namespaces are the other big pain point for sharing code between Clojure and ClojureScript. ClojureScript has different syntax for requiring macros than Clojure. To use these macros in a .cljc file, you’ll need Reader Conditionals in the namespace declaration.

I saw Vesa Karvonen had a neat trick for this, taking advantage of the fact you can have multiple :require’s in one Clojure ns form. N.B.: ClojureScript ns forms can only have one :require in them.

(ns poc.cml.sem
  (#?(:clj :require :cljs :require-macros)
    [poc.cml.macros :refer [go sync!]])
  (:require
    [poc.cml :refer [choose wrap]]
    [poc.cml.util :refer [chan on put!]]))

Here is a larger example showing more conventional usage from a test in route-ccrs

(ns route-ccrs.schema.ids.part-no-test
  (:require #?(:clj  [clojure.test :refer :all]
               :cljs [cljs.test :refer-macros [is]])
            #?(:cljs [cljs.test.check :refer [quick-check]])
            #?(:clj  [clojure.test.check.clojure-test :refer [defspec]]
               :cljs [cljs.test.check.cljs-test :refer-macros [defspec]])
            #?(:clj  [clojure.test.check.properties :as prop]
               :cljs [cljs.test.check.properties :as prop 
                       :include-macros true])
            [schema.core :as schema :refer [check]]
            [route-ccrs.schema.ids :refer [PartNo]]
            [route-ccrs.generators.part-no
             :refer [gen-part-no gen-invalid-part-no]]))

Sente uses CLJX for sharing code between Clojure and ClojureScript. I’ve rewritten the main namespace to use reader conditionals. Notice that we’ve used the splicing reader conditional to splice the vector into the parent :require. Notice also that some of the requires are duplicated between :clj and :cljs.

(ns taoensso.sente
  (:require
    #?@(:clj  [[clojure.string :as str]
               [clojure.core.async :as async]
               [taoensso.encore :as enc]
               [taoensso.timbre :as timbre]
               [taoensso.sente.interfaces :as interfaces]]
        :cljs [[clojure.string :as str]
               [cljs.core.async :as async]
               [taoensso.encore :as enc]
               [taoensso.sente.interfaces :as interfaces]]))
  #?(:cljs (:require-macros 
             [cljs.core.async.macros :as asyncm :refer (go go-loop)]
             [taoensso.encore :as enc :refer (have? have have-in)])))

In this example, we want to be able to use the rethinkdb.query namespace in Clojure and ClojureScript. However we can’t load the required rethinkdb.net in ClojureScript as it uses Java sockets to communicate with the database. Instead we use a reader conditional so it’s only required when read by Clojure programs.

(ns rethinkdb.query
  (:require [clojure.walk :refer [postwalk postwalk-replace]]
            [rethinkdb.query-builder :as qb :refer [term parse-term]]
            #?(:clj [rethinkdb.net :as net])))

;; snip...

#?(:clj (defn run [query conn]
      (let [token (get-token conn)]
        (net/send-start-query conn token (replace-vars query)))))

Exception handling

Exception handling is another area that benefits from reader conditionals. ClojureScript now supports (catch :default) to catch everything and this may come Clojure too, however you will often still want to handle host specific exceptions. Here’s an example from lemon-disc.

(defn message-container-test [f]
  (fn [mc]
      (passed?
        (let [failed* (failed mc)]
          (try
            (let [x (:data mc)]
              (if (f x) mc failed*))
            (catch #?(:clj Exception :cljs js/Object) _ failed*))))))

Splicing

I don’t see a lot of uses yet for the splicing reader conditional outside of namespace declarations, but to get really meta, lets look at the tests for reader conditionals in the ClojureCLR reader. What might not be obvious at first glance is that the vector of [:a :b :c] is actually being spliced into the parent wrapping vector.

(deftest reader-conditionals
     ;; snip
     (testing "splicing"
              (is (= [] [#?@(:clj [])]))
              (is (= [:a] [#?@(:clj [:a])]))
              (is (= [:a :b] [#?@(:clj [:a :b])]))
              (is (= [:a :b :c] [#?@(:clj [:a :b :c])]))
              (is (= [:a :b :c] [#?@(:clj [:a :b :c])]))))

File organisation

There isn’t a clear community consensus yet around where to put .cljc files. For libraries I think in most cases it will make sense to just have one src directory where .clj, .cljs, and .cljc files go. In applications it may end up being simpler to have src/clj, src/cljc, and src/cljs directories.

Conclusion

At the time of writing, I’m not aware of any way to use .cljc files in versions of Clojure less than 1.7, nor is there any porting mechanism to preprocess .cljc files like CLJX does. For that reason library maintainers may need to wait for a while until they can safely drop support for older versions of Clojure and adopt reader conditionals.

UPDATE: There is now a leiningen plugin cljsee which can preprocess cljc files in the same way that cljx did, to output .clj and .cljs files. This might be a good option if you want to keep backwards compatibility with older Clojure versions.

If you have any other interesting uses for reader conditionals, let me know and I’ll update this post to add them.

Lastly I’d like to offer my congratulations to everyone who worked on Reader Conditionals, and its predecessor Feature Expressions. There was a lot of thought, effort, and discussion put into this, and I think that it has produced something useful, extensible, and long-lasting.


Want more like this?

Sign up to my newsletter to hear more about Clojure, Distributed Systems, and programming.

* indicates required

The James Mickens Collection

James Mickens works at Microsoft Research. Amongst his more serious work, he has written some hilarious papers for Usenix, and given some funny talks. I’ve collected a selection of my favourite quotes here, but you should really read and watch them all from start to finish.

Videos:

Usenix papers:

The Night Watch

When you debug systems code, there are no high-level debates about font choices and the best kind of turquoise, because this is the Old Testament, an angry and monochromatic world, and it doesn’t matter whether your Arial is Bold or Condensed when people are covered in boils and pestilence and Egyptian pharaoh oppression. HCI people discover bugs by receiving a concerned email from their therapist. Systems people discover bugs by waking up and discovering that their first-born children are missing and “ETIMEDOUT ” has been written in blood on the wall.

The Saddest Moment

Listen, regardless of which Byzantine fault tolerance protocol you pick, Twitter will still have fewer than two nines of availability. As it turns out, Ted the Poorly Paid Datacenter Operator will not send 15 cryptographically signed messages before he accidentally spills coffee on the air conditioning unit and then overwrites your tape backups with bootleg recordings of Nickelback. Ted will just do these things and then go home, because that’s what Ted does. His extensive home collection of “Thundercats” cartoons will not watch itself. Ted is needed, and Ted will heed the call of duty.

Mobile Computing Research Is a Hornet’s Nest of Deception and Chicanery

So, your phone just starts doing stuff, all the stuff that it knows how to do, and it’s just going nuts, and your apps are closing and opening and talking to the cloud and configuring themselves in unnatural ways, and your phone starts vibrating and rumbling with its little rumble pack, and it will gently sing like a tiny hummingbird of hate, and you’ll look at the touchscreen, and you’ll see that things are happening, my god, there are so many happenings, and you’ll try to flip the phone over and take out the battery, because now you just want to kill it and move to Kansas and start over, but the back panel of the phone is attached by a molecule-sized screw that requires a special type of screwdriver that only Merlin possesses, and Merlin isn’t nearby, and your phone is still rumbling, and by this point, you can understand the rumble, it’s a twin language that you and your phone invented, and the phone is rumbling, and it’s saying that it’s far from done, that it has so much more that it wants to do, that there are so many of your frenemies that it wants to “accidentally” call and then leave you to deal with the social ramifications, and your phone, it buzzes, and you think that you see it smiling, and you begin to realize that land-line telephones were actually a pretty good idea.

This World of Ours

My point is that security people need to get their priorities straight. The “threat model” section of a security paper resembles the script for a telenovela that was written by a paranoid schizophrenic: there are elaborate narratives and grand conspiracy theories, and there are heroes and villains with fantastic (yet oddly constrained) powers that necessitate a grinding battle of emotional and technical attrition. In the real world, threat models are much simpler. Basically, you’re either dealing with Mossad or not-Mossad. If your adversary is not-Mossad, then you’ll probably be fine if you pick a good password and don’t respond to emails from [email protected] If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https://. If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled with tumors, they’re going to hold a press conference and say “It wasn’t us” as they wear t-shirts that say “IT WAS DEFINITELY US”.

To Wash It All Away

The fourth and seventh errors represent uncaught JavaScript exceptions. In a rational universe, a single uncaught exception would terminate a program, and if a program continued to execute after throwing such an exception, we would know that Ragnarok is here and Odin is not happy. In the browser world, ignoring uncaught exceptions is called “Wednesday, and all days not called ‘Wednesday.’” The JavaScript event loop is quite impervious to conventional notions of software reliability, so if an event handler throws an exception, the event loop will literally pretend like nothing happened and keep running. This ludicrous momentum continues even if, in the case of the seventh error, the Web page tries to call init() on an object that has no init() method. You should feel uncomfortable that a Web page can disagree with itself about the existence of initialization routines, but the page is still allowed to do things with things. Such a dramatic mismatch of expectations would be unacceptable in any other context. You would be sad if you went to the hospital to have your appendix removed, and the surgeon opened you up, and she said, “I DIDN’T EXPECT YOUR LIVER TO HAVE GILLS,” and then she proceeded with her original surgical plan, despite the fact that you’re apparently a mer-person. Being a mer-person should have non-ignorable ramifications in the material universe. Similarly, if a Web page thinks than an object should be initialized, but the object has no initialization method, the browser shouldn’t laugh about it and then proceed under the assumption that the rest of the page is agnostic about whether its objects are composed of folly.

The Slow Winter

Perhaps the processor could run multiple copies of each program, comparing the results to detect errors? Perhaps a new video codec could tolerate persistently hateful levels of hardware error? All of these techniques could be implemented. However, John slowly realized that these solutions were just things that he could do, and inventing “a thing that you could do” is a low bar for human achievement. If I were walking past your house and I saw that it was on fire, I could try to put out the fire by finding a dingo and then teaching it how to speak Spanish. That’s certainly a thing that I could do. However, when you arrived at your erstwhile house and found a pile of heirloom ashes, me, and a dingo with a chewed-up Rosetta Stone box, you would be less than pleased, despite my protestations that negative scientific results are useful and I had just proven that Spanish-illiterate dingoes cannot extinguish fires using mind power.