Download all dependencies

Often in a CI environment, you want to download all of the dependencies for a project before building the project. In Leiningen, you can use lein deps to do this.

You can use clojure -P to Prepare the project with the Clojure CLI. This downloads libs and caches the classpath.

Note that you should avoid running clojure -P in parallel on new environments (like CI). When running clojure -P for the first time, it will set up files in ~/.clojure and clone gitlibs. If you run multiple processes doing this simultaneously, you can get race conditions that throw errors.

View the dependency tree

When debugging dependency issues, it’s often helpful to see which library pulled in a transitive dependency.

Leiningen can print the dependency showing the resolved transitive dependencies.

$ lein deps :tree
 [clansi "1.0.0"]
 [clj-kondo "2022.03.09" :scope "test"]
 [babashka/fs "0.1.2" :scope "test"]
 [cheshire "5.10.2" :scope "test"]
 [com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.12.4" :scope "test" :exclusions [[com.fasterxml.jackson.core/jackson-databind]]]
 [com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.12.4" :scope "test" :exclusions [[com.fasterxml.jackson.core/jackson-databind]]]
 [tigris "0.1.2" :scope "test"]
 [com.cognitect/transit-clj "1.0.324" :scope "test"]
 # ...

clojure has a similar command, though with a slightly different output. In contrast to Leiningen, clojure prints the entire dependency tree. Dependencies clojure didn’t select get an X in front of them, and a short reason like :older-version or :excluded. These views can be helpful when debugging why you didn’t get the version you expected.

$ clojure -Stree
org.clojure/clojure 1.11.0
 . org.clojure/spec.alpha 0.3.218
 . org.clojure/core.specs.alpha 0.2.62
org.clojure/java.classpath 1.0.0
clansi/clansi 1.0.0
org.clojure/data.json 2.4.0
org.slf4j/slf4j-simple 2.0.0-alpha1
 . org.slf4j/slf4j-api 2.0.0-alpha1
org.owasp/dependency-check-core 7.0.2
 . us.springett/cpe-parser 2.0.2
 X org.slf4j/slf4j-api 1.7.30 :older-version
 . com.vdurmont/semver4j 3.1.0
 X org.slf4j/slf4j-api 1.7.36 :older-version

View the Classpath

While dependency trees can help track down dependency problems, sometimes you need to see the actual classpath.

Tip: I like to use | tr ':' '\n' | sort to get a sorted list of the classpath entries.

You can do this in Leiningen:

lein classpath
# or
lein classpath | tr ':' '\n' | sort

With clojure:

clojure -Spath

Both Leiningen and clojure have more commands for dependencies; you can read more about them at Dep Expansion, Deps and CLI, and lein deps.

In Maven you can specify a dependency on a version with <classifier> like this:

<dependency>
    <groupId>io.netty</groupId>
    <artifactId>netty-transport-native-epoll</artifactId>
    <version>4.1.74</version>
    <classifier>linux-x86_64</classifier>
</dependency>

To specify this dependency with Clojure’s deps.edn, you need to use the syntax <group>/<artifact>$<classifier>:

{:deps 
  {io.netty/netty-transport-native-epoll$linux-x86_64 
    {:mvn/version "4.1.74.Final"}}}

Others have also posted their thoughts on Heart of Clojure: Fork This Conference, The people’s conference, The hallway track conference, Community with lots of heart, and A courageous conference?.

Here were some things that I think made Heart of Clojure so special. If you’re running a conference, consider stealing some of these ideas.

• There was an site which listed activities around the event. Anyone could host an activity and others could join. Having it on a website meant everyone knew what was happening around the conference. I felt like this helped Heart of Clojure feel very welcoming to all of the conference goers, especially those who were less outgoing.
• One of the nights I joined an activity for an adventurous dinner. People got split into groups of 6-8 with a reservation at a restaraunt in Leuven. This broke up friend groups and introduced me to a bunch of new people. The restaurant we went to had their own brewery, and I got to try the flavours of a beer soup, beer stew, and beer crème brûlée!
• Yulia Startsev’s talk To loosen up, to put together set the tone for the conference and captured the spirit of mixing technology and the humanities. Many of the talks and conversations referenced her talk throughout the rest of the event.
• Leuven was a perfect town to have the conference. It was large enough to have a good nightlife, but small enough that you could walk between the conference venue, the bars, restauruants, and hotels in 10-15 minutes.
• In the months before the conference, Jan Stępień paired up new speakers with more experienced speakers. This was my first conference talk, and I got paired up with Tiago Luchini. Tiago was a very experienced public speaker and gave me great feedback on my talk. Having to show someone my presentation helped me get it into shape well before the conference. I think Jan’s work here contributed to the high quality of the talks at the conference.
• I’m not a vegeterian, but I really appreciated that all of the breakfast and lunches were vegeterian. I can imagine it would have made things very easy for vegeterians. The food was delicious and healthy tasting.
• There was ample time left around the talks so that you didn’t need to choose between the ‘hallway track’ and the conference track. On the Saturday there was a siesta break between 12pm and 4pm. You could do one of the activities, take a nap, or just chat with others. I was a bit skeptical before the conference that this was going to be too long, but it worked out really well.
• The lightning talks on Saturday broke up the afternoon and left space for serendipity to strike. Connie’s talk on waffles, Vim, and building her own language captured the audience.
• On the Saturday evening after the conference ended, lots of people stayed behind to help pack down the auditorium. This made the job much easier.
• There was a large contingent of people from Berlin, and they were all incredible. Berlin feels like it has a really vibrant community, one day I’d like to visit it.

Heart of Clojure was a very special event. I can’t imagine how much work it was for Arne Brasseur, Martin Klepsch, and all of the other helpers before, during, and after the event but it paid off. Heart of Clojure felt like a very polished event, not something being put on for the first time. Thanks to everyone who organised, helped out, spoke, sponsored, and attended the conference. I hope there is another Heart of Clojure in the future, if you get a chance to go, I highly recommend it.

If you see profiles like this in a project.clj, here is what is happening:

:dev  [:project/dev  :profiles/dev]
:test [:project/test :profiles/test]
:profiles/dev  {}
:profiles/test {}
:project/dev  { ... }
:project/test { ... }

Leiningen has a feature called Composite Profiles. On startup, if Leiningen sees a vector of keywords for a profile, it will lookup each keyword as a profile, and merge them together. For the :dev profile, Leiningen will merge the values of :project/dev and :profiles/dev.

If you want to add any custom settings or dependencies for your own use, you can place them into the :profiles/dev or :profiles/test in your ~/.lein/profiles.clj. If either of these are set in the user’s profiles.clj, they will override the empty :profiles/dev map specified in the project.clj. You need an empty map for :profiles/dev {} in the project.clj, because otherwise Leiningen will complain about a missing profile.

[net.danielcompton/defn-spec-alpha "0.1.0"]

A quick peek at defn-spec:

(ds/defn to-zoned-dt :- ::zoned-date-time
  [instant :- ::instant
   zone-id :- ::zone-id]
  (ZonedDateTime/ofInstant instant zone-id))

One of the features in Schema that I always appreciated was inline function schemas, using a schema.core/defn macro.

When spec was released it had many similarities to Schema, but one thing it didn’t have was a way of expressing specs inline with your function definition. Spec only supported defining function specs separately with fdef. This does have some advantages, it forces you to think carefully about changing your specs, and to be aware of possibly breaking consumers. While this is valuable, not all code is written to these constraints, and I found having fdef’s separate from the function definition had a number of downsides.

When writing Clojure, I noticed that I often resisted writing specs for functions. After thinking about it I realised that I didn’t want to duplicate information from the defn into the fdef. It’s not a huge deal, but it was enough to deter me from writing specs on code that was being heavily modified. This is a really useful time to have basic specs on your functions, so that you can catch refactorings gone wrong early.

I created defn-spec to increase the locality of the spec definitions, and to reduce the activation energy to start adding specs to your codebase. defn-spec copies the syntax (and implementation) of Schema’s defn macro. This has the advantage of adopting a proven design, familiarity for many Clojurists, and the ability to work with existing tooling that understands the Schema defn macro.

Benefits and tradeoffs

Like all things in life, defn-spec has benefits and tradeoffs:

Benefits

• Makes it easy to incrementally spec your functions. It lowers the activation energy needed to add a spec, perhaps just for a single arg or return value.
• Makes it easier to see more code on one screen.
• Makes it harder for your specs to get out of sync with the function definition, as they are linked together. Depending on how you instrument your specs, it may be possible to go quite some time before you realise that your specs are broken.
• Avoids repeating all argument names in the s/cat form.

Tradeoffs

• For some specs, particularly complex :args specs with many branches, it may be simpler to define the fdef separately. defn-spec is designed for the 80-90% of Clojure functions that have simple argument lists and return types.
• Making it easier to change specs means that it is easier to accidentally break your callers. If your fdef is defined separately, it forces you to think more about growing the spec.

This is similar to Orchestra’s defn-spec macro, but allows for optionally only speccing part of the function, and matches the well-known Schema defn syntax. Try them both out though, and see which one works best for you. defn-spec is still missing some features from the defn macro like destructuring, but they are on the roadmap to be added soon. I’m releasing this early to get feedback from other users.

defn-spec will follow Spec’s release cycle, and there will be a new set of namespaces and artifacts for spec-alpha2 and beyond. If you have features/bug reports, feel free to post them on GitHub.

Some comments have been lightly edited for spelling, clarity, and brevity.

Update: Alex Miller has posted a few responses to some of the comments I highlighted below, and some suggestions for the next steps that people can take to help.

Error messages

Error messages have been a top complaint in Clojure for a long time, I think since the very first survey. CLJ-2373 introduced improvements to error messages in Clojure 1.10. A number of comments complained about error messages this year, but none of the complaints mentioned 1.10’s changes. Given that 1.10 was released on 17 December 2018, and the survey ran from 7-22 January 2019, it seems likely to me that many of the people complaining haven’t tried the latest improvement to Clojure’s error messages.

• I appreciate all the work that’s been done in improving error messages. Thanks to Alex Miller and others!
• Thanks for all of your hard work on Spec and the error messages in general (often caused by minor typos).
• 1.10 alleviates most of the problems I found with error messages. Since this was my single biggest concern I had with Clojure, I just want to say thank you for listening!
• Just give me the Clojure filename and line where the exception happened, please. I beg of you.
• Better error messages. Please!
• Great to see the improved error messages in Clojure 1.10 - it was my number one issue in all previous Clojure surveys, but not this one :-)

Spec

Spec has been around for two and half years, but is still in alpha. A number of comments referenced Rich’s recent Maybe Not talk about coming changes to Spec. It was great to see Rich putting in to words some of the problems I had felt. It feels like the community is in a transitional phase with spec, where people are still not quite sure where things are going to land.

• Clojure spec tooling would be nice! Because it’s hard to learn and visualize–and it’s not obvious how to integrate it with normal deftest tests. It has a massive amount of IDE integration potential though.
• I’d really love spec to feel a little more complete. … I’ve got a lot of love for spec as a project and use it heavily at work, but I feel it’s shadowed at the moment by some ambiguity about what changes might be on the way.
• Spec has great potential, but really bad UX. If you are new to Clojure it’s very hard to figure out what’s wrong. Spec made Clojure errors worse.
• Spec should have better tool integration and more static analysis facilities.
• Really excited to see the direction spec is heading, can’t wait to see the next iteration.
• Thank you again, for all that you have given to us. Looking forward to the improvements in spec from “Maybe Not?”! I am certainly feeling the pain points they address, and currently work around them by generating heaps of variations on essentially the same spec per fdef.
• I couldn’t put my finger on it, but it always felt a bit awkward to use specs and had to juggle the spec and what the spec meant at a particular point in time during the flow of my software. Thanks Mr. Hickey for the presentation during Conj, it finally makes sense! I’m looking forward to the next iteration of spec.

Docs

Documentation has continued to be a sore spot for beginners. API docstrings were often cited as being too hard to understand, instead people preferred to see examples at clojuredocs.org. The documentation at clojure.org has grown this year with a number of great guides added, particularly Programming at the REPL.

• Concerning docs, I’m seldom able to really make use of the Clojure API docs. Since examples help me more to understand what a function does, I rely more often upon the examples provided by the community at clojuredocs. However, the quality of the examples vary and it would be helpful if there was a set of curated examples.
• Clojure is great, but the main website, docs, examples and tutorials are not good. I would invest time in improving it as it has become a barrier for the language.
• I wish we had a better documentation generator and a better culture of documentation. I really miss the documentation in the Haskell ecosystem (and five years ago, I never thought I would be saying that!). For example, the Haskell community generally expects detailed documentation of data structures, including detailed asymptotics, e.g. Data.Sequence. This complaint extends to many of Clojure’s included features, too. Headline features like STM have incredibly sparse documentation (about one printed page with the semantics only hinted at), so I’m left guessing at what I can actually do with these things.
• The beginner support is exceptionally good. I think I would appreciate more intermediate to advanced support, in the form of books, courseware etc. to carry through to advanced Clojure.
• I love Clojure. The docs are phenomenal. The improvements to the ClojureScript site and getting started and the CLI tools have been huge.
• As a beginner I really struggle to understand the api docs. I tend to use third-party websites more often.
• It would be great to know things like when to use a record vs a map. Intermediate guides would be nice. Also a guide on effective idiomatic Clojure like golang provides.
• Please work on improving the ease of use. The doc strings for the core language are third rate, poorly written, cryptic and unhelpful. The error messages (although presented more nicely in 1.10) are still the same terrible error messages as they have been for years.
• I’d ask for more examples in documentation and provide more tutorials like Scala School or Elixir School for Clojure, for newbies like me.
• Beginner experience still needs more work. Please consult with first-time programmers and teachers who teach them.

Startup time

Startup time has been a perennial complaint, and there haven’t been any major changes here.

• This long-standing issue of startup time + poor runtime performance (out of the box, without tinkering) + insanely high memory usage is becoming the major drawback of the language for me (and us). We are planning to move to a faster language (probably Rust or Go).
• I would like to use Clojure (not CLJS) in AWS Lambdas for a side project but the painfully slow startup time prevents me from doing so. Improving startup times will do more than just improve developer happiness, it will make Clojure a much better tool for serverless computing.

Marketing/adoption/staffing

Clojure continues to grow in business adoption, but hiring developers has been one of the top three things preventing people from using Clojure for the last three years. I’ll add that one way that companies could address the mismatched supply and demand of Clojure developers is being more open to remote workers. Update: Alex Miller also suggested training people in Clojure, which I thought about when preparing the post, but forgot to include in the final copy.

A new option in the survey this year was “Convincing coworkers/companies/clients”; it was ranked number one in preventing people from adopting Clojure. In my opinion, this is a significant issue to be addressed if Clojure wants to grow its market share further. This probably needs more research to find out what the barriers are to convincing people. Elm is proof that exotic languages can be adopted widely, we should learn from what they’ve done.

• We’d all benefit from wider Clojure take up. I worry about the view from the outside looking in. Why is it not attractive to more?
• My biggest problem after one year of intense Clojure learning/coding is to find a Clojure job as remote (because there are zero Clojure jobs in Taiwan). Regardless of how good I am at writing functional code, finding a remote job (outside of the US) is hard.
• Very passionate evangelist 5+ years – It’s surprising how hard it is to have some folks even consider Clojure. Rubyists are particularly baffled: “ok what do you mean REPL driven – we have pry!”, etc.
• I co-founded a startup company and we picked Clojure as our primary language and are very happy with the decision. A nice note is that almost none of our hires knew Clojure before, they all learned it quickly and like it! I don’t see a lack of Clojure talent as a concern at all.
• Clojure is thoughtfully designed and stable. It’s a dynamic, functional lisp that can actually be sold to the bosses (A sentence I never thought I would write).
• Clojure has a great community and great technology but it would be super helpful in selling the language if there were more efforts put into marketing the language and perhaps having “blessed” solutions using Clojure from Cognitect; similar to what Lightbend does with Scala.
• Thank you for all the hard work you all have been and continue to put into Clojure. We’ve built our entire new startup on Clojure, and was able to hire an amazing team within just a year by tapping into the local, tight-knit community.

Language

There were again more suggestions for improvements to existing language features, or development of new ones.

• Please, take a look once again at Dunaj Project or similar ideas. I believe that the community still have many great ideas which could be adopted by Clojure and wouldn’t bring up any breaking changes. And we need better IDE for sure.
• Please spend time improving clojure.test.
• When I dream, I dream of seeing a slime-style backtrace in Clojure.
• ClojureScript namespace definition <-> :require <-> macro & reader conditionals - and how these interact with each other is seriously non-trivial; especially so for macros you intend to use in Clojure too. Simplification there would be welcome, if possible.

Language Development Process

2018 had a lot of discussions about Clojure’s development process, and the relationship between the core team and the community. I was very curious to see how widely those feelings were reflected in the free-form comments from respondents. After compliments (~230), this was the most common comment type, with ~70 responses (positive, negative, and in-between) on Clojure’s language development process (out of 563 total comments). While many people had issues with the development process, there were also many people who were supportive of how Clojure is currently developed.

• Grateful for the careful stewardship of Clojure and ClojureScript by Rich Hickey and Cognitect.
• Clojure has strong opinions on how software should be built. It needs to keep those strong opinions as it’s these opinions that differentiate it and make it attractive. Keep up the great work!
• Clojure is Amazing and you guys do a FANTASTIC job!!! Ignore the critics and keep doing things just like you have been. Thank you for this wonderful language!!
• Closed core development and disappointment with spec have made made many friends leave Clojure/Script last year. Let’s make 2019 better.
• I love Clojure, and an extremely grateful to Rich, Stu, Alex, etc for their continuing stewardship and work of this fantastic gift. With respect to recent kerfuffles about community involvement, I simultaneously understand and respect Rich’s position, and I am also concerned that some very respected Clojure developers who have contributed a lot (IMHO) feel disconnected and may be drifting away from the community.
• About the governance of Clojure by Cognitect: thank you all! You’re doing it right.
• Managing expectations in 2018 is about 10 years too late. Please give a signal that the core team recognises that, and also how much the community is worth to Clojure as a language before dismissing any critics to the (previously very unclear) contribution/lifecycle process.
• I love the principled stand taken by the clojure/core team about silly arguments erupting in the “community”. I really appreciate that the core is so well thought out and there is a valid resistance to adding “stuff” in there.
• I appreciate Alex’s insights into the Clojure development process, and what he’s working on. I also appreciate the all of the core team’s work that makes Clojure such a great language to use!
• Thanks for recent improvements inline with community wishes. Still makes me sad we don’t have a bigger core team utilizing the skills of (some) of the community that could do things like e.g. actively improve core.async.
• Clojure is a wonderful language; however, we are becoming worried about the future prospects of the language because of the way that Cognitect handles community feedback about the direction of the language.
• /2018 was the year that Rich finally explained how he felt about the open source community around Clojure. For many this was disappointing, but I’m glad that he finally explained how he felt, because at least it is now in the open and people can make decisions with full information. Five years ago, Clojure could afford to lose key contributors because the language was growing and new people were coming along. Clojure is now at a very different place and position, and it’s no longer growing at the same rate, or has the same mindshare. I’ve invested many years, and lots of time into the Clojure community. Up until this year, I thought I would have a long career with Clojure, but I am starting to look around at other language communities for ones that are a better fit for my values.
• Even though I agreed with the substance of Rich Hickey’s “Clarity” gist, it is concerning to me that some of the more prolific library maintainers are leaving the community. I’m sure that had it been me, I would have responded even less graciously. Still, there’s an element of community management that goes beyond just being correct.
• After using Clojure professionally for 7 years, I am hedging my bets by involving myself in the Rust community. While Rich’s comments may be lauded by the greater online community, it has had a large impact on my personal consideration on the state of Clojure. … But, as a member of the community, my concern is that the user base of Clojure cannot grow larger without this concept of ownership changing. This conflict, at the end of the day, is a conflict between what my hopes for Clojure were and the interests of Cognitect … But it seems that Clojure has remained what Rich originally intended it to be. … The conflict is simply that, for some reason, I had assumed my interests were in line with Cognitect’s. It has become clearer, year after year and especially this year, that this is not the case. It’s very likely this misunderstanding is a mistake on my part. Looking back, I see no reason that I should have assumed this other than it is generally the trajectory of most languages focus on growth. For this reason, I will likely exit this community as many others have. I don’t have any hard feelings or regret being part of it. I am very appreciative to all of the hard work that has gone into Clojure and regretful that I did not contribute nearly enough during my time writing it.

Community

People seem to mostly enjoy the Clojure community, but others have had negative experiences, often citing an elitist attitude from some community members. Losing the history from Clojurians Slack came up several times. I’m not sure if everyone is aware of the Clojurians Slack Log?

• … the community has been aggressive/hostile from our experience.
• Love the community. Everyone has been really helpful. I wish it was more diverse and wrote more stuff down. A lot seems to get shared face to face which is great, but makes a really high barrier to entry.
• I’m a bit worried about the long-term viability [of Clojure]. I wouldn’t be so worried about the language and ecosystem not being friendly to newcomers, it’s clearly distilling many years of software practitioner’s experience and thus is opinionated.
• The only bad/ugly part of the Clojure ecosystem is the use of (unsearchable)Slack as the main knowledge base.
• I really enjoy this community. Despite the occasional war of words, generally it seems to be filled with helpful, deep thinking people who are fun to interact with. If we were known for one thing only, I’d like it to be that!
• Thanks so much to the Cognitect team and everyone who helps to make the Clojure community better. I understand that people can be very vocal about the things they dislike and not so much when it comes to appreciate the things that work nicely. Being able to use Clojure professionally to build software that solves problems is really exciting and I’m grateful for it.
• I suggest moving off of slack to a more accessible chat system. Losing history is a bad thing. Check out discord or matrix or gitter or mattermark or any other number of tools made for this purpose.
• I’m concerned that there is too much of a disconnect between the core developers and the wider community. … I think it would be great if the core acknowledged some of the work of Clojurists Together. I think the new clj-commons organisation is a great idea and I’d like to see it better promoted and supported.
• The tone in the community has changed. I think this is very sad.
• The community and the prominent people around it are OUTSTANDING. It is really something to behold. This was my first motivation. I am staying for the good language and tools and etc. but the people behind it are definitely the main treasure here.

Libraries

People are still looking for a curated on-ramp into Clojure web development, a ‘Rails for Clojure’. There are a number of frameworks and templates here, but they don’t seem to be hitting the spot. I’m not sure whether that is because of limited mindshare/marketing, limited documentation, or the scope and quality of the frameworks. Having used Clojure for many years, I no longer find it difficult to find and assemble the pieces that I need for projects, but I definitely remember finding this being difficult when I started. Data Science and ML were places that people saw a niche for Clojure. Several people hoped for improvements on core.async, it still has a number of rough edges.

• I would love to see a way to get open source tools, such as Cider, sustainable support. I’m worried about all you open source maintainers getting burned out and not creating these awesome tools. I don’t know what this looks like yet. [Ed. I’ve got just the program for you]
• I know that Clojure promotes a build-it-yourself type of mentality, which is nice, but it would be great if it had standard implementations for things like webapps (auth, security, etc.) that were leading the way for alternatives to follow. It’s ok to be opinionated, if it works properly.
• One huge opportunity that is still being missed is the field of data science – Clojure(+Clojurescript) has the potential to become the best platform for data research ever. In that field, more than others, it is important to be beginner-friendly.
• Clojure needs a Rails-like “killer app framework” that provides novices a training-wheels included opinionated approach to building applications with Clojure. The roll your own / pick your poison approach to Clojure is a barrier to many.
• core.async could use some love with closing long-open bugs and documentation on best-practices and how to build bigger systems with it. Or maybe it just needs to be officially deprecated? As an outside observer, it feels like it’s in limbo by the clojure-core team. Either way, we’re currently using Zach Tellman’s manifold abstractions a lot to great effect.
• I much prefer the library over framework approach taken in the Clojure community, but I do miss the rapid development which RubyOnRails allows, i.e. conventions.
• … I think a curated collection of recommended third party libraries would help a lot of teams.

Other compilation targets

People have been experimenting with the Graal compiler to produce static binaries without needing a JVM. At this point that seems like the strongest option for people wanting to use Clojure without the JVM. Better Android support was also requested.

• Clojure doesn’t seem to have any plan for Android integration, this is pretty sad considering how much computing power is employed there, and how big of a showcase mobile development is :(
• I think that the limitations of their respective host languages are holding Clojure and ClojureScript back.
• Is there a Clojure project that compiles to WebAssembly?
• So many people want a native Clojure. We need a native Clojure without JVM. Is there a plan about this?
• The advent of GraalVM, and the resulting ability to build Clojure executables that startup up quickly has been a game changer. The slow startup time and need to use a JVM has been a real impediment to Clojure’s adoption as a general purpose language. It would be even better if someone would pursue a Clojure native x86_64 compiler which would generate native executables, but that is probably not going to happen. Ensuring that Clojure can run well with GraalVM is probably the most we can hope for, and there is plenty of work to be done there.
• Difficulties in running on Android may end up forcing me to port all the existing desktop applications off of Clojure to pure Java/Kotlin solutions.
• Targeting two of the most popular VMs is great, but being able to get native performance is always a wish, like LLVM.

Typing

For the past few years people have been less and less worried about types. I suspect this is mostly due to spec.

• Love Clojure. But at two Java shops over four years, the lack of static typing (the ability to command-click identifiers in IntelliJ and navigate to their source, the ability to refactor) has prevented me from winning over colleagues. TBH it’s also reduced my own enthusiasm since the prospect of me working on our 1.6MM LoC project without static typing honestly scares me.
• I’ve selected “no static typing” but matching Haskell is not my intent. Lack of static guarantees is what bothers me. If it were possible to reject programs that are guaranteed to fail, that would be an improvement. For example: (clojure.string/replace nil #"a" "b") or (clojure.string/replace "abc" #"a").

ClojureScript

Shadow CLJS was mentioned by many people as having been a great part of their workflow. For those not using Shadow, externs remain challenging.

• shadow-cljs made me continue developing ClojureScript + started using it in backend services. It provides significantly better support for NPM libs than lein-cljsbuild that I’ve been using before.
• Without shadow-cljs we wouldn’t have been able to use clojurescript because we found it impossible to get the npm packages we needed to work.
• It’s not at a terrible state right now, but I’m at least slightly worried about more divergence between Clojure and ClojureScript in the future.
• shadow-cljs does great job on compiling clojurescript and npm integration, but it more and more diverges from the standard compiler. I would love to see shadow-cljs merged into the mainstream clojurescript compiler. In general: I would like to have better, easier way to compile clojurescript without setting tons of flags.
• Feel like cljs could make an amazing, even-better version of dash (maybe powered by Reagent/re-frame).
• Although work has been done on externs, I find venturing into JavaScript interop fraught with peril. I would be very happy if interop would just work, same as Java interop. Most frustrating is that development builds work but releases don’t. Catching these problems early would seriously mitigate the issue. For now I’m using cljs-oops library, which works very well, but it feels kind of a hack. Overall I’m very content with ClojureScript though!

Setup/tooling

Clojure’s tooling has continued to improve. This year lots of work has been done adding plugins or expanding functionality of the new official Clojure tools. clj on Windows was asked for by many people. Maintainer burnout seems like a significant risk to the Clojure tooling ecosystem.

• I’m happy to see that many have realised we need to find ways to better support the development of CIDER, a crucial part of Clojure development infrastructure. Seems like the lead dev has been burning out recently, and many of us depend on his (and a few others’) goodwill.
• It has been a challenging first year in obtaining high productivity with the tools. Would be great to see a simple but capable tool set get some focus, e.g. Atom or VSCode.
• I no longer want to work with CIDER or Cursive, rather, I’d like to work with REBL + a REPL as my IDE. I believe we can have a renaissance of Clojure tooling if prepl becomes demystified and/or we add a tap-repl, something like tap> to the Clojure repl. This would allow tooling to be added and removed dynamically.
• Please port clj to Windows!
• clj for windows please.
• We need clj under Windows. Is it possible to release it?

Compliments

As always, the vast number of comments were compliments to Clojure and the core team.

• Keep up the good work! 👍
• Clojure is beautiful, functional and concise. It really rejuvenated my love for programming.
• I absolutely love Clojure 💚
• I really appreciate the well thought out design of Clojure. We feel that Clojure gives us distinct advantages while providing a language with minimum disruption.
• 5 years later and I still enjoy writing Clojure. Thanks to all those who make it extremely productive and fun to use!
• I really appreciate all the work Cognitect and the community abroad put into making Clojure and the Clojure community a joy to work with. Keep up the good work!
• It’s my 11th year with Clojure. What a journey. Thanks, Rich and everybody.

Conclusion

Clojure is a great language, and people are very enthusiastic about it. Its adoption in business continues to grow. There are a number of areas for improvement still, especially if the Clojure community wants to grow further. Not all of this work needs to go through Core though, particularly in areas of documentation and guides, libraries, and tooling.

I worry about the Clojure community losing key contributors. There are a few linchpins holding a lot of things together; if we lose them, it will be hard to come back from. If you don’t want to see this happen then please support the libraries and tools you rely on by contributing code, documentation, issue triage, or money.

Introduction

In principle, I’m not opposed to the idea of Reagent using Preact. It has some cool features and I like that it is small (although in comparison to the total compiled size of a normal CLJS app it’s probably a wash). If Preact worked 100% out of the box with Reagent with no code changes required then I would have no issues with someone swapping out the React dependency for a Preact one and calling it a day. If there are only a few minor tweaks to Reagent required to pass the tests, then again I don’t really have any issues with that. I suspect that even if you have 100% of the tests passing, there will still be issues, as Reagent was built around React, and may not have tests that would cover the difference in behaviour between React and Preact.

Abstraction

It looks like it may not be a pure lift and shift to support Preact. If that’s the case then we run into a bigger issue: abstraction. Reagent was written and built around the ideas and in the context of React. There are assumptions (probably tens or hundreds) built around React’s API and possibly implementation details too.

Adding abstraction adds a large cost because you can no longer program against a concrete API and implementation, you now have to consider two. There are three numbers in computer science, 0, 1, and many. We would be moving from 1 to many, and that takes work.

An aside: recently at work, we were looking at moving a legacy system from only supporting dates in the past to also be able to support dates in the future. This should be straightforward right? We talked to the programmers responsible for it and they couldn’t guarantee that it would work, nor whether supporting future dates would be easy or hard. In the building of that (or any) system, hundreds of simplifying assumptions are made around the context that the system is going to be built in.

It is a very common pattern to have different backends and I don’t see any downsides to it.

I can’t think of a single example of a system with multiple backends that didn’t have any downsides to it, e.g. ORMs, HTML/CSS/JS, Java. There may be some, but they would be the exceptions that prove the rule. Everything has a cost, the question is whether there is a benefit that outweighs the cost. It is much harder to remove something from software than to add it, which is why we should be certain that the benefits outweigh the costs.

While Preact strives to be API-compatible with React, portions of the interface are intentionally not included. The most noteworthy of these is createClass() … https://preactjs.com/guide/switching-to-preact#3-update-any-legacy-code

Reagent currently uses createClass. There are workaround options provided, but this is an example of some of the API differences between React and Preact which you need extra compatibility layers to support. Do we know if the compatibility layer works 100% correctly?

A possible future if Preact support is merged now

As a thought experiment, let’s assume that Preact is in Reagent with some kind of compatibility shim. Preact already has several performance optimisations that people can take advantage of:

customizable update batching, optional async rendering, DOM recycling and optimized event handling via Linked State. - (from Preact homepage)

Wouldn’t you want to be able to take advantage of those in your application? I certainly would. Now to do so, you may run into issues because the compatibility shim layer that was written was encoded around default assumptions of React, and they may not apply to Preact. Do we have to rework the shim layer, or lower level Reagent API stuff? Who is going to do that work? Who is going to review it and merge it?

Let’s consider the reverse. Perhaps in some new React version, Facebook comes out with a new API which is faster or better suited to Reagent’s style of rendering, so we want to switch to that [since writing this they came out with Hooks. These may be added to Preact also, but this isn’t certain and neither is the time-frame]. However that new model may not work with Preact. Again, we’re in a bit of a pickle: Preact users want to be carried along with Reagent and get the benefits of new Reagent work, but it may not be easy or possible to support the new API for them. Now what?

Consider everyday development on Reagent. Reagent’s source code is built around a very detailed understanding of React and is highly optimised. If Preact was supported too, then developers would probably need to gain an understanding of Preact too.

At the moment, Preact has one main contributor, it has been around for 1.5 years. React has many contributors. I’d estimate there are 100+ people with very deep knowledge of React. It’s been around (in public form) for 3.5 years. In general, the JavaScript community does not have a reputation for long-term support of projects. What happens if development slows/stops on Preact and the compatibility layer isn’t kept up to date? It is much harder to remove something than it is to add it. Who decides when/if to remove Preact from Reagent at a future date?

These are all hypotheticals, but I hope this demonstrates that the extra abstraction provided by supporting two VDOM layers doesn’t come for free. At the very least, it consumes extra brain cycles when testing and developing Reagent, extra support and documentation costs from users wanting to use one or the other, as well as extra indirection when running and debugging apps using Reagent.

The Innovators Dilemma

If you haven’t already, I highly recommend reading “The Innovators Dilemma” by Clayton Christensen. One of the key points he makes in that book is the difference between integrated and modular products and when to develop each kind.

CHRISTENSEN: When the functionality of a product or service overshoots what customers can use, it changes the way companies have to compete. When the product isn’t yet good enough, the way you compete is by making better products. In order to make better products, the architecture of the product has to be interdependent and proprietary in character.

In the early years of the mainframe computer, for example, you could not have existed as an independent contract manufacturer of mainframe computers, because the way they were made depended upon the art that was employed in the design. The way you designed them depended upon the art that you would employ in manufacturing. There were no rules of design for manufacturing.

Similarly, you could not have existed as an independent maker of logic circuitry or operating systems or core memory because the design of those subsystems was interdependent. The reason for the interdependence was that the product wasn’t good enough. In every product generation, the engineers were compelled by competition to fit the pieces of the system together in a more efficient way to wring the maximum performance possible out of the technology that was available at the time. This meant that you had to do everything in order to do anything. When the way you compete is to make better products, there is a big competitive advantage to being integrated. … In order to compete in that way, to be fast and flexible and responsive, the architecture of the product has to evolve toward modularity. Then, because the functionality is more than good enough, you can afford to have standard interfaces; you can trade off performance to get the advantages of speed and flexibility. These standard interfaces then enable independent providers of pieces of the system to thrive, and the industry comes to be dominated by a population of specialized firms rather than integrated companies.

• Clayton M. Christensen, The Thought Leader Interview

I would argue that we are still very much at the point where the current VDOM libraries aren’t good enough yet. They aren’t yet ready to be commoditised, and the best option is to tightly integrate.

Options from here (with some conjecture)

1. Someone can make a PR to Reagent to add support for Preact. It will probably take a while to get merged because it is a significant change. Once it is merged and released, there will probably need to be several rounds of revisions before it is ready to go. Because Reagent moves relatively slowly, this will take a while.

   Reagent also has a large number of production users, so new releases need to be well tested and stable. Adding Preact to the mix is going to slow this down further.

2. Someone can make a fork of Reagent (let’s say it’s called Preagent). You can run wild experimenting with what is the best way to use Preact in Preagent, take advantage of all of the great features Preact has, and have a much faster turnaround time for releasing and using it. You will be able to work out what is the right API and integration points for Preact because you have room to experiment with it, without the weight and responsibility of bringing the rest of the Reagent users along with you.

   At some point in the future, you could review merging Preagent back into Reagent, given all that you now know. You would also have the weight of evidence on your side where you can demonstrate the benefits of Preact and can show how many users want Preact. This would let you make a much better case for including Preact, give you what you want in the meantime, and likely provide a higher quality integration in the future.

   Alternatively, you may decide that Preagent is better served going its own way and integrating more closely with Preact. This is also a good option.

Abstraction is not free

The point I have been trying to drive through this post is that abstraction is not free. Over-abstraction is a common anti-pattern and it saps productivity. I had a friend who recently left a Clojure job and started a Java one. He quipped to me about how he’d forgotten what it was like to trace code through five layers of abstraction to get to the concrete implementation. As programmers, we’re trained to solve problems by adding layers of abstraction, but that isn’t always the best way to solve the problem.

Lastly, this isn’t a personal attack on you or your ideas. I’m all for innovation in ClojureScript webapps and I think that it is worth investigating Preact and how it could work in the ClojureScript ecosystem 😄. I’m not against Preact. I would consider using it if there was a measurable benefit. I’m just suggesting that the best way to go about this is probably not to integrate it into Reagent as the first step.

I first had the idea of producing a Clojure podcast around January 2018. I didn’t have the time for it then, but the idea kept swirling around the back of my brain. When Michael Drogalis’s Pyrostore was acquired, I wanted to hear more about the acquisition, so I contacted Michael to see if he wanted to talk about it. That became the first episode.

My goal for The REPL is to talk with people about the technical details of the projects they are working on. The Clojure community has a ton of interesting people doing really creative work with Clojure. I’ve really enjoyed talking with a bunch of people already, and I’m looking forward to talking to more of them in the future. You can find it therepl.net. It’s available on Apple Podcasts, RSS and all of the other common podcasting apps.

Some comments have been lightly edited for spelling, clarity, and brevity.

Error messages

Error messages have been one of the top complaints about Clojure since the surveys started, and this year they have gotten worse with the introduction of specs on core macros. I’m not the only one who has noticed this either. In 2015 Colin Fleming gave a talk on improving Clojure’s error messages with Grammars. In previous surveys there was hope that spec would be able to use this approach to improve error messages; Ben Brinckerhoff has recently shown a proof-of-concept that gives some similar (excellent) error messages in Expound.

• My primary wish would be for more helpful error messages (e.g a la Elm) in Clojure and Clojurescript.
• I was excited about spec initially; however, I find the error messages so large, noisy and unhelpful that I don’t want to use it. To put it another way: when there’s a spec failure, my heart sinks; knowing that the next few minutes are probably going to be frustrating and painful.
• Spec was supposed to help with error messages, but it just made them worse. Stuff like (let [f/x 42] f/x) now emit insane error messages. But it looks like the Clojure dev team has moved on to improving command line tools now? Can we please just get error messages fixed before a ton of people get frustrated and jump ship? The current state of errors in Clojure is an embarrassment to the community.
• Error messages is a big show stopper. I am trying to get Clojurescript adopted for web and react-native development withing our organization for few years, but I always see the same pattern: people get some error during project setup/configuration and get stuck forever because they have no idea what the error means at all and how get it fixed. Please please improve error reporting. :)
• A big problem for new starters, an issue that has been singled out time and time again over the years, is poor error messages that lead to difficult debugging of code structure. That Clojure 1.9 has actually made these error messages harder to understand is a disaster. […] I hope I’m wrong and that I’ll understand the grand plan eventually.
• Please please please please enhance the error messages. An incredible error reporting would make this language out of this world, and that would make the biggest difference.

Spec

Comments ranged from loving spec to disliking it. There were also requests for more guidance on how to apply spec to real-world use-cases and how to integrate it into whole programs.

• OMG Spec is amazing. It’s dramatically changing the way I program. I’ve been starting new projects out by writing specs before nearly any other code. The result is that I am able to extend my hammock time to the keyboard, and think through my domain model with composable, high level descriptions implemented as specs. The fact that these high level descriptions then become so useful in so many different ways – generative testing, value conformation for data api parsing, more useful error messages, etc – is a stunning thing to behold. I’m excited to see how the fruits of these ideas permeate through our and other communities.
• The presence of “spec” on some of those lists made me want to rate everything else “important” so I could show how unimportant it is. […]
• spec is awesome, but its dogmatism leaves us needing tools like spec-tools, which makes working with spec clunky.
• There needs to be more guidance, development and reliance on spec from both Cognitect and community.

Docs

Documentation in all forms has been another perrenial complaint but this is starting to improve. clojure.org continues to add more guides this year on Programming at the REPL and Higher Order Functions among other things. Martin Klepsch’s cljdoc holds a lot of promise to improve the state of third-party documentation, by automatically generating and publishing docs for published JARs.

• We should take example of how Rust community/documenation is run: BIG focus on newcomers, features MUST be perfectly documented before being shipped. This serve advanced users as well. No offense, but Rich Hickey should understand what good documentation is, or delegate completely to people who do so that he can focus and making a even greater language (his strong point).
• I feel the true weak points are confusing error messages, not enough friendly guides/tutorials for the early to beginner Clojurist, and a lack of solid CLJ/S example projects. These are key because it is difficult to know the right way without “blessed” examples.
• Please provide a good set of tutorials on Javascript Interop such as how to translate a ES6 or higher JS file into ClojureScript.
• The documentation is really awful - incomplete, vague, no examples, no semantic precision - not up to the standards of what one normally sees for programming language documentation.
• Please improve the docstrings. They are the worst of any programming language I have ever used.
• Really, the core documentation is not written for beginners or even intermediates
• What I miss most is a book which teaches how to approach problems with Clojure. I mean primarily not “best practices” or “design patterns” or language independent algorithms. Nor a kind of language reference or few lines long mini examples, but complex, advanced examples with detailed explanations about the pros and cons of certain solutions applied/not applied.

Startup time

Startup time continues to be an issue for people. As serverless computing becomes more mainstream, this may end up pushing people from Clojure to ClojureScript, or elsewhere.

• Startup time is really really really really really long
• Startup time never used to be an issue for me, as my services were long running and persistent. Lately however, I’ve been using Docker for jobs. Startup time becomes an issue here. Continued improvements in total app startup time will pay dividends going forward.
• Startup time regression. Clojure 1.9 startup time is 1/3 longer then Clojure 1.8 (1.6s vs 1.1s on my humble machine).

Marketing/adoption/staffing

There were lots of comments this year in this category. Even after 10 years, Clojure still unknown or seen as being extremely niche to many people, especially among business stakeholders. Hiring seems to be less of an issue than in years past. There is still a large contingent of programmers looking to work with Clojure in remote positions, although it’s hard to say whether this proportion is any higher than in other language communities.

• I recently hired a programmer with no Clojure experience. She was writing lovely idiomatic Clojure within a month. The biggest difficulty was overcoming her initial fear of parentheses.
• To use Clojure(Script) more it would help if there was marketing for it. It’s still largely unknown, and therefore hard to sell to other companies.
• There is always an uphill battle in pushing Clojure as an enterprise solution, this seems to mostly occur to lack of marketing. Non-technical decision makers care about branding.
• I’ve found that some folks really dislike lisps passionately. One of our vendors indicated that his customers and employees were highly critical of the use of Clojure as an extension point.
• Unfortunately, this year my organization in ceasing its use of Clojure for production projects after many years of using it. Staffing and on-boarding external resources are the 2 main reasons for doing so.
• Hard to convince managers at work to use Clojure because of the perception that its a fringe language
• I wish it was easier to find a remote (non US-only) remote Clojure/Clojurescript job for people with no major Clojure experience :)
• It’s very important to address the enterprise software community. We need formal tools, formal approaches, standard IDEs, SonarQube, Eclipse, Jenkins, e.t.c.
• I believe “JavaScript converts” will be an important source of growth for Clojure moving forward.
• We find Clojure a joy to use and hiring Clojure engineers or those excited and capable of learning Clojure has been a big advantage, contrary to common wisdom. Clojure positions tend to self select for strong candidates.

Language

• Enterprise Java tends to lean heavily on reflection to pull components together (e.g., Spring, annotation-driven JUnit and JAX-RS). To incorporate Clojure into those environments, lots of component-specific wrappers are needed. It is easy to slip a little Java into a Clojure codebase, but this has made the reverse difficult.
• I would love to see some more work on improving clojure.test. It works, but I wish there was less stateful macro magic and more flexibility with fixturing and test structure
• I’m increasingly finding that I’m writing more cljc than clj or cljs but find it really frustrating that cljs seems to be a second citizen to clj. It would be much better if there was actually a core Clojure library/api that was actively supported by both, with clear variances between them.
• I would love to see some work towards using JVM MethodHandle’s to obviate the need for the compiler’s direct linking. Seems we could have our var cake, and eat it too.
• We have a relatively large codebase and 20+ build targets (fat jars/advance compilation js files). we’d like better AOT support and ability to incrementally compile this code base as most changes only affect a few targets. right now AOT compile is pretty black box and not very amenable to building tooling.

Language Development Process

This hasn’t changed in a long time, and I don’t see it changing in the future, but it has been persistently highlighted by many people as a big issue.

• For long term success of Clojure/ClojureScript, it’s important for the Clojure core team & project to be more open and welcoming like other open source projects. The view from outside of the Cognitect bubble is that the language and community aren’t healthy at all, and I fear for it’s long run viability.
• The core team seems to be extra cautious and move really slowly on the language and core libraries. And the roadmap for releasing new versions is really opaque to the community.
• Cognitect has continued to be good technical stewards of Clojure, but poor stewards of the relationship between the core team and the rest of the community. Many (most?) senior Clojure programmers want nothing to do with contributing to Clojure or any projects owned by Core (like the site) because the process is designed to frustrate and prevent meaningful contributions.
• Cognitect need to climb a few rungs down the ivory tower and engage with the community as if it were in fact made up of human beings trying to do their jobs.
• I don’t know what direction the core team is taking Clojure, and the top issues (startup time, poor error messages, lack of involvement with the community and numerous improvements to Java interop in JIRA being ignored) have not changed significantly over the past 5 years. I never felt welcomed to contribute, and seeing how Skummet and other large efforts from the community were largely ignored was not encouraging.
• More attention to JIRA issues/patches/contributions would be great. […] I painstakingly write a patch and jump through the JIRA hoops; but there it languishes. The dev team’s complete silence is not encouraging. Makes me feel more of an outsider, less of a part of the community.

Community

By and large, people found the community welcoming, and personally I have found it to be one of the best things about working with Clojure. However there is a persistent undercurrent of eliteness from some Clojure programmers which really puts people off the language.

• I find Clojure a great language, but I haven’t noticed any increase in market (or mind) share. Quite the opposite, it seems to be going nowhere, possibly dying a slow death I hope I’m wrong, but, for now anyway, I’ve given up on it.
• Thanks, Alex et al! I’m still extremely happy with Clojure, and appreciate the community. I worry a bit about the possibility of clj/s losing user share and dwindling, which I didn’t worry about a year ago. I hope like hell that that won’t happen, and try to do at least a bit toward preventing it.
• That defn podcast is bloody marvelous
• Being in the Clojure community since around 1.2, I loved the language then, and I like the direction, it’s been taking since. The community is great, very friendly and even its most controversial figures are usually polite, intelligent people. Heck, even the local troll from the mailing list is occasionally helpful, stays on point in his threads and sticks with the conversation, even after blowing stuff up.
• clojure.org changes so infrequently (like once a year, if that?) that I’m concerned the language is not being enthusiastically supported and will soon fade away. It makes me nervous to invest time and money in a language that seems to have no vitality.
• Clojure conferences lack diversity in both attendees, speakers, and programs. I hope the merging of Conj and West will improve that this year.
• People who give to the Clojure community are role models. I’m serious about this. I’ve woken up and thought “what would Bruce Hauman do” more than once. I talk with my daughters about the Clojure community.
• Cognitect is doing an awesome job, making programming fun again! Alex Miller is very helpful.
• Effort community makes to promote diversity, particularly in terms of women is good, although there is a long way to go.
• I am doing Clojure for some time now and what does not bring me joy is the fact that the message to other communities (other languages) is clear: “we are better and smarter”. […] I think we can do better without thinking that we are.
• I didn’t want to tick the unpleasant communities cause it is not […] lately [however], […] I started noticing a kind of “elitism” especially from people that are frequent speakers or started working early with the language.
• I have multiple colleagues, all great developers, all former Clojure developers. Whenever Clojure comes up they never fail to mention how alienating the “smarter than you” attitude sometimes present in the community can be.
• I’m worried about the perceived shrinking of the Clojure community
• I’ve gotten to know a few more female developers using Clojure in the past year, and have learned that some parts of the community are scaring/driving women away. This includes unpleasant interactions on Slack and working to avoid certain people at Conj.
• Not to put too much stock in the oversimplified Gartner hype curve model, it does seem like Clojure is passing through what they call the trough of disillusionment.
• Please consider supporting Clojurists Together, even launching initiatives for the community to help through it.

Libraries

More guidance on how to compose and select libraries remains a common common issue for people in this section, as well as improving the state of library documentation.

• The plethora of web app (both client & server side) frameworks is both bad and good. Too many to choose from (no standard like rails) and also lots of options. Also, limited security options, especially w.r.t. use of session state, async handlers & server-less designs.
• I frequently find myself frustrated by looking for a library that will help me do “X”. There are many choices out there, each of which does some subset of what one might want in that arena, but a different subset.
• It is sad that I often find interesting Clojure or ClojureScript libraries that are abandoned and haven’t been updated for a long time
• Need more quality open source projects for machine learning and data visualization libraries. Great to see the advances from conj but for a language that has a front end tool built in, there is no excuse for someone not to build a great visualization library for data science.
• No de-facto web framework, Pedestal is in dire need of documentation, Schema is deprecated and Spec isn’t ready, New CLI tools in Clojure 9 competing with Leiningen/Boot. All of these complaints have something in common: it looks like the Clojure core team is actively hostile towards the larger community. All community-built tools/libraries are open to attack and replacement by the core team and there’s always fear of a major destabilizing revolution everyday.
• Several widely used library started to stall… sustainability might become a concern… [Ed: hopefully Clojurists Together can help here]

Other targets

As in previous years, there are a smattering of people asking for alternative platforms to target other than JS and the JVM. LLVM, Go, and Web Assembly all had interest from people.

• Is it posible to use Clojure with the Go ecosystem? That would be amazing
• It is hard to watch Go get so much mind-share. Clojure on the JVM is great, but it would sure be nice if I could build a Clojure application that didn’t need the JVM.
• I still long for a low-level oriented Clojure dialect to be able to produce small clj executables with low overhead and systems programming capacity, but one can dream!
• webassembly support? and native support? (like a scala-native)

Typing

Requests for static typing were down from previous years. I attribute that mostly to spec gaining wider adoption and understanding.

• Clojure needs to make a better case for why static typing is not necessary for maintainable, large, multi-contributor codebases and how the REPL dev process helps with that.
• I still love Clojure as a whole more than any other. I’d love to address long-term maintenance. The difficulty of stepping back into a codebase after some time can be more challenging than with Haskell or PureScript. I wish the compiler could do more for me. Spec is huge though, really enjoying it. It still takes a good amount of discipline to use well, but I’m thankful for it.
• I used Clojure for one work project because a coworker recommended it. Overall it was a good experience, but the lack of a type system made maintaining that particular project too difficult. Tools like Plumatic Schema and Typed Clojure helped in some ways but didn’t end up pulling their weight. Regardless, the Clojure service continues to run today and it’s better than the Ruby service it replaced.

ClojureScript

ClojureScript continues to improve, although getting the tooling setup remains a pain point, as does integrating NPM modules. There are recent improvements on that front though, so this may not be such an issue in 12 months time. I have heard lots of good things about shadow-cljs, it seems to be a strong tool to investigate for ClojureScript usage, especially if you work with a lot of NPM modules.

• ClojureScript has had a serious push this year… ;)
• For CLJS, we really need to get npm deps working. Right now it’s hard to get right, or at least there’s no up-to-date tutorials available..?
• I find I spend more time than I should configuring my development environment so it works well. This is particularly bad in React Native, but cljs/figwheel can still be frustratingly complex to get right.
• Lack of async/await support in CLJS is holding it back. The JS community has settled on promises and async/await as the primary asynchronous patterns, and although core.async is useful for certain things, shoehorning existing JS code into it is very kludgy.
• Nothing has made a bigger improvement to my ClojureScript workflow in the past couple of years than shadow-cljs.
  • Intelligent externs inference, plus simple ^js or ^clj tagging for anything that’s missed
  • Virtually trouble free usage of other JavaScript modules /npm
  • Way faster cljs compile flow due to smart caching of intermediate artifacts
  • Self-hosted dependencies handled very well
• NPM support in ClojureScript seems to be a little bit tricky and not working all the time for all the libs. But has been improved a lot in the past.
• Probably my biggest frustration has been trying to use libraries from JS in a full-on ClojureScript front-end app.

Setup/tooling

Tooling on the Clojure side seems to be somewhat less of an issue this year, but setting up ClojureScript projects with all the things that you need remains an issue for many people.

• Clojure is now mature and stable enough to start focusing in tooling. Many JVM tools are not Clojure specific and for people coming to Clojure from non-jvm languages diagnosing problems is a barrier (more with systems in production). From GC issues, to debugging exceptions, to attaching to a process in runtime to peek at some memory or patch something. It was pretty easy to do with gdb in linux, it should be much more easily done with Clojure.
• Fast track to CLJS web dev for junior devs, in one piece in one place. There is not that much essential knowledge/tooling needed to get started that it can’t fit in a CreateReactApp-like solution. Too much content, too many sources, don’t help. No need to be exhaustive.
• Git-based deps seem like a bad idea. clj build tool is otherwise cool, if a bit unnecessary given the prevalence and quality of leiningen, along with the stability afforded through maven central/clojars.
• I think we should pay more attention to interactive programming. Like what Bret Victor proposes, beyond the REPL.
• Making changes to libraries then having to jar them just to test is hard (checkouts never worked for me (or others) with cljs). I hope Alex Miller’s recent work will fix that.
• Once Figwheel and Reagent and tests were set up, my experience as a developer was very positive - barring some bad times with advanced Closure optimization bugs. Really happy with ClojureScript myself, but it would be hard to convince a friend to start using it, because there is no clear or standard way to develop.

Compliments

As always, the survey was dominated by positive comments. Here were some of my favourites.

• Clojure has ruined every other language for me. Every time I have to do something in another language, I always wish it was in clojure. It’s just so easy to get things done in clojure. I love the language and hope to use it forever.
• Clojure’s still the language I find myself turning to most frequently when given a choice. It just feels natural, the toolkit simple and yet powerful enough to express pretty much anything at a whim.
• I’ve been using Clojure professionally for at least 4 years and these have been the most productive years of my career. I’m looking forward to using spec more fully. I’m looking forward to seeing what improvements come to the language in the next few years and have no desire to leave. Thanks so much for this language, community, and way of working.
• This. Is. The. Best. Language. Ever. High-level programming done right. Thank you Rich.

Conclusion

When I went to write this years summary, I re-read the previous posts to remind myself of where we were a few years ago. While some areas have improved, I was struck by how many of the comments could have been lifted from previous years survey results. There was a worrying trend in the community and adoption sections that people perceive Clojure to be shrinking. Perception doesn’t always match reality, and it may be (as a commenter noted) that Clojure is entering the trough of disillusionment phase of the Gartner Hype Cycle. However I think there are a lot of low-hanging fruit around that could significantly improve the experience and ecosystem around Clojure without changes to the core language.

Cache-Control rules everything around me

The first thing to do was to add a Cache-Control: no-cache header to all static file responses. Despite the name, no-cache tells the browser it can cache files, but must always validate them with the server before using them. If the browser’s cached version is up-to-date, a compliant HTTP server should return a 304 Not Modified response, otherwise it serves the new file.

If you don’t provide a caching header to an HTTP response, the browser can choose its own caching behaviour. The browser’s caching heuristics are much more aggressive than you want in development, and lead to the weird caching behaviour I was seeing.

I thought this had fixed the issue, but occasionally I would still notice stale files were being used. After looking closely at the compiled output files, I made a surprising discovery.

ClojureScript copies file modification times

ClojureScript (as of March 2018) copies the last-modified date of ClojureScript source files to the compiled JavaScript target files. This is so that the Clojure compiler can detect changes to source files. JavaScript from the Closure compiler (e.g. goog.base), gets a modification time that matches the time it was compiled.

Using a Last-Modified date header for caching is often useful, but was causing two problems here:

• Closure compiled JavaScript doesn’t change from run to run, so caching based on the last modified date will cause it to be unnecessarily re-downloaded.
• JavaScript compiled from ClojureScript that uses macros from other Clojure files will receive the modification date of the consuming ClojureScript file, even if the macro files have changed more recently and triggered a recompilation. This was leading to the second round of (very confusing) caching issues that I saw.

To avoid these issues, I recommend removing the Last-Modified header from the response when in development.

ETags

To knock both problems on the head once and for all (hopefully), I added a CRC32 checksum based ETag to Figwheel for static file responses. I packaged this up in a library ring-etag-middleware so that other projects could also use it.

Serve 304 Not Modified responses

At this point the browser will check with the server for every ClojureScript file, on every pageload. However, this causes all of the files to be downloaded each time, even if they haven’t changed. The last step is to add ring’s ring.middleware.not-modified/wrap-not-modified middleware. This will return a “304 Not Modified” response if the ETag provided in the If-None-Match request header matches the ETag header in the response.

Summary

As best as I can tell, this has completely solved all of the odd caching issues that I was seeing, while still keeping the app snappy to load by reusing as much of the cache as possible. If you are serving ClojureScript files in development and not using Figwheel, I recommend you follow these three steps:

1. Set a Cache-Control: no-cache header
2. Add an ETag to your static file responses
3. Remove the Last-Modified header
4. Wrap your responses in ring.middleware.not-modified/wrap-not-modified or the equivalent in your Clojure web framework.

If I’ve missed anything here, please get in contact and I’ll update the post.

Pros to some predicates returning nil

• The main argument for some predicates returning nil is that the predicate still returns a falsy value. Idiomatic Clojure code usually doesn’t need to distinguish between nil and false.
• The docstrings for qualified-keyword? for example says: “Return true if x is a keyword with a namespace”. It doesn’t say anything about what happens if it doesn’t have a namespace, so returning nil or false are both technically valid interpretations.
• Not coercing the output from qualified-keyword? into a boolean is faster. On my computer qualified-keyword? takes roughly 3.5 ns to run, and a version that returns a boolean value takes around 6.5 ns. In absolute terms they are both pretty small though.

The first two points aren’t strong arguments for returning nil, rather they argue that it doesn’t matter whether the functions return nil or false.

Cons to some predicates returning nil:

• The biggest downside to this change is that it breaks a core convention that Clojure and the community has held around predicates, namely that any function ending in ? returns true or false. This is my biggest concern about the changes. You can find this expectation in:

  • Programming Clojure: “A predicate is a function that returns either true or false. In Clojure, it is idiomatic to name predicates with a trailing question mark, for example true?, false?, nil?, and zero?.” - Programming Clojure, 2nd Ed., Page 27.
  • Clojure’s own library coding standards: “Use ‘?’ suffix for predicates. N.B. - predicates return booleans”. That page also has the disclaimer: “Rules are made to be broken. Know the standards, but do not treat them as absolutes.”
  • The community Clojure Style Guide: “The names of predicate methods (methods that return a boolean value) should end in a question mark (e.g., even?).”
  • All of the Clojure standard library functions that end in a ? return a boolean value. Anyone learning Clojure would be justified in assuming that all functions that end in ? return a boolean value if that’s the convention they have always seen.
  • A mailing list thread on predicates in the aforementioned Programming Clojure book.
  • A mailing list thread about this exact question, whether all Clojure functions that end in ? should return a boolean value.
  • Tutorials Point: “Predicates are functions that evaluate a condition and provide a value of either true or false.”
  • StackOverflow on naming rules: “The main function naming conventions seem to be … Use ? to indicate a predicate that returns true or false: sequential?”

• The second biggest problem is that qualified-keyword? can return three values. “How is that?” you might be asking, “Didn’t you just say that the functions return true or nil?”. I thought so too. However as I was looking at the implementation of these functions, I realised that qualified-keyword? can return false as well, if not given a keyword. For example:

    user=> (qualified-keyword? ::abc)
    true
    user=> (qualified-keyword? :abc)
    nil
    user=> (qualified-keyword? 'abc)
    false
  

  It is fairly common to use group-by or juxt with a predicate function to partition a collection into two groups. If you use any of the new qualified* functions, you will end up with three groups. Based on Alex Miller’s tweet, it seems like this was considered and accepted.

• If these predicates are used at the boundaries of systems or in interop with Java, then it would be easy to forget that the qualified* predicates may return nil, and end up passing null into a Java method as a parameter when you meant to pass false. You could also return false under certain input when you expected the function to only return true or nil, or return nil when you were expecting the result to be a boolean. These kinds of bugs can be very subtle, especially with Clojure’s falsy handling.

• ClojureScript code is a lot smaller and enables more optimisations when it can guarantee that a boolean value is returned.

• Clojure gets similar benefits from the JVM when it can mark the function as returning a Boolean.

• Historically, Clojure has made a distinction between functions that return truthy and those that return true. There are predicate functions in core that return a truthy value like some and every-pred, but they don’t end in a ?. This change starts to dissolve that distinction.

Summary

Personally, I don’t see a strong reason to keep the current behaviour of the new qualified predicate functions, but you might see things differently. Whatever the outcome, I’m glad that in Clojure we don’t have to name all of our boolean functions with an “Eh” suffix.

Update

Alex Miller suggested to file a ticket about this behaviour, so I opened CLJ-2141. After some discussion, a patch was made to coerce the output of these new functions to a boolean.

In his summary, Justin said that 62% of the responses were positive. That number sounded low to me (but was presumably calculated by SurveyMonkey). I would have estimated closer to 80% after reading through them all. A little under one quarter of the 2420 survey respondents left a comment.

I’m reusing most of the same categories from last year so interested readers can compare with last years post. Some comments have been lightly edited for spelling, clarity, brevity, and niceness.

Error messages

Error messages were the biggest pain point for people last year. This year there were still many complaints about error messages and stack traces, but a lot of people are waiting to see what spec brings. There was a lot of discussion about spec error messages several months ago, but it seems to have gone quiet. I’m really hopeful that spec will improve error messages.

This year Figwheel has started providing very sophisticated reporting of compile errors, and configuration problems. It has made a massive difference to my ClojureScript workflow. Thanks Bruce!

Please Please Please can we get better error messages. …

ClojureScript has come a long way, but I still encounter errors that are totally perplexing. Better debugging would help enormously.

Error messages are very frustrating and it is hard to find where the error occurred and why. Errors coming from underlying Java code are only somehow relevant to the Clojure that produced them. Programming with dynamically typed language rocks when inferred types align with your idea, but when it’s not, things get fuzzy.

Please, please, please, please, improve error messages. This would single handedly improve: new user onboarding, community feedback (let’s admit it - it is a small community), rapid iterations, usage of the REPL, usage of spec and testing, and other areas. …

I consistently see error messages being the biggest barrier for learners of Clojure, and even more so for ClojureScript

I think a concerted focus on improving Clojure’s error reporting & handling would benefit both newcomers to the language and experienced developers alike and would result in considerably less wasted time & effort.

Docs

Last year clojure.org had just been revamped and open sourced. I wrote at the time:

I have high hopes that it will become a central source for high quality, official documentation and ease the pain for newcomers.

clojure.org has had some contributions from the community, but it still doesn’t have the breadth of material needed to be a central source. I would love to see something like Python’s web documentation for Clojure. Renzo Borgatti is writing a mammoth book “The Clojure Standard Library” which will cover the entire Clojure standard library.

… Please improve the documentation. Recently I heard someone describe Clojure documentation as “comically terse”.

A simple way to get someone up and running that is a standard for the community would be great. There is nothing more frustrating than telling new people to “Google” for a solution and pick one you like… They never get further than reading the docs.

Clojure appeared to be a nice language when I started with it and I do not regret this decision. The flip side is that I had hoped that the poor documentation would have gone away by now – 1.4 was the first version I used and it does not seem to have improved. That’s a shame. Documentation helps new users and really, hinders nobody.

Clojure docs are too terse. They assume that the reader already fully understands the concepts being described, and merely needs a refresher. …

Clojure’s community is surprisingly small given it’s output. This leaves documentation and tutorials sparse because they require time and effort that is in short supply. …

I think both clojure.org and clojurescript.org could use a better “Get Started” experience. It’s our best opportunity at a first impression but we rely too heavily on third party docs. For example, on cljs.org the reference section starts with compiler options. Doesn’t really capture the imagination.

Setup/tooling

Again, Figwheel has been leading the way in improving tooling on the ClojureScript side. Tooling is still a pain point, but seems to be less so than in previous years. For Clojure editors, Cider, Cursive, and vim-fireplace are all used pretty much the same amount as last year, but Atom has entered fourth place, presumably coupled with proto-repl.

One thing we’re used to from C#/Java that would be awesome to have is a better unit testing framework with IDE integration. …

The build toolchain for cljs has been a pain. All of my learning has come from cloning public repos and seeing how they do it.

A huge thanks to the community for invaluable tools like Leiningen, Parinfer and the various IDE plugins like ProtoREPL or Cursive. And of course huge thanks to the core team for excellent stewardship of Clojure itself.

While tooling for Clojure is improving there needs to be a lot more resources devoted to it. I am currently in a Java shop and the only realistic option is Cursive.

Startup time

Startup time was another persistent complaints this year. Unfortunately there wasn’t always enough information about whether startup time was a problem in development or production. There are several good options for avoiding constantly restarting Clojure REPLs in development (and in production), but perhaps for newcomers these are too complicated to setup, or they don’t know about them?

Love Clojure … hate its startup time!

Love the language and will keep at it. Primary frustration is with the start up time and tooling. In particular, I’m finding it difficult to quickly test the backend and frontend of my application with Boot.

… We love functional programming, but hate Clojure’s long startup times. We do what we can to minimize this (using test-refresh, REPLs, etc.), but it is still very painful, and we find ourselves twiddling our thumbs waiting for a restart several times a day. Similarly, our Clojurescript compiles take ~60 seconds (:simple optimizations) or ~30 seconds (:none optimizations). This slows us down substantially. Coming from Python, where compile and startup times are sub-second, this is the biggest complaint from our team.

Clojure would be perfect with faster startup time and a little less memory usage. Seriously I don’t even mind the java stack traces.

REPL startup time is all I care about. I use emacs+cider, and it takes an age. Otherwise, I am completely happy with Clojure :)

Marketing/Adoption

Justin specifically mentioned marketing and adoption as a common theme amongst the feedback:

One relatively stronger theme this year was the need for better marketing for the purposes of expanding or introducing Clojure within organizations, which is a great area for contribution from the entire community.

In my opinion, one of the areas where Clojure could improve on marketing is on the main Clojure and ClojureScript websites. I think Scala in particular has done a really good job of this. Elm and Rust also do really well in presenting the main reasons why you would want to use their languages. The community isn’t able to help with this, as contributions to style and templates are not currently wanted.

Last year I mentioned Cognitect’s case studies on Clojure adoption, this year Juxt has done some great work with their case studies of companies using Clojure in Europe.

Anecdotally, I’ve seen fewer people coming along to the Auckland Clojure Meetup than in previous years, although we’re working on a fairly small sample size. I’m not sure what (if anything) to make of that.

I would do 100% of my development in Clojure if the enterprise-y company I worked for allowed it. “Reason for not using Clojure as much as you would like”? Management not on board.

I’d love to see more “serious” support for ClojureScript: when I present ClojureScript to colleagues, it looks like a thing supported by several enthusiasts rather than a platform supported by a company.

Clojure is very solid. As a frontend developer, a stable build system, DCE and syntax stability are very valuable for me. I failed to convince my CTO on Clojure and instead we stuck to ES6/ES7. The main concern of management was hiring and training people. Although the learning curve for Clojure is easy, people still have a perception that LISP syntax is esoteric and difficult to pick up for someone completely new to the ecosystem. This myth has to be busted on a larger scale.

Language is great, community is great, the “marketing” is not that great. You guys have a great language and you are struggling to sell it.

[I] would love to see better marketing from Cognitect and other Clojure centric shops. Selling Clojure to a Java shop is not easy. It can and should be easier. (and simple, but I’ll take “easier” here first)

As an average developer fluent in Java, I would love to use Clojure more, but the biggest hurdle for me is the lack of peers interested in Clojure. At the local meetup people report the same situation at their organisations. Pity, because Clojure is super cool! Thank you.

Love Clojure, really. Can’t convince my peers that it’s worth investing the time to learn it though.

The Clojure community needs a “Rails killer,” and only Arachne holds serious promise for that.

Clojure/ClojureScript ecosystem could benefit from stronger stewardship from Cognitect to propel it into the mainstream with a focus on a killer app or a industry domain similar to the way Light Bend has been focusing on enterprise and reactive system to promote the Scala ecosystem. …

I’m in Shanghai and we barely have full-time Clojure developers in China. We formed online groups, but I guess we need help to push it to the next level.

And the age old developer question:

How can we get management to understand what they don’t understand?

Language Development

Like last year, there are still concerns about the contribution process, and the opaque decision making process for Clojure’s development.

Engagement of community is still sub-optimal, there are talented individuals who could be engaged more and productively so without opening it up to the peanut gallery.

I wish that the core Clojure development process was more flexible and friendly. I (and all the Clojure developers I know) have pretty much given up on trying to make suggestions or tickets about Clojure because I usually wind up banging my head against the wall and getting frustrated. The core team are under no obligations to explain themselves or their motivations, but they would save a lot of frustration from new community members wishing to contribute if there was a prominent document outlining the philosophy, and what kinds of changes and suggestions are and aren’t welcome.

A more open approach to development and contributing to Clojure would be appreciated. I’d love to contribute to Clojure but the mess of a lot of the code in Clojure and the JIRA-based contribution model is a barrier. I know that’s what Rich likes and see the rest of Cognitect are happy contributing in this way but I’d love to see something more open and approachable.

The slow rate at which JIRA issues are addressed remains a frustration, and provides a disincentive for contributing. Features that Cognitect cares about get rushed into the language half-baked, while well-thought out patches from years ago languish.

A little concerned over how ‘arbitrary’ some design decisions seem to be and how inflexible the core team is. Running instrumented post-condition specs is such an obvious idea for example but it has been deemed not necessary “from on high”.

The issue tracker is well tended during the earlier workflow stages. There seems to be a bottleneck later on. I hope 2017 will bring the project’s founders to modes of community contribution that are more efficient of the art and tradecraft of the many.

Community

I would say that Clojure’s community is one of it’s greatest assets. There were mostly positive comments for the community, though still some concerns. The Clojurians Slack has been a great resource for a lot of newcomers, and spawned many channels for libraries and topics. IRC is still around, but seems to be less popular, and doesn’t have the breadth of topics that Clojurians does.

Alex Miller is doing a great job addressing the community.

CLJS has come a long way!! to the point we can use it for production development! this community is great! Justin_Smith stands out as ridiculously helpful on IRC … if the world (this community) had more folks like him, it’ll elevate all of our games!

The thing that makes me wary of continuing to invest in Clojure professionally is how few paid opportunities there are for women and people of color. I don’t know what the exact problem is - they are left of networks, don’t have an opportunity to build their skills, or just straight-up sexism and racism - but it is definitely a problem. The current state of things means I will likely not be able to be on teams with women or people of color, and that is a big turn-off for me.

Clojure has the best community after PostgreSQL, in my opinion. I came for the language but I stayed for the community.

Be more inviting to new-comers.. so they stop learning Ruby.

Libraries

A “Rails for Clojure” was a common request so people can start delivering value quickly. Arachne was commonly raised as one possibility to provide this.

Clojure/ClojureScript it’s very cool, but not very practical nor efficient in many typical business case scenarios. I’m missing a full stack framework like Grails or Ruby on Rails or Django: there’s nothing like that for the Clojure world: it’s not about a project template but about the quick prototyping (with the customer by your side) these frameworks allow.

… There is also no good Clojure Web Development story: Pedestal looks interesting, but not actively developed and unsupported, Arachne is years away from being complete and useful, Luminus/Duct are initial scaffold project generators. It’s hard to delivering some value quickly - user have to dig through all underlying libs and how they work together.

… Clojure web development needs a “Rails” to overcome the chicken-and-egg problem of getting enough developers to be able to hire for it. …

I hope to write a lot more Clojure. No, I hope to do a lot more with less Clojure. And looking forward to using Arachne.

Other targets

Many people are looking to run Clojure on alternative platforms or reduce host dependence. LLVM was a common request. Some are looking to move away from the JVM, while others are looking to move into environments that Java and JavaScript aren’t suited for. This is an interesting request, though one of Clojure’s original goals (and still on the Clojure homepage) is to be a hosted language. I’m not sure whether it’s possible for these two desires to ever meet? Graal and Truffle are also interesting possibilities.

The biggest thing I’ve noticed for beginners on forums / blog comments is they are afraid of the JVM. …

I’ll never rewrite all my Scala into Clojure unless some sort of Cognitect Clojure OFFICIAL LLVM effort is made. Scala Native is [beating] Clojure.

Clojure’s hosted nature is its biggest downfall, in my opinion. As a systems programmer looking to lisp + immutability for saner concurrency, the cost of Clojure’s startup time and memory usage is very high. …

The one thing that I think would drive Clojure to a new level of adoption across the industry would be for someone to do an actual LLVM implementation of Clojure. Not a “close variant” of Clojure, but real Clojure. This would probably yield some performance boost beyond Java, but more importantly it would open up Clojure to environments that are simply not amenable to Java (or Javascript) for whatever reason. Some people have moved in this direction, but either have not completed the work, or have implemented their own “variants” of Clojure. What I wish someone would do is implement something with LLVM that is at least as close to Clojure as Clojurescript is.

I really think an opportunity is being missed by dismissing the c++ ecosystem, and I think that something that should be given somewhat serious consideration to by somebody would be to leverage LLVM with JIT and get a clojurescript running in that setting. There are significant legacy environments to be liberated there.

I would love for there to be some sort of compile-to-bundle with Clojure such that you have a single artifact that doesn’t require an external JVM installation. Like Go programs. An embedded JVM (perhaps).

Typing

As we’ve all learned this year, people are worried about types. Spec is the biggest movement in this direction, and the number of type related comments has dropped a lot since last year.

I know that Spec should help a lot with the error messages once we move to 1.9, although people coming from a non-Java background still have a lot of learning to do.

It is just sad that people who love Clojure (Colin Fleming, Steve Yegge, Adam Bard), have to use Kotlin sometimes instead (better startup times, static typing and Java interop like fast multidimensional arrays)

I hope Spec will find its way into tooling, especially Cursive.

Looking forward to learn and use spec

I’d most like to see improvements to the Clojure compiler. Error messages, performance, better Java interop and simple static type checking seem most important.

Compliments

As I mentioned at the start, the overwhelming majority of comments had positive things to say about Clojure. Alex Miller also deserves a special mention for his work with the community, and maintaining the Clojure infrastructure.

I sincerely thank Clojure(Script) community to make LISP revolution back to live stage and making us realize the importance of focusing on core things which matter

I have been programming since 1979. Clojure is a work of art.

The community has been unbelievably great and I hope it stays true to its roots! Keep up the good work and I can’t wait to see where Clojure/Script goes next!

The community is amazing, with a special shoutout to Alex Miller for being such a great part of it.

Happy Clojure user for over 4 years. Still loving every minute :-).

Finally, I found a comment that I think sums up the general feeling of the community (and my own):

No one and nothing is perfect, but I (for one) am very appreciative of the work that has been done and is on-going around Clojure/ClojureScript and the community in general. People tend to ‘harp’ on the things that aren’t just right (in their opinion) and forget about all the things that are amazing. I just want to say thanks, and keep up the superb work! :-)

I think the future of Clojure is bright.

The major changes in a nutshell:

• There are now no limits on process count
• Datomic Starter is now limited to one year of updates, then you need to pay $5,000 for Datomic Pro, or stay on the current version when your license expires.
• Datomic Starter can now run the Memcached cache.
• All Datomic Pro licenses are now $5,000, and annual maintenance is also $5,000/year.

If I’ve got any of the following wrong, please get in touch via email and I’ll update my post.

The old pricing model

Datomic launched in March 2012 with a paid option and in July 2012 added Datomic Free. In November 2013, Cognitect launched Datomic Pro Starter Edition. The old pricing page is on archive.org here. This model was easy to understand, as it mapped well to existing database licensing patterns of ‘perpetually free with limitations’ or ‘paid without limitations on a per node basis’.

Datomic Free

• Free (as the name indicates)
• Limited to storing data in memory on the transactor, or on the transactor’s disk.
• Freely redistributable (i.e. in on-premise or open source software)
• Only able to run two peers (clients) and a single transactor. If the single transactor fails, then you won’t be able to write to the database until another one starts.
• Community support

Datomic Pro Starter Edition

• Free
• Support for all storage backends (SQL, DynamoDB, Cassandra, Riak, Couchbase, Infinispan)
• Limited to 2 peers and a single transactor
• Not redistributable
• Community support
• Limited to 1 year of maintenance and software updates, though you were able to renew your Datomic Pro Starter Edition license each year for free (more on this later).
• When your updates expire, you can continue using that version of Datomic, but you won’t be able to use any future versions.

Datomic Pro

• Paid
• Support for all storage backends
• Able to run a second transactor in HA standby to take over if the first one fails or for rolling updates.
• Ability to use Memcache to cache segments rather than all peers needing to talk directly to the storage backend.
• Support is included while maintenance is current.
• Pricing scaled linearly from $3,000 for five processes (this includes peers and transactors) up to 30 processes for $16,000. You could upgrade later to higher tiers for the price difference between the old license and the new license.
• Annual maintenance for support and updates was half of the upfront license price
• If an organisation had special needs (more processes, custom EULA, 24x7 support, redistributing Datomic e.t.c.) then you could talk to Cognitect to negotiate terms.

The new model

All versions of Datomic apart from Free now support unlimited peers, high availability, Memcached support, and all storages (more on storage later). This is a significant change, as you can now use Datomic in an almost unlimited fashion for free. There is also a new Client API which is suitable for smaller, short-lived processes, like microservices and serverless computing. The changes are smart, as it frees users up from having to architect their systems around their Datomic license limitations. The new pricing model rearranges the tiers. There is now Datomic Free (unchanged), Datomic Starter, Datomic Pro, and Datomic Enterprise.

Datomic Starter

• Similar intent to the previous Datomic Pro Starter
• Free
• Maintenance and updates limited to 1 year. However, based on discussion on Hacker News, it seems that you can no longer renew your Datomic Starter license. This means that you will need to pay for Datomic Pro to get updates after one year. You can still use whichever version was available when your updates expire. Discussion in the #datomic Slack channel on clojurians matches up with this too.
• Update: Alex Miller said that you can sign up for multiple Datomic Starter licenses for different systems you’re running.

Datomic Pro

• Similar intent to the previous Datomic Pro
• $5,000/year per system including maintenance and updates. For organisations using 16 processes or less, then maintenance will be more expensive (previously $1,500 - $4,500 depending on process count).
• 2 Day Business-Hours-Only Support. The former Datomic Pro didn’t have a published SLA for support, but I suspect that this is just formalising what was previously there.

Datomic Enterprise

• Enterprise integration support (professional services)
• Negotiated license terms
• 24x7 Support
• While this is a new tier in the pricing grid, these options were available as a “Contact Us” note on the former Datomic Pro.

Other changes

Datomic has a snazzy new documentation site. It also looks like as part of the licensing changes, Riak, Couchbase, and Infinispan are now considered legacy storage and are only available to be used under an enterprise license. Standard editions of Datomic only support SQL, DynamodDB, and Cassandra. This change hasn’t been mentioned on the Datomic mailing list or release notes, but probably will be soon.

Datomic has deprecated the REST API in favour of the new Client API.

There is also a new customer feedback portal where you can suggest features. You can access this after logging in to my.datomic.com.

Summary

If you are a Datomic Pro user then your maintenance is probably going to be higher, although in absolute terms it’s still not a lot compared to developer salaries. If you were on Datomic Pro Starter and want to stay current, then you are now looking at moving Datomic Free, or paying $5,000/year for Datomic Pro. If you were using Riak, Couchbase, or Infinispan then it seems like you’ll need to get Datomic Enterprise.

Datomic from the beginning has always felt like a database that understood the Cloud, and the zeitgeist of computing. It supported AWS DynamoDB and CloudFormation from early on, and their architecture has always felt well suited to cloud systems. The license changes to accommodate the trend towards microservices and serverless computing are a continuation of that.

The title is a joke (as is his), I don’t really think there are people worried about types, but it is interesting to see a small but steady flow of type or type related (schema, spec, e.t.c.) articles relating to Clojure each week.

Clojure

Clojure generates a Java class for every defrecord that you create. You need to :import them, just like you would a standard Java class. Lets say we have a small music store app that sells vinyl:

(ns myapp.cool-music)

(defrecord Vinyl [speed])

;;; meanwhile ...

(ns myapp.music-store
  (:import [myapp.cool_music Vinyl]))

(instance? Vinyl v)
  
;; or

(ns myapp.music-store
  (:import [myapp.cool_music.Vinyl]))

(instance? myapp.cool_music.Vinyl v)

There’s a few things to note here:

1. Because Clojure is generating Java classes, they follow the naming conventions of Java classes, where dashes get converted to underscores.

2. You should probably prefer the first example where the record is aliased, over the second one where the record is fully qualified.

3. Clojure dynamically generates the class for the record when that namespace is required (ignoring AOT compiling). If your code never requires the namespace that the record lives in, then it won’t be created. This will cause ClassNotFoundException errors when you try to import the class. In our trivial example, that would mean changing the ns import to:

    (ns myapp.music-store
      (:require [myapp.cool-music]
      (:import [myapp.cool_music Vinyl]))
   

ClojureScript

ClojureScript generates the code for a record in the namespace it is defined in. You can require, refer, and alias a record just like you would any other function in a ClojureScript namespace.

(ns myapp.cool-music)

(defrecord Vinyl [speed])

;;; meanwhile ...

(ns myapp.music-store
  (:require [myapp.cool-music :refer [Vinyl]))

(instance? Vinyl v)

;; or

(ns myapp.music-store
  (:require [myapp.cool-music :as m))

(instance? m/Vinyl v)

;; or

(ns myapp.music-store
  (:require [myapp.cool-music))

(instance? myapp.cool-music/Vinyl v)

There are no hidden gotchas with requiring records in ClojureScript. At one point ClojureScript supported importing records like Clojure, but this was backed out later.

Clojure and ClojureScript

If you want to work with Clojure and ClojureScript (and let’s face it, who doesn’t?) then you can use Reader Conditionals to require a record in both language variants at the same time.

(ns myapp.music-store
  (:require #?(:clj [myapp.cool-music]
               :cljs [myapp.cool-music :refer [Vinyl]]))
  #?(:clj
     (:import [myapp.cool_music Vinyl])))

(instance? Vinyl v)

I’ve recently been working on a new ClojureScript application as part of a contract, and I was digging around for things to polish before launch. The app was mostly fast, but I noticed that when the main list of content got to around 40 items, it was a little bit slow to render. I also noticed that it seemed like it got almost twice as slow when I added another 10 items. At this point, you might already be having alarm bells go off in your head suggesting what the problem was likely to be. I didn’t, so I dived into the code to look at the part of the app rendering the main list.

Analysis

I looked over the code path that rendered the list, and wrapped time around a few suspect pieces of code. After a few checks, I found that a sort-by function in the view was the slow part, though it wasn’t immediately clear why sorting a list of 40 items would take a second. We were using a custom comparison function with sort-by to order items by state (flagged, unread, read, etc.), then reverse date order (newest items first).

sort-by takes a custom sort key function. Our comparison parsing the ISO 8601 timestamp into a date, then subtracting the current Unix time from the parsed Unix time to get an integer. The lowest numbers are the most recent dates and would be sorted first. I suspected that the date parsing could be the problem, but I wasn’t really sure. As an experiment, I disabled all of the date parsing, and returned the string directly for comparison. My sorting was the wrong way around, but it went from taking 1000 ms to 10 ms, a factor of 100x speedup!

A fix

A standard sort of the dates (which were in ISO 8601 order, e.g. 2016-04-02T08:24:31+00:00) sorted the oldest dates as the first ones in the list. After a few minutes thinking, I remembered I had recently read the clojure.org guide on comparators. In it, it discusses a reverse comparator:

(fn [a b] (compare b a))

This comparator is exactly the same as a normal comparator, but it will return the opposite result to what the normal one would. Passing this comparator to sort-by kept the 100x speedup, but sorted in the correct order. The list rendered in 10-15 ms, and was now plenty fast enough.

One question remained though, why was the list getting so much slower to render as I added a few more items to it? Of course reading this now, you Dear Reader are probably thinking to yourself, “aha! JavaScript’s sorting algorithm will be O(n log(n)), and the slow comparison function will therefore be called O(n log(n)) times.” It took me a bit more thinking than you (I didn’t have a blog post explaining why my code was slow to work from), but I got to this conclusion in the end too.

I really enjoyed this debugging session, as it is not very often that I can both speed up real world code by 100x, and get exposed directly to algorithmic complexity issues. A++, would debug again.

Specifying standard Clojure types is fairly straightforward, but I recently needed to specify that a value was a goog.date.UtcDateTime. I did a bit of searching on Google, didn’t find anything. I then looked in the company codebase and found it. Here’s a minimal example:

(ns myns
  (:require [goog.date.UtcDateTime]))

(def Date js/goog.date.UtcDateTime)

Error messages

Error messages have been a pain point with Clojure since the first Clojure survey in 2010. Colin Fleming’s talk on Improving Clojure’s Error Messages with Grammars represents a bright spot on the horizon, and I hope that some variant of this comes to Clojure and ClojureScript. I’d highly recommend watching the talk if you haven’t already.

I’d love to see elm-like approach to hard tasks like improving on error reporting and documentation. Clojure team is so skilled that it won’t bother even with low hanging fruit. But it could be hurtful in the long run.

Regarding potential future enhancements, improved error reporting would be very helpful. This has been a major stumbling block for our junior developers in particular. Too many error messages require a high level of understanding of Clojure internals to understand. Overall, I’m very appreciative of the well-thought-out design decisions in both Clojure and its ecosystem.

Please fix/finish transitive AOT, and please make error messages better. I know error messages continue to receive this sort of “well, if you wanna make ’em better, patches welcome…” treatment, but I think it’s a mistake to keep beating up on them, insinuating they don’t really matter. As someone who’s spent a fair amount of time debugging Clojure programs, some of them do matter, and I think the wrong message is being sent to would-be contributors about the value in improving them.

Better error messages from macros! It can be done with PEGs quite easily.

Please improve error messages - Elm developers are a good example to follow.

2+ years with Clojure. First one was strong, but the community has been solving non-problems (transducers, reducers) instead of improving error messages and general debugability.

Highest on my list of annoyances are NullPointerExceptions missing stack traces. They they disproportionately more time to debug that other failures. We rarely see them in production but during development they are a constant scourge.

Clojure has the problem that many languages have — the people capable of providing a better UX aren’t interested in solving that problem. (Myself included.) We could learn from the Go core team’s willingness to solve common UX problems (fast compiler, auto-formatter) and other projects like Elm’s compiler work to support good error messages.

Docs

clojure.org has undergone a massive visual refresh between the survey being taken, and the results being released. I have high hopes that it will become a central source for hiqh quality, official documentation and ease the pain for newcomers.

Community & resource wise, everything things feel extremely federated, in a bad way. Documentation This is where I had the worst experience with Clojure. Documentation is extremely federated. To understand clojure.zip & clojure.core.logic took me so much work […]

Many things that disappoint you about Clojure at first (“why errors are so awful?”, “why the contribution process is so hard?”, " why don’t you implement this one feature I like? “), they fade away as you learn Clojure and Clojure Way more. Still, it would nice to have a sort of explain-like-I’m-five FAQ somewhere on clojure.org that covers these non-obvious points so that people skip the frustration part and understand the reasoning behind those decisions immediately.

CLJS could use better official documentation.

I still don’t know how to get externs working with either lein or boot!! Evaluating code in vim doesn’t always work. Documentation on google are just not reliable because they are outdated or assumes too much pre-knowledge. Applying what’s read in the docs most often do not work, or I just don’t understand. The documentation must really be much more comprehensive and not assume the readers are clojure pros.

Documentation! The clojure.org documentation is so terse that it is unusable for beginners. The clojuredocs.org documentation is confusing for someone who wants the learn the basics. Some examples are too cryptic and appears to be written by very smart people showing off edge uses of the feature they’re describing.

Clojure needs more outreach. Clojure.org or clojuredocs.org needs more tutorials, application-development documentation pointers (For example: “here are your choices if you want to make webapps with Clojure”).

Very happy with progress on cljs.js (tho remains sparsely documented). Much of my pain is related to the compilation step. Lots of errors still happen only after compilation, & not easy to know how to split large code base and load only necessary modules, etc.

Re. documentation: Features added to clojure.core usually have far inferior documentation on initial release than many libraries. Sometimes this is eventually remedied by the community, but sometimes not. This has been the case for reducers, core.async, the new threading macros, and transducers. Even the core library functions didn’t have much example-based documentation until the community took it upon themselves to make clojuredocs. […] Actual documentation, with practical examples, would lead to faster and more widespread adoption of new features.

We just had a Clojure user group meeting here in PDX with several new faces, new to Clojure. […] One person pointed out how the Clojure home page makes no mention of Leiningen, the de-facto standard build tool. The references to running Clojure apps with “java -jar …” are that much more intimidating for anyone coming from a non-Java background. Some of these issues were well described in the “ClojureScript for Skeptics” talk at the Conj. Consensus: people love, or want to love, the language (I do!) but the rough edges have not been smoothed out, and it is still intimidating to get started in Clojure. – Howard

Setup/tooling

Setup and tooling has historically been another pain point in Clojure. In the ClojureScript world, Figwheel seems to be solving a lot of hard problems (REPL, reloading code) in a relatively easy way to set up, and I’m sure it will get even better this year. As an alternative to emacs, Cursive (which I use) is a great tool for getting a full Clojure development environment setup with minimal hassle.

By far, the biggest complaints I hear from friends and at meetups is the startup time and the work required to get a ClojureScript workflow set up.

Clojurescript project setup remains really involved.

Tooling tooling tooling

[…] EMACS SETUP: IMPOSSIBLE I used and liked emacs 15 years ago and prefer to use emacs for clojure development now, but after a full day of trying I still can’t get all the emacs pieces together. It’s too complex, there are too many half-baked and out-of-date how-to examples. Will the real viable emacs setup tutorial please stand up?

Cursive and it’s debugger are a Godsend - I couldn’t ever convince myself to use Emacs and all the alternatives were inferior. Cursive made writing and debugging Clojure a really pleasant experience. Same with boot in Clojurescript world - getting lein + cljbsuild + figwheel working well was a pain. Boot+boot-cljs+boot-reload+boot-cljs-repl is a pleasure. I think those two things will do a lot good towards Clojure and Clojurescript adoption.

Marketing/Adoption

Convincing coworkers to adopt Clojure also was a pain point for some of the respondents, and additional marketing materials could help here. Cognitect’s case studies are good, and could be ported to the official Clojure website.

The clj / cljs / om / figwheel websites make Clojure appear like a niche ecosystem. Seems to be time for marketing!

needs better marketing. clojure has bad marketing (great product, bad marketing) typesafe has good marketing (complex product, good marketing) 10gen has great marketing (bad product, great marketing) it is hurting companies wide adoption: “Clojure? Um.. What is it you said?” “Scala, yea, I’ve heard good things about it, it’s solid.” “MongoDB? Oh, yea my teams tell me it’s really good.”

Love clojure and clojureScript. Where I need help is on the “evangelical” side. If we had more people singing the praises of clojure that would help my business secure more clojure business and everyone else’s work as well. This brings in more money and talent which can then be reinvested into tooling, systems, turn-around development time,etc. just look at what’s happened over in the JS world once their reputation improved.

Overall, very happy with the current direction of Clojure. What feels missing is awareness in the market about the goods of is as a technology, as the biggest impediment with using Clojure more has been finding customers willing to embrace the risk. Alternatives like Scala managed to have a better name within the industry.

Clojure deserves a better website and documentation.

Still trying to persuade co-workers to use Clojure more.

Community

In my opinion, the Clojure community is one of Clojure’s greatest assets. I’ve found it to be overwhelmingly positive, and there are a massive amount of innovative libraries coming from a very young community. There were also some concerns about the contribution process, and how many of the decisions about Clojure happen inside Cognitect.

Many thanks to Rich, Stu, David, Alex and many other people from community and Cognitect. You are doing amazing job. I love the culture of community you created around Clojure: open community with emphasis on learning, design, thinking hard and solving the right problem. It is inspiring place to be in.

What an amazing language. Relatively frequent, consistently stable releases. A pleasure to use. A friendly, smart community. I feel very lucky to be a Clojure user! Thank you for all of your hard work.

The Clojure contrib process frustrates me more than any technical or community aspect of the language.

Clojure gets a lot right, but as has been repeatedly discussed the pace of evolution and the maintainership’s dim view of 3rd party non-bugfix work flatly leads to worthy but minor work such as type predicates going largely ignored and certainly unmerged. In most open source projects, contributors can impact priorities by giving of their time to support work which isn’t high priority. The work which Nola Stowe did on clojure.string was awesome and I think we can see more of that if Alex et all allocate more capacity to working with contributors.

I found some of the recent back-and-forth about tuples to be very disheartening—if I were Zach Tellman I would decide not to bother attempting to make improvements to the core of Clojure. […]

i have become a little worried about the future of the language. The core team are way ahead in terms of design and concepts, but seem to lack any kind of empathy with beginners.

Things I think should be a priority for the long term future of Clojure: […] 3. More open core development process. I worry about the long term future of Clojure when it appears to be driven by internal decisions within a single company (i.e. Cognitect). I am uncomfortable with this approach, and am hesitant to commit fully to Clojure as a language until this becomes more open. Maybe a “Clojure Foundation” would be a good idea?

Thanks to everyone again for another great year in CLJ(S)land. Best community for any language.

I participate in a local meetup group which has helped greatly and I find the community very welcoming (especially being a woman, I’m not finding as many issues as I do with other communities). Thanks!

Keep up the great work! Its been a joy to participate in this community :)

I would like to thank entire Cognitect staff for creating such a great environment and community.

Libraries

Although there are some guides like Clojure Toolbox, awesome-clojure and the Luminus useful libraries list, it seems like there are still pain points around library discovery.

Finding good libraries has not been difficult, but it would be helpful to have some mechanism in the community for evaluating which libraries are viable in the long term as well as which are “best in class”, so to speek. In many cases, I want a library that will provide a useful component for a multi-year project. The library does what I need and is available, but the modifications on github are 3+ years ago. Should I invest in using this library in my core code? Are there better, more current and maintained options? And so forth. Some community aggregation that makes it easy to evaluate libraries in this sense would be very helpful. […]

Libraries Not sure where the main place is to find clojure libraries. Some are in maven (with no ability to evaluate the library). Clojars also feels incomplete and hard to evaluate. The landing page http://clojure.org/libraries even suggests checking maven, clojars, and github, it feels like a dead end. Google is better resource than these, because you can find blogs evaluating the libraries.

Typing

I was surprised to see so many comments about static typing, but it was a recurring theme. They can roughly be divided into three groups:

1. People who were interested in structural typing (Prismatic Schema), or gradual typing (core.typed).
2. People who were getting push back from their organisation for using a dynamic language.
3. People who wanted a type system like Haskell’s.

There seems to be a lot of interest in core.typed but people are waiting at the sidelines until it reaches more maturity.

I was surprised to see that the survey doesn’t include reference to usage of typing or contract tools like core.typed or Schema. I’m interested in understanding how widespread their usage is. Schema’s been an important tool in helping to encourage Clojure’s more widespread use at our enterprise and I’ve been largely happy with it.

Clojure and ClojureScript are amazing projects and the community is pushing the edges of programming today (Om Next, Datomic, cljc). I’ve been close to using both many times, but at this point I don’t think I could leave the expressive syntax or type systems in Haskell and PureScript. I wish they had the same community and dev experience, and am interested in using the Clojure community’s ideas there.

People in my organization feel strongly that dynamic typing won’t scale to projects with more than a few developers. I am concentrating on small projects. Also, the learning curve for Java devs is a bit steep. Like anything else, one has to really want it.

The two things holding back Clojure/ClojureScript use at work are corporate policy and a strong fear of weakly typed production languages by the team.

I love clojure but I also like static type systems. Don’t know if its possible to two in a way that works well. Haskell and scala are languages I like for its type system

The discussion we have been having lately though is around static typing. I have a Haskell background (and a Phd in type systems, even), so I’m aware of the benefits they can provide but I don’t think they’re a silver bullet either. None-the-less, improved static support seems incredibly useful. A gradual-typing approach does seem ideal, allowing you to specify either types up front, or to add annotations as the design solidifies, perhaps. I’m eagerly watching Ambrose’s work, and perhaps prismatic-schema is enough – it would be great to get some of the core community’s thoughts on the matter though.

I really, really, really want to see some form of official support for static typing. This could be core.typed, but it needs to be blessed by the core Clojure dev team and development to allow core.typed (and other future tools) to better integrate with the compiler is necessary. Type annotations provide helpful documentation and (optional) static type checking can help speed up development for those already annotating for documentation purposes by detecting type errors before executing programs. I see the lack of support for (optional) static typing as Clojure’s biggest weakness and the main reason I have considered moving to a different language.

Compliments

Like Justin said, over half of the responses were happy or positive. Here’s a few to finish on an upbeat note. I’m looking forward to seeing what 2016 brings for Clojure, and especially ClojureScript. The tooling story seems to be coming together quite well. If enhanced error messages make it into Clojure 1.9, and the documentation story improves, a lot of the pain in using Clojure could be minimised.

(= “Clojure” “Awesome!”) => true

Thanks for everything that you do. :)

Love it! Thank you for everything that you do.

I just love programming in clojure.

Very nice thing. I like to program in this Lisp a lot

I would never have written PartsBox.io if it wasn’t for Clojure and ClojureScript. Only these tools allow for such a productivity increase that a single person can write a complete medium-size application.

You rock guys, I love you!!!

Onward and upward!