Daniel Compton

The personal blog of Daniel Compton - Projects

Announcing The REPL podcast

I sent the first newsletter for The REPL two years ago, and I have really enjoyed writing it. I’ve learnt a lot from what people have written and created to share with the Clojure community. Sometimes when I’m writing the newsletter I’ve thought “That’s fascinating, I’d love to hear more about the technical details of that”. Now I have an outlet for doing just that.

I first had the idea of producing a Clojure podcast around January 2018. I didn’t have the time for it then, but the idea kept swirling around the back of my brain. When Michael Drogalis’s Pyrostore was acquired, I wanted to hear more about the acquisition, so I contacted Michael to see if he wanted to talk about it. That became the first episode.

My goal for The REPL is to talk with people about the technical details of the projects they are working on. The Clojure community has a ton of interesting people doing really creative work with Clojure. I’ve really enjoyed talking with a bunch of people already, and I’m looking forward to talking to more of them in the future. You can find it therepl.net. It’s available on Apple Podcasts, RSS and all of the other common podcasting apps.

On Abstraction

Almost two years ago, there was a Github issue on reagent (a ClojureScript React wrapper), suggesting that Preact be added as a substitute for React. I wrote up a fairly long comment about why I didn’t think this was a great idea (at least not immediately). React’s recent announcement of the new hooks feature made me think about it again. I’ve republished it here with a few edits for context and time.


Introduction

In principle, I’m not opposed to the idea of Reagent using Preact. It has some cool features and I like that it is small (although in comparison to the total compiled size of a normal CLJS app it’s probably a wash). If Preact worked 100% out of the box with Reagent with no code changes required then I would have no issues with someone swapping out the React dependency for a Preact one and calling it a day. If there are only a few minor tweaks to Reagent required to pass the tests, then again I don’t really have any issues with that. I suspect that even if you have 100% of the tests passing, there will still be issues, as Reagent was built around React, and may not have tests that would cover the difference in behaviour between React and Preact.

Abstraction

It looks like it may not be a pure lift and shift to support Preact. If that’s the case then we run into a bigger issue: abstraction. Reagent was written and built around the ideas and in the context of React. There are assumptions (probably tens or hundreds) built around React’s API and possibly implementation details too.

Adding abstraction adds a large cost because you can no longer program against a concrete API and implementation, you now have to consider two. There are three numbers in computer science, 0, 1, and many. We would be moving from 1 to many, and that takes work.

An aside: recently at work, we were looking at moving a legacy system from only supporting dates in the past to also be able to support dates in the future. This should be straightforward right? We talked to the programmers responsible for it and they couldn’t guarantee that it would work, nor whether supporting future dates would be easy or hard. In the building of that (or any) system, hundreds of simplifying assumptions are made around the context that the system is going to be built in.

It is a very common pattern to have different backends and I don’t see any downsides to it.

I can’t think of a single example of a system with multiple backends that didn’t have any downsides to it, e.g. ORMs, HTML/CSS/JS, Java. There may be some, but they would be the exceptions that prove the rule. Everything has a cost, the question is whether there is a benefit that outweighs the cost. It is much harder to remove something from software than to add it, which is why we should be certain that the benefits outweigh the costs.

While Preact strives to be API-compatible with React, portions of the interface are intentionally not included. The most noteworthy of these is createClass() … https://preactjs.com/guide/switching-to-preact#3-update-any-legacy-code

Reagent currently uses createClass. There are workaround options provided, but this is an example of some of the API differences between React and Preact which you need extra compatibility layers to support. Do we know if the compatibility layer works 100% correctly?

A possible future if Preact support is merged now

As a thought experiment, let’s assume that Preact is in Reagent with some kind of compatibility shim. Preact already has several performance optimisations that people can take advantage of:

customizable update batching, optional async rendering, DOM recycling and optimized event handling via Linked State. - (from Preact homepage)

Wouldn’t you want to be able to take advantage of those in your application? I certainly would. Now to do so, you may run into issues because the compatibility shim layer that was written was encoded around default assumptions of React, and they may not apply to Preact. Do we have to rework the shim layer, or lower level Reagent API stuff? Who is going to do that work? Who is going to review it and merge it?

Let’s consider the reverse. Perhaps in some new React version, Facebook comes out with a new API which is faster or better suited to Reagent’s style of rendering, so we want to switch to that [since writing this they came out with Hooks. These may be added to Preact also, but this isn’t certain and neither is the time-frame]. However that new model may not work with Preact. Again, we’re in a bit of a pickle: Preact users want to be carried along with Reagent and get the benefits of new Reagent work, but it may not be easy or possible to support the new API for them. Now what?

Consider everyday development on Reagent. Reagent’s source code is built around a very detailed understanding of React and is highly optimised. If Preact was supported too, then developers would probably need to gain an understanding of Preact too.

At the moment, Preact has one main contributor, it has been around for 1.5 years. React has many contributors. I’d estimate there are 100+ people with very deep knowledge of React. It’s been around (in public form) for 3.5 years. In general, the JavaScript community does not have a reputation for long-term support of projects. What happens if development slows/stops on Preact and the compatibility layer isn’t kept up to date? It is much harder to remove something than it is to add it. Who decides when/if to remove Preact from Reagent at a future date?

These are all hypotheticals, but I hope this demonstrates that the extra abstraction provided by supporting two VDOM layers doesn’t come for free. At the very least, it consumes extra brain cycles when testing and developing Reagent, extra support and documentation costs from users wanting to use one or the other, as well as extra indirection when running and debugging apps using Reagent.

The Innovators Dilemma

If you haven’t already, I highly recommend reading “The Innovators Dilemma” by Clayton Christensen. One of the key points he makes in that book is the difference between integrated and modular products and when to develop each kind.

CHRISTENSEN: When the functionality of a product or service overshoots what customers can use, it changes the way companies have to compete. When the product isn’t yet good enough, the way you compete is by making better products. In order to make better products, the architecture of the product has to be interdependent and proprietary in character.

In the early years of the mainframe computer, for example, you could not have existed as an independent contract manufacturer of mainframe computers, because the way they were made depended upon the art that was employed in the design. The way you designed them depended upon the art that you would employ in manufacturing. There were no rules of design for manufacturing.

Similarly, you could not have existed as an independent maker of logic circuitry or operating systems or core memory because the design of those subsystems was interdependent. The reason for the interdependence was that the product wasn’t good enough. In every product generation, the engineers were compelled by competition to fit the pieces of the system together in a more efficient way to wring the maximum performance possible out of the technology that was available at the time. This meant that you had to do everything in order to do anything. When the way you compete is to make better products, there is a big competitive advantage to being integrated. … In order to compete in that way, to be fast and flexible and responsive, the architecture of the product has to evolve toward modularity. Then, because the functionality is more than good enough, you can afford to have standard interfaces; you can trade off performance to get the advantages of speed and flexibility. These standard interfaces then enable independent providers of pieces of the system to thrive, and the industry comes to be dominated by a population of specialized firms rather than integrated companies.

I would argue that we are still very much at the point where the current VDOM libraries aren’t good enough yet. They aren’t yet ready to be commoditised, and the best option is to tightly integrate.

Options from here (with some conjecture)

  1. Someone can make a PR to Reagent to add support for Preact. It will probably take a while to get merged because it is a significant change. Once it is merged and released, there will probably need to be several rounds of revisions before it is ready to go. Because Reagent moves relatively slowly, this will take a while.

    Reagent also has a large number of production users, so new releases need to be well tested and stable. Adding Preact to the mix is going to slow this down further.

  2. Someone can make a fork of Reagent (let’s say it’s called Preagent). You can run wild experimenting with what is the best way to use Preact in Preagent, take advantage of all of the great features Preact has, and have a much faster turnaround time for releasing and using it. You will be able to work out what is the right API and integration points for Preact because you have room to experiment with it, without the weight and responsibility of bringing the rest of the Reagent users along with you.

    At some point in the future, you could review merging Preagent back into Reagent, given all that you now know. You would also have the weight of evidence on your side where you can demonstrate the benefits of Preact and can show how many users want Preact. This would let you make a much better case for including Preact, give you what you want in the meantime, and likely provide a higher quality integration in the future.

    Alternatively, you may decide that Preagent is better served going its own way and integrating more closely with Preact. This is also a good option.

Abstraction is not free

The point I have been trying to drive through this post is that abstraction is not free. Over-abstraction is a common anti-pattern and it saps productivity. I had a friend who recently left a Clojure job and started a Java one. He quipped to me about how he’d forgotten what it was like to trace code through five layers of abstraction to get to the concrete implementation. As programmers, we’re trained to solve problems by adding layers of abstraction, but that isn’t always the best way to solve the problem.

Lastly, this isn’t a personal attack on you or your ideas. I’m all for innovation in ClojureScript webapps and I think that it is worth investigating Preact and how it could work in the ClojureScript ecosystem 😄. I’m not against Preact. I would consider using it if there was a measurable benefit. I’m just suggesting that the best way to go about this is probably not to integrate it into Reagent as the first step.