When I started this project, I was experimenting with shadow-cljs to see if I would be able to make an Atom package that would auto reload, run tests on ClojureScript, and se how far could I push ClojureScript in an Atom package.
Now, some months later, I’m seeing the package being used by a bunch of people, and I even discovered some bugs in UNREPL! Now, on this post, I’ll discuss a little bit more in detail the design decisions and my vision on the future of the project.
Chlorine is a clojure and ClojureScript atom package. It connects to a socket repl (opened via lein, boot, clj, shadow-cljs, lumo, or even REBL) and then upgrades the repl to be programmatically oriented with unrepl. Unrepl only works on Clojure, so for ClojureScript we use other techniques. Also, socket repl is a stream protocol, so to emulate a “request-response”, we need to coordinate things so Atom (and other editors) can react to commands and know exactly what’s the correct response for each command sent.
Design decisions
The choice for UNREPL was mostly because there is almost no documentation about prepl so far. Also, Socket REPL is literally everywhere: on Clojure , on ClojureCLR, on Lumo and Plank. Also, I wanted a better way to use ClojureScript, and I still have nightmares trying to use it over nREPL… and with Socket REPL things work fine.
Also, when I started the project Clojure 1.10 was just alpha software, and UNREPL offers us insanely good support for lazy lists, big strings, and other things that I wanted to use out of the box. One of the problems I’m still facing is coordination of evaluate/response, but this will probably be solved after a bunch of other fixes I’ll try.
So, first thing I’ve tried was to find some kind of library that could parse UNREPL protocol, some kind of “client” to a socket repl+unrepl “server”. The problem is that I needed it to be in ClojureScript, that’s the language that most editors run today, and found none. Then, I’ve started a project called “repl-tooling”, and while working on it I simply lost track of everything. The code was a huge over-engineering and I simply couldn’t see the end of it.
Then I simply changed approaches-I fired up Atom and generated a package. Deleted all Javascript code and configured shadow-cljs to build a node library. Then, I configured on package.json the main file, and began hacking. Now, that was much better, and had better results. There are still very bad code on repl-tooling just because I still didn’t fix the over-engineering that I began…
Release often, release fast!
I would like for Chlorine to be better written, with more tests, before I could publish it. But one day, I was writing a ClojureScript project for my current job and was using Chlorine-and, after a while, I realized that my support for ClojureScript was so solid that I hadn’t had any problem and I didn’t even realize I wasn’t working in Clojure!
Then, I published a very unstable 0.0.1 version. Fixed lots of bugs, and published a 0.0.2. I think that on 0.0.3, Sean Corfield saw a thread on slack and made lots of PRs to support the workflow he uses, and then helped with bugfixes and deprecations. He also told me that Chlorine is working fine and he even pushed support to work with windows and REBL.
As a programmer, some times I want to be proud of the code I write. In practice, what matters is the end result. The code is not beautiful, but it’s usable; there are still lots of bugs, but it works most of the time to be useful. So, why delay the release? Every new publish of Chlorine adds a little more functionality, a little refactoring, and I can learn what works or not, in practice.
Integration Tests
Most Atom packages I see have little to no tests. I’ve been there multiple times too, and the idea behind it is simple: it’s really hard!
So, what I’ve done is an integration test, with Node.JS’ spectron library that fires up a real Atom instance and then tests everything that appears on the screen. This means that tests run MUCH slower and that I have multiple false-negatives. But then, with Docker and a CI server (Travis, for now) I can test that release builds are working, and if something fails I can retry the build to see if it was a false negative. Sounds like a lot of work, but this inspires me to (1)-put most code that I want to unit-test on repl-tooling (in that way I’ll be able to reuse it from multiple editors, so that’s what I want anyway) and (2)-keep PRs and other things safe from simple breakages (I never published a package version that broke evaluation. Not even once).
This approach made me capture at least four errors that were merged on master that would break how exceptions are shown and how evaluation on CLJS works. They are a bit of a pain to run, but it pays off.
Unit tests
There are almost no unit tests on Chlorine. This is expected: I want to push most, if not all, code on REPL-Tooling library, and that one is unit-tested. The idea is that every editor should use repl-tooling and that we’ll have extension points to editors in the future.
Project Future
In the future I intend to port all code that currently Atom uses to REPL-Tooling. Then, I’ll probably port it to other editors like VS Code (or VS-Codium), to see if the idea of a library with all tooling works correctly. There are also some other things that are on radar:
Discovery
This is still not implemented, but the idea behind REPL-Tooling is that all refactoring libraries like Cider could be detected when we connect to REPL, and then we can register on Chlorine commands for everything. Probably the API should be something like “connect to REPL” => “Retuns a map with supported commands” or “Returns a Core.Async channel that’ll allow us to register commands”.
Some commands will always be present: “evaluate-aux”, “evaluate”, “break” and “disconnect”. The difference between “evaluate” and “evaluate-aux” is that the former is to run user’s code, and the latter for tooling. Because of the way UNREPL works, we always need two connections so that we can cancel a pending command; we use AUX for tooling too, for example, to refactor code, autocomplete, refresh, and other things.
Other editors
I will not lie: I’m unsure about Atom’s future. On my work’s machine, one with 8gb of RAM, Atom alone consumes about 8~10% of RAM, more than my JVM and my browser. Also, with Microsoft buying Atom, I have serious doubts that they’ll support both editors. I don’t think Atom’s end will be the same as LightTable, but I think it’ll be always a “second class citizen” on MS world.
So, the first thing I’ll do after porting most commands to REPL-Tooling will be support VS Codium (an open-source fork of VS Code). Then I can see what problems it brings, and maybe solve some of then.
Code Exploring
I want Chlorine to be a tool for exploring. I’ll see if I can add some debug commands, or watch expressions, or something like this. I also want to make it a tool that will help write code: so, refactorings, detection of unused imports, linter, they are all on the long term radar.
But one thing I really want to do is to allow users to see Java and Javascript objects. For example, when I evaluate a code, if it returns a Java object, I want to show getters, maybe setters, methods, and in some way help programmers to go right into the code without needing to rely on documentations, javadocs, goto definition that will use javap to parse a Java representation of that object… I want it to “just work”.
This also could apply to Clojure objects: integration with spec/schemas is something I’m planning. Also, other things that can help are automatic definition of a nested part of code (so that we can explore the inner workings of a deep nested map, for example). I still don’t know how far can I push this, through..
User extensions
I don’t want Chlorine to be just “works for me™” software. I would love for users to extend the functionality of result exhibition.
Let’s suppose a code returns a Hiccup data structure. Would it help to see it as vectors? Probably not. But we’re on a rich editor platform, so we can render HTML! Maybe add a tab on inline results that renders the result as HTML. Or as a react component. The possibilities are many, and the idea is to allow users to register new visualizations if possible.
Conclusion
Chlorine is just on the beginning of its life. There are still multiple things that I want to try, and there are still lots of things to explore.
When I started the project, I never planned REBL integration, for example, and now it works correctly. I also never imagined that I would use it so early in the development process, and again, I’m using it even on ClojureScript (with a self-hacked autocomplete too!), even to develop Chlorine itself. So, things are pretty stable, and we’re still ust in the beginning.