A new Chlorine is almost done!

For the last few months, I’ve been working on a new version of Chlorine (now that Pulsar is on a good track). Here I want to share what to expect of this new version.

For a quick TL;DR; it’ll have better ClojureScript (shadow-cljs) support, fix some bugs related to evaluating code and the REPL being “locked” without returning the results or evaluating new commands, and have better configurable parameters. Things will probably break a little bit – it’s not a non-breaking-change, unfortunately – but hopefully it’s good enough that people will want to migrate to the new version.

NRepl Embrace

Socket REPL never gained traction; prepl also didn’t. Chlorine will drop support for Socket REPL, and with that, will also drop support for Clojerl and Clojure.CLR (for now). CLR support needs to fix some issues with their nREPL server (mainly because we can’t evaluate reader conditions like #?(:clr ...) and Chlorine uses quite a lot of these) but I hope this change can bring us better support with modern tools that have nREPL middlewares. The new Chlorine also allows us to evaluate nREPL operations so if one wants to integrate with some middleware, it is now completely possible and easy.

For Clojerl, I’m thinking about writing a simple nREPL server so that Chlorine can use it instead. The old Socket REPL support for Clojerl was incredibly buggy anyway.

Shadow-CLJS support

“Old” Chlorine connected to a Socket REPL, then we had to connect to “embedded ClojureScript”. This confused people, and worst yet, sometimes it didn’t work, and nobody knew why.

On this new version, when we connect to nREPL (the only option now) it will connect to Shadow too. But it gets better – it will also detect which builds are running, if they are compiling or not, and it’ll detect which Javascript runtimes are connected to the build – that means, if you have two, three browsers open in the ClojureScript code, you can choose where you want to run your code (hopefully, in the future, I can add a way to make it run in multiple runtimes at the same time!)

Also, errors now appear inside the editor (they already did, but it was behind a feature flag) so you can see if you have an error in your code, and quickly jump to that code. It also automatically resolves Javascript promises, even promesa ones (again, it was behind a feature flag in the past) and if the promise ends up with an error/rejection, it will also display it in a red color to explain it’s an error (this one is new).

Bugs fixed

A lot of old problems happened because Chlorine tried to use Socket REPL. Evaluating Datahike code locked the plug-in; evaluating Datomic code could lock the plug-in; evaluating Babashka code could lock the plug-in; the list goes on…

Another issue is with autocomplete. In some very rare cases, it was possible for autocomplete to stop working if compliment (the library) was removed from the classpath after a REPL reload. This is also fixed.

Finally, there is a very interesting edge-case that was fixed too – when evaluating code in Chlorine, sometimes we got into a situation when we need REPL information to do stuff. Most notable situation is when we evaluate “interactive results” and then want to do something else. In Chlorine, this meant that interacting with the result inside the inline result – the one that’s below the code you evaluated, inside the text editor itself – would be evaluated in the namespace of that editor, for example; whereas interacting inside the console would use the user namespace. This was not so bad, with the exception of… exceptions. Clojure stacktraces sometimes say core/some-function.clj. So… what is this core/? Well, it depends on the namespace you’re currently in. Sure, so what’s this namespace? Well, in inline evaluations, we knew what it was. But in the console tab…

Newer Chlorine fixes this by making everything dependent on the Pathom library. Sound strange, but if you saw my post about Pathom you know that it’s possible to define the “initial entity”. So what we do is simple: before evaluating anything we get the current context – REPL, evaluator, editor, namespace, contents, etc – and then we “cache” this. Newer evaluations will always use this cached entity, so in a way, the console tab’s result knows the previous state of things.

Finally, promise resolving now shows errors. In the past, you simply never got any result, meaning that it stuck in “pending” status forever. Problem is that considering we do show when it’s resolved, then it was actually misleading. Another thing is that now errors are “localized” – in older versions, either the whole result was an error, or it was not; now, each element can be an error, like it appeared in the previous screenshot.

Finally, it wasn’t really a bug, but the console tab is faster. Like, way faster – because I figured out (too late, I guess?) that everything being rendered at the console is just text (sometimes ANSI char codes) so why the hell I was using Reagent for that?

Bugs added, and features removed

Socket REPL support is gone, unfortunately. For Javascript, some inspection things like showing the methods, attributes, etc of pure JS stuff is also gone (but probably will be re-added later). All the REBL stuff is also gone, together with the “reload on save” (and reload in general) – these need to be configured via config. It’s better this way – most apps reload in a different way, so it’s difficult, close to impossible, to have a sane “one size fits all” situation

Because Chlorine will only support nREPL, this also means it’ll need a new serialization protocol. I didn’t find any errors on the serialization that I invented for this process, but that was not battle-tested yet, so it remains to be seen if this will have bugs or not (who am I kidding, of course there will be).

An unfortunate thing is that nREPL is not “generic” anymore, meaning we can’t connect to things we did support like HyLang and others. Probably these will be re-added later, though…

Finally, a not-so-surprising thing – this new version will be Pulsar-only. You won’t be able to use it in Atom, even if you installed from source. I’m using new APIs that are only present on Pulsar.

Config

The config is being massively changed. It’ll support multiple namespaces, it’ll support some arbitrary JS imports and requires, it’ll have namespace support, etc. But that’s not the most exciting thing… the most exciting thing is that it’ll be REPL-connected.

That’s right – autocomplete, doc for var, evaluations, they will work inside the config files and they will render exactly the same as Clojure/ClojureScript, but it’ll run the internal interpreter (SCI) that Chlorine uses for evaluation. That opens up the possibilities for multiple amazing things, like being way easier to make interactive results and other stuff.

Another amazing thing is that now the interpreter shares the config file with the interactive results. That basically means if you def something inside the interactive result, it’ll be available at the config file as a var – which translates, obviously, for better debug support. Another amazing thing is that we can use arbitrary Node.JS libraries inside interactive results and even function that were defined in the config files (like helper functions) in our interactive results. This opens the door for multiple possibilities, and it’s something that was bothering me for a long time (that being, Chlorine having this amazing “configure everything you can!” feature but being actually hard to configure stuff). Finally, this also mean that UI components can be written in the config file (or even redefined to look different) and then be used in the interactive results!

Think about, for example, a component that renders a map, but prioritizes some fields. This could be used in place of the default “map” implementation, so that it’s easier to read huge maps!

Future work

Right now, I’ll probably publish Chlorine with the features mentioned above. In the future, I’ll probably try to integrate Kindly-Advice so that we could, inside the config, configure that every map that have some specific situation will use a different render – and that is possible now because of the rework in the Console tab.

Also… Portal. Portal renders EDN in an amazing way, so I’ll probably add an option to use Portal instead.

But what I think it’ll be one of the more interesting things I want to do is to add FlowStorm support. Basically, I am still figuring out the details, but the idea is to have “tracing” inside Chlorine. Imagine LightTable’s watch expressions, but better – something that can trace the whole system without being explicitly told to do that, and then show the results inline, close to the code, displaying what a function received, what it returned, and all the intermediate steps if possible. Better yet if we can, somehow, put inside the console tab the actual code path that the evaluation took – so if we decided to record something, and then made some action in the actual app running somewhere else we could see exactly what code ran in that moment. Hopefully I can get a prototype sooner than later.

And finally – nREPL again. There are many languages that are offering nREPL support. I want to check if I can integrate something incredibly different, like Ruby for example, maybe not in Chlorine but a new plug-in that shares most of the code with it.

The future is bright, and I am really excited for this new milestone!