It’s been a while since I’ve been working on Lazuli – the editor plug-in for Pulsar (and probably VSCode in the near future) where I try to bring the Clojure REPL experience – that is, running code inside your editor and printing the result inside your editor, without having to copy-paste between two different tabs – to other languages.
If you didn’t read the previous post, the TL;DR; is: Lazuli is basically “Chlorine, but for more languages than Clojure”. Ruby was the first target, for two reasons – first, because I decided to go back to the Ruby/Rails stack; and second, because it’s probably one of the easiest languages to implement that on.
Since then, I’ve added support for Python, and I’ve been experimenting with JavaScript and Elixir. Each of them ended up being its own can of worms, and I think it’s worth writing down what I found – because some of the “obvious” solutions turn out not to work at all.
A quick refresher on watch points
I explained watch points in the previous post, but here’s a very short version: a watch point is a binding around a method or function – all the local variables, self/this/whatever the language calls it, class variables, and so on. When you evaluate something in the editor, the nREPL tries to match a watch point at that line, so if you evaluate self.some_method it knows where you are – the instance, the class variables you have defined – and it evaluates the code as if it was running inside your already-running process, be it a webserver, an app, etc.
I originally called them “watch points” precisely so I would not bind the name to a single language. The concept is the same everywhere – only the implementation changes. And that’s what this whole post is about – the different levels of pain each language gives you when trying to implement them.
Python – the easy one
At my job I started to work with Python, so I did the same thing I did for Ruby: implemented an nREPL server, and added support for parsing the code in duck-repled. Then, at the nREPL layer, I had to implement some kind of “watch point”, but using what Python offers me.
For Ruby, I used binding to define how the server captures the context. For Python, I used the frame object that the setprofile API gives you – it plays essentially the same role. Both languages support manually-defined watch points and automatically-defined ones, and they map surprisingly well to the same concept. Updating the watch points (and patch, and imports) are harder in Python because in Ruby, you can evaluate something passing binding as a parameter – you can’t do that in Python, and instead you need to convert the frame to locals and globals “dict” objects, pass those around, and then replace the watch point with the result (so that it’s not a frame anymore, but a dict containing the values). But it works, although it does feel like cheating (you are essentially faking an environment to the Python eval function) and sometimes can cause some weird issues – for example, let’s suppose I have this code:
class Example:
def m1(self, a, b):
return a + b
def m2(self, a, b):
return a - b
Supposing I have a watch point on m1, then I add an import re at the first line, before the class. That will “alias” an re inside m1, but not inside m2. Lazuli supports “patching” so I can “patch” m1 to use a new implementation, but if I do evaluate a code that adds a watch point on m2 later… it’ll gladly say that re doesn’t exist. The reason is that the “import” only exists for things that have watch points (it’s confusing, I know) because namespaces in Python don’t get profiled and don’t get their own watch points (so when you evaluate an import, it’ll only update existing watch points, not new ones that might appear later – and for the interpreter, that current namespace can’t be changed).
The printing part of Python was actually easier than Ruby. In Ruby, there is no one-size-fits-all for inspect – so many gems override how things are printed that I gave up trying to intercept the output, and instead I decided to parse the Ruby result and instance variables, properties, and everything else into a structure based on what Ruby actually gives me. Not perfect, but it works. In Python, not many people change the way outputs are presented, so the parser-based approach was way less painful (I decided to implement a structure similar to hiccup, but containing what most languages offer – so arrays and strings are supported, and nothing else. To say that it’ll print a “number” in the screen, you return ["number", "10"] for example).
So – Python: mostly a rerun of Ruby, with fewer surprises. Good.
Then I decided to try JavaScript.
JavaScript – the DevTools Protocol saves us (almost)
JavaScript is where things start getting complicated.
The whole idea of Lazuli, and of the nREPL I’m building, is that you’re running your normal code, and you just add a thin server that can somehow connect to whatever is running right now and then evaluate commands. You don’t change anything in your codebase – you add a require, import, or similar, and whatever you bring needs to be as unintrusive as possible – for example, it can’t depend on additional libraries, because otherwise you need them installed locally, and that’s not how some languages work anyway (the first version of the Ruby nREPL depended on a BEncode library, and that was a no-go – I would have to include the nREPL into the Gemfile for example).
So, what are the main problems in JavaScript? Well, you can’t just “connect into” a JS virtual machine – you need to open a server. But if you’re in a web app, you can’t start a local socket in your machine (or any server for that matter) your webpage needs to be connected to a server, and it needs to be a WebSocket. But the socket needs to be running in the server that is offering the JavaScript, so completely unacceptable (and even if it was, it would not work – it would need to “rewrite” the JS before sending to the browser, but a server can, and will, send multiple JS files – which ones is the “right one” to connect to the websocket?)
To solve that, Lazuli uses the DevTools Protocol. Both Node.js and the browsers support it – you can tell a browser that DevTools is available for your localhost machine, and, surprisingly, that just works (well, it opens a devtools channel for every page but you can just filter for the one you want and that will work). But we gain access to more things by using this protocol – one of them being the debugging API, which is actually how we add watch points to the JavaScript nREPL.
But, as always, there’s a problem. Well, actually, three of them.
Problem 1: source maps
JavaScript is not a compiled language, but most people use some kind of bundler. If you’re using React, for example, you’ll write JSX inside your JavaScript, and that will get transformed – changing lines, changing names, wrapping things.
So when you get an exception (or when you’re trying to figure out where a watch point should go), you have to parse that through a source map. And unfortunately, for the Lazuli project, we can’t escape this. There are ways to avoid minifying code – in fact, on Pulsar we actually do that – but it’s simply not how most JavaScript development works nowadays. The solution is to try to do a “reverse source map”. Essentially: if I evaluate something like a in my editor, that has to map to something that was already transpiled to the final JavaScript code, and then I evaluate that transpiled name over there. It works, but it’s flimsy and still not perfect – for example, supposing a const value = 10 gets transpiled to var a=10, then later the bundler found that this variable isn’t referenced anymore and decided to reuse const otherValue = 20 to a=20 – the “original” value, containing 10, will be lost forever, and there’s no way we can avoid this. Most bundlers don’t do that in development, luckily, but it’s still an issue that might happen.
Problem 2: functions that don’t exist
The second problem is way more complicated. And it was a surprise for me.
If you have three functions in a file, and only two of them are used, then the third one will never exist. Not “will not be called” – literally does not exist. For some reason, it seems that either the JavaScript engine garbage collects the function even if it’s in a top-level global namespace, or – more likely – it simply never actually compiles it to bytecode. For example, in the code below:
function unused(a, b) {
return a + b
}
function functionOne(a, b) {
return a / b
}
export function functionTwo(a, b) {
return functionOne(a + b, 2)
}
Now, supposing I have a watch point on functionTwo. Changing the code of functionTwo to call functionOne twice works; inspecting a and b on both functionTwo and functionOne works too. Trying to call parseInt inside functionTwo also works, but trying to call unused won’t work – ever. Even just typing unused and evaluating it will return undefined, because the JS virtual machine simply “erases” that function.
This might pose a problem for future Lazuli features – if the whole point is to be able to interactively evaluate anything, and if half of what you wrote isn’t there anymore, that’s a hole can of works. That might be possible to mitigate with the same technique as Python, though – maybe we can “fake” it by making “evaluate top block” produce a “global-ish” identifier, then update manually each watch point so that this identifier is in scope – but it’s still kind of hard to make it work.
Problem 3: namespaces and patching
And that’s not all. JavaScript is a very different language in the context of namespaces and files – there’s some magic going on around ESM that’s different from CommonJS and other stuff, and it’s close to impossible to “patch” a function. There’s a Chrome DevTools API that will try to patch things, but it only works with some very strict constraints that are not really clear what they are (and I mean very strict – for example, adding a new line won’t work).
One thing I’m thinking about is to make a Babel transformer that injects functions with their own inspect handlers, and somehow keeps a global state of these functions so that they can never be garbage collected. This might work, but I don’t yet know if it’s a good idea – but if it does, we could have Lazuli working perfectly in a JavaScript environment, with just some changes to whichever bundler you’re using (it’s not fully unintrusive as I wanted, but if it’s the only way, that might be the path).
Elixir – the interesting one
Elixir is the last language I’m working with, because I find it very interesting, and I’m checking whether it’s possible to replicate the Ruby experience there.
Some things seem easy. Some things are hard. And some things might be impossible.
Private functions are invisible
The first problem is that private functions in Elixir are not visible to my plugin. Elixir has some ways to get the bindings and the environment around a call, and context – but because I don’t know Elixir that well yet, I don’t fully understand:
- what’s the reason for using one of those over the other,
- why we have to use both sometimes,
- and how (or if) I can capture the whole context, including private functions.
For now, seems that private functions (defined with defp) are not “compiled” into the bindings and the env. That complicates things, because I can get the “local variables” part of my watch point, but if I try to call a public function, that works… but private ones don’t. I could, theoretically, re-create the private function as a “public” one while I am evaluating the watch point just to know what is the result, but then I get into the second problem:
The VM is immutable, kinda
Another problem with Elixir is that the BEAM is immutable – kinda. You can’t just patch a function and make every caller in the runtime use the new version. That literally isn’t possible at the VM level. You can have some of this approach with GenServer and other constructs, but I don’t fully believe that this will actually patch stuff in production the way REPL-driven development expects – because your code needs to be a “GenServer” already, so it’s essentially easier to just save the file and hot-reload it.
The issue is – a REPL-driven development is meant to be a way to test solutions without needing to save the file. I can write “fragments”, evaluate them, patch functions, evaluate them, repeat, until I’m satisfied with the output – then I will save the code and allow either the hot-reload of my environment do its magic, or reload the whole file and check if I didn’t introduce any bug. With this approach, a “evaluate top block” might not even be useful for Elixir, honestly.
Bindings only exist at call time
Elixir does have the bindings of a function – but only when the function is called. So you know which local parameters are passed in, you know which global ones are used, but you don’t actually know what are the variables that you create over time inside the function.
Python, with locals and globals, have the same problem. But we can bypass that by storing the frame object, and when I evaluate the code in the editor the first time then it’s converted to the dicts. Unfortunately, this won’t work in Elixir, and I’m not sure it’ll work at all, ever – even if we had some way to capture “just before the end of the function”, we would still capture only when nothing crashed – and capturing when something crashed is probably the biggest reason for a watch point now.
Shadowing
And here’s the one that might complicate things a lot: variables are shadowed. Suppose I have this code:
module Something do
def example() do
a = 10
fun = fn -> a + 1 end
a = 20
fun2 = fn -> a + 1 end
end
end
If we define a watch point on this example function, then start to evaluate line per line, we’ll update the watch point with a=10, fun=<function>, then we’ll update the a to be 20, and then define a fun2. The issue is – fun.(), in this case, will return 11 – because that was the value of a at the time – and fun2.() will return 21 for the same reason. This is the happy path… but if I want to understand why the value of fun was 11, I can try to select its inner body – that is, a + 1 – and then evaluate it. BUT – watch points are, unfortunately, tied to a single point in a specific file and line, and they are updated when we evaluate code – meaning that evaluating that selection would return 21, confusing things.
The solution could be to add some “metadata” for each evaluation, meaning that it’ll redefine where that variable was defined, and then evaluate could capture the scopes but only consider variables defined before or at the current line. I don’t think this is easy or simple to do (considering that changing the code inside the editor also needs to update where variables were defined and also the editor will need to be “deletion and change aware” (if we rewrite a to a1, what happens?) and there might be even more trade-offs that I didn’t think about. Considering that I don’t know Elixir that well that might cause such a huge amount of bugs that this might be too difficult for now.
Final thoughts
Lazuli is growing, and the project – in my view, at least – is quite interesting. It might be bringing some of the superpowers from Clojure REPL-driven development that people love to other languages.
I’m still not comfortable with the level of tests that I have, because I really want to avoid breaking existing REPLs when I update the plug-in, and vice versa – and I also want to keep adding languages, some of which I don’t personally use at work (I don’t use Python anymore, and I only use Elixir for personal experiments).
Another thing about Lazuli is that it sometimes feels like it’s moving in the diametrically opposite direction of what people want to do nowadays. Lazuli is a project to bring the code and the developer closer together – to the point that you’re evaluating code live while you’re typing it. But we live in a world of LLMs, where people want to distance themselves from the code. I, honestly, don’t think that’s the right position to be betting on – especially when LLMs are not perfect yet, and I don’t know if they ever will be. There is still a lot of misinformation, broken promises, and hype around AI-assisted coding – some people trust too much on the ability of the AI agents. Some are even comparing source code to assembly language and to the binary that runs on your computer, which is completely absurd – compilers are (supposedly) deterministic, and LLMs are not – by design.
So maybe, if the whole LLM boom proves to be a great mistake, and people end up with millions (or even billions) of lines of code that they don’t understand, and it simply doesn’t work… maybe Lazuli can help in the future.
Or maybe I’m completely wrong and I’ll move to a different position.
But here’s the thing: I still believe that as developers, we need to understand our creation – be it written by us, or by a machine.