For the last two months, I have been busy searching for a new job, unfortunately. But also, because of this, I was able to evolve on lots of projects that I have been working in a while. So in this quick post I want to explain a little bit what I did, what is happening, and hopefully somebody might be interested in some very interesting improvements.
First thins, I already wrote about Chlorine in my last post. This week I was able to remove all the dead code that I had in the previous version, and I’m closer than ever to publish a new version of that I will call “Version 2.0”. Unfortunately, this version will be lacking some features that the previous version had, and I’m not really sure if I’m going to add this back or not.
But the thing is, because the new Chlorine is more configurable and the config file is easier to write, I hope people can add these features in user-space, so I can decide later if I’ll re-include them or not (it’s always easier to add stuff than remove what people already depend on).
Pulsar
While I didn’t advance too much in Pulsar, there were some improvements. First, we might be close to kiss goodbye to IndexedDB as the backend for saving the editor state – specifically IndexedDB was used to save individual files that were open in the editor, changes that were not saved yet, etc.
The reason for the change is how fragile IndexedDB is – Electron bumps can cause the state to disappear, we can’t open two Pulsar instances at the same time in two different Electron versions (sometimes even at the same version), and we can’t backup the database; finally, if something is wrong, we can’t even debug the problem.
The new backend is in sqlite3 – a know database that it’s easy to handle and debug, and backup. It’s also easier to change, so if someone wants to open a Pulsar with a specific configuration, for example, it’s now possible.
Finally, there was another Electron bump at the experimental branch (where I fixed a new issue), and I am close to publish a fork of Hydrogen, so people can use the new Pulsar in the latest Electron to handle Jupyter notebooks.
Generic-LSP
It is quite amazing how freaking hard is to make autocomplete work. Surprisingly, the “difficult” part is to know what to replace – for example, suppose you have the word hey-there in the editor, and you trigger an autocomplete. There are many ways that autocomplete can handle suggestions: C++ offers a “replace range” (that is the easy one, unfortunately Pulsar still doesn’t fully support this option) so you know where you should replace; Clojure and Ruby offer just the suggestion, but in Clojure, hey-there-person is a possible suggestion, and that means you need to replace the whole part – hey-there – with the full suggestion, where in Ruby, you would replace only there.
This might seem straightforward – supposedly, you just replace whatever a “word” mean in the language, right? Well… no – few people that I know would consider Foo::Bar as a “single word”, and yet, in C++, it is what is expected to be replaced for Generic-LSP…
There are also worse problems – in some cases, we need to keep some info from the “completions” to ask for “more info” – and this is also unsupported in Pulsar, so I used a cache. In fact, I had to use some very weird ideas to make the whole “what do I need to replace” work, but it seems that it’s finally coming together.
Parinfer-Plus
Parinfer-plus was an experiment I did a long time ago to try to integrate Parinfer-Rust in Pulsar (in the WASM version). It was both a very successful idea, and a clumsy one. It is faster than the original Parinfer and I did support some stuff they didn’t have (for example, the ability to paste text, in some cases, and have it be reformatted) but I found that it is painfully slow when the file is too big (like, 3,000 lines for example). The reason is simple, and it might surprise: WASM.
WASM conversion is slow. To convert a JS object to WASM, and back, is too costly. So I tried to make Parinfer-Plus only handle the “top block” of the app. It also didn’t work well – it’s actually pretty hard to capture the “block” in a way that accepts incomplete or incorrect text (which you inevitably will have when you’re inferring where a parenthesis need to be). That… is bad, but the worst part is that it simply didn’t “fail and stop inferring”, it actually “inferred wrongly” – like, never closing a paren – and literally nothing I could do would convince Parinfer to infer it…
… so I’m back at the “slow” version, but I found some other stuff – that Parinfer and Paredit do conflict with each other when I’m using VIM-Mode. Great… so now the next step will probably be to make a single package that both infers AND is a paredit. Maybe even re-use some of the Chlorine code to be able to infer partial stuff. Not many good news here, but at least it’s a start.
Lazuli (name can change)
I have started to work on a Chlorine port for… Ruby. It might never work, but I’ll try (especially because my next work will probably be Ruby indeed).
It might seem strange, but the idea is to use Tree-Sitter to capture “top-blocks” in a way that makes sense. Considering that Ruby is “patch-able” – meaning that I can evaluate parts of a class – maybe I could “extract” only a single method, and evaluate as if I’m defining only that method on a specific class – something like:
module Foo
class Bar
def initialize
@a = 10
end
def hello
puts "Hello"
end
end
end
If the cursor is at the same line as puts ..., maybe I can extract the whole class around and send something like:
module Foo
class Bar
def hello
puts "Hello"
end
end
end
Instead of the whole class, and that will be a “top block”. For autocomplete, I’ll probably use the fact that instance_methods exist, etc… it might not work, or it might be clumsy (probably it will) but I’ll try anyway. Hopefully things will work!
Future work
I want to make Squint’s REPL work with Chlorine; I also want to re-add support for nREPL implementations that I removed now with Orbit. Finally, I am thinking about implementing nREPL for Clojerl. Hopefully, this will be possible.
Or… I might get a job in the meantime, and that will become backlog for a future time. Which is, honestly, what I hope for…