Implementing a nREPL Client

Some people asked me on how did I implement the nREPL client on Chlorine. So, I’m writing this post!

nREPL is a simple protocol. It uses “sessions” that are used to isolate evaluation contexts and other things (for example, on Chlorine every connection connect to two REPLs: one “primary” and one “auxiliar” that is used to run commands like autocomplete / goto var definition, and so on). On nREPL, this isn’t necessary: you just connect to a single REPL and use two different sessions. Just for the record, because the way Chlorine works, I didn’t implement it like this (because I would have to rewrite lots of code – maybe in the future).

Now, to explain the protocol, let’s separate things in parts: the first thing I do (and I was already doing in the past) is to connect to a socket. Then, I looked at the documentation for nREPL to understand how to send and receive commands to the REPL. Now, there are some details on the way that every operation is implemented that I simply ignored because it was not necessary to understand then from the perspective of my application…

A “handler”, on nREPL, is some operation. So, for example, to create a new session you issue a clone operation. To evaluate, the command is eval, and to interrupt, interrupt. And these are the three only options I’ve implemented, so far, as they are sufficient for Chlorine to work.

To issue these operations, you need some kind of “transport”. By default, nREPL supports “bencode” – it is a really simple protocol that serializes integers, strings, lists and dictionaries. And, that’s all. So, benconde in a nutshell:

• To encode integers, you put i<your-integer>e
• To encode strings, you put the number of bytes of the string, followed by : and the string. So, to encode somestr you will put 7:somestr. To encode naïve, it’s 6:naïve (because the ï is 2 bytes in UTF-8)
• To encode lists, l<each-element-serialized>e
• To encode dictionaries, you put d<key><val><key><val>...e

And that’s all. Because of the nature of sockets, you can (and probably will) receive fragments of a command, so you need to somehow cache the results until you get the full serialized object. It’s so simple that the full code is less than 100 lines!

So, to evaluate code, you first need to create a new session – Chlorine does it by issuing a {"op" "clone"} to the REPL (or, on bencode format, d2:op5:clonee). Then, if this command does not return what is expected, it’s not a nREPL but a regular Socket REPL one; but, if does return what’s expected, then the result will come into a map format, with a new-session value. Chlorine will cache this session-id and use it to evaluate code. Now, the eval operation does not need a session-id, but it becomes easier to interrupt evaluations if you do have one. So, every eval command will be sent using the format:

--> {"op" "eval", "session-id" "the-id", "code" "(+ 1 2)", "id" "eval1"}
<-- {"id" "eval1", "value" "3", "session" "the-id", "ns" "user"}
<-- {"id" "eval1", "status" ["done"], "session" "the-id"}


; To interrupt:
--> {"op" "interrupt", "session-id" "the-id"}
<-- {"id" "eval1", "status" ["done" "interrupted"], "session" "the-id"}

Now, the results will come with keys out (for a STDOUT print), err (for a STDERR print), result (for an evaluation result) and ex (for an exception). Also, in case of an interruption, status will contain interrupted. Finally, Chlorine is ignoring the done status. Obviously, every line here is bencoded, but it could be not – and that’s where things become interesting! If the server supports, you can serialize your messages in EDN, JSON, Transit, or whatever – the protocol does not care.

Now, for next versions, I’m thinking of a way parse the result of an evaluation in a better way. Currently, things can break if we try to parse results from the value field. I’ll probably “invent” a simple way to serialize these results, so maybe I can even lazily parse then – so, big results inside Atom will not lock the editor, on any REPL implementation!