REPL-Tooling Clients

Chlorine, Clover and Clematis are all implementations of the same library: REPL-Tooling. In this post I will show you how to create a new implementation of it in a way that’s completely disconnected from any editors, so you can grasp the general concepts.

Suppose I want to do an implementation for an editor that doesn’t run JavaScript – so it’ll connect by some kind of socket. In this example I’m going to use WebSockets because… why not?

We’re going to create a shadow-cljs node project and add repl-tooling as a dependency. We will also had some more dependencies: mostly ws for websockets and the same react libraries that we use for reagent (react, create-react-class and react-dom) – repl-tooling still needs reagent, and probably in the future I will split it into two different libraries (one for the REPL handling and other for the visual rendering part). This supposedly is not to much of a problem because ClojureScript compiler will probably remove these parts in the dead code elimination process anyway. So, our package.json file will just be like this:

{
  "name": "ws-repl",
  "devDependencies": {
    "shadow-cljs": "^2.8.83"
  },
  "dependencies": {
    "create-react-class": "^15.6.3",
    "install": "^0.13.0",
    "react": "^16.12.0",
    "react-dom": "^16.12.0",
    "ws": "^7.2.1"
  }
}

And our shadow-cljs.edn file:

{:source-paths ["src"]

 :dependencies [[repl-tooling "0.4.0"]]
 :builds {:node {:output-to "index.js"
                 :target :node-script
                 :main ws-repl.core/main}}}

The first step is when someone connects to the WebSocket. Then, we’ll just create a connection to the client, and send a list of supported commands – for now, is just the “connect” command:
(more…)

Ubiquitous interface – how to integrate things in Clojure

Inspired by this thread on Reddit, I decided to write a little bit about my experience integrating things in Clojure.

The first thing to understand is that Clojure have an ubiquitous interface: EDN. And it is important to understand what this means. In the beginning, I made this mistake of “Death By Specificity” on my now abandoned Relational project: to abstract things that don’t need to be abstracted.

But can we do even better? How about we de-abstract (concretize? Is this a real word?) things that are already abstracted?
(more…)

(check (my-code) => (needs :tests))

So, yesterday I made a talk (in Portuguese only, unfortunately) about the difficulties of testing Clojure and ClojureScript code. Specifically, I think the most problematic issue is the lack of “custom matcher libraries”, and how the default error messages are kinda bad and don’t help you identify the problems.

Then, on Clojurians’ #announcements Slack channel, I found that clojure.test Expectations library have a new version. So, why not integrate it on my Check library, and maybe continue developing it?

What is check?

Midje is too magic. Clojure.test is too little. Thinking about findind a “middle ground” I’ve started the “check” project, and I’m using it to test my personal projects like Chlorine, Clover, REPL-Tooling and Paprika. The problem is that, while the API is stable, but it still doesn’t do all the things I want.
(more…)

Why I tend to avoid core.async?

It’s no surprise that I don’t like core.async very much. For starters, it make my functional composition looks like imperative programming again. There’s also multiple issues that you need to be aware of (like, don’t use async/put! because you will have problems, deadlocks that are difficult to predict, go blocks don’t compose over functions so you loose lots of helper macros like delay).

But the most important reason is that most of the time, I’m working in ClojureScript. And it’s impossible to migrate callback to core.async.

Well, you may be tempted to write something like:

(js/someFunction "i'm async" "lol" #(async/put! some-channel %))

And one day or another you’ll have the dreadful Assert failed: No more than 1024 pending puts are allowed error. There are multiple ways around this problem, but none of then work if you can’t lose messages.
(more…)

Quick Post – native apps with ClojureScript

React apps now run on the desktop with Electron. They also run on the desktop with react-native-desktop, or proton-native. In this simple post, I’ll share my (very small!) experience with proton-native:

First, you need Shadow-CLJS. Because of the npm integration, you can literally just:

npm install proton-native react react-dom create-react-class

And then add [reagent "0.8.1"] to dependencies on shadow-cljs.edn file. Then, configure a :node-script target and start to build things. This means that your shadow-cljs.edn file will be:

{:source-paths ["src" "test"]
 :dependencies [[reagent "0.8.1"]]
 :builds {:desktop {:target :node-script
                    :main demo.app/init
                    :output-to "target/index.js"}}}

(more…)

My Atom editor configuration for working with Clojure/Script, revisited

Sometime ago, I did a post on how I work with Atom to develop Clojure and ClojureScript projects. It is in Portuguese, so I’m gonna re-visit the subject and also update with my current workflow.

There are two packages that I have to install to work with Clojure: lisp-paredit and chlorine. Without lisp-paredit, when I start a newline, the indentation gets all sorts of problematic. I use it on “strict mode” and use the tools to slurp/barf forward only. As for chlorine, it is needed to have autocomplete, evaluation, show documentation, goto var definition and so on. Last, I use also parinfer so I can remove whole lines of text and parinfer will infer the correct closing of parenthesis for me (most of the time at least).

Now, how exactly do I work with Clojure? When you use lein or boot, you’ll get a nREPL port. This is not the port you use with Chlorine, so I need a bit more of work. I can’t just start a REPL with lein repl or clj, I need to inform the tool to open a socket-repl server. The JVM option needed is: '-Dclojure.server.myrepl={:port,5555,:accept,clojure.core.server/repl}'. So, the commands below are what I use with lein or clj:

JAVA_OPTS='-Dclojure.server.myrepl={:port,5555,:accept,clojure.core.server/repl}' lein repl

or

clj -J'-Dclojure.server.myrepl={:port,5555,:accept,clojure.core.server/repl}'

This will open a REPL at port 5555 (or I can change the port if necessary). Then, it’s time to fire up the Atom’s command palette and select “Connect Clojure Socket REPL”, put 5555 on the port, and connect. Then, I’ll use “Refresh Namespaces” or “Load file” command to load my latest version of code into the REPL, and start working.
(more…)

Chlorine, Clematis, REPL Tooling

Recently I began to migrate some of the code on Chlorine to REPL Tooling, so things may become more testable. I already found multiple bugs in this approach and began an integration test methodology using Electron (I say that there are three kinds of developers in the field of automated tests – the ones that don’t test, the ones that do, and test freaks. I’m probably the later).

In the beginning, the idea would be to rewrite the renderer of results and exceptions so that I could fix lots of bugs that they have. So far, it’s progressing slowly, but I already can render datomic results in a meaningful way (the last renderer was interpreting as Java objects, which they indeed are) and fix some bugs with tags and other issues.

Now, one of the things that I wanted in the new renderer is the ability to copy/paste results in a meaningful way – as I’m using UNREPL, there are lots of tags that need to be reinterpreted, like #unrepl/browseable for datafy. These need to be “translated” to something useful for rendering on screen, with links to navigate inner details, but for copy-paste they should have a textual representation that makes sense (probably the hard part).

The results of all these experiments are quite interesting – when you evaluate something and th result is a Java object, there’s a link ... that can be used to navigate inside the object: it’ll show getters, the current class, methods that the object supports, so things like “goto definition for Java object” or “Javadocs” aren’t really necessary.

Then, one of my friends asked for a Vim option. Just for fun, I started Clematis, a port of Chlorine to NeoVim. And it seems they things are progressing faster than I thought!
(more…)

Chlorine is thriving!

When I started this project, I was experimenting with shadow-cljs to see if I would be able to make an Atom package that would auto reload, run tests on ClojureScript, and se how far could I push ClojureScript in an Atom package.

Now, some months later, I’m seeing the package being used by a bunch of people, and I even discovered some bugs in UNREPL! Now, on this post, I’ll discuss a little bit more in detail the design decisions and my vision on the future of the project.

Chlorine is a clojure and ClojureScript atom package. It connects to a socket repl (opened via lein, boot, clj, shadow-cljs, lumo, or even REBL) and then upgrades the repl to be programmatically oriented with unrepl. Unrepl only works on Clojure, so for ClojureScript we use other techniques. Also, socket repl is a stream protocol, so to emulate a “request-response”, we need to coordinate things so Atom (and other editors) can react to commands and know exactly what’s the correct response for each command sent.

Design decisions

The choice for UNREPL was mostly because there is almost no documentation about prepl so far. Also, Socket REPL is literally everywhere: on Clojure , on ClojureCLR, on Lumo and Plank. Also, I wanted a better way to use ClojureScript, and I still have nightmares trying to use it over nREPL… and with Socket REPL things work fine.

Also, when I started the project Clojure 1.10 was just alpha software, and UNREPL offers us insanely good support for lazy lists, big strings, and other things that I wanted to use out of the box. One of the problems I’m still facing is coordination of evaluate/response, but this will probably be solved after a bunch of other fixes I’ll try.
(more…)

My fading frustration with ClojureScript

I’ve talked about at another post on how ClojureScript frustrates me, mostly because I was doing some Node.JS work and Figwheel simply wasn’t working correctly. Now, it’s time to revisit these points:

A little update: I talked a little with Thomas Heller, Shadow-CLJS creator, and he pointed me some issues with this article, so I’ll update it acordingly

Tooling

Figwheel and Lein are not the best tools to work with ClojureScript. Since I discovered shadow-cljs, things are working way better than before: I can reload ClojureScript code from any target, and I’m even experimenting with Hubot (and it works really fine too). The only thing I’m missing is my profiles.clj file, but I can live with that (and I can always use Shadow with Lein if I need profiles.clj).

Also, I’m working on a new package for Atom (and in the future, for another editors too) called Chlorine. One of the ideas is to offer better ClojureScript support (we have Autocomplete now!), using Socket REPL for solutions (even self-hosted REPLs like Lumo and Plank) and even wrap UNREPL protocol in Clojure. So far, is still in the very beginning but things are looking promising!

The stack

Forget Figwheel at all: Shadow-CLJS is probably the best tooling for ClojureScript ever. It auto-reloads ClojureScript code for the browser, for node.js, for node modules, and it integrates with almost everything that you want. It controls release optimizations, have sensible defaults, and even have post-compile hooks (so you can hook Clojure code to do something after some compilation phases). Also, it integrates with node-modules (no more maven-wrappers for JS libraries!) and have some warnings when you use some kind of ClojureScript code that would break :advanced compilation. And, let’s not forget that you can control the refresh reload phase, it adds a userful :include-macros in ns form (that will include all macros from the namespace being required), and controls exports in a sane manner. But first let’s begin with the feature that I found most useful: :before-load-async.
(more…)