Maurício Szabo

Putting REPL-Tooling to test – VSCode!

Sometimes you make some tooling because you want to use it; sometimes, to experiment; and sometimes to test the waters.

The last tooling in that I did was one of these cases – now there’s a port of Chlorine to Visual Studio Code called Clover!

Now, when I started the project I imagined that VSCode would not have all the features that I have in Atom, nor all the APIs that I want to use – for example I didn’t have any hope of having the inline results in this version.

The thing is, I wasn’t expecting it to be so bad! To begin with, the API: is not really that bad documented, but compared to the Atom, it is incredibly weak. The first thing is that they expect you to use TypeScript so there’s little to no documentation on how to represent objects in pure JavaScript; for example in some cases you can use pure objects, on others you have to instantiate a TypeScript class in the JavaScript code. Also, there are multiple parts of the documentation when they just give you the type signatures and little (or even no) explanation (and let’s make a little detour here: what’s the deal with some people that use static types, that they expect you to understand how any API work just by showing the types of the functions?).

The second part is that VSCode expects you to fit your plug-in infrastructure on what they offer – so, some functionalities will land on the “peek definition” API, others on “Code Lens” and so on. The problem is that they don’t explain what’s a “code lens” for example nor give you any screenshots of the functionality in action – mostly the documentation is some code examples in GitHub repositories so you have to download, install the example extension on your machine and then run it to simply understand how something works.

The second hard part is that you can’t test the API in the devtools – in fact the devtools is almost useless because when you have an error, the stacktrace will point you to a minified JS code in the VSCode internal API. Also, some exception messages are completely obscure and some log errors on the devtools but things work fine on the editor. Well, to summarize: it’s completely useless to depend on the errors.

But the worst part, at least for REPL-Tooling, is that you can’t change the UI of VSCode in any way – and this means no pop-ups, no new elements, no console in the editor, nothing – the only way you have that you can extend the visual components is by implementing a webview – and by webview I am not saying an “electrom webview” when you can access all the Node.JS APIs – I am saying a simple web page when you have to pass your data to and from the editor using JSON. And and that’s all there is – no Date, no JavaScript classes and, of course no Clojure objects.
(more…)

By Maurício Szabo, ago

Complex testing – the saga

While I’m developing Chlorine, sometimes I need to test multiple specific implementations of lots of really complicated stuff like REPL state, connection, async stuff (as the project is ClojureScript on Node.JS, all I/O treatment is via callbacks) and the complicated nature of rendering multiple different object types on Atom editor. I had multiple regression errors, then I’ve created some “acceptance” tests (these open up a real Atom editor and evaluate commands on it). The problem with these tests should be obvious: they are slow, and I mean REALLY SLOW, and they need a real Atom editor, lots of setups, and because Atom is not really predictable on its actions (sometimes you connect REPL and it changes the focus on the editor) there are lots of unnecessary interactions on the editor just to have less false-negatives.

Now, as I told before, I’m developing Chlorine together with REPL-Tooling, a library that should contain all tooling for any editor capable of running Javascript to run a port or Chlorine. There’s still too much on Chlorine that relies on internal Atom APIs (for example, detection of the beginning and end of forms is one, detection of namespace’s forms is other) but time is passing and more and more is being moved to REPL-Tooling, and as soon as the detection of forms is on REPL-Tooling (and is stable) there should be possible to port most Clojure parts to REPL-Tooling, and then I can think on how to refactor the ClojureScript part, test it, and then Chlorine will be a very easy project to port. Also, as a proof-of-concept there’s Clematis for NeoVIM (it’s still on the very beginning and nothing much happened after I wrote about it here), and also an “electron fake editor” that I’m using as test.

Wait, What?
(more…)

By Maurício Szabo, ago

Heart of Clojure

It all started with the idea to send a talk to Heart of Clojure, based on my work on Chlorine. Now, the talk wasn’t selected, but I decided to go to the conference even then. Being on Brazil, this meant about 19 hours of travel, so without thinking too much I bought the ticket to the conference and decided what do to latter about the travel, where to stay, and how many days I would stay. The first idea was to simply go there to the conference, but I imagined it would be very tiring, so came a strange and incredible idea that I’ll let to another post.

Now, the conference. In fact, it was different from everything I ever saw. First of all, it was amazing – not only because of the technical talks, but the whole experience of being in Leuven, meeting new people, using English as the first language everywhere, trying to understand a little of Dutch (spoiler alert: it didn’t work), and so on…
(more…)

By Maurício Szabo, ago

Rails’ ActiveRecord – the bad and the ugly

I’m known to not be a big fan of ActiveRecord. No, that would be a simplification: I probably hate ActiveRecord and think it adds more problems than it solves, specially after I began to work with functional programming and saw how difficult, if not utterly impossible, is to make ActiveRecord models behave like immutable structures or separate (and maybe even predict) the I/O from the rest of the code.

The ActiveRecord pattern (not the GEM) was created to hide SQL details from the users. The Gem elevates this to extremes: you never know when a query is issue, what query is issued (unless you check the logs), and sometimes a later clause modifies the way previous clauses work. Also, to extend ActiveRecord, you need to rely on monkey-patches and other internal implementation details, and there are API changes that seem innocent but are tremendously dangerous.

Now, what I want to do in this post is to elaborate the bad and the ugly parts. I’m not gonna talk about the “good parts” because we already know: auto-discovery of fields, fast prototyping, simple CRUDs, and so on. One could argue that this “easy setup, fast prototyping” is not worth the amount of technical debt you’ll have later, but let’s focus on the bad parts instead:
(more…)

By Maurício Szabo, ago

Object-oriented after working with functional programming

Sometimes I need to divide my knowledge in phases: when I started to program, I felt that C/C++ were the best languages to make a software. Then, I’ve began to work with Ruby and learned Test-Driven Development. I divide my knowledge “before TDD” and “after TDD”, because it literally changed the way I could think about software and tests in general. Now, I’ll probably make the same division: “before functional programming” and “after FP”, and I only discovered that things have changed this week: I just discovered I don’t know how to do object oriented programming anymore – and to be honest, it’s not really a bad thing: I also saw in practice how object-oriented programming expects you to write lots of boilerplate in most situations.

This realization came to me when I was working in a peculiar kind of problem – it’s basically a ruby code that needs to connect on messaging system and, when it receives a message, needs to persist it in a relational database. The problem is that this specific system have a strong consistency requirement, so the message could only be updated on DB before a specific event happens: after that, we can’t change it anymore, except for some specific fields. Also, as we’re working with messaging, we can receive the message multiple times, out of order, and all of those strange things that happens on Internet-land. So, to summarize:

  1. The system will receive a message
  2. It’ll UPSERT a record on DB, just with some identification fields (they’ll never change, and if they do, they characterize another message to be persisted – think about an external id)
  3. It’ll BEGIN a transaction and SELECT ... FOR UPDATE my just upserted message
  4. It’ll find, on another table, the rest of the fields on the message
    1. If this record doesn’t exist, we’ll create it (and log)
    2. If it does exist, we’ll check additional info, and decide: or we ignore it (and log), or we upsert (and log), or raise an error

Now, for the implementation. Because consistency is a must have on this system, I don’t want to expose for future programmers (be it myself or some of my friends) some code that’ll induce me to errors: after all, who wants to do with_consistency_check(msg) { Message.upsert!(msg) } when I can just do Message.upsert!(msg), right? So, I don’t want to allow people to be able to modify, find, update, or anything else outside of the with_consistency_check (or whatever name I decide). So, how to do it?
(more…)

By Maurício Szabo, ago

Chlorine, Clematis, REPL Tooling

Recently I began to migrate some of the code on Chlorine to REPL Tooling, so things may become more testable. I already found multiple bugs in this approach and began an integration test methodology using Electron (I say that there are three kinds of developers in the field of automated tests – the ones that don’t test, the ones that do, and test freaks. I’m probably the later).

In the beginning, the idea would be to rewrite the renderer of results and exceptions so that I could fix lots of bugs that they have. So far, it’s progressing slowly, but I already can render datomic results in a meaningful way (the last renderer was interpreting as Java objects, which they indeed are) and fix some bugs with tags and other issues.

Now, one of the things that I wanted in the new renderer is the ability to copy/paste results in a meaningful way – as I’m using UNREPL, there are lots of tags that need to be reinterpreted, like #unrepl/browseable for datafy. These need to be “translated” to something useful for rendering on screen, with links to navigate inner details, but for copy-paste they should have a textual representation that makes sense (probably the hard part).

The results of all these experiments are quite interesting – when you evaluate something and th result is a Java object, there’s a link ... that can be used to navigate inside the object: it’ll show getters, the current class, methods that the object supports, so things like “goto definition for Java object” or “Javadocs” aren’t really necessary.

Then, one of my friends asked for a Vim option. Just for fun, I started Clematis, a port of Chlorine to NeoVim. And it seems they things are progressing faster than I thought!
(more…)

By Maurício Szabo, ago

Chlorine is thriving!

When I started this project, I was experimenting with shadow-cljs to see if I would be able to make an Atom package that would auto reload, run tests on ClojureScript, and se how far could I push ClojureScript in an Atom package.

Now, some months later, I’m seeing the package being used by a bunch of people, and I even discovered some bugs in UNREPL! Now, on this post, I’ll discuss a little bit more in detail the design decisions and my vision on the future of the project.

Chlorine is a clojure and ClojureScript atom package. It connects to a socket repl (opened via lein, boot, clj, shadow-cljs, lumo, or even REBL) and then upgrades the repl to be programmatically oriented with unrepl. Unrepl only works on Clojure, so for ClojureScript we use other techniques. Also, socket repl is a stream protocol, so to emulate a “request-response”, we need to coordinate things so Atom (and other editors) can react to commands and know exactly what’s the correct response for each command sent.

Design decisions

The choice for UNREPL was mostly because there is almost no documentation about prepl so far. Also, Socket REPL is literally everywhere: on Clojure , on ClojureCLR, on Lumo and Plank. Also, I wanted a better way to use ClojureScript, and I still have nightmares trying to use it over nREPL… and with Socket REPL things work fine.

Also, when I started the project Clojure 1.10 was just alpha software, and UNREPL offers us insanely good support for lazy lists, big strings, and other things that I wanted to use out of the box. One of the problems I’m still facing is coordination of evaluate/response, but this will probably be solved after a bunch of other fixes I’ll try.
(more…)

By Maurício Szabo, ago

The perfect programming language (for me)

Disclaimer: the perfect programming language does not exist. Even if it did, different people want different things, so probably the ideas in this post would not reflect the ideas from different people. With that being said, let’s start with a little background:

I usually prefer dynamic languages. There are also moments when I miss static typing, but most of the time the “code/solution exploration” and a good REPL (and code design) do the job of reasoning about the shape of my data.

Except when it doesn’t. Then, things get ugly. Real fast. There’s always bad code that you need to work with, even one that you wrote about six months ago, and now you’re dumbfounded, looking at the code trying to remember what the hell were you thinking when you wrote the code and why you did think it was a good idea to wrote it that way at all. It happens with everyone. And that’s when static typing can (and will) help: it reasons about your data. You can have a variable named a, but at least you know it have the fields b: String, c: Int, whatever that means. But it helps.

Maybe we could have a language that allows you to turn on/off the typing whenever you wanted? With better REPL support? So, this would be my dream language to work with.
(more…)

By Maurício Szabo, ago

My fading frustration with ClojureScript

I’ve talked about at another post on how ClojureScript frustrates me, mostly because I was doing some Node.JS work and Figwheel simply wasn’t working correctly. Now, it’s time to revisit these points:

A little update: I talked a little with Thomas Heller, Shadow-CLJS creator, and he pointed me some issues with this article, so I’ll update it acordingly

Tooling

Figwheel and Lein are not the best tools to work with ClojureScript. Since I discovered shadow-cljs, things are working way better than before: I can reload ClojureScript code from any target, and I’m even experimenting with Hubot (and it works really fine too). The only thing I’m missing is my profiles.clj file, but I can live with that (and I can always use Shadow with Lein if I need profiles.clj).

Also, I’m working on a new package for Atom (and in the future, for another editors too) called Chlorine. One of the ideas is to offer better ClojureScript support (we have Autocomplete now!), using Socket REPL for solutions (even self-hosted REPLs like Lumo and Plank) and even wrap UNREPL protocol in Clojure. So far, is still in the very beginning but things are looking promising!

The stack

Forget Figwheel at all: Shadow-CLJS is probably the best tooling for ClojureScript ever. It auto-reloads ClojureScript code for the browser, for node.js, for node modules, and it integrates with almost everything that you want. It controls release optimizations, have sensible defaults, and even have post-compile hooks (so you can hook Clojure code to do something after some compilation phases). Also, it integrates with node-modules (no more maven-wrappers for JS libraries!) and have some warnings when you use some kind of ClojureScript code that would break :advanced compilation. And, let’s not forget that you can control the refresh reload phase, it adds a userful :include-macros in ns form (that will include all macros from the namespace being required), and controls exports in a sane manner. But first let’s begin with the feature that I found most useful: :before-load-async.
(more…)

By Maurício Szabo, ago