Clojure

Exposing Clojure to Ruby and other languages – Java objects in C

In the first post of this series we decided to compile a Clojure library to a native shared library (with the GraalVM native-image command) and we used “isolates” so that a “global variable” in Clojure (and Java) would appear as if they’re “local” in Ruby. Then on the second post of the series we decided to make things local, and instead of “making a resolver then storing it in a global variable” we decided to “make a resolver and return it to Ruby”, and to abstract the nature of “callbacks” we used a CFunctionPointer in the Java side, and we sent the “block” from Ruby as an “opaque object” in Java, using GraalVM’s VoidPointer object.

Now, we need to do the opposite – we need to return a “resolver” to Ruby, and then we need to make a “List of Resolvers” in Ruby-side, and send it to Clojure-side somehow. These Clojure/Java objects (a Resolver, and a List of Resolvers) will also be “opaque objects” in C (meaning – we’ll receive them as void*, because they can’t be represented in C-land, nor in Ruby-land), but only in C – in Java-land (and Clojure, by definition) they’ll need to retain their types. The object GraalVM provides for this task is an ObjectHandle.

We’ll also fix a lot of memory leaks, so let’s move on!
(more…)

By Maurício Szabo, ago

Exposing Clojure to Ruby and other languages – Callbacks

At the first part of this series I showed how to expose some Java methods (that delegates to Clojure functions) to a shared library, so that the shared library could be exposed in Ruby via the C bindings. There is a lot of “cutting corners” on the first implementation, for example, we only transmitted strings that would be serialized and de-serialized between languages, and the fact we’re using GraalVM “isolates” to allow multiple instances of “VMs” to exist, so we can fake having multiple objects, etc. In this post, I will fix some of these issues, and then show how to expose complex objects.

So, here’s a quick overview of the Clojure library I want to expose: the library allows you to define “resolvers”, which are functions. The difference between a resolver and a “normal function” is that the resolver always expects and returns a “Map” (HashMap in Ruby). The resolver also have some “metadata” that explains, at least, the dependencies: meaning, what are the keys that the “input” of the resolver must have, and what are they keys that the “output” will return.

The implications of this are huge, although they might not appear so – the idea is that, instead of wiring up things manually (like, for example, querying a database, getting the result, sending to some external service, get the result of the service, do something else) you just define the “dependencies” – for example, “I need the user’s login and email for this external service” as a “resolver” – if hashmap doesn’t have it, you’ll have a different resolver saying “Query the database and return the user’s login and email” and so on.
(more…)

By Maurício Szabo, ago

Exposing Clojure to Ruby and other languages

Recently, I’ve been missing Pathom (from Clojure) in the Ruby world. So I decided to check if I could expose an arbitrary Clojure library to Ruby, using the GraalVM’s “Substrate VM” (popularly known as native_image).

The way I tried to do this was to expose a shared library to C, then making a Ruby extension that consumes this shared library and then finally “normalize stuff” into Ruby. This… honestly, worked way better than I expected, and because I needed a lot of help from AI tools, guesswork, and reading some very weirdly-written Java code from Oracle (and auto-generated Javadocs) I decided to make a post explaining the whole process.

The whole experience started as weird, then easy, then VERY hard, and right now… I’m not sure where it will be, because I’m kind of changing the whole project’s core ideas (the reasons will be clear while you read this post). So let’s dive in
(more…)

By Maurício Szabo, ago

Quick Post – Multiple Shadow-CLJS builds at the same runtime

One of the greatest limitations of ClojureScript is how you can’t have two ClojureScript codebases, running on “development mode”, in the same runtime. In the browser, or in Node.JS, this is not a big problem, but it is one in some other situations – like browser extensions (you might be developing two extensions at the same time), node libraries (again, same case) and… well, editor plug-ins for Pulsar, or VSCode.

That… might not be a problem anymore.

If you use Shadow-CLJS (and you should) then you can just change your target to :npm-module (or :esm if you’re in the browser). That will create a lot of files but the important one is cljs_env.js. What this file does, is bootstrap the Google Closure Compiler and make some assignments for Shadow and for ClojureScript namespaces.

In the beginning of the file, you’ll see this line:

var CLJS_GLOBAL = global;
var $CLJS = CLJS_GLOBAL;

Just change it to:

global.some_unique_identifier = {}
var CLJS_GLOBAL = global.some_unique_identifier;
var $CLJS = CLJS_GLOBAL;

And that’s done.
(more…)

By Maurício Szabo, ago

Goodbye, Chlorine. And welcome, Lazuli!

So, if you’re not familiar with Chlorine, or any of my tooling, let me get you a refresher. If you don’t want to read it all, the TL;DR; is – Chlorine is a Socket REPL plug-in for the Clojure, and ClojureScript languages. It also supports nREPL, although to a lesser extend, and it works on the Atom editor. It also contains a VSCode version called Clover, and I tried to port it to NeoVIM and I called it Clematis.

And now, it’s gone.

For historical reasons, I wanted to rename the project for a while – so that Chlorine would stay as a “socket REPL for the Atom editor” and the new plug-in would be the “Chlorine:Next” or something like this. I tried to like the name Alya, but in the end I never did…

… and then, I stopped working with Clojure. Like, at all. For a while I haven’t touched a single line of Clojure code, although I still use ClojureScript on my personal plug-ins. But ClojureScript have a weird problem – I can’t develop two plug-ins written in ClojureScript at the same time, so I need to compile everything, except for one, plug-in for “release”. Which is annoying to say the least, so that’s why I started the “Star Ring” metapackage – to make all plug-ins a “single, unified codebase”. Unfortunately, this have its own fractal of problems that I won’t discuss now.

So not that I’m now working with Ruby, I needed better tooling for Ruby otherwise I would go crazy. Leaving the “REPL-Driven Development” of Clojure behind is a nightmare, because it’s such a great tool and it somehow “grows” in you in such a way that you can’t imagine working in a codebase when you can’t evaluate your code all the time. And that’s where Lazuli appears… but again, because of the problems with not being able to develop Lazuli and Chlorine at the same time, Chlorine’s development stalled… and I decided to stop working on it to work in Lazuli.
(more…)

By Maurício Szabo, ago

Performance optimizations WITHOUT Reagent – part 3 (and probably final)

This is probably the last post about these improvements. At the last post, I showed how I threw every performance optimization I tried away (combine-leafs, other stuff, etc) just to throw it all away because React didn’t like what I did.

So, after a lot of time, I decided to do the horrible, insane thing that I did a lot more than I wanted – rewrite the rendering process of Chlorine to not use Reagent. The reason was actually quite simple: I wanted things to be simpler to implement, and honestly, Reagent was getting in my way. Like, A LOT.

Before I continue explaining the performance improvements that I am doing, I have to say something – in the first version of the rendering code (the one that I was trying to optimize at my last posts) I had a huge beginner’s luck – the actual rendering time is amazing for most cases, and while it is slow for some very specific situations, it’s actually not that bad; but there were also some weird trade-offs that I had to do and I cannot use them anymore.

The reason being, Chlorine started to become very strong in the customization code and “interactive renderers”. If you’re not aware, you can configure Chlorine to add custom commands and other amazing stuff, just by opening up a config file, writing some ClojureScript code, and you’re good to go. Unfortunately, also at the first versions of Chlorine, sometimes you would evaluate a code – let’s say a list of a thousand elements – and you get back only 10 or 20 elements. Worse yet, you would not get back a “List”, but an object called an “Incomplete List”. Which was not ideal, and basically made the whole “customize your editor” difficult to say the least.
(more…)

By Maurício Szabo, ago

Porting Clojure libraries to Ruby – Part 2

Last post I gave an introduction on how I’m working with GraalVM to port Clojure things to Ruby. Now, it’s time to handle the hard parts.

The first thing is – some stuff simply doesn’t work. Pathom3 (the library I wanted to port) depends on Guardrails, which requires core.async, which starts a thread pool, which GraalVM doesn’t like. At all.

And the second things is how to serialize some weird stuff like callbacks. So let’s start with the first part – how to make a “Pathom3 shared library” without GraalVM screaming at us
(more…)

By Maurício Szabo, ago

Porting Clojure libraries to Ruby

Since I started working with Ruby, in some situations I miss some of the Clojure libraries – especially killer ones like Pathom.

Unfortunately, rewriting the whole library in a different language is completely out of the question, especially because I will reintroduce some of the bugs that are already fixed, and will not have some of the more important features like parallel processing and other stuff.

But it’s not an “end of the world” situation. GraalVM have a very important feature that is to compile Java code to native, using what it’s usually called the “native-image” or, to be more precise, SubtrateVM. I also knew that you could generate shared libraries with this approach, but I never actually did anything and in my last attempts – every time I tried the only result I had was lots of frustrations and weird errors.

So that’s why I’m very proud and very surprised to announce that I have a Pathom working in Ruby right now.

But more on that later – it’s probably not yet ready to be used in production; but what I want to share is how I actually made it work, then challenges I faced (and how to handle them) and how can other people do the same. Finally, I am writing this post to make it easy for other people to do the same – honestly, the whole thing was awfully hard but not because it’s actually hard – but because the documentation is lacking so much that, at some times, I even though things I did were not possible.
(more…)

By Maurício Szabo, ago

Clojure, and Ruby

I’m gonna make a bold claim here. As a language, Clojure is much better than Ruby. But, as a means of working together with other people, Ruby can be better, and Clojure can be a nightmare.

Here’s the problem with this debate: in Ruby it’s easier to actually bootstrap something because of the Rails framework. Clojure doesn’t have anything like that, but the code tends to get less worse as time goes by – as a functional language where everything is immutable by default, there’s less “surface” that’s touched by a piece of code. For example, suppose you have a function – in Clojure, changing that function only affects callers of that function, whereas in a mutable language, the function can mutate an object/value and now everywhere that uses that same object can be affected.

Another reason, that might be counter-intuitive is because of the real framework itself. When an application grows, you might find that you need less constraints and more freedom to do stuff that was not defined originally. Unfortunately, if you are using Rails (or any other framework), this means that you have to find ways around the framework, because it’s a kind of an “all or nothing” – it’s not that easy to basically do something outside of it and then bring that into your workflow, into your web servers and things like that (for example, I once had to integrate a SQS message queue on Rails, to listen to some message and deliver a notification via websocket. It was such a bad experience that we did a Node.JS app that did just that).
(more…)

By Maurício Szabo, ago