Ruby

Exposing Clojure to Ruby and other languages – Java objects in C

In the first post of this series we decided to compile a Clojure library to a native shared library (with the GraalVM native-image command) and we used “isolates” so that a “global variable” in Clojure (and Java) would appear as if they’re “local” in Ruby. Then on the second post of the series we decided to make things local, and instead of “making a resolver then storing it in a global variable” we decided to “make a resolver and return it to Ruby”, and to abstract the nature of “callbacks” we used a CFunctionPointer in the Java side, and we sent the “block” from Ruby as an “opaque object” in Java, using GraalVM’s VoidPointer object.

Now, we need to do the opposite – we need to return a “resolver” to Ruby, and then we need to make a “List of Resolvers” in Ruby-side, and send it to Clojure-side somehow. These Clojure/Java objects (a Resolver, and a List of Resolvers) will also be “opaque objects” in C (meaning – we’ll receive them as void*, because they can’t be represented in C-land, nor in Ruby-land), but only in C – in Java-land (and Clojure, by definition) they’ll need to retain their types. The object GraalVM provides for this task is an ObjectHandle.

We’ll also fix a lot of memory leaks, so let’s move on!
(more…)

By Maurício Szabo, ago

Exposing Clojure to Ruby and other languages – Callbacks

At the first part of this series I showed how to expose some Java methods (that delegates to Clojure functions) to a shared library, so that the shared library could be exposed in Ruby via the C bindings. There is a lot of “cutting corners” on the first implementation, for example, we only transmitted strings that would be serialized and de-serialized between languages, and the fact we’re using GraalVM “isolates” to allow multiple instances of “VMs” to exist, so we can fake having multiple objects, etc. In this post, I will fix some of these issues, and then show how to expose complex objects.

So, here’s a quick overview of the Clojure library I want to expose: the library allows you to define “resolvers”, which are functions. The difference between a resolver and a “normal function” is that the resolver always expects and returns a “Map” (HashMap in Ruby). The resolver also have some “metadata” that explains, at least, the dependencies: meaning, what are the keys that the “input” of the resolver must have, and what are they keys that the “output” will return.

The implications of this are huge, although they might not appear so – the idea is that, instead of wiring up things manually (like, for example, querying a database, getting the result, sending to some external service, get the result of the service, do something else) you just define the “dependencies” – for example, “I need the user’s login and email for this external service” as a “resolver” – if hashmap doesn’t have it, you’ll have a different resolver saying “Query the database and return the user’s login and email” and so on.
(more…)

By Maurício Szabo, ago

Exposing Clojure to Ruby and other languages

Recently, I’ve been missing Pathom (from Clojure) in the Ruby world. So I decided to check if I could expose an arbitrary Clojure library to Ruby, using the GraalVM’s “Substrate VM” (popularly known as native_image).

The way I tried to do this was to expose a shared library to C, then making a Ruby extension that consumes this shared library and then finally “normalize stuff” into Ruby. This… honestly, worked way better than I expected, and because I needed a lot of help from AI tools, guesswork, and reading some very weirdly-written Java code from Oracle (and auto-generated Javadocs) I decided to make a post explaining the whole process.

The whole experience started as weird, then easy, then VERY hard, and right now… I’m not sure where it will be, because I’m kind of changing the whole project’s core ideas (the reasons will be clear while you read this post). So let’s dive in
(more…)

By Maurício Szabo, ago

Goodbye, Chlorine. And welcome, Lazuli!

So, if you’re not familiar with Chlorine, or any of my tooling, let me get you a refresher. If you don’t want to read it all, the TL;DR; is – Chlorine is a Socket REPL plug-in for the Clojure, and ClojureScript languages. It also supports nREPL, although to a lesser extend, and it works on the Atom editor. It also contains a VSCode version called Clover, and I tried to port it to NeoVIM and I called it Clematis.

And now, it’s gone.

For historical reasons, I wanted to rename the project for a while – so that Chlorine would stay as a “socket REPL for the Atom editor” and the new plug-in would be the “Chlorine:Next” or something like this. I tried to like the name Alya, but in the end I never did…

… and then, I stopped working with Clojure. Like, at all. For a while I haven’t touched a single line of Clojure code, although I still use ClojureScript on my personal plug-ins. But ClojureScript have a weird problem – I can’t develop two plug-ins written in ClojureScript at the same time, so I need to compile everything, except for one, plug-in for “release”. Which is annoying to say the least, so that’s why I started the “Star Ring” metapackage – to make all plug-ins a “single, unified codebase”. Unfortunately, this have its own fractal of problems that I won’t discuss now.

So not that I’m now working with Ruby, I needed better tooling for Ruby otherwise I would go crazy. Leaving the “REPL-Driven Development” of Clojure behind is a nightmare, because it’s such a great tool and it somehow “grows” in you in such a way that you can’t imagine working in a codebase when you can’t evaluate your code all the time. And that’s where Lazuli appears… but again, because of the problems with not being able to develop Lazuli and Chlorine at the same time, Chlorine’s development stalled… and I decided to stop working on it to work in Lazuli.
(more…)

By Maurício Szabo, ago

Porting Clojure libraries to Ruby – Part 2

Last post I gave an introduction on how I’m working with GraalVM to port Clojure things to Ruby. Now, it’s time to handle the hard parts.

The first thing is – some stuff simply doesn’t work. Pathom3 (the library I wanted to port) depends on Guardrails, which requires core.async, which starts a thread pool, which GraalVM doesn’t like. At all.

And the second things is how to serialize some weird stuff like callbacks. So let’s start with the first part – how to make a “Pathom3 shared library” without GraalVM screaming at us
(more…)

By Maurício Szabo, ago

Porting Clojure libraries to Ruby

Since I started working with Ruby, in some situations I miss some of the Clojure libraries – especially killer ones like Pathom.

Unfortunately, rewriting the whole library in a different language is completely out of the question, especially because I will reintroduce some of the bugs that are already fixed, and will not have some of the more important features like parallel processing and other stuff.

But it’s not an “end of the world” situation. GraalVM have a very important feature that is to compile Java code to native, using what it’s usually called the “native-image” or, to be more precise, SubtrateVM. I also knew that you could generate shared libraries with this approach, but I never actually did anything and in my last attempts – every time I tried the only result I had was lots of frustrations and weird errors.

So that’s why I’m very proud and very surprised to announce that I have a Pathom working in Ruby right now.

But more on that later – it’s probably not yet ready to be used in production; but what I want to share is how I actually made it work, then challenges I faced (and how to handle them) and how can other people do the same. Finally, I am writing this post to make it easy for other people to do the same – honestly, the whole thing was awfully hard but not because it’s actually hard – but because the documentation is lacking so much that, at some times, I even though things I did were not possible.
(more…)

By Maurício Szabo, ago

My last experience with Ruby

On my last job I tried to go back to Ruby programming language. Not really by choice – but just because it was a language and was familiar with, and my last two jobs in Clojure didn’t really work out for me (not because of the language, really). I even imagined that some things would be easier in Ruby, specially while working with relational databases (something I really did miss while I was working in the Clojure language).

At the time, in my opinion, Rails was still a great framework – maybe a little too complex. And I imagined that somehow there could be better ways of doing Ruby code that could use the best parts of Rails and avoid the worst (like ActiveRecord, for example, maybe even substitute it with Sequel or ROM).

No a year and a half later I’m back to Clojure – and I will probably never go back to Ruby. And the reason is quite simple: I was working with Ruby doing exactly the same code that I did about three years ago. Rails didn’t change that much, but still was a huge mess when you had to upgrade from one version to another. The same problems I had in the past kept happening again and again, and it felt like a huge step backwards for me.
(more…)

By Maurício Szabo, ago

Rails’ ActiveRecord – the bad and the ugly

I’m known to not be a big fan of ActiveRecord. No, that would be a simplification: I probably hate ActiveRecord and think it adds more problems than it solves, specially after I began to work with functional programming and saw how difficult, if not utterly impossible, is to make ActiveRecord models behave like immutable structures or separate (and maybe even predict) the I/O from the rest of the code.

The ActiveRecord pattern (not the GEM) was created to hide SQL details from the users. The Gem elevates this to extremes: you never know when a query is issue, what query is issued (unless you check the logs), and sometimes a later clause modifies the way previous clauses work. Also, to extend ActiveRecord, you need to rely on monkey-patches and other internal implementation details, and there are API changes that seem innocent but are tremendously dangerous.

Now, what I want to do in this post is to elaborate the bad and the ugly parts. I’m not gonna talk about the “good parts” because we already know: auto-discovery of fields, fast prototyping, simple CRUDs, and so on. One could argue that this “easy setup, fast prototyping” is not worth the amount of technical debt you’ll have later, but let’s focus on the bad parts instead:
(more…)

By Maurício Szabo, ago

Understanding Ruby’s Awesome Nested Set

Recently, I saw people migrating from Awesome Nested Set to Ancestry. The reasons are simple – Ancestry is very simple, it just need a new string field in your table, and it’s easier to reason about. So, why should I even consider to use an alternative?

The answer is simple, and I’m going to quote H. L. Mencken: For every complex problem there is an answer that is clear, simple, and wrong.

And the reason that using Ancestry is a bad idea is simple: it doesn’t reflect good design. It treats a single field as a multi-valored column that keeps the ancestry of your object. This means that there’s no way to fetch all records AND their parents in a single query, or fetch all records AND their children, because we need to split the string (ruby-side) and then create a query (also ruby-side). So, in this post, I’ll show some tricks of what we can do with Awesome Nested Set, or even beter, how does it implements the tree pattern (and why it calls it a “set”, not a “tree”).

First, when we think about categories and sub-categories, we think like this:

But this is, in fact, a terrible way of representing trees in SQL. So, the alternative, is to transform it in a group of sets: Parent 1 (P1, for short), is a super-set containing subsets C1 and C2 (Children 1 and Children 2, for short). Each of the children have its own grandchildren, so C1 is a super-set containing G1, and C2 is a super-set containing G2 and G3.


(more…)

By Maurício Szabo, ago