Right now, I’m working in a game project in Clojure. I don’t really know how it will turn out, but for now I’m just trying to learn a better way of making games.
While working in this project, I found out that my game was consuming a lot of memory. I’m using play-clj library, and I know that it creates a lot of small objects for each render cycle, so that was my first guess.
So, I plugged in a VisualVM in my running game to understand what was happening. In the beginning, nothing seemed to make sense: the heap grew, then was released, the correct and normal cycle of any Java application. Then, I tried a memory profiling and a memory dump. Then, things became interesting.
There were a lot of float[] objects popping up, as I would expect – play-clj uses floats to position elements on the screen, and all the time I found myself trying to coerce doubles to floats. But there was something even stranger there was consuming a lot of memory: instances of java.lang.Method.
For those who don’t know, Clojure interoperability with Java relies on reflection when it can’t resolve a type. To resolve a type means that Clojure can be certain that, at run time, that a specific identifier will be a specific type. So, for the following code:
(ns example.core) (defn sum-abs [a b] (Math/abs (/ a (float b)))) (defn only-abs [a] (Math/abs a))
The first method call will use reflection because it knows that the result of a sum will always be a float. The second one has no idea if it will be called with a number or not, so it relies on reflection. It may seem strange, as we’re calling Math/abs, but remember that in Java we can have different methods with the same name, differing only on type signature.
So, to resolve the type, we’ll need type hints. But first, we can test if our code is using reflection using lein check.
This command will try to compile our code and warn if it could not resolve some specific method call. In this case, its output is:
Compiling namespace example.core Reflection warning, example/core.clj:7:3 - call to static method abs on java.lang.Math can't be resolved (argument types: unknown).
Then, we can add our type hints:
(ns example.core) (defn sum-abs [a b] (Math/abs (/ a (float b)))) (defn only-abs [^double a] (Math/abs a)) ; OR (defn only-abs [a] (Math/abs ^double a))
Why does it matter?
First of all, performance issues. Although the method call itself can be fast, everytime we have to fall down to reflection we need to inspect the object’s class, then search for a method that matches the name that we’re calling. Then, we need to check each argument’s type, and select a method that matches all parameter’s type too. When we add type hints to all arguments, we can just delegate the method call to the JVM – and that’s as fast as we can get.
The second reason is memory allocation, and that was what was getting in my way. For each render cycle that my game had, it needed to resolve a bunch of reflection calls. This meant a lot of Method objects being created to just make a single call, then being discarded. I don’t really know how JVM discards Method objects, and I don’t really know if it reuses the same Method for the same reflection call, but since I’ve added type hints to everything, my code ran with less than half the memory it was using before – and no more leaks or my Linux machine killing my process because it was using too much memory.
The downside
Obviously, you lose the dynamic nature of Clojure. If you sign a function with a specific type hint, you can’t pass anything of different type to it – even if the code would compile and run. In the above case, when we pass an int to only-abs, Clojure will coerce it to double, and so even if we call (only-abs -10), the method’s return will be 10.0 – the int will be coerced to double before the call. Other specific case is when we have two different classes with the .draw, method, with exactly the same signature – using hints, we will not be able to pass any object to our code and use Clojure’s interop to figure which one to call – we will be tied to only one type.
Another problem is the simple fact that Clojure will not check types at compile time – so, we have all downsides of a static typed language, without any of the upsides. Worse yet – Clojure’s types are the same as Java’s, so it will accept nils in most places where a specific object is needed – it won’t even protect us from dreadful null pointer exceptions.
Now, one of the upsides is that type hints are accepted in places where we don’t expect a language to accept a type. For instance, we can use hints when deconstructing a collection, making possible to have a Vector with each argument of a different type (without downgrading it to a List of Objects). Another great upside is that Clojure will try to inflect return types and other things for us, so most of the time we need very few hints to speed up things!