Performance optimizations in Reagent – part 2

Have I mentioned that it’s a nightmare to install React Devtools in Electron? Turns out, it’s worth it. But it’s still a pain. If you want to try, you’ll need to first install the extension in a chrome-based browser (like Brave, the one I use) and then install it by code. At Chlorine, what I did was to copy the whole extension folder to a devtools directory and then changed the script to load electron to the script below (see the BrowserWindow.addDevToolsExtension):

const path = require('path')
const {app, BrowserWindow} = require('electron')

app.on('ready', () => {
  let browser = new BrowserWindow({
    width: 900, height: 600,
    webPreferences: {nodeIntegration: true}})
  browser.loadURL(path.join('file://', __dirname, '/index.html'))

  BrowserWindow.addDevToolsExtension('./devtools')
})

Originally, the React Devtools was installed at ~/.config/BraveSoftware/Brave-Browser/Default/Extensions/fmkadmapgofadopljbjfkapdkoienihi/4.21.0_0 (tilde is my home folder). If you’re trying to install on Atom editor, you’ll have to open developer tools and issue the following script:

r = require('remote')
r.BrowserWindow.addDevToolsExtension('./devtools') // or the right directory

This command will fail, but somehow when you reload Atom, the React Devtools will be there. There’s only one feature that I use – highlight which elements were updated. And that’s where our performance tuning starts again.

Combining elements

Turns out that in the last post, I’ve decided to combine elements. It works fine, but for “root” elements there’s a catch: as it’s a tree data structure, when I combine elements React loses the ability to individually update a single component. That means that if you render [[1] [2] [3]], and opens up the root, then tries to open [2] (a sub-element), it’ll update THE WHOLE TREE instead of only [2]. That’s a BIG performance hit when you’re rendering big structures, and a moderate one when you’re rendering medium ones. Suppose, for example, that you’re rendering about 500 elements. The whole process ends up in less than 300ms – not bad, really.

But then, opening every single element will also run in 300ms. If I didn’t do that, elements would open in 80ms or less. So, yeah – big performance hit. What now?

So I decided to try some approaches. All of the following code will use the same data: it’s the result of (->> (range 4000) (map (constantly ["1" 2 3])) vec) (so, a vector with 3 sub-elements).

Don’t batch at all when it’s a “root” element

All interaction happens at the “roots” of the rendering. So, nothing better than just not batch when it’s in the root, right?

Turns out that this makes the “root” rendering REALLY slow, but the leafs REALLY fast. For the test data, that makes the root take 4.42s, and the leafs take 180ms. Great for the leafs, bad for the root. There’s also a 230ms garbage collector at the end of the rendering, so if possible I would like to avoid this option

Combining elements in batches

Instead of combining ALL elements into one, combine a group of elements – like, 500 – into one. This means that when you interact with the leafs, you’ll pay a performance price (because you’ll not only update the leaf, but also the other 499 elements) but it’s lower than combining everything. So I decided to try – and the code was really easy to do too (oh, the joys of ClojureScript) – instead of doing (->> elements (map #(html-for {:root? true} %)) (combine-leafs [:br])), I added a partition-all:

(->> elements
     (partition-all 500)
     (map (fn [elements]
            [(fn []
               (->> elements
                    (mapv #(html {:root? true} %))
                    (combine-leafs [:br])))])))

This… was a completely disappointment. The rendering went down to 3.83s, with the same GC pause. But leafs now take 260ms to render. So I pay a high price to a low improvement on the root rendering. Not good at all.

Pagination

Creating the React elements is not slow – what’s slow is rendering them on the interface. With that idea, I decided to paginate results. When someone opens a very big element tree, what if it shows only the first 100 elements, and then add a link to show more? With the idea in mind, I decided to try. Again, it’s incredibly simple to do it – just add, on the current state, a key/value :loaded-elements that defaults to 100, and then use:

(->> elements
     (map (fn [e] [html {:root? true} e]))
     (take (:loaded-elements @state)))

And then add a link to update the state, incrementing 500 elements. That’s all. With this, the root renders in astonishingly 457ms! Leafs take 85ms to render, and asking for “more” data takes 1.22s – the best performance for EVERYTHING so far! So, next versions of Chlorine (and Clover) will have a kind of pagination on ClojureScript, Clojerl, etc so they will render faster.

Other performance improvements

While I was doing all the improvements, I also tried other ways of combining leafs. The combination of leafs can be as complicated as I want, and while I could combine more, in the end it’s tricky, it would require A LOT of testing for a minimum performance improvement, and it would be a wasted effort considering that I’m only doing it when things are not expanded so…

But what I did try was to improve performance of some Clojure constructs that I know are slow. In the first version of combine-leafs I used destructuring to get the first and the rest of elements. combine-leafs is currently not used to render the tree-like structure anymore, so I just benchmarked the first rendering – I did not benchmark the open/close of the tree, nor the leafs. I did three different renderings, and compared their results. The first, unoptimized version, rendered in 1.35s, 630ms, and 550ms.

Then I tried to memoize, equivalent? – that’s one function that’s called multiple times for almost the same elements over and over. And it did change the render time – to 1.26s, 632ms, and 599ms – it got WORSE! But then I looked at the source of memoize, and saw that it does use atom. I tried to write my own version of memoize using volatile!, and now it did render in 1.13s, 545ms, and 495ms. So, even considering that atom and volatile! are almost the same in ClojureScript, it did make a difference!

I decided, them, to try different approaches to combine-leafs. First, to avoid creating sub-elements by using transducers in Clojure. With this change, the whole process rendered in 1.04s, 550ms and 504ms – not much difference, to be honest… The last thing I did was try to refactor combine-leafs to use vectors instead of regular collections – that way I could use subvec and nth 0 instead of the slower versions of first and rest. This gave me rendering times of 1.05s, 560ms, and 487ms – the fastest implementation after everything “warms up”, but not that far from the original implementation – it’s not fast enough so that the user will see any difference, and also I lost a little bit of flexibility because combine-leafs now needs a vector every time, so it’s a tricky trade-off.

Throwing it all away!!! 😭

It was fun while it lasted, but in the end, React does not like what I did. Combining leafs that way made the whole process incredibly slow for every interaction except the first render, so I ended up just “streaming” the rendering process – that is, instead of (map <my-element> children), I did (take <n> children) and then changed the n inside a setTimeout until it was equals, or greater, than the number of children I had to render. That made the first render take 400ms (REALLY fast) and every secondary render take about 500ms (also fast) in a way that users’ can’t even see the difference. So, that’s where it ended up.

So, while it was a fun experiment to work with pure data, combine different things, and try new ideas, the libraries are still JS – they don’t like what I did, and they also don’t understand what happened. If, instead of React, the code was some pure ClojureScript, that made diffs between hiccup elements instead of virtual DOMs, and somehow rendered only what changed on the real DOM, them probably that original technique could work. In fact – that may be an interesting idea for a library, for someone that wants to avoid react in the future?

In the meantime, it was fun knowing a little bit more about performance in ClojureScript, learning about bench-marking React apps, and be able to see how to debug rendering inside a react app – it was really fun, and it’s a tool I’ll surely use in the future.

So a little table to compare everything:

“Micro” optimizations on combine-leaf, with percents compared to unoptimized version:

And for the code to render root and leafs in a faster way, all compared to the current implementation of Chlorine: