Programming Languages

CommonJS is not a problem – the Javascript ecosystem is

Recently I came across this post saying that CommonJS is hurting the JavaScript world. And while I do agree that the CommonJS specification was not a good one, I also disagree a lot with the article. As someone who have strong opinions about the subject, I decided to write a post about it.

So, here’s the problem: supposing that you have a legacy system that’s years old, and that basically is modular, meaning that people write extensions, or libraries, to that. Like for example, NodeJS’ CommonJS. Now, here’s the problem: if that legacy system is flawed, and you need to change somehow, and your solution is “let’s do something that is better in any way than the older model, it’s faster, etc, but also completely incompatible with the old stuff” – meaning that we have to rewrite everything to be able to be used on this new format – then you’re doing it wrong.
(more…)

By Maurício Szabo, ago

Clojure does NOT need a “definite web framework”

Few times I feel inclined to answer for a post. There are actually lots of posts that say that Clojure needs “the web framework” like Rails or Django or Phoenix so it’ll be better. Sometimes these ideas have value, sometimes don’t, and sometimes they are simply nonsense at all.

I recently was shown a post that was basically nonsense, and to keep things neutral, I’m not going to link it here. I want to expose why I think these ideas are nonsense, and how a person that’s new to Clojure could think about this lack of frameworks, how to handle this situation, and what to expect from the language.

So, first things first: Clojure is made to be a hosted language. Which means it expects you to use Java libraries, or Javascript ones, if you’re in ClojureScript. Sure, in Javascript things are more difficult mostly because of the weird ecosystem that expects everything to be transpiled first, but most of the time you can use interop just fine.

All this to say: you don’t need to wrap libraries. Most people I know, me included, said sometime that Clojure needs more libraries that wrap over other other things, and while I still would love to just stay on “Clojure-land” most of the time, it’s better to have a stable Java library that we can use over Clojure than an old, unstable, and severely lacking behind features that wrap the Java one to have a “Clojure feel”. Most of the time, you can have a “Clojure feel” with few lines of code anyway.
(more…)

By Maurício Szabo, ago

ClojureScript vs Core.Async debate – the last updates.

So, my old post made to Reddit, and as I expected, lots of people complained about my conclusion. I still find damaging that Clojure community have this huge energy on defending “core” solutions, even when they are low-level, impractical, or (in this case) does not work well, if it works at all. But, well, I decided to clarify some of the posts, and answer some of the comments that people told me would, supposedly, “solve” the problem. I already answered these questions on other forums, so I’ll try to re-use some of my comments:

Interop with <p!

Some people asked about interop with <p!. Honestly, when I made the article, this option did not exist. Even then, a Promise can be rejected and resolved with arbitrary data, so to translate this code to ClojureScript means capturing the original result of <p!, check if it’s an ExceptionInfo, with the right type, and extract the right part. It’s also harder to compose IMHO (like, with Promises, you can chain actions, then if any of them fail, you can catch the error and the promise is back to the “success” phase). But at least, now it’s less complicated – although I would say, if you’re going to add a library to your project, why not add funcool/promesa instead of core.async? Remembering that promesa is both faster than core.async, and its constructions can work with any arbitrary data, not only promises, so you can treat all code as if it’s async (or sync), without needing to remember which ones are promises, and which ones are not…

Just .pause and .resume constantly

This works, indeed. For example, with sockets:

(.on socket &quot;data&quot; (fn [data]
                     (.pause rs)
                     (async/put! chan data #(.resume socket))))
(.on socket &quot;end&quot; #(async/close! chan))

But not all callback-based async code in ClojureScript allow you to pause events. Node.JS Socket do, WebSocket in browser don’t. Also, NPM package pg, when you query with a cursor, also will not allow it. In this case, there’s nothing you can do, really – you will either drop messages, or an exception will happen.

Use offer!

(go (>! and put! both are fragile with callbacks, because you can’t really “park” or “block” threads in Javascript. Keep put!ing messages in a channel, and you’ll hit an exception “No more than 1024 pending operations” (or something like that). So people asked me to use offer! or make a sliding buffer, etc. This doesn’t really solves the problem if you can’t drop messages – and I do have lots of cases when this happens. There were some people that told me that dropping messages is part of the life, and I wholeheartedly disagree – there are LOTs of situations when you can’t loose anything – bank transactions, socket messages (you can’t reconstruct the payload if you lost part of it), downloads, etc. There are ways to mitigate this, for example:
(more…)

By Maurício Szabo, ago

Advent of Code in SQL – Day 01

Ok, this year I decided to try the Advent of Code. But to level up my SQL game, I decided to do it purely in PostgreSQL. No stored procedures, no tables, just pure logic over SQL (or, at least, PostgreSQL-flavored SQL). I’ll also try to not use PG specific datatypes like JSONB or Arrays/Ranges, but I can’t promise this – the challenge is already hard with the way it is.

So, I decided that the input will be a single table called input with a single record and field, called input. This means that I have to split the input in some way – I’m currently using regexp_split_to_table(input.input, '\n') to generate a table with all the lines when this makes sense.

So, day 1. The first thing is to generate the input:

WITH input AS (SELECT &#039;199
200
208
210
200
207
240
269
260
263&#039; AS input)

The Day 01 problem is kinda simple: you compare a row with the next one and when the next one is greater, you add 1 to a result. So in this case, we do need to split the input into numeric rows. There are two ways of doing this: you either first split then cast, or split and cast in a single query. The first one is easier, so let’s go with that:

WITH depths_as_str as (SELECT regexp_split_to_table(input.input, &#039;\n&#039;) depth FROM input)
  , depths as (SELECT depth  :: integer FROM depths_as_str)

(more…)

By Maurício Szabo, ago

Performance optimizations in Reagent – part 2

Have I mentioned that it’s a nightmare to install React Devtools in Electron? Turns out, it’s worth it. But it’s still a pain. If you want to try, you’ll need to first install the extension in a chrome-based browser (like Brave, the one I use) and then install it by code. At Chlorine, what I did was to copy the whole extension folder to a devtools directory and then changed the script to load electron to the script below (see the BrowserWindow.addDevToolsExtension):

const path = require(&#039;path&#039;)
const {app, BrowserWindow} = require(&#039;electron&#039;)

app.on(&#039;ready&#039;, () =&gt; {
  let browser = new BrowserWindow({
    width: 900, height: 600,
    webPreferences: {nodeIntegration: true}})
  browser.loadURL(path.join(&#039;file://&#039;, __dirname, &#039;/index.html&#039;))

  BrowserWindow.addDevToolsExtension(&#039;./devtools&#039;)
})

Originally, the React Devtools was installed at ~/.config/BraveSoftware/Brave-Browser/Default/Extensions/fmkadmapgofadopljbjfkapdkoienihi/4.21.0_0 (tilde is my home folder). If you’re trying to install on Atom editor, you’ll have to open developer tools and issue the following script:

r = require(&#039;remote&#039;)
r.BrowserWindow.addDevToolsExtension(&#039;./devtools&#039;) // or the right directory

This command will fail, but somehow when you reload Atom, the React Devtools will be there. There’s only one feature that I use – highlight which elements were updated. And that’s where our performance tuning starts again.
(more…)

By Maurício Szabo, ago

The magical wonders of programming with pure data

I’ll start with a disclaimer – if you’re developing in ClojureScript because of performance, you’re doing the wrong thing. Although a Clojure code can be really close to Java performance, ClojureScript simply can’t match JS performance – specially because in Javascript, the fundamental data structures like arrays, sets, and objects are implemented in a faster language, and as of today, no “user space” data structure can get even close the “native” ones. And, let’s not forget that JS is slower than Java.

So, if you want to beat JS performance in ClojureScript, you have to make use of what ClojureScript offers you – the REPL, the fast experimentation, and the wonderful experience of programming with data. So, this is a tale of a huge performance improvement that I’m currently working in Chlorine.

Rendering EDN

In Chlorine and Clover, when you evaluate something, the result will be rendered in a tree-like way at the console. This works quite well for Clojure, because UNREPL makes structures that are too big small, and you can query for more data. So far, so good.

In ClojureScript, Clojerl, Babashka, etc, things are not so good. Big data structures will be rendered fully on the view. They can lock or crash your editor, they can occupy all your memory, etc. The reason for that is that tree structures are hard – when you’re rendering the “parent”, you don’t know what the children will be. Currently, in Chlorine, rendering (range 80000) locks my editor for 4 seconds until it calculates everything, layouts all data, etc… and I wanted to change this.

Reagent vs Helix vs React

I was always intrigued on how much performance hit I get when I use Reagent’s Hiccup data structure instead of React objects – after all, there must be some performance problems, right? After all, that’s the promise of Helix – to not pay this performance hit because you’re closer to what React wants.

So, first things first: profiling Javascript inside Electron IS A NIGHTMARE. I decided to install React Devtools extension on Electron (inside the Atom editor) but that got me into so much trouble, false-positives, wrong profiles that I ended up deciding against it and simply used the “performance” tab to do my profiling. There, I could see batching.cljs hogging my performance, so the next move was easy – move away from Reagent and use Helix.

Except… that Helix uses some macros and I ended up doing the code in React. My testcase was simple: render a vector of 80000 elements and see how it would perform, obviously without all the bells and whistles that Chlorine offers today (otherwise this experiment would be waaay longer). And that’s where things get surprising: with Reagent, I was hitting 1800ms of scripting, and about 120ms of rendering. With React… 1650ms of scripting, and about 200ms of rendering. I decided to do more benchmarks and probably because of OS caching, warm-up, or whatever, the results got even closer, with Reagent sometimes performing better than React – but still too slow.
(more…)

By Maurício Szabo, ago

The state of JS in 2021

So… I recently found out that I had no space left on my SSD. When I tried to debug what was consuming my space, my projects folder were the culprit. How? I mean, I take photos, store them in RAW format, so how source code can be consuming more than my photos?

Turns out that by running rm -Rf (find -name 'node_modules'), (FISH shell) I was able to free 20gb of space!

So… let’s see how bizarre is the Javascript ecosystem in 2021.

Strangely big libraries

Sometimes, when I have to generate projects, they ask me to install yo. Yo is just a CLI to run generators – it does not include any generator. Yo occupies 89mb of space in my disk. Installing the react-webpack (manually by running npm i generator-react-webpack, because otherwise yo will try to install globally) means that your node_modules now occupies 180mb.

Let’s thing a little bit about this number: Windows XP, minimalist install without any add-on, requires 1.5gb of space. So a SINGLE GENERATOR occupies 12% of a full operating system.

So, create-react-app is more gentle – it just installs under 5.3mb. Generating a react app creates a node_modules folder with… 242mb. One of the dependencies is global-modules, that is just a single file with an if to define how to call things. It’s also only used in a single file – react-dev-utils/printHostingInstructions.js – in the folllowing code:
(more…)

By Maurício Szabo, ago

The power of Pathom

Most people don’t know the power of Pathom. Most people that I know of think that Pathom is about graphs.

They are wrong.

I mean, yes, in the end you’ll have a graph, with dependencies, but that’s not the point. The point of Pathom is the ability to transform your code into a soup or attributes. It’s also probably the best usage of qualified keywords for me, if not the only one that justifies the downsides (I’ll not enter in details here – just know that having to convert from qualified keywords to unqualified multiple times is not fun at all).

The name “soup of attributes” may not be a beautiful one, but believe me – it’s incredible. The idea is quite simple – instead of trying to handle all possible conversions from multiple sources to multiple destinations, you just define which attributes can be computed in terms of others, and Pathom does the rest. As always, things are better with examples, so let’s go.

I had to work on a system that somehow had strange rules – it needed to generate a bunch of text files for different companies. Each company expected a different file name and different fields. To generate the file, we had to accumulate data that came from a payload, from an external system that we called via REST API, and also from some data we had on our database. To make things worse, some companies would expect some of the data that was returned from REST on the filename, and there were also some state changes – like, if a file was already processed, the company would send us a return file, and we had to read some content of this file, move this file to another directory, renaming the file in the process.
(more…)

By Maurício Szabo, ago

Your editor as a query

Some time ago I found out this post: https://petevilter.me/post/datalog-typechecking/. It stroked me as really interesting – because it was an idea to make some IDE not depend on some internal API that only works for a single editor, but somehow make a tooling that’s query able by datalog. This datalog Read more…

By Maurício Szabo, ago