The perfect programming language (for me)

Disclaimer: the perfect programming language does not exist. Even if it did, different people want different things, so probably the ideas in this post would not reflect the ideas from different people. With that being said, let’s start with a little background:

I usually prefer dynamic languages. There are also moments when I miss static typing, but most of the time the “code/solution exploration” and a good REPL (and code design) do the job of reasoning about the shape of my data.

Except when it doesn’t. Then, things get ugly. Real fast. There’s always bad code that you need to work with, even one that you wrote about six months ago, and now you’re dumbfounded, looking at the code trying to remember what the hell were you thinking when you wrote the code and why you did think it was a good idea to wrote it that way at all. It happens with everyone. And that’s when static typing can (and will) help: it reasons about your data. You can have a variable named a, but at least you know it have the fields b: String, c: Int, whatever that means. But it helps.

Maybe we could have a language that allows you to turn on/off the typing whenever you wanted? With better REPL support? So, this would be my dream language to work with.
Continue reading

Posted in Clojure | Tagged , , , | Leave a comment

My missing frustration with ClojureScript

I’ve talked about at another post on how ClojureScript frustrates me, mostly because I was doing some Node.JS work and Figwheel simply wasn’t working correctly. Now, it’s time to revisit these points:

A little update: I talked a little with Thomas Heller, Shadow-CLJS creator, and he pointed me some issues with this article, so I’ll update it acordingly

Tooling

Figwheel and Lein are not the best tools to work with ClojureScript. Since I discovered shadow-cljs, things are working way better than before: I can reload ClojureScript code from any target, and I’m even experimenting with Hubot (and it works really fine too). The only thing I’m missing is my profiles.clj file, but I can live with that (and I can always use Shadow with Lein if I need profiles.clj).

Also, I’m working on a new package for Atom (and in the future, for another editors too) called Chlorine. One of the ideas is to offer better ClojureScript support (we have Autocomplete now!), using Socket REPL for solutions (even self-hosted REPLs like Lumo and Plank) and even wrap UNREPL protocol in Clojure. So far, is still in the very beginning but things are looking promising!

The stack

Forget Figwheel at all: Shadow-CLJS is probably the best tooling for ClojureScript ever. It auto-reloads ClojureScript code for the browser, for node.js, for node modules, and it integrates with almost everything that you want. It controls release optimizations, have sensible defaults, and even have post-compile hooks (so you can hook Clojure code to do something after some compilation phases). Also, it integrates with node-modules (no more maven-wrappers for JS libraries!) and have some warnings when you use some kind of ClojureScript code that would break :advanced compilation. And, let’s not forget that you can control the refresh reload phase, it adds a userful :include-macros in ns form (that will include all macros from the namespace being required), and controls exports in a sane manner. But first let’s begin with the feature that I found most useful: :before-load-async.
Continue reading

Posted in Clojure | Tagged , , , , , , , | Leave a comment

A new way to write Clojure on Atom

This will be just a simple anouncement: I’m publishing my (still very crude!) plug-in for Clojure’s Socket REPL on Atom.

Chlorine was an experiment that grew more than I could imagine: I wanted a better way to write ClojureScript on Atom, and that’s the reason I made Clojure-Plus (also, I wanted to seamlessly integrate CIDER and other refactoring/helper libraries on my workflow). But there were lots of bugs, problems, with my approach: I was using CoffeeScript (the default language on Atom packages, at the time) to write my plug-in. This meant lots of conversions from Clojure to JS, and every single bug I needed to reload atom to re-activate my plug-in.

Then, I tried to work with Figwheel. Atom packages are a mix of Node.JS with Browser target: figwheel was simply a lost cause (lots of problems when reloading when it didn’t knew what to reload, inconsistent state, flaky compilation failures that sometimes did work, sometimes not, without any logic). When I migrated to Shadow-CLJS, things worked a lot better.

Then, Microsoft bought Github. Atom always had a bunch of things that I did not like (quasi-semantic versioning, lots of feature requests that were simply ignored, multiple bugs that were not being addressed, performance issues) and with Microsoft buying Github (and already having an electron-based editor – VSCode), I’ll not wait to see Atom becoming obsolete. Most of the people that I work with don’t use Atom anymore, and it’s kinda difficult to see people working with Clojure with Atom too, so I started to think on a better way to port my work to other editors.

Then, it came the idea: most editors nowadays can run Javascript. Maybe I could work with ClojureScript and make a layer that would be used by all editors, and just code the parts that are specific for Atom in a separate plug-in. Then, it came the idea for REPL Tooling.

Then, I ended up too far on the rabbit hole: I lost track multiple times of what I was doing on REPL Tooling. So, for now, I’m developing both packages in parallel (if you look at Chlorine repository, you’ll see a submodule registered) so that I could code a tool, integrate immediately on Atom, then see the results in real time. This became a much better workflow (at the expense of unit tests for now). And then, one day, I was working on a ClojureScript project on my current job, and after multiple auto-completes, evaluates, and other things, I realized that my plug-in was good enough to be usable (and in a way that I would prefer to use it instead of other solutions).

So, I’m publishing it. Expect bugs (lots of it!), but it’s a start. I’ll try to release every new feature as soon as possible (possibly in an automated way – who knows?), and there are still multiple things to do (go-to definition, for instance, is completely absent) but I’m sure that there are lots of interesting things to happen!

Posted in Atom, Clojure | Tagged , , , , , , | Leave a comment

Atom packages with ClojureScript – upgraded

Last time I talked about plug-ins in atom with ClojureScript, I was using Weasel. Since then, I tried figwheel but it never worked as good as I wanted.

Then, I’ve decided to try shadow-cljs. Also, with shadow, we can build a node library instead of a “generic node application”, and this helped a lot in my current tests. So, right now, I’m going to show how I am developing the next version of Clojure Plus plug-in, and what to expect in the future.

Also, I must add that this is so far the best experience ever on creating Atom packages, so I’ll probably stick with ClojureScript for every future package too (I just need a way to make atom’s spec tests work better with ClojureScript – I’m thinking about using a helper library or something).

Preparation

First, you’ll use atom to create a package. It doesn’t matter if you produce a CoffeeScript or Javascript version, because we’ll delete all source files.

On package.json, we’ll modify the entrypoint: on the key "main":..., we’ll write "main": "./lib/main".
Continue reading

Posted in Clojure | Tagged , , , , , | Leave a comment

Understanding Ruby’s Awesome Nested Set

Recently, I saw people migrating from Awesome Nested Set to Ancestry. The reasons are simple – Ancestry is very simple, it just need a new string field in your table, and it’s easier to reason about. So, why should I even consider to use an alternative?

The answer is simple, and I’m going to quote H. L. Mencken: For every complex problem there is an answer that is clear, simple, and wrong.

And the reason that using Ancestry is a bad idea is simple: it doesn’t reflect good design. It treats a single field as a multi-valored column that keeps the ancestry of your object. This means that there’s no way to fetch all records AND their parents in a single query, or fetch all records AND their children, because we need to split the string (ruby-side) and then create a query (also ruby-side). So, in this post, I’ll show some tricks of what we can do with Awesome Nested Set, or even beter, how does it implements the tree pattern (and why it calls it a “set”, not a “tree”).

First, when we think about categories and sub-categories, we think like this:

But this is, in fact, a terrible way of representing trees in SQL. So, the alternative, is to transform it in a group of sets: Parent 1 (P1, for short), is a super-set containing subsets C1 and C2 (Children 1 and Children 2, for short). Each of the children have its own grandchildren, so C1 is a super-set containing G1, and C2 is a super-set containing G2 and G3.


Continue reading

Posted in Ruby | Tagged , , , , , | Leave a comment

Clojure with SQL databases

There are times when a specific piece of technology captures my attention in such a way that I feel the need to study more, and do things with it. Since I began to study Clojure, and after two jobs working with it, I’m still in love with it – I think it matches what I expect in a language most of the time, and also matches my repl-driven-development workflow.

So why I’m not considering it for new projects? Even personal ones?

When I need to prototype something really fast, I end up going for Ruby, with Sinatra (sorry rails, but you’re a terrible choice for me). I saw that in my last project, I worked with Sequel, postgresql, and Sinatra. For tests, I used VCR and recorded the external api calls that I had to make.

Well, I could probably prototype a simple VCR that would record a specific function call in Clojure. Sinatra, I could use Compojure. As for Sequel… Or ActiveRecord, for that matter…

There are a few libraries that can be used to communicate with SQL: Korma, that doesn’t support select ... for update and have some implicit connection problems and other issues, HoneySQL that interprets Clojure maps and converts then to SQL, and lots of other libraries that simply ask you to write your query and then, somehow, call then for you. I didn’t care too much for the later, after all, if I want to write queries I prefer to not depend on a library that does it to me – I know that there are people that don’t like to have SQL in the middle of the code, but I don’t see any problem with it.
Continue reading

Posted in Clojure | Tagged , , , , , | 2 Comments

To be wrong, even when you’re not

There’s a time when we must assume that we are wrong – even when we are doing the right thing. At my last job, I realized that it was happening with me.

After Nubank, I began to work on a company that, in a way, was a bet in my career – most of the systems in this new company were written in C# (a language that I had never written a single line), there was a huge codebase without a single line of test, the systems did not run on the developer’s machines, and one of the reasons that I was interested in working with then was to organize this codebase, in Clojure, with microservices. Also, to coach the development team, so we would migrate away from monolithic systems to a better architecture, with tests, microservices, and multiple programming languages.

In other words, to apply what I learned at Nubank in this company.

It was a long journey, first to be able to make the code, at least partially, compile on my Linux machine (the default configuration of our development machines was Windows with Visual Studio). And no, there wasn’t simply a matter of swimming against the flow that made me want to work on Linux: it was simply because the mandatory configuration of our Windows workstations was a very slow machine with a very slow antivirus and monitoring systems (to see if anyone was installing pirate software). Also, we had no access to USB, could not download external software, and our antivirus sometimes identified our git blobs as “suspicious software”, effectively corrupting our code every git commit. My machine booted in 10 minutes, and I had to wait another 20 for Visual Studio to be responding (and I’m not faking these numbers).

Then, the migration to Clojure. It wasn’t really that difficult to write the code. The problem was to publish it – the company did not want to use docker, nor containers, nor continuous integration/delivery. We migrated our code from TFS to Bitbucket to GitLab, then to a self hosted GitLab, then back to Bitbucket. Also, we had to fight – a lot – to be able to access our Q/A docker environment, as we had no access to production, staging, or homologation environments. We caught lots of bugs only in production, because we simply had no idea on how the calls for external APIs worked (there was not a single mocked/stubbed external system, and the very few external services that did offer sandboxed environments were so misconfigured that even API versions were different).

Then, after a long fight, more than a year later, we lost.
Continue reading

Posted in Work and Jobs | Tagged , , , , , , , , | Leave a comment

Don’t be a Copperfield

After a year and a half working in other languages, I’m back to Ruby. And, one of the things that I was amazed at Clojure is how simple things were.

Back to Ruby, I’m really surprised on how people overcomplicate things.

There’s no perfect language, nor perfect community of languages. In Clojure, there are fewer abstractions, which is not exactly a good thing – if you, let’s say, just want to create a simple page to show data from a database, you’ll find it extremely tedious to do it in Clojure, where in Ruby/Rails it’s just a few lines of code away. Ruby’s moto is “programmer happiness”, and this reflects in every library that they write.

But this kind of higher abstraction pays off with time. At least where I live, working with Ruby means working with Rails almost all the time. There are no “big competitions” for Rails or ActiveRecord (Sequel is a close one, but at the time of this post, AR have 10 times more downloads than Sequel), and other web frameworks are mostly “Rails-like”. But what bugs me most is “magic”.
Continue reading

Posted in Ruby | Tagged , , , , | Leave a comment

Two new libraries in Clojure

Last week, I was looking to some old code I wrote in my last job and my spare time. Then, I’ve decided to publish two new libraries for Clojure and ClojureScript.

One is Paprika, available in Clojars at version 0.1.0-SNAPSHOT.

The other is Check, also available in Clojars (but at version 0.0.1-SNAPSHOT).

The reason for the early publishing is to push forward some simple libraries to fix a simple problem that I had while working with Clojure: the absence of abstractions.
Continue reading

Posted in Clojure | Tagged , , , , , , , , , | Leave a comment

My frustration with ClojureScript

Last time I worked with Clojure, I was working in a multi-language team, and somebody wanted to do microservices with Node.JS. We already had a library for microservices with Clojure, and I wanted to help then. We did a simple proof of concept where we migrated parts of our libraries to use ClojureScript too, and we did rewrite some tests so things would work correctly in both platforms (even in Node.JS, when we made a “translation layer” for it to work).

Then, I’ve found out how difficult is to test ClojureScript code, and made a bunch of macros to help me test async code, with timeouts and other tweaks. Now, I’m trying to extract this library to be usable for multi platforms.

And then, the headache began:

The Stack

We can use ClojureScript to generate JS for the browser, or for Node.JS. Unfortunately, when we’re trying to generate code for Node.JS, some options need to be added, some removed, and we have a bunch of problems with ClojureScript libraries being exported as Google Closure Compiler libs, in place of Node.JS’ require statement. Also, it’s not really clear how to expose ClojureScript code to JavaScript, as every specific optimization configuration uses some strange code to call functions.

In our experience, :optimizations :simple worked best for Node.JS. :advanced broke our code in ways that we did not understand why, :whitespace doesn’t work with Node.JS output, and other bizarre issues I found. Also, to avoid leaking goog global variable, I used :output-wrapper true, one simple flag for cljsbuild that it’s incredibly difficult to see at documentation.

Then, one of the things I wanted to do is to use the same API for both Clojure and ClojureScript. This means, mostly, that I wanted to re-utilize all code that I could using .cljc files.

And there’s the catch: ClojureScript works really well with .cljc files… unless, of course, you skip the part where macros must be written in Clojure, requires are incredibly different between each platform, and different levels of optimizations give us different results when we’re using cljc and reader conditionals… so, let’s start with the first:

The Macro Problems

Let’s say that I want to write a macro for, let’s say, an arrow for assertion, much like Midje, using some external library. So, instead of saying (is (= "foo" my-str)), I want to be able to say (check my-str => foo). For more information, check my library check (where I used the wonderful expectations library to make my asserts):

; In check/core.cljc
(ns check.core)

#?(:cljs (require '[some-cljs-assert :as assert-cljs])
   :clj (require '[some-clj-assert :as assert-clj]))

(defmacro check [actual arrow expected]
  (assert (= arrow '=>))
  #?(:cljs
     `(is (assert-cljs/assert ~expected ~actual))
     :clj
     `(is (assert-clj/assert ~expected ~actual))))

Except that this doesn’t work. When we’re compiling for ClojureScript code, this will interpret the file as a Clojure file, so our reader conditionals will try to require different code, and will use the clj version of our macro.

There is a way to tell ClojureScript, when requiring macros, that it’s supposed to use ClojureScript version, but is hacky.

The “Hacky” Version

This would create a macro that, at macro-expansion time, will use the correct version for Clojure or ClojureScript. Except that it doesn’t work correctly too: ClojureScript is very strict about their requires, and while I was programming check, I found that in development mode (no optimizations, figwheel running, etc) it worked correctly. But I also want to be able to run this code on a CI, so I’ve created a cljsbuild profile for test, and that didn’t work: it gave me an error saying SEVERE: : ERROR – required namespace never provided.

The odd thing is that this namespace changed over time: sometimes, it was “cljs.core.async” (that I also use on my library), other times were “expectations”, etc. And the really odd thing is that it only occured when I tried to use :simple optimization, so when I was using, :none, things were working fine.

The really ugly

Sometimes, there’s simply no way to work around these limitations. In my project, I’ve tried multiple possibilities to allow the namespace check.async for both Clojure and ClojureScript. Unfortunately, there wasn’t a single option that I was aware of that made this possible: when I tried to use a cljc file, I had to use the horrible if-cljs macro, and that made Closure Compiler crazy with errors. I had to require specific namespaces for Clojure and ClojureScript, alias then with different names, and even then things didn’t work-I was still getting errors. Then, I’ve just gave up and created a new namespace: check.async-cljs, so that the macros would live in a different clj file.

And yet, that didn’t work either!

Somehow, I was still getting required namespace never provided. To solve it, I needed to create a check/async_cljs.cljs file, and put the following trivial code:

(ns check.async-cljs
  (:require [clojure.string :as str]
            [cljs.core.async :as async]))

Why? Because it informs to ClojureScript that I want to require these namespaces in my ClojureScript file, when I’m using the macros from check/async_cljs.clj. It simply doesn’t matter that I already required these same namespaces, with these same aliases, in my .clj-it’s a Clojure file, and it’s ignored by cljsbuild.

Also, another problem is that when I’m using my check library, there’s no easy way to “require” things in a way that works for both platforms. Again, if I try to (require 'name.space) with reader conditionals, depending on optimization level, Closure Compiler will not understand that it needs to add that library to the final JS file.

Conclusion?

Although the idea that Clojure and ClojureScript, in a single project, it’s amazing, I simply wouldn’t recommend when there’s a time box to make your project work. Seriously, it’s a lot of work with lots of issues that I simply don’t know why they exist.

Althrough slow, I would probably still prefer a experimental flag on cljsbuild to be able to write macros in ClojureScript. This would probably slow a lot the compilation process, but it’s would make things easier to program. Also, figwheel is an amazing technology, but we need to be aware that it allows invalid ClojureScript code to be compiled and tested, so it’s essential that we always have a cljsbuild profile for testing.

Posted in Clojure | Tagged , , , | 3 Comments