The state of JS in 2021

So… I recently found out that I had no space left on my SSD. When I tried to debug what was consuming my space, my projects folder were the culprit. How? I mean, I take photos, store them in RAW format, so how source code can be consuming more than my photos?

Turns out that by running rm -Rf (find -name 'node_modules'), (FISH shell) I was able to free 20gb of space!

So… let’s see how bizarre is the Javascript ecosystem in 2021.

Strangely big libraries

Sometimes, when I have to generate projects, they ask me to install yo. Yo is just a CLI to run generators – it does not include any generator. Yo occupies 89mb of space in my disk. Installing the react-webpack (manually by running npm i generator-react-webpack, because otherwise yo will try to install globally) means that your node_modules now occupies 180mb.

Let’s thing a little bit about this number: Windows XP, minimalist install without any add-on, requires 1.5gb of space. So a SINGLE GENERATOR occupies 12% of a full operating system.

So, create-react-app is more gentle – it just installs under 5.3mb. Generating a react app creates a node_modules folder with… 242mb. One of the dependencies is global-modules, that is just a single file with an if to define how to call things. It’s also only used in a single file – react-dev-utils/printHostingInstructions.js – in the folllowing code:

function printStaticServerInstructions(buildFolder, useYarn) {
  console.log('You may serve it with a static server:');
  console.log();

  if (!fs.existsSync(`${globalModules}/serve`)) {
    if (useYarn) {
      console.log(`  ${chalk.cyan('yarn')} global add serve`);
    } else {
      console.log(`  ${chalk.cyan('npm')} install -g serve`);
    }
  }
  console.log(`  ${chalk.cyan('serve')} -s ${buildFolder}`);
}

It literally is just used to know if it will print npm or yarn on the console. AND THAT’S ALL!

Strange dependencies

So, a react app will pull: emoji-regex and emojis-list. Why? Well, to… encodeStringToEmoji. And to somehow get the “string-width”, so that this code:

sw = require('string-width')
sw('foo😀') // returns 5
'foo😀'.length // instead of this code, that returns 5
// Irony?

Literally useless dependencies

It pulls for-in and get-own-enumerable-property-symbols. The implementations for both of them is:

  for (var key in obj) {
    if (fn.call(thisArg, obj[key], key, obj) === false) {
      break;
    }
  }
// and

export default (object: object): symbol[] => Object
  .getOwnPropertySymbols(object)
  .filter((keySymbol): boolean => Object.prototype.propertyIsEnumerable.call(object, keySymbol))

hex-color-regex generates (yes, GENERATES) a regex to match colors; indexes-of is bizarre in its own way: the README says that its “a 5 line module” in haiku format; There’s also its own share of is-<something> libs: in fact, 44 of them. There’s is-regex and is-regexp; is-plain-obj and is-plain-object; There’s… is-negative-zero.

IS
NEGATIVE
ZERO!!!

Math should be proud! It’s used by es-abstract to implement StringGetOwnProperty. If you look at the spec, StringGetOwnProperty(S, P) must get the property P from S, provided that P is a… STRING. So, why not replace the dependency with P === "-0"?

There’s a library called del. It… deletes a file using glob strings. It uses 8 dependencies, including is-path-cwd, is-path-in-cwd, is-path-inside, and path-is-inside. All these libraries are used to implement safeCheck – a simple sanity check. In facts, is-path-in-cwd is literally isPathInside(path, process.cwd()), and is-path-cwd is, surprisingly, path.resolve(path_) === process.cwd(), with a sanity check for Windows systems.

There’s also a library that “Transform SVG into React Components”. It’s an webpack plug-in, and depends on some filesystem libs, a YAML parser, and pulls a helluva lot of “svgr” libs that have, AT MOST, 25 lines of code to do menial tasks like “adding an attribute”, “remove an attribute”, “remove an empty attribute”, and “replace an attribute”. I’m not kidding – these are separate libraries that do basically what a single DOM API is capable of doing. And it gets worse: these SVG libraries pull a library that transforms SVG into a React component, and one that transforms a SVG into JSX. And, one that transforms SVG into JSON. Again – none of these libraries are pulled from React – they all are pulled from react-scripts, to… run tests? I simply have no idea. I can’t fathom why we must have a full serializer/deserializer of SVG from/to multiple formats in a “Hello, World” app.

What’s even more weird? Most of these dependencies are pulled from react-scripts. By the name of this library, it seems that it’s just a library to be able to run react-scripts <something>. It’s probably not – or at least, I hope.

Strange-ish cases

has-value depends on get-value and has-values. I really have no idea why do we need a library that allows you to get if a nested element contains values inside of it, one that gets deeply nested object passing a string as the path, and why a singular lib depends on a plural one.

Lots of libraries have to check if a type is equal to other. I really have no idea why, and probably it’s because of testing. Speaking of test… the create-react-app don’t produce a package.json with devDependencies – everything is inside the dependencies key, probably because it’s supposed to work on a browser env.

As for testing, it includes @testing-library/jest-dom, @testing-library/react and @testing-library/user-event. None of these include jest – no even jest-dom. The dependency that DO include jest is… react-scripts. So, if I want to use another test framework, I loose the JS compilation and bundling (because these are on the react-scripts), the babel and webpack configs, the watch, etc.

Jest itself pulls some strange dependencies – it pulls JSDom that also pulls a CLI options parser (WHY?). In fact, why even Jest is installed? Can’t the user choose what he want to use to test libraries? I suppose that the reason for this multitude of deps is because of snapshot testing, a tool that I find almost a criminal offense (it’s so brittle that there’s literally a keystroke that says “every snapshot that’s incorrect, delete them and re-generate”. If you have a keystroke to say “my test is wrong, ignore it and then re-generate the assertions” I can only imagine that there’s something wrong with the kind of test).

There are also lots of libraries makes node’s fs library easier to use. Some of the dependencies use these libs – others don’t. These libs also can monkey-path the original fs library, and I really hope they don’t use these features on a simple Hello, World app. Even all things considered, there are lots of patterns to avoid this: functional composing, adapter, facade, etc – why these libraries don’t use any of them is an exercise in itself. The problem: these libraries are used for multiple other projects, so if one of them use the monkey-patching feature…

Changes in the ecossystem

First, came npm. And it was BAD. Well, yeah, it still is, but it was waaay worse – some libraries only installed with a specific version of npm, and it was close to impossible to make a node.js app works two weeks after it was written. Now, it’s a little more stable – at least, only libraries that depend on binaries can break. Then came yarn, and pnpm. yarn is better, more usable, but pnpm is the only one that makes a global cache of your libraries, so you don’t install the same 300mb of node_modules over EVERY SINGLE PROJECT that uses React.

For example: for my tooling libraries all depend on repl-tooling. My node_modules for REPL-Tooling occupies 200mb. The same is copy-pasted in Chlorine, Clover, and Clematis (because they all must depend on the same libs). Right now I’m experimenting on porting Chlorine to CodeMirror, and thinking about a different take on VSCodium’s plug-ins. If I do add these 2 new repos to the list, I’m occuping a Windows XP minimal instalation just for JS dependencies. You know what’s weird? When I pack these libraries on Chlorine, it occupies 2,85MB of space. So – I have 200mb that will become less than 3MB when dead code is removed (well, that if you minify also. If you don’t, then the size is 8MB, or 4% of the original size)…

So, libraries. People used to use Prototype, but it monkey-patches core libraries, so they migrated to underscore with jQuery. There was an epoch where Angular was famous, then died when the maintainers decided to create a new framework that’s almost completely incompatible with Angular and called it Angular 2. Then came React, and Vue, and both are good in their own ways – but state is hard to manage, so came Redux… even then it became complicated, so people migrated to TypeScript with Flow (but then the Flow creators told everybody “this tool is really opinionated” so people went back to TypeScript) and now we have TypeScript over JSX that indirectly manage state over Redux…

As for bundling, there’s webpack that’s incredibly hard to config. The fact that it changes API every major version makes it even worse, because now you have to keep track of these changes. There’s also parcel, rollup, terser, and now there’s also esbuild, that is written in go. In fact, esbuild have a webassembly version.

Oh, webassembly. Yeah, originally it was a JS with weird things that became a binary format, and now is both the binary format and a companion JS that exposes some APIs. You must call an API to compile WASM into something that you can work with, and it’s FAST. But the compilation must happen on different places depending if you are on Node or on the Browser, and the API is slightly different and… well, you get the idea. In fact, exactly the same as everything JS in the past years, you probably have to change things to be able to run your .wasm file if you haven’t touched it in a while, so… here we are again.

From JS to JS

Let’s not forget about babel – an absolute required tool to transform your JS into… JS. Because why add macros to a language if you can write a compiler that receives JS with weird things and transform these weird things into valid JS? Oh, and it’s a compiler pipeline, so it means that if your compiler sees some weird thing that it doesn’t understand, it NEEDS TO ignore it and pass it unchanged to the next compiler in the pipeline, so that compiler can do its work. Sounds weird? That’s exactly what happens if you use Flow, or Typescript, with React’s JSX. And, obviously, the extension for Flow files and for JSX is… .js.

Imagine explaining this to someone that was never exposed to Javascript in their life?

You: Yeah, so, you create a file hello.js that’s a Javascript sourcecode.
Student: Great! So can I run this on any Javascript interpreter?
You: Not really, because although it’s a Javascript file, it’s not really Javascript, so you have to write a config for Babel, a compiler for JS.
Student: Ah, and Babel will compile this Javascript file into a binary?
You: No, into another Javascript file that’s valid Javascript, so you can run it
Student: Now can I run everywhere?
You: Well, mostly. You can run on Node, that’s the backend. For frontend, you have to pack everything in a single file with webpack.
Student: Right, but if I want to run on the backend, can I run the babel output?
You: Yes, if the files use CommonJS format.
Student: Wait, if it’s CommonJS, then shouldn’t be… common? I mean, everyone uses?
You: Yeah… but only Node.JS uses, browser doesn’t support, other JS machines don’t also support, and there’s this whole issue that we’re migrating away to ES Modules
Student: Ok, so if I migrate to ES Modules, can I use it anywhere?
You: Errr, not really, again, only on Browser, or Node after a specific version and only if you add a key to package.json
Student: WHAT? So if I want to use the new feature I make it incompatible with everything else?
You: Pretty much.
Student: Why are we doing this again?

So… that’s the question. What are we doing?

And one more thing

Recently, “generators” happened. It’s a new syntax that allows one to “generate” values:

function* generator(i) {
  yield i;
  yield i + 10;
}
const gen = generator(10);

console.log(gen.next().value);
// expected output: 10
console.log(gen.next().value);
// expected output: 20

I’m not really sure why this is a good idea, to be honest. What’s different than:

function gen(i) {
  const values = [i, i+10]
  return function() { return values.shift() }
}

const gen = generator(10);
console.log(gen());
// expected output: 10
console.log(gen());
// expected output: 20

Andd….. they defined a new concept, called the “async iterators” and “async iterables”. The syntax is bizarrely confusing, doesn’t seem to “fit” with normal Javascript, and to traverse it it’s awkward to say the least:

const asyncIterable = {
  [Symbol.asyncIterator]() {
    return {
      i: 0,
      next() {
        if (this.i < 3) {
          return Promise.resolve({ value: this.i++, done: false });
        }

        return Promise.resolve({ done: true });
      }
    };
  }
};

So… how did we come to this?