The power of Pathom

Most people don’t know the power of Pathom. Most people that I know of think that Pathom is about graphs.

They are wrong.

I mean, yes, in the end you’ll have a graph, with dependencies, but that’s not the point. The point of Pathom is the ability to transform your code into a soup or attributes. It’s also probably the best usage of qualified keywords for me, if not the only one that justifies the downsides (I’ll not enter in details here – just know that having to convert from qualified keywords to unqualified multiple times is not fun at all).

The name “soup of attributes” may not be a beautiful one, but believe me – it’s incredible. The idea is quite simple – instead of trying to handle all possible conversions from multiple sources to multiple destinations, you just define which attributes can be computed in terms of others, and Pathom does the rest. As always, things are better with examples, so let’s go.

I had to work on a system that somehow had strange rules – it needed to generate a bunch of text files for different companies. Each company expected a different file name and different fields. To generate the file, we had to accumulate data that came from a payload, from an external system that we called via REST API, and also from some data we had on our database. To make things worse, some companies would expect some of the data that was returned from REST on the filename, and there were also some state changes – like, if a file was already processed, the company would send us a return file, and we had to read some content of this file, move this file to another directory, renaming the file in the process.

So, let’s break down some cases:

1. Simple Company INC requires a filename containing <current-date>-payload.txt
2. Complex Company INC requires a filename containing <current-date>_<our-db-id>_payload.txt
3. Bizarre Company SA requires a filename containing <current-date>-<our-db-id>-<some-data-from-rest>-payload.txt
4. Bizarre Company SA ALSO requires that we read the filename <current-date>-<our-db-id>-<some-data-from-rest>-payload.txt.ret, get some data from inside of it, and then move this .ret file to a specific folder, based on the contents of the file

If it’s not yet clear, I’ll say – that project was a nightmare! But let’s see how to implement this code in Pathom: I’ll just generate the code to check for the filename for specific companies for now – starting for the simplest case:

(require '[com.wsscode.pathom3.connect.operation :as connect]
         '[com.wsscode.pathom3.interface.eql :as eql]
         '[com.wsscode.pathom3.connect.indexes :as indexes])

(connect/defresolver current-date []
  {:time/now (java.time.ZonedDateTime/now java.time.ZoneOffset/UTC)})

(connect/defresolver filename [{:keys [time/now]}]
  {:file/name
   (let [f (java.time.format.DateTimeFormatter/ofPattern "yyyy-MM-dd-HH-mm")];
     (str (.format now f) "-payload.txt"))})

(defn eql [query]
  (-> [current-date filename]
      indexes/register
      (eql/process query)))

We decide that a “datetime” is a different resolver, so that we’ll use the same “date/time” if it appears multiple times. We also decide that :file/name depends on the :time/now attribute (when you destructure the attribute on Pathom, it’ll auto-detect that these keys are prerequisites). Then, we register both “resolvers” on Pathom, and that’s it – if we need to get the current data, we can use (eql [:editor/filename]) and it’ll give us the filename.

Now, for the Complex Company INC. We need to (1) – somehow “seed” some data to Pathom (that we’re requiring info for one company or other) and (2) – pass a database connection to it. Both are quite simple: for the database, we can assoc it on the first parameter (that Pathom calls the “environment”) and the seed is just a pre-requisite that we can pass as an additional attribute for the query. We then add a resolver for the :db/id:

(connect/defresolver get-info-from-db [env {:keys [person/kind]}]
  {:db/id (get-in @(:db env) [kind :id])})

(defn eql [seed query]
  (-> [current-date
       filename
       get-info-from-db]
      indexes/register
      (assoc :db (atom {"Personal" {:id 10} 
                        "Company" {:id 20}}))
      (eql/process seed query)))

We then could change our filename resolver to depend on :db/id, but this would be wrong. As we saw, for the “Simple Company INC”, we don’t need the ID at all. So, how can we solve this issue?

The answer is simple: we re-design resolvers a little bit so that they don’t have conflicting names, and make a resolver for the DB only. Now, instead of querying just by :file/name, we’ll query for, let’s say, {:company/simple [:file/name]}, so it’ll give us the filename of “Simple Company INC”. Also, complex company needs some additional information, so we’ll have to give it as an “initial entity”:

(connect/defresolver simple-filename [{:keys [time/now]}]
  {:company/simple
   {:file/name
      (let [f (java.time.format.DateTimeFormatter/ofPattern "yyyy-MM-dd-HH-mm")];
        (str (.format now f) "-payload.txt"))}})

(connect/defresolver complex-filename [{:keys [time/now db/id]}]
  {:company/complex
   {:file/name
    (let [f (java.time.format.DateTimeFormatter/ofPattern "yyyy-MM-dd")];
      (str (.format now f) "_" id "_payload.txt"))}})

(connect/defresolver get-info-from-db 
  [{:keys [db]} {:keys [person/kind]}]
  {:db/id
   (get-in @db [kind :id])})

For this last resolver, we have two parameters. The first one is the “environment”, a map that can have arbitrary keys and values, as well as information about Pathom (that we can mostly ignore). In this resolver, we’re also expecting a db, and for the sake of simplicity, let’s say that’s just a Clojure atom that contains the person kind as key, and the value is the data for that kind. We’ll assoc it on the EQL, so we’ll have to change our query a little bit (both because we need to pass this “database” to the environment, and because we need to consider the entity we’ll be passing as initial parameters of our query):

(defn eql [entity query]
  (-> [current-date
       simple-filename
       complex-filename
       get-info-from-db]
      indexes/register
      (assoc :db (atom {"Personal" {:id 10}
                        "Company" {:id 20}}))
      (eql/process entity query)))

We can query our data with:

(eql {:person/kind "Company"}
     [{:company/complex [:file/name]}])
; => {:company/complex {:file/name "2021-05-27_20_payload.txt"}}
(eql {:person/kind "Company"}
     [{:company/simple [:file/name]}])
; => {:company/simple {:file/name "2021-05-27-15-25-payload.txt"}}

And now for the company that needs us to read their RESTFUL data:

(connect/defresolver restful-data [{:keys [db/id]}]
  ;; Simulate we're trying to get some data from REST
  {:restful/data
   (do
     (Thread/sleep 500) ;Delay...
     {:status 200 :body {:secret-stuff (str id "imasecret")}})})

(connect/defresolver bizarre-filename
  [{:keys [time/now db/id restful/data]}]
  {:company/bizarre
   {:file/name
      (let [f (java.time.format.DateTimeFormatter/ofPattern "yyyy-MM-dd")];
        (str (.format now f) "-"
             id "-"
             (-> data :body :secret-stuff)
             "-payload.txt"))}})

If you look at the code above, you’ll see that we have two places that are depending on our DB data. Pathom is intelligent to make that query only once, so the DB will not be queried two times (one for the bizarre-filename, and other for the restful-data). Now, if we want to implement the logic for the return file, it’s INCREDIBLY simple: we just need to depend on the “filename”, read things, and that’s it:

(connect/defresolver bizarre-return-filename
  [{:keys [file/name return-file/secret-stuff]}]
  {:return-file/name
   (str "/return/" secret-stuff "/" name)})

(connect/defresolver return-file-contents
  [{:keys [file/name]}]
  {::connect/input [:file/name]}

  {:return-file/secret-stuff
   (-> name (str "-ret") slurp parse-contents)})

(defn eql [entity query]
  (-> [current-date
       simple-filename
       complex-filename
       get-info-from-db
       bizarre-filename
       restful-data
       return-file-contents
       bizarre-return-filename]
      indexes/register
      (assoc :db (atom {"Personal" {:id 10}
                        "Company" {:id 20}}))
      (eql/process entity query)))

There’s something that may be seem confusing, that’s why we’re “flattten” here and not on :file/name, before. The reason is that Pathom works with entities and sub-entities: in this case, :bizarre/company is an entity, so every attribute inside :bizarre/company will be used as inputs for others. There’s no problem on it, except if you want to somehow support “multiple return files from multiple companies”. In this case, there are some solutions that we could use, for example, together with the filename return the :company/kind or something that would allow us to conditionally get the files.

Other thing that may seem strange is that we’re not checking for bugs. What if there’s no filename? Well, no worries – Pathom captures the error, and marks that resolver as “invalid” for this query, so you don’t even need to treat the bug. This, together with :connect/priority (a feature that’s still on its early stages) makes working in a “let it break” manner awesome – if something fails, but you can get that information from another path, no worries – Pathom will try to solve multiple paths.

Even more magic!

Now, suppose that people think that your code is too slow – maybe the RESTFUL API or the DB is slowing down, or whatever. So they “batch” some data for you, trying to group similar entries, and it’s your job to make the code correctly. On other approaches, you probably would have to change code to “pre-cache” some attribute, and then dismiss it after the batch was processed. On Pathom? You just make the right query, combining the power of “sub-entities” – you just need give a key for each data, and make the query:

(eql {:person/one {:person/kind "Company"}
      :person/two {:person/kind "Company"}
      :person/three {:person/kind "Company"}
      :person/four {:person/kind "Personal"}
      :person/five {:person/kind "Personal"}}
     [{:person/one [{:company/bizarre [:file/name :return-file/name]}]}
      {:person/two [{:company/bizarre [:file/name]}]}
      {:person/three [{:company/complex [:file/name]}]}
      {:person/four [{:company/simple [:file/name]}]}
      {:person/five [{:company/simple [:file/name]}]}])
; =>
; {:person/one {:company/bizarre {:file/name "2021-05-27-20-20imasecret-payload.txt"}}, 
;  :person/two {:company/bizarre {:file/name "2021-05-27-20-20imasecret-payload.txt"}}, 
;  :person/three {:company/complex {:file/name "2021-05-27_20_payload.txt"}}, 
;  :person/four {:company/simple {:file/name "2021-05-27-20-25-payload.txt"}}, 
;  :person/five {:company/simple {:file/name "2021-05-27-20-25-payload.txt"}}}

And this will only query the database once, and also only query the RESTFUL once too. It’s also guaranteed that they will all have the same timestamp too. There are still lots of other great features: you can “batch” a resolver (so instead of querying one by one, you can query people at the same time, avoiding the famous N+1 problem), you can have plug-ins, you can use it in ClojureScript (and it’ll resolve promises for you, so you write everything as if it was synchronous) and a parallel resolver is coming so Pathom will be able to run things in parallel!

So, although Pathom is still about graphs, and there are people that call it “GraphQL for the backend”, in the end it’s a great tool that may be a killer feature in some projects. There are some tools that I am quite sad that I only tried recently, and I’m 100% certain that Pathom is one of these!