I recently took a foray into ClojureScript after having come from a background in Clojure development. I was surprised to find that setting up a REPL was quite non-trivial when compared to Clojure. In this post, I’ll make an attempt at grokking the basic architecture for ClojureScript’s REPL implementation.

Overview

My primary source of confusion as a ClojureScript beginner is that there is seemingly no equivalent to lein repl that immediately launches you into a command-line REPL for your project with no additional configuration. Instead, the current best practice involves use of third-party plugins that entail various modifications to your application code or Leiningen configuration to enable REPL functionality. I should say that once you understand what’s going on, it doesn’t seem very complex anymore, but to a beginner it’s rather opaque.

Before I get into how things actually work, I think it’s a fun exercise to imagine how a hypothetical ClojureScript REPL could work. Then, based on the intuition we get from that, we’ll see how things are actually wired up.

In an ideal world, here’s what I would like to see. When I load my web app in the browser, the ClojureScript runtime on the browser automatically starts a background REPL server waiting for forms to evaluate. From the command line, I can execute something like lein do-something from within the project directory, which fires up a client that automatically detects and connects to the REPL server running on the browser. When I type forms into the command line REPL, they’re forwarded to the browser for evaluation. If I refresh the browser, this is all transparently handled by the client. I should not have to manually restart my command-line REPL session.

In fact, this is almost the way things actually work! However, there are some details we need to fill in.

1. We need to decide what requests that server responds to and how to format those requests.
2. We can’t actually start a server on some browsers, so we need some way of “faking it.”

Both requirements must be fulfilled to realize the system I’ve described.

In this post, I’ll describe the approach taken by ClojureScript proper – it requires no external plugins, which is why I call it the Vanilla REPL.

Another approach, which I’ll defer to a future post, uses external plugins and has many more bells and whistles. But let’s ignore that for now.

The Vanilla REPL

ClojureScript itself comes with a REPL implementation. Due to its lack of external dependencies, it’s the simplest to get started with.

How does Vanilla REPL meet both needs I listed above?

1. Use a special protocol and implementation different from the Clojure REPL. In particular, by sending raw Javascript code to the browser.
2. Use long-polling with CrossPageChannel.

In the remainder of this post, I’ll describe the details of these two points. Before we get to that though, I need to make a clarifing point.

An Aside: Terminology

Before I get into specifics, I need to make precise some terminology I’ll be using. When I say backend, I mean ClojureScript code running inside the browser. Note that this is somewhat unconventional since code running in a browser is generally called frontend code. In this particular context since the browser is serving responses to REPL requests, the browser is actually the backend.

Ok, now let’s get back to it.

REPL ↔ Browser Communication Protocol

The following Clojure code, when run from your terminal of choice, will fire up a REPL session connected to a ClojureScript application running in the browser.

(require '[cljs.repl :as repl] '[cljs.repl.browser :as browser]) (repl/repl (browser/repl-env))

Obviously, it only works if you’ve already got a ClojureScript application running in your browser and waiting to accept a new REPL connection attempt via (repl/repl (browser/repl-env)).

As you can see, there are two required namespaces: cljs.repl and cljs.repl.browser.

cljs.repl implements the REPL’s command line interface in Clojure-land. Here’s the relevant code that implements the body of the read-eval-print loop. It’s part of cljs.repl/repl*.

read-eval-print (fn [] (let [input (binding [*ns* (create-ns ana/*cljs-ns*) reader/resolve-symbol ana/resolve-symbol reader/*data-readers* tags/*cljs-data-readers* reader/*alias-map* (apply merge ((juxt :requires :require-macros) (ana/get-namespace ana/*cljs-ns*)))] (read request-prompt request-exit))] (or ({request-exit request-exit :cljs/quit request-exit request-prompt request-prompt} input) (if (and (seq? input) (is-special-fn? (first input))) (do ((get special-fns (first input)) repl-env env input opts) (print nil)) (let [value (eval repl-env env input opts)] (print value))))))] github source

The key bit is on line 835: (eval repl-env env input opts). It evaluates a form that has been read from the REPL. We’re not quite ready to answer how it works yet, but we’ll get back to it.

The code in cljs.repl is backend-agnostic. By this, I mean that it has no notion of the backend that will actually evaluate forms typed into the REPL. It might be a browser-hosted application. It might be something running in a node.js instance. It might be something else. All that cljs.repl/repl knows is what it’s told via its input argument repl-env, which implements the protocol cljs.repl/IJavaScriptEnv (a.k.a, a REPL environment). Here is the definition of IJavaScriptEnv.

(defprotocol IJavaScriptEnv (-setup [repl-env opts] "initialize the environment") (-evaluate [repl-env filename line js] "evaluate a javascript string") (-load [repl-env provides url] "load code at url into the environment") (-tear-down [repl-env] "dispose of the environment")) github source

There’s nothing complicated here. The protocol just has implementations for four basic operations: -setup, -evaluate, -load, and -tear-down. Their implementations are backend-dependent, but a client of this protocol need not worry about their implementation.

ClojureScript provides implementations of IJavaScriptEnv for various backends. You can find them here. In our case, we only care about cljs.repl.browser, which defines the REPL environment for a browser-hosted application. (browser/repl-env) creates a REPL environment that, at a high level, contains the parameters needed to communicate with the browser-hosted backend:

{:host "localhost" :port 9000 :working-dir (->> [".repl" (util/clojurescript-version)] (remove empty?) (string/join "-")) :serve-static true :static-dir (cond-> ["." "out/"] output-dir (conj output-dir)) :preloaded-libs [] :optimizations :simple :src "src/" ... github source

An instance created via (browser/repl-env) knows how to evaluate raw Javascript via function (-evaluate [repl-env filename line js]) declared by IJavaScriptEnv. Its implementation boils down to a call to cljs.repl.server/send-and-close (implemented here), that encodes a Javascript string as a bytestream, then sends it to the browser backend for evaluation.

At this point, then, perhaps it’s more clear what’s going on with that call to eval in cljs.repl/repl*. It leads to a call to -evaluate from cljs.repl.browser, which itself leads to an HTTP “request” to the browser.¹

Faking a Browser-hosted “Server” via Long-Polling

To fake a server running on the browser, we use the classic long-polling technique for push notifications. In a nutshell, we start an HTTP server, then load the ClojureScript application in the browser. Upon initialization, the ClojureScript application makes a request to the HTTP server. Rather than immediately respond to the request, the HTTP server holds onto the connection and doesn’t send the response until it has something to say – in particular, the response is a ClojureScript form that you have typed into the REPL.² The ClojureScript application evaluates the form, then makes a new HTTP request to the server that contains the result of the evaluation. The net effect of all this is an inversion of the standard client-server model: HTTP requests masquerade as responses, and HTTP responses masquerade as requests!

Here’s an example of the browser-side code that connects to the HTTP server.

(ns browser-hosted-app.core (:require [clojure.browser.repl :as repl])) (repl/connect "http://localhost:9000/repl")

Here’s the Clojure code that implements the HTTP server, found in cljs.repl.server:

(defn- handle-connection [opts conn] (let [rdr (BufferedReader. (InputStreamReader. (.getInputStream conn)))] (if-let [request (read-request rdr)] (dispatch-request request conn opts) (.close conn)))) (defn- server-loop [opts server-socket] (when-let [conn (try (.accept server-socket) (catch Throwable _))] (.setKeepAlive conn true) (.start (Thread. ((ns-resolve 'clojure.core 'binding-conveyor-fn) (fn [] (handle-connection opts conn))))) (recur opts server-socket))) (defn start "Start the server on the specified port." [opts] (let [ss (ServerSocket. (:port opts))] (.start (Thread. ((ns-resolve 'clojure.core 'binding-conveyor-fn) (fn [] (server-loop opts ss))))) github source

The browser-side repl/connect function comes from clojure.browser.repl:

(defn connect "Connects to a REPL server from an HTML document. After the connection is made, the REPL will evaluate forms in the context of the document that called this function." [repl-server-url] (let [repl-connection (net/xpc-connection {:peer_uri repl-server-url})] (swap! xpc-connection (constantly repl-connection)) (net/register-service repl-connection :evaluate-javascript (fn [js] (net/transmit repl-connection :send-result (evaluate-javascript repl-connection js)))) (net/connect repl-connection (constantly nil) (fn [iframe] (set! (.-display (.-style iframe)) "none"))) (bootstrap) repl-connection)) github source

A Complication: Same-Origin Policy

The browser makes a cross-origin request whenever it performs a long-poll to our local HTTP server. This is a problem because it violates the same-origin policy enforced by browsers that prevents a webpage from making a request to a page hosted at a different domain. To get around this limitation, Vanilla REPL uses the CrossPageChannel class from the Google Closure API, which essentially exploits the fact that a parent is allowed to communicate with a child iframe, even if the iframe hosts a page from another domain. In our case, that child iframe is hosted by the local HTTP server. This funny business is encapsulated in namespace clojure.browser.net. For example, the call to net/xpc-connection on line 7 constructs a CrossPageChannel instance under the hood.

Conclusion

At this point, we’ve covered the basics of how ClojureScript’s built-in REPL works. There are obviously gobs of implementation details that I have elided, but hopefully with this brief introduction you can feel comfortable digging deeper into the source code if you like. If not, I hope this post has at least dispelled some of the magic behind browser-hosted REPLs. I know when I first started it seemed a bit like magic to me.