Load Node.js modules in browsers via lobrow

[2011-11-09] dev, javascript, jsmodules, jslang

Update 2011-11-19: Bridging the module gap between Node.js and browsers

Node.js has the advantage of letting you use JavaScript on client and server. Thus, it is a major nuisance that you can’t put portable code into a file that can be loaded on both platforms. This post presents a solution.

The problem

One of Node.js’ advantages is that it lets you write the frontend (client) and the backend (server) of a web application in the same programming language. The term universal means that something works on both platforms. When it comes to universal code such as data structures and utilities, it is disappointing that you cannot package it as a file that can be loaded universally. The problem is aggravated by unit tests, because you equally have to write them in a platform-specific manner. Node.js calls its files modules. We want those modules to be universal, to work on browsers, too. The lobrow library lets you do that. The remainder of this post explains how it works.

Node.js modules

The Node.js documentation contains the following example: Given a directory with two files main.js and circle.js.

The contents of main.js:

    var circle = require('./circle.js');
    console.log('The area of a circle of radius 4 is '
                + circle.area(4));

The contents of circle.js:

    var PI = Math.PI;

    exports.area = function (r) {
        return PI * r * r;
    };

    exports.circumference = function (r) {
        return 2 * PI * r;
    };

lobrow allows you to load a module such as main.js in a web browser. It also loads modules that that module depends on (etc.).

lobrow

Getting started. lobrow is a library that you can download on GitHub. It is a single file, to be included via a script tag. The following HTML fragment shows how to use it to load a Node.js from a file mylib.js.

    <script src="lobrow.js"></script>
    <script>
        lobrow.onload(["./mylib"],
            function (mylib) {
                ...
            });
    </script>

Comments:

The second argument is optional. You can thus put all of your client-side logic into a module.
lobrow.onload() waits until all the modules listed in the array are ready and the onload event has fired for window. If you don’t want to wait for onload, you can use lobrow.require(), which has the same signature.

Module names. lobrow supports a subset of Node’s module names. The file extension ".js" is always omitted. There are three kinds of names:

Global module names: a single identifier, without dots or slashes. Example: "bar".
Such names can refer to built in “core” modules. If not, Node searches directories relative to the current module. For example, if the current module is /home/joe/js/foo.js and imports the module "bar", then Node searches for the following files:
```
    /home/joe/js/node_modules/bar.js
    /home/joe/node_modules/bar.js
    /home/node_modules/bar.js
    /node_modules/bar.js
```
lobrow lets you map global module names to paths (as searching the file system in the above manner makes less sense in a browser). That allows one to use a single name for a module that has a browser-specific and a Node-specific implementation. The value of lobrow.global_names is used to resolve global module names:
- The result of a resolution is either a path to a script to be loaded or an object (which is the module).
- The resolution can be performed by an object (mapping module names to paths or objects) or a function (taking a module name and returning a path or an object).
Relative module names: "./" or "../" (repeated one or more times), followed by a sequence of identifiers separated by slashes. Examples: "./bar", "./bar/baz", "../bar", "../../bar".
Their names interpreted as paths relative to the path of the current module. Example: resolving relative module names.
```
    /home/joe/js/foo.js – path of current module
    
    ./bar     → /home/joe/js/bar.js
    ./bar/baz → /home/joe/js/bar/baz.js
    ../bar    → /home/joe/bar.js
    ../../bar → /home/bar.js
```
Absolute module names: a slash followed by one or more identifiers separated by slashes. Examples: "/bar", "/bar/baz".
The given name is used as an absolute path to find the JavaScript file.

Accessing globals. On browsers, you can access the global object (with all global variables in it) via the variable window or via this – if you are at the top level. On Node.js, you are never at the top level and window does not make sense. Therefore, it introduces the variable global to do so. In lobrow, global is available as a module-local variable.

How it works

In contrast to Node.js, script loading must be asynchronous on a web browser. For example, you can do the following on Node.js:

1:   doSomething()
2:   var func = require("./lib").func;
3:   func();

In a web browsers, loading files is always performed asynchronously: First, the current code has to terminate (“run to completion”), then you can load new data. That means that you cannot stop in line 2 and wait until the loading has finished. When loading a module in the browser, lobrow works around this problem by extracting all arguments of require. It can then load all of the required modules before executing the requiring module. To hand in the imports, it loads the module as text, wraps the following code around it, evaluates it and invokes the resulting function with the appropriate arguments.

    function (require,exports,module,global) {
        // module code goes here
    }

Having to search for require arguments is not a very clean solution, but at least it also works for minified code, because neither the global function require nor its string arguments will affected by minification.

Limitations

Warning: lobrow is just a proof of concept. It has only been tested on Firefox, Chrome, and Safari.
If you comment out a require() invocation then its module will still be preloaded. Differentiating between active and inactive code would make the extraction process much more complicated.
Performance: Things become slow if there are many modules. For deployment, you can switch to one of the compilation solutions listed in the section “related work”. They compress multiple Node.js modules into a single file that can be loaded more quickly in browsers.

lobrow loads scripts via XMLHttpRequest which has several restrictions. First, you cannot load scripts from another web site (only script tags allow you to do that). Second, while XMLHttpRequest works for file: URLs, there are limitations:

Chrome: needs to be started with the command line option --allow-file-access-from-files
Firefox: can only access files in the current directory (and below), but not in the parent directory of the main HTML file.

Universal unit tests

Jasmine already allows you to write universal unit tests. With lobrow, those become even simpler: you simply put your universal tests into a Node.js module. For browsers, you only need to slightly modify the standard Jasmine runner (an HTML file):

    <script type="text/javascript" src="../lobrow.js"></script>
    <script type="text/javascript">
        lobrow.onload(["./strset.spec"], // the tests (*)
            function() {
                // Jasmine code
            }
        );
    </script>

lobrow.onload() ensures that Jasmine is run after strset.spec.js has been loaded. Line (*) is the only custom (test-specific) part of the runner. For Node.js, you can use jasmine-node to run the same tests:

    $ jasmine-node strset.spec.js

Related work

Jake Verbaten and morgain de la fey fowle contributed a list of related work: All of the following solutions allow you to run Node.js modules on browsers.

Adapt Node.js modules by wrapping code around them:
- sea.js
- RequireJS
Compile Node.js code (at the command line or on-demand via a server)
- Browserify
- modul8
- require
- Ender

As far as I can tell, all of these solutions require either a compilation step or a modification of the Node.js code. That makes sense, because it can increase performance during deployment. For development, only lobrow allows you to use Node.js code directly (without any extra steps or tools), which is simpler.

Conclusion

Compared to other comparatively clumsy solutions for making modules work on both Node.js and browsers, lobrow is quite elegant. Even better would be if Node.js supported Asynchronous Module Definitions, because those are much better suited for web browsers – no need to extract the names of required modules from the source code to preload them. Thankfully ECMAScript 6 (code-named ECMAScript.next) will bring us native modules which will eliminate all of the hacks that are currently needed to achieve universality.