In a web app of mine, I wanted to let end users run multi-module JavaScript that they enter via text fields. It turns out that this simple idea is relatively difficult to implement. In this blog post, I’ll explain how to do it. It is less polished than usual – I mainly wanted to get the knowledge out there.
eval()
the text string.The web app itself is bundled via webpack.
Before we can bundle, we need a way to pass the modules to Rollup as strings (vs. files in a file system). That can be achieved via a plugin:
function createPlugin(moduleSpecifierToSourceCode: Map<string,string>): Plugin {
return {
name: 'BundleInputPlugin', // this name will show up in warnings and errors
resolveId(id) {
if (moduleSpecifierToSourceCode.has(id)) {
// Tell Rollup to not ask other plugins or check the file system to find this ID
return id;
}
return null; // handle other IDs as usually
},
/** Override source code by returning strings */
load(id) {
const source = moduleSpecifierToSourceCode.get(id);
if (source) {
return source;
}
return null; // other ids should be handled as usually
},
};
}
Now we can bundle the code:
const mainModuleName = '__theMainModule__';
const regexPrefixSuffix = /^var __theMainModule__ = ([^]*);\n$/;
export interface BundleDesc {
code: string;
sourceMap: SourceMap;
}
export async function createBundle(
moduleSpecifierToSourceCode: Map<string,string>,
entrySpecifier: string): Promise<BundleDesc> {
try {
const bundle = await rollup({
input: entrySpecifier,
plugins: [createPlugin(moduleSpecifierToSourceCode)],
});
const {output} = await bundle.generate({
format: 'iife',
name: mainModuleName,
sourcemap: true,
});
const chunkOrAssetArr = [...output];
if (chunkOrAssetArr.length !== 1 || chunkOrAssetArr[0].type !== 'chunk') {
throw new Error();
}
let {code, map} = chunkOrAssetArr[0];
const expressionMatch = regexPrefixSuffix.exec(code);
if (!expressionMatch) {
throw new InternalError('Unexpected bundling result: ' + JSON.stringify(code));
}
code = expressionMatch[1];
if (!map) {
throw new InternalError();
}
return {
code,
sourceMap: map,
};
} catch (err) {
if (err.code === 'PARSE_ERROR') {
// Handle syntax errors
} else {
throw err;
}
}
}
Even for “global” modules such as 'assert'
, we need to provide strings. webpack’s raw-loader can help here:
//@ts-ignore
import srcAssert from 'raw-loader!../modules/assert.js';
That allows us to develop modules for the user’s code with the same tools as the web app (even though the web app exists at a meta-level, relative to the user’s code).
Rollup supports several bundle formats (AMD, CommonJS, ES module, etc.). The Rollup REPL helps with exploring the different bundle formats. The best format for our purposes is “iife”. The following code is an example of this format:
var myBundle = (function (exports) {
'use strict';
const foo = 1;
exports.foo = foo;
return exports;
}({}));
If we remove 'var myBundle = '
at the beginning and the semicolon at the end, we can eval()
this code.
If Rollup finds syntax errors, it throws instances of Error
that include the following properties:
{
code: 'PARSE_ERROR',
loc: {
file: './test.mjs',
column: 8,
line: 1,
},
frame: '1: iimport a from 'a';\n ^\n2: import * as b from 'b';',
}
.frame
is a string that shows the lines surrounding the syntax error and points to its location.
I had to import Rollup as follows in order to not lose the static type information in my IDE:
import { Plugin, rollup, SourceMap } from 'rollup';
Alas, that imports from the Node.js version of Rollup, not from the browser version. I fixed that by adding the following entry to webpack.config.js
:
resolve: {
alias: {
'rollup': 'rollup/dist/rollup.browser.es.js',
}
},
To execute the bundle, a simple eval.call(undefined, str)
suffices. Why eval.call()
and not eval()
? The former is a safer version of the latter (details).
If the evaluated code throws an exception, we get stack traces that look as follows:
Error: Problem!
at Object.[foo] (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:7:13)
at Object.bar (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:10:19)
at Object.get baz [as baz] (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:13:12)
at f (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:17:9)
at g (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:20:5)
at eval (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:22:3)
at eval (eval at <anonymous> (first-steps-bundle.js:33), <anonymous>:28:2)
at eval (<anonymous>)
at first-steps-bundle.js:33
Error: Actual: 6, expected: "blank"
at equal (eval at executeCode (first-steps.js:NaN), <anonymous>:28:19)
at Object.eval [as code] (eval at executeCode (first-steps.js:NaN), <anonymous>:40:7)
at runTests (eval at executeCode (first-steps.js:NaN), <anonymous>:12:26)
at eval (eval at executeCode (first-steps.js:NaN), <anonymous>:44:16)
at eval (eval at executeCode (first-steps.js:NaN), <anonymous>:48:2)
at executeCode (first-steps.js:77)
The stack trace includes line numbers for the web app and line numbers for the end user’s code. I could not find any stack trace parsing library that parsed such stack traces correctly. Hence, I did some quick and dirty parsing via a regular expression.
The <anonymous>
line numbers refer to the bundled code. We can use the library source-map to get the original module names and locations:
import { SourceMapConsumer } from 'source-map';
const result = SourceMapConsumer.with(sourceMap, null,
async (consumer) => { // (A)
const result = [];
for (const l of parsedStackTraceLines) {
const {source, line, column} = consumer.originalPositionFor({
line: l.lineNumber,
column: l.columnNumber,
});
result.push(···);
}
return result;
});
By wrapping the code contained in the callback (line A), SourceMapConsumer.with()
ensures that the resources it creates are disposed of, after they are not needed, anymore.
Internally, source-map uses Wasm code. I had to use version 0.8.0-beta.0 in order to load that code from the web app bundle.
The JavaScript code looks like this:
//@ts-ignore
import arrayBuffer from 'source-map/lib/mappings.wasm';
(SourceMapConsumer as any).initialize({
'lib/mappings.wasm': arrayBuffer,
});
Thanks to webpack and the arraybuffer-loader, we can import the Wasm code into an ArrayBuffer. We need the following entry in webpack.config.js
:
module: {
rules: [
{
test: /\.wasm$/,
type: 'javascript/auto',
loader: 'arraybuffer-loader',
},
],
},