Skip to main content

Roadroller: Flattens Your JavaScript Demo

Roadroller is a heavyweight JavaScript packer designed for large demos of at least 10 KB in size, like js13kGames. Depending on the input it can provide up to 10% additional compression compared to Zopfli. Try it online!

Roadroller is considered “heavyweight” unlike typical JS packers such as JSCrush or RegPack, because it is quite resource intensive and requires both a considerable amount of memory and a non-negligible run time. The default should work for most devices, but you can configure both aspects as you need.

Quick Start

In addition to the online demo, Roadroller is available as an NPM package:

$ npx roadroller input.js -o output.js

You can also use Roadroller as a library to integrate with your build pipeline.

import { Packer } from 'roadroller';

const inputs = [
    {
        data: 'console.log("Hello, world!");',
        type: 'js',
        action: 'eval',
    },
];

const options = {
    maxMemoryMB: 150,
};

const packer = new Packer(inputs, options);

// this typically takes about a minute, can be omitted if you want.
packer.optimizeSparseSelectors();

const { firstLine, secondLine } = packer.makeDecoder();
console.log(firstLine + '\n' + secondLine);

Usage

By default Roadroller receives your JS code and returns a compressed JS code that should be further compressed with ZIP/gzip (or more accurately, DEFLATE). Ideally your JS code should be already minified, probably using Terser or Closure Compiler; Roadroller only does a minimal whitespace and comment suppression.

The resulting code will look like this:

_='Zos~ZyF_sTdvfgJ^bIq_wJWLGSIz}Chb?rMch}...'
t=12345678;M=1<<17;w=[0,0,0,0,0,0,0,0,0,0,0,0];p=new Uint32Array(12<<21).fill(M/4);/* omitted */;eval(c)

The first line is a compressed data. It can contain control characters  (U+001F) that might not render in certain environments. Nevertheless you should make sure that they are all copied in verbatim.

The second line is a compressor tuned for this particular input. By default the decompressed data immediately goes through eval, but you can configure what to do with that.

The first line is very incompressible unlike the second line, so ideally you should compress two lines separately. This is best done by using ADVZIP from AdvanceCOMP or the aforementioned Zopfli. It is also possible but not recommended* to put the first line to a separate file and load it with a separate <script> tag.

* This is not recommended because any additional file to ZIP incurs at least 88+2n bytes of overhead where n is the length of file name and that overhead mostly negates the additional compression. Instead put everything into a single file and run ADVZIP or Zopfli.

Input Configuration

Each input can be further configured by input type and action. In the CLI you put corresponding options before the file path.

Input type (CLI -t|--type TYPE, API type in the input object) determines the preprocessing step to improve the compression. Dropping a file to the input window also tries to detect the correct input type.

  • JavaScript (js) assumes a valid JS code. Automatically removes all redundant whitespace and comments and enables a separate modelling for embedded strings. This also works for JSON.
  • Text (text) assumes a human-readable Unicode text that can be encoded in UTF-8. This can also be used for JavaScript code that should not undergo preprocessing.

Input action (CLI -a|--action ACTION, API action in the input object) determines what to do with the decompressed data.

  • **Evaluate** (`eval`) evaluates the decompressed JavaScript code. If there are multiple inputs there should be exactly one JavaScript input with evaluate action, since subsequent inputs will be decompressed in that code. The resulting value is always a code string, which may include decoders for subsequent inputs.
  • **Write to document** (`write`) writes a decompressed string to `document`. Typically used with HTML.

Output Configuration

Optimize contexts (CLI -O|--optimize 1, API Packer.optimizeSparseContexts) searches for better modelling parameters. If parameters are already given the optimizer will try to improve upon that. Parameters are solely related to the compression ratio so you can try this as many as you can afford.

  • The additional argument in the CLI indicates the level of efforts, which can be 0 (do nothing) or 1 (use the default setting, takes about a minute); other values are reserved for the future expansion. The resulting parameters are printed at the end which can be reused for faster iteration.

  • While not strictly required, Packer.optimizeSparseContexts in the API strongly recommends the use of arrayBufferPool in the options object. Otherwise the optimization can run slower especially with larger memory. The pool can be created via new ArrayBufferPool().

Maximum memory usage (CLI -M|--max-memory MEGABYTES, API maxMemoryMB in the options object) configures the maximum memory to be used both for compression and decompression. Increasing or decreasing memory usage only affects the compression ratio and not the run time. The actual memory usage can be as low as a half of the specified due to the internal architecture. The default is 150 MB.

Advanced Configuration

Chosen contexts (CLI -S|--selectors SELECTOR,SELECTOR,..., API sparseSelectors in the options object) are actual modelling parameters, one number for each context. Each bit of numbers is set if the particular order is used for given sparse context: 5 = 101(2) for example would correspond to the sparse context (0,2). There is no particular limit for the number, but Roadroller only considers up to 9th order for the optimization process.

Tips and Tricks

  • The current algorithm slightly prefers 7-bit and 8-bit inputs for the decoder simplicity. You can still use emojis and other tricks that stuff many bits into Unicode code points, but the compression ratio might be decreased. Keep in mind that Roadroller is already doing the hard work for you and you might not need to repeat that.

  • The compressed JS code doesn’t do anything beyond computation and the final action, so you can do anything before or after that. The online demo for example inserts a sort of splash screen as a fallback.

  • Roadroller, while being super effective for many inputs, is not a panacea. Roadroller is weaker at exploiting the duplication at a distance than DEFLATE. Make sure to check ADVZIP/Zopfli out.

Compatibility

Roadroller itself and resulting packed codes are ECMAScript 2015 (ES6) compatible and should run in every modern Web browser and JS implementation. Implementations are assumed to be reasonably fast but otherwise it can run in slower interpreters. MSIE is not directly supported but it works fine (slowly) after simple transpiling.

Roadroller and packed codes extensively use Math.exp and Math.log that are implementation-approximated, so there is a small but real possibility that they behave differently in different implementations. This is known to be a non-issue for browser JS engines as well as V8 and node.js as they use the same math library (fdlibm) for those functions, but you have been warned.

By comparison, the Roadroller CLI assumes more from the environment and probably requires Node.js 14 or later (only tested with 16).

Internals

Roadroller is mostly possible due to the progress in data compression algorithms as recent as 2010s:

  • Bytewise rANS coder, adapted from Fabien Giesen’s public domain code.

  • Logistic context mixing, which is a type of neural network specifically designed for the data compression.

  • Sparse context models up to 9th order. Models are tuned for each input with simulated annealing. (You may have noticed that this entire architecture is similar to Crinkler, but Roadroller uses a faster and possibly better parameter search algorithm.)

The minimal JS code for this algorithm was initially adapted from a golf.horse submission by Hasegawa Sayuri (public domain). The main difference is that Roadroller implements hashed contexts and thus order 3+ context models.

License

The Roadroller compressor proper is licensed under the MIT license. In addition to this, any decoder code produced by Roadroller, that is, everything in the second line is put in the public domain.