JS ARToolKit performance

Have you ever wondered how to measure the performance of ARToolKit for JavaScript (JSARToolKit) inside the web-browser? I’ve done it, and the following article will outline how I did it.

ARToolKit for JavaScript

JSARToolKit has been around for a few years now. There even is a direct integration with Three.js and aframe called AR.js. ARToolKit for JavaScript can basically do the same as the regular ARToolKit with the big advantage that ARToolKit for JavaScript runs in the browser and doesn’t require the installation of an app. This is made possible leveraging a technology called Emscripten. Emscripten transpiles C-Code into JavaScript code. Using this transpiled code, and a bit of glue code and voila you’ve got ARToolKit for JavaScript which is running inside your web-browser.

ARToolKit for JavaScript performance

When you are building an augmented reality app for the web performance becomes a very critical component. After all, you don’t want to present your users an experience that runs at less than 20 FPS. To ensure that everyone that uses ARToolKit for JavaScript enjoys it I’ve written some performance tests to measure the performance for video stream processing and single image processing.

How to write performance tests

Fortunately, I didn’t have to start on a green field because I had already written test cases for JSARToolKit. So all I had to do was take the existing functional tests and rewrite them to measure the actual performance during processing time.

This is the actual performance test:

QUnit.test("PTV: performance test video", assert => {
    const testDone = assert.async();

    performance.mark('start video measure');  //1. First performance mark
    const done = () => {
        performance.mark('cleanup-done');
        performance.measure('Cleanup time', 'cleanup', 'cleanup-done');
        performance.measure('Test time', 'start video measure', 'cleanup-done');
        const measures = performance.getEntriesByType('measure');
        const csv = Papa.unparse(JSON.stringify(measures));
        console.log(csv);
        testDone();
    };
    assert.timeout(this.timeout);
    assert.expect(0);
    const success = (arController, arCameraParam) => {
        performance.mark('getUserMediaARController-success');
        performance.measure('Start videostream','start video measure', 'getUserMediaARController-success');

        arController.loadMarker('./patt.hiro',(markerId) => { 
            performance.mark('loadMarker-success');
            performance.measure('Load marker','getUserMediaARController-success', 'loadMarker-success');

            //2. Process the open video stream
            for(var i = 0; i <= 100; i++) {
                performance.mark('process-' + i + ' start');
                arController.process(this.video);
                performance.mark('process-' + i + ' done');
                performance.measure('process video','process-' + i + ' start', 'process-' + i + ' done');
            }

            performance.mark('cleanup');
            arController.dispose();
            done();
        });
    };

    const error = error => {
        done();
    }

    const config = {
        onSuccess : success,
        onError : error,

        cameraParam: './camera_para.dat', // URL to camera parameters definition file.
        maxARVideoSize: 640, // Maximum max(width, height) for the AR processing canvas.

        width : 640,
        height : 480,

        facingMode : 'environment'
    }
    this.video = ARController.getUserMediaARController(config);
    document.body.appendChild(this.video);
});

Performance analysis step-by-step

To measure the performance the JavaScript internal performance API is used. The first performance mark (// 1. xxx) measures the time for the complete test run. Then I define a done-function that measures the clean-up time and closes the measure for the complete test run. The success function measures the actual video stream processing performance. Here you can see several performance marks that measure different states during the JSARToolKit setup and configuration process.

The most important measure is the line marked //2. xxx here I’m measuring how long it takes to process 100 frames. Processing is defined as copying the video frame over to the Emscripten transpiled JavaScript code, run the ARToolKit marker detection, identify the markerId, if marker is detected return position of the marker.

Performance results

Using the above-described process I was able to exactly determine the time it takes to configure ARToolKit and run the actual marker tracking.

JSARToolKit performance

Outlook

As you can see above there are tests for a WebAssembly version of JSARToolKit and for JSARToolKit without WebAssembly. This is another topic but while writing the tests I also enabled WebAssembly support for JSARToolKit. Make sure to check back here to get the insights about how I made that possible and don’t hesitate to ask questions below.