JavaScript usage is not limited to client-side code in browser or NodeJS powered server-side code. Many JVM based projects are using it as internal scripting language. Testing this sort of functionality is neither straightforward nor standard. In this post I intend to demonstrate an approach for testing JavaScript in server-side JVM environment using mature tools like Jasmine, Spock and Nashorn.

Using JavaScript as scripting engine inside JVM application has significant difference comparing to client-side coding. And unfortunately there’s no industrial standard tools nowadays for testing it.

Regarding existing approaches found in Internet I’d like to highlight following disadvantages

  • lack of integration with build and continuous integration tools (Maven, Gradle, Jenkins, etc.)

  • insufficient cooperation with IDEs

    • no possibility to run single suite or test from IDE

    • unable to view test execution reports from IDE

  • tight coupling to browser environment

  • no possibility of using customized JavaScript executors

As far as I’ve seen most of the projects test their embedded business scripts by calling JS engine runner, passing script under test to it and doing assertion by inspecting side-effects on engine or mocks after script execution.

Those sort of approaches usually have similar drawbacks

  • hard to stub or mock something in JS code, usually ending up hacking on JS prototype

  • need too much orchestration for mocking environment for script

  • hard to organize tests into suites and report test execution errors

  • previous causes creation of custom test suite frameworks for particular project

  • not leveraging existing JavaScript testing tools and frameworks

So, driven by the need for comfortable embedded JavaScript testing in JVM projects I’ve created this sample setup. To fulfill our goals next tools will be used.

  • Jasmine is one of the most known TDD/BDD tools for JavaScript

  • Spock is great testing framework for JVM powered by Junit and Groovy

  • Nashorn is modern scripting engine introduced in JDK8

Customized JavaScript runner (Nashorn based)

There’s no need to conform standards in non-browser JS environments, so usually developers extend scripting engine with custom functions, built-in variables etc. It is extremely important to use exactly the same runner both for production and testing purposes.

Let’s consider we have such customized runner, accepting script name and map of predefined variables as parameters and returning resulting value of the executed script.

JavaScriptRunner.java
public class JavaScriptRunner {
  public static Object run(String script, Map<String, Object> params) throws Exception {
    ScriptEngineManager factory = new ScriptEngineManager();
    ScriptEngine engine = factory.getEngineByName("nashorn");
    engine.getBindings(ScriptContext.ENGINE_SCOPE).putAll(params);
    return engine.eval(new InputStreamReader(JavaScriptRunner.class.getResourceAsStream(script))); (1)
  }
}
1 script source is searched in classpath.

Jasmine setup

To start using Jasmine framework we need

  • Jasmine 2.1.3 distribution available as Maven artifact from WebJars project.

  • custom bootstrap script, since Jasmine doesn’t support JVM based platforms

jasmine2-bootstrap.js
var loadFromClassPath = function(path) { (1)
  load(Java.type("ua.eshepelyuk.blog.Jasmine2Specification").class.getResource(path).toExternalForm());
};

var window = this;

loadFromClassPath("/META-INF/resources/webjars/jasmine/2.1.3/jasmine.js");
loadFromClassPath("/jasmine/jasmine2-html-stub.js"); (2)
loadFromClassPath("/META-INF/resources/webjars/jasmine/2.1.3/boot.js");
load({script: __jasmineSpec__, name: __jasmineSpecName__}); (3)

onload(); (4)

jsApiReporter.specs(); (5)
1 helper function resolving script path from classpath location.
2 Nashorn specific code adjusting Jasmine for non-browser environments. Not a part of Jasmine distribution.
3 loading test suite source code, see next section for details.
4 faking browser load event, that should trigger test suite execution.
5 this value will be returned as script result.

Transform Jasmine report into Spock tests

Having JS executor and bootstrap script for Jasmine we could create JUnit test to iterate over suite results and check if all are successful. But it will become a nightmare to understand which particular test had failed and what is the reason of failure. What we’d really like to have is ability to represent each Jasmine specification as JUnit test, so any Java tool can pick up and inspect the results. Here why Spock could be the answer to the problem, with its Data Driven Testing that allows developer to declare list of input data and for each item of that dataset new test will be created and executed. This is very similar to Parametrized Test of Junit but much more powerful implementation.

So the idea will be to consider Jasmine test suite results obtained after running bootstrap script as array of input data, whose every item will be passed to Spock test. Then test itself will provide assertion to report successful and failed tests properly, i.e. assertion should check status of Jasmine specification.

  • if status is pending or passed, this means specification is either ignored or successful

  • otherwise Spock test should throw assertion error, populating assertion exception populated with failures messages reported by Jasmine

Jasmine2Specification.groovy
abstract class Jasmine2Specification extends Specification {
  @Shared def jasmineResults

  def setupSpec() {
    def scriptParams = [
        "__jasmineSpec__"    : getMetaClass().getMetaProperty("SPEC").getProperty(null), (1)
        "__jasmineSpecName__": "${this.class.simpleName}.groovy"
    ]
    jasmineResults = JavaScriptRunner.run("/jasmine/jasmine2-bootstrap.js", scriptParams) (2)
  }

  def isPassed(def specRes) {specRes.status == "passed" || specRes.status == "pending"}

  def specErrorMsg(def specResult) {
    specResult.failedExpectations
        .collect {it.value}.collect {it.stack}.join("\n\n\n")
  }

  @Unroll def '#specName'() {
    expect:
      assert isPassed(item), specErrorMsg(item) (3)
    where:
      item << jasmineResults.collect { it.value }
      specName = (item.status != "pending" ? item.fullName : "IGNORED: $item.fullName") (4)
  }
}
1 exposing source code of Jasmine suite as jasmineSpec variable, accessible to JS executor.
2 actual execution of Jasmine suite.
3 for each suite result we assert either it is succeeded, throwing assertion error with Jasmine originated message on failure.
4 additional data provider variable to highlight ignored tests.

Complete example

Let’s create test suite for simple JavaScript function.

mathUtils.js
var add = function add(a, b) {
  return a + b;
};

Using base class from previous step we could create Spock suite containing JavaScript tests. To demonstrate all the possibilities of our solution we will create successful, failed and ignored test.

MathUtilsTest.groovy
class MathUtilsTest extends Jasmine2Specification {
    static def SPEC = """ (1)
loadFromClassPath("/js/mathUtils.js"); (2)
describe("suite 1", function() {
  it("should pass", function() {
    expect(add(1, 2)).toBe(3);
  });
  it("should fail", function() {
    expect(add(1, 2)).toBe(3);
    expect(add(1, 2)).toBe(0);
  });
  xit("should be ignored", function() {
    expect(add(1, 2)).toBe(3);
  });
})
"""
}
1 actual code of Jasmine suite is represented as a String variable.
2 loading module under test using function inherited from jasmine-bootstrap.js.
Test results from IntelliJ IDEA
Figure 1. Test results from IntelliJ IDEA

IntelliJ Idea language injection

Although this micro framework should work in all the IDEs the most handy usage of it will be within IntelliJ IDEA thanks to its language injection. The feature allows to embed arbitrary language into file created in different programming language. So we could have JavaScript code block embedded into Spock specification written in Groovy.

Language injection
Figure 2. Language injection

Pros and cons of the solution

Advantages

  • usage of industry standard testing tools for both languages

  • seamless integration with build tools and continuous integration tools

  • ability to run single suite from IDE

  • run single test from the particular suite, thanks to focused feature of Jasmine

Disadvantages

  • no clean way of detecting particular line of source code in case of test exception

  • a little bit IntelliJ IDEA oriented setup

P.S.

For this sample project I’ve used modern Nashorn engine from JDK8. But in fact there’s no limitation on this. The same approach was successfully applied for projects using older Rhino engine. And then again, Jasmine is just my personal preference. With additional work code could be adjusted to leverage Mocha, QUnit and so on.

Full project’s code is available at My GitHub