While building our new widget, we ran a massive number of tests over it. Seriously: we’ve created hundreds of automated and manual tests for it, even covering all items described in our documentation.
The reason is because our widget is our most appealing selling product: it’s a 5 minute integration, it has an awesome UX, and it works beautifully on mobile.
Automated tests are only good if you run them, that’s why we use a CI tool to run our specs after every code commit. For CI tool, we like Shippable. Why? It’s a service (with several reasonable plans, including a free one), works smoothly out of the box (you don’t need to set up a Jenkins server, yay!), it has integrations with GitHub, Slack (and several others), it’s customizable and has nice documentation, it can returns number of passed tests, coverage, etc.
specs. And it’s popular and well maintained.
So the idea: every time someone pushes code to your GitHub repository, a CI tool will run our test suite, returning green or red according the test results.
After get our specs running with
jasmine in Shippable, we were just missing the proper report (with number of passed tests) and coverage.
Coverage we’ve fixed with
istanbul node module.
But for reporting, we found an issue: current version of
jasmine lost the ability to provide reports in JUnit XML format, when compared with old
Adding back additional reporters
There is a node module called
jasmine-reporters that adds some additional report formats to
jasmine. But the integration is not smooth: you have to write additional code in your test suite to add the reporters.
Jasmine XML Reporter
In order to make our specs produce the JUnit XML reporter back in a smooth way (no code changes required), we’ve created a node module:
It restores 2 lost abilities to
- ability to provide results in JUnit XML format (using the
- an option to specify an output dir while running from the command line (
Simple as any node module:
npm install jasmine-xml-reporter
To run tests with command line output only (default behavior, same as use
To run tests and generate a JUnit XML report:
To run tests, generate a JUnit XML report and place the output in a specific folder:
./node_modules/jasmine-xml-reporter/bin/jasmine.js --junitreport --output=shippable/testresults/
npm test is already an alias to
./node_modules/jasmine-xml-reporter/bin/jasmine.js, then run with:
npm test -- --junitreport --output=shippable/testresults/
And now we can see the number of passing and failing (for shame!) specs right in Shippable/Slack/email etc.
We hope that other developers find this module useful. With all the open source software we use, it feels good to give a little back.