Karma is a JavaScript test-runner built with Node.js and meant for unit testing.
The Protractor is for end-to-end testing and uses Selenium Web Driver to drive tests.
Both have been made by the Angular team. You can use any assertion-library you want with either.
Screencast: Karma Getting started
related:
pros:
- Uses node.js, so compatible with Win/OS X/Linux
- Run tests from a browser or headless with PhantomJS
- Run on multiple clients at once
- Option to launch, capture, and automatically shut down browsers
- Option to run server/clients on development computer or separately
- Run tests from a command line (can be integrated into ant/maven)
- Write tests xUnit or BDD style
- Supports multiple JavaScript test frameworks
- Auto-run tests on save
- Proxies requests cross-domain
- Possible to customize:
- Extend it to wrap other test-frameworks (Jasmine, Mocha, QUnit built-in)
- Your own assertions/refutes
- Reporters
- Browser Launchers
- Plugin for WebStorm
- Supported by Netbeans IDE
Cons:
I'm totally unqualified to comment on mocha.js's features, strengths, and weaknesses,
but it was just recommended to me by someone I trust in the JS community.
List of features, as reported by its website:
- browser support
- simple async support, including promises
- test coverage reporting
- string diff support
- javascript # API for running tests
- proper exit status for CI support etc
- auto-detects and disables coloring for non-ttys
- maps uncaught exceptions to the correct test case
- async test timeout support
- test-specific timeouts
- growl notification support
- reports test durations
- highlights slow tests
- file watcher support
- global variable leak detection
- optionally run tests that match a regexp
- auto-exit to prevent "hanging" with an active loop
- easily meta-generate suites & test-cases
- mocha.opts file support
- clickable suite titles to filter test execution
- node debugger support
- detects multiple calls to done()
- use any assertion library you want
- extensible reporting, bundled with 9+ reporters
- extensible test DSLs or "interfaces"
- before, after, before each, after each hook
- arbitrary transpiler support (coffee-script etc)
- TextMate bundle
This no longer exists, redirects to sequential.js instead
Yolpo is a tool to visualize the execution of javascript. Javascript API developers are encouraged to write their use cases to show and tell their API. Such use cases forms the basis of regression tests.
Futuristic test runner with built-in support for ES2015. Even though JavaScript is single-threaded, IO in Node.js can happen in parallel due to its async nature. AVA takes advantage of this and runs your tests concurrently, which is especially beneficial for IO heavy tests. In addition, test files are run in parallel as separate processes, giving you even better performance and an isolated environment for each test file.
- Minimal and fast
- Simple test syntax
- Runs tests concurrently
- Enforces writing atomic tests
- No implicit globals
- Isolated environment for each test file
- Write your tests in ES2015
- Promise support
- Generator function support
- Async function support
- Observable support
- Enhanced asserts
- Optional TAP o
utput
- Clean stack traces
A JavaScript test-runner built with Node.js. Very modular and flexible. It comes with its own assertion library, but you can add your own if you like. The assertions library is decoupled, so you can also use it with other test-runners. Instead of using assert(!...)
or expect(...).not...
, it uses refute(...)
which is a nice twist imho.
A browser JavaScript testing toolkit. It does browser testing with browser automation (think JsTestDriver), QUnit style static HTML page testing, testing in headless browsers (PhantomJS, jsdom, ...), and more. Take a look at the overview!
A Node.js testing toolkit. You get the same test case library, assertion library, etc. This is also great for hybrid browser and Node.js code. Write your test case with Buster.JS and run it both in Node.js and in a real browser.
Screencast: Buster.js Getting started (2:45)
pros:
- Uses node.js, so compatible with Win/OS X/Linux
- Run tests from a browser or headless with PhantomJS (soon)
- Run on multiple clients at once
- Supports NodeJS testing
- Don't need to run server/clients on development computer (no need for IE)
- Run tests from a command line (can be integrated into ant/maven)
- Write tests xUnit or BDD style
- Supports multiple JavaScript test frameworks
- Defer tests instead of commenting them out
- SinonJS built-in
- Auto-run tests on save
- Proxies requests cross-domain
- Possible to customize:
- Extend it to wrap other test-frameworks (JsTestDriver built in)
- Your own assertions/refutes
- Reporters (xUnit XML, traditional dots, specification, tap, TeamCity and more built-in)
- Customize/replace the HTML that is used to run the browser-tests
- TextMate and Emacs integration
Cons:
- Stil in beta so can be buggy
- No plugin for Eclipse/IntelliJ (yet)
- Doesn't group results by os/browser/version like TestSwarm *. It does, however, print out the browser name and version in the test results.
- No history of previous test results like TestSwarm *
- Doesn't fully work on windows as of May 2014
* TestSwarm is also a Continuous Integration server, while you need a separate CI server for Buster.js. It does, however, output xUnit XML reports, so it should be easy to integrate with Hudson, Bamboo or other CI servers.
https://github.com/jquery/testswarm
TestSwarm is officially no longer under active development as stated on their GitHub webpage. They recommend Karma, browserstack-runner, or Intern.
This is a behavior-driven framework (as stated in quote below) that might interest developers familiar with Ruby or Ruby on Rails. The syntax is based on RSpec that are used for testing in Rails projects.
Jasmine specs can be run from an html page (in qUnit fashion) or from a test runner (as Karma).
Jasmine is a behavior-driven development framework for testing your JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM.
If you have experience with this testing framework, please contribute with more info :)
Project home: http://jasmine.github.io/
QUnit focuses on testing JavaScript in the browser while providing as much convenience to the developer as possible. Blurb from the site:
QUnit is a powerful, easy-to-use JavaScript unit test suite. It's used by the jQuery, jQuery UI, and jQuery Mobile projects and is capable of testing any generic JavaScript code
QUnit shares some history with TestSwarm (above):
QUnit was originally developed by John Resig as part of jQuery. In 2008 it got its own home, name and API documentation, allowing others to use it for their unit testing as well. At the time it still depended on jQuery. A rewrite in 2009 fixed that, now QUnit runs completely standalone.
QUnit's assertion methods follow the CommonJS Unit Testing specification, which was to some degree influenced by QUnit.
Project home: http://qunitjs.com/
Another great tool is sinon.js by Christian Johansen, the author of Test-Driven JavaScript Development. Best described by himself:
Standalone test spies, stubs and mocks
for JavaScript. No dependencies works
with any unit testing framework.
The Intern Web site provides a direct feature comparison to the other testing frameworks on this list. It offers more features out of the box than any other JavaScript-based testing system.
A new but yet very powerful testing framework. It allows snapshot based testing as well this increases the testing speed and creates a new dynamic in terms of testing
Check out one of their talks: https://www.youtube.com/watch?v=cAKYQpTC7MA
Better yet: Getting Started
The Quick Answer
One very important point to bring up is that there are two flavors of Behavior Driven Development. The two flavors are xBehave and xSpec.
xBehave BDD: SpecFlow
SpecFlow (very similar to cucumber from the Ruby stack) is excellent in facilitating xBehave BDD tests as Acceptance Criteria. It does not however provide a good way to write behavioral tests on a unit level. There are a few other xBehave testing frameworks, but SpecFlow has gotten a lot of traction.
xSpec BDD: NSpec
For behavior driven development on a unit level, I would recommend NSpec (inspired directly by RSpec for Ruby). You can accomplish BDD on a unit level by simply using NUnit or MSTest...but they kinda fall short (it's really hard to build up contexts incrementally). MSpec is also an option, there has been a lot of work put into it, but there are just somethings that are just simpilier in NSpec (you can build up context incrementally in MSpec, but it requires inheritance which can become complex).
The Long Answer
The two flavors of BDD primarily exist because of the orthogonal benefits they provide.
Pros and Cons of xBehave (GWT Syntax)
Pros
- helps facilitate a conversations with the business through a common dialect called (eg. Given ...., And Given ...., When ......, And When ..... , Then ...., And Then)
- the common dialect can then be mapped to executable code which proves to the business that you actually finished what you said you'd finish
- the dialect is constricting, so the business has to disambiguate requirements and make it fit into the sentences.
Cons
- While the xBehave approach is good for driving high level Acceptance Criteria, the cycles needed to map English to executable code via attributes makes it infeasible for driving out a domain at the unit level.
- Mapping the common dialect to tests is PAINFUL (ramp up on your regex). Each sentence the business creates must be mapped to an executable method via attributes.
- The common dialect must be tightly controlled so that managing the mapping doesn't get out of hand. Any time you change a sentence, you have to find method that directly relates to that sentence and fix the regex matching.
Pros and Cons of xSpec (Context/Specification)
Pros
- Allows the developer to build up context incrementally. A context can be set up for a test and some assertions can be performed against that context. You can then specify more context (building upon the context that already exists) and then specify more tests.
- No constricting language. Developers can be more expressive about how a certain part of a system behaves.
- No mapping needed between English and a common dialect (because there isn't one).
Cons
- Not as approachable by the business. Let's face it, the business don't like to disambiguate what they want. If we gave them a context based approach to BDD then the sentence would just read "Just make it work".
- Everything is in the code. The context documentation is intertwined within the code (that's why we don't have to worry about mapping english to code)
- Not as readable given a less restrictive verbiage.
Samples
The Bowling Kata is a pretty good example.
SpecFlow Sample
Here is what the specification would look like in SpecFlow (again, this is great as an acceptance test, because it communicates directly with the business):
Feature File
The feature file is the common dialect for the test.
Feature: Score Calculation
In order to know my performance
As a player
I want the system to calculate my total score
Scenario: Gutter game
Given a new bowling game
When all of my balls are landing in the gutter
Then my total score should be 0
Step Definition File
The step definition file is the actual execution of the test, this file includes the mappings for SpecFlow
[Binding]
public class BowlingSteps
{
private Game _game;
[Given(@"a new bowling game")]
public void GivenANewBowlingGame()
{
_game = new Game();
}
[When(@"all of my balls are landing in the gutter")]
public void WhenAllOfMyBallsAreLandingInTheGutter()
{
_game.Frames = "00000000000000000000";
}
[Then(@"my total score should be (\d+)")]
public void ThenMyTotalScoreShouldBe(int score)
{
Assert.AreEqual(0, _game.Score);
}
}
NSpec Sample, xSpec, Context/Specification
Here is a NSpec example of the same bowling kata:
class describe_BowlingGame : nspec
{
Game game;
void before_each()
{
game = new Game();
}
void when_all_my_balls_land_in_the_gutter()
{
before = () =>
{
game.Frames = "00000000000000000000";
};
it["should have a score of 0"] = () => game.Score.should_be(0);
}
}
So Yea...SpecFlow is cool, NSpec is cool
As you do more and more BDD, you'll find that both the xBehave and xSpec flavors of BDD are needed. xBehave is more suited for Acceptance Tests, xSpec is more suited for unit tests and domain driven design.
Relevant Links
Best Answer
That's because there isn't any.
That's wrong. BDD and TDD have absolutely nothing whatsoever to do with testing. None. Nada. Zilch. Zip. Nix. Not in the slightest.
Unfortunately, TDD has the word "test" in pretty much everything (not only in its name, but also in test framework, unit test,
TestCase
(the class you tpyically inherit from),FooTest
(the class which typically holds your tests),testBar
(the typical naming pattern for a test method), plus a lot test-related terminology such as "assertion" and "verification") which leads some people to believe that it actually does have something to do with tests. So, some smart people said: "Hey, let's just change the name" to remove any potential for confusion.And that's what BDD is. It's just TDD with any test-related terminology replaced by examples-of-behavior-related terminology:
assert
→should
BDD is just TDD with different words. If you do TDD right, you are doing BDD. The difference is that – provided you believe at least in the weak form of the Sapir-Whorf Hypothesis – the different words make it easier to do it right.