Using Tests and Mocks ​

Use Test Driven Development ​

Test driven development is a software development technique where you follow these steps:

1. Write a test that reflects how the code is intended to be run.
2. Write code that allows the test to compile, and run it to make sure it fails for the reason you expect.
3. Write code to make the test pass.
4. Refactor the code ensuring the tests still pass.
5. Repeat from step 1.

Unit Tests ​

The body of a unit test consists of three steps (this isn't Rust specific and is considered good practice for testing in any language):

1. Arrange: Setup the data or state required for testing
2. Act: Run the code under test
3. Assert: Ensure the result is what was expected

Writing Unit Tests in Rust ​

Place unit tests in the file with the code they're testing. Create a module (with the mod) keyword at the bottom of each file, and annotate it with the #[cfg(test)] attribute. Then, annotate each unit test with a #[test] attribute. For example,

#[cfg(test)]
mod tests {
	use super::*;
    #[test]
    fn test1() {
		// snip
    }
}

#[cfg(test)]
mod tests {
	use super::*;
    #[test]
    fn test1() {
		// snip
    }
}

In Rust, there are a few types of tests that can be used:

1. Assert tests: These tests use one of assert!(), assert_eq!(), or assert_ne!() to check that the result of some code is what we expect. Note that you can pass custom failure messages after the expression or variables being asserted.
2. Panic tests: These are tests that ensure that the program panics when some code is used incorrectly. To use these type of tests, add the #[should_panic] attribute to the test. To make these tests more precise you can also add an expected sub-string to the attribute, such as #[should_panic(expected = "Cannot divide")]. In this case, if the test panics with a message of "Cannot divide by zero", then this test will pass, otherwise it will fail.
3. Result tests: These tests operate similar assert tests, but instead return a Result type. If during the execution of the test an Err variant is returned, the test fails. This type of test allows you to use the ? operator to conveniently return an error variant, thus failing the test.
4. Ignoreable tests: These tests can be of the same type as any of the above, but can be optionally ignored by annotating it with #[ignore]. To run only the ignored tests, use cargo test -- --ignored.

For example, the snippet below shows three different ways you can write and control unit tests.

#[cfg(test)]
mod tests {
    use super::*

    #[test]  // An 'assert' test
    fn test_one() {
        assert_eq!(2 + 2, 4);
    }

    #[test]
    #[should_panic]  // A 'panic' test
    fn test_two() {
        let b = True;
        let x = "string cannot add to" + b;
    }

    #[test]  // A 'result' test
    fn test_three() -> Result<(), String> {
        if 2 + 2 == 4 {
            Ok(())
        } else {
            Err(String::from("Doesn't equal four"))
        }
    }

    #[test]
    #[ignore]  // An 'ignored' test
    fn test_four() {
        assert("Hello world".contains("ello"));
    }

}

#[cfg(test)]
mod tests {
    use super::*

    #[test]  // An 'assert' test
    fn test_one() {
        assert_eq!(2 + 2, 4);
    }

    #[test]
    #[should_panic]  // A 'panic' test
    fn test_two() {
        let b = True;
        let x = "string cannot add to" + b;
    }

    #[test]  // A 'result' test
    fn test_three() -> Result<(), String> {
        if 2 + 2 == 4 {
            Ok(())
        } else {
            Err(String::from("Doesn't equal four"))
        }
    }

    #[test]
    #[ignore]  // An 'ignored' test
    fn test_four() {
        assert("Hello world".contains("ello"));
    }

}

Note, the line use super::* brings objects in the parent module into the scope of the tests module. This allows private functions to be unit tested. While some methodologies argue that private functions shouldn't be tested, Rust remains agnostic on this matter and allows developers to test private functions if they like.

Running the Unit Tests ​

To run your tests, issue

$ cargo test

$ cargo test

This commands compiles your code in test mode and runs a binary that executes any functions annotated with #[test]. Other functions in the test module can can still be useful for shared setup logic.

There are a number of command line flags that can be set to adjust how the tests are compiled and run. Flags that are impact the testing compiler are passed directly (e.g. cargo test --doc) while flags that impact the resulting binary are seperated by -- (e.g. cargo test -- --nocapture). Some of the most useful flags are given below.

Framework Pattern for Unit Tests ​

By using non-test functions as helpers to the testable ones, a test framework pattern is easily implemented in Rust unit tests.

Example

You want to use a library from logging in your unit tests. The crate's log and env_logger together with will do the trick.

use log;
use env_logger;

env_logger::init();

// -- snip --

#[cfg(test)]
mod tests {
    fn init() {
        // Enables logging in tests
        let _ = env_logger::builder().is_test(true).try_init();
    }

    #[test]
    fn test_1() {
        init();
        log::debug!("Debug message from test 1")
    }

    #[test]
    fn test_2() {
        init();
        log::debug!("Debug message from test 2")
    }

use log;
use env_logger;

env_logger::init();

// -- snip --

#[cfg(test)]
mod tests {
    fn init() {
        // Enables logging in tests
        let _ = env_logger::builder().is_test(true).try_init();
    }

    #[test]
    fn test_1() {
        init();
        log::debug!("Debug message from test 1")
    }

    #[test]
    fn test_2() {
        init();
        log::debug!("Debug message from test 2")
    }

Then make sure the RUST_LOG environment variable is set to debug.

RUST_LOG=debug cargo test -- --show-output

RUST_LOG=debug cargo test -- --show-output

This way you can use shared setup logic in the init() function to setup logging capabilities directly in each test.

Integration Tests ​

Put integration tests in the tests/ directory. Files in the *tests/ folder will only bo compiled when the cargo test command is issued. Test functions in integration tests still need to be annotated with #[test] and the code being tested has to be brought into scope, but the #[cfg(test)] attribute isn't required. Each file in the *tests/* directory is compiled as a seperate crate.

Framework Pattern for Integration Tests ​

By separating utility code into another crate in the tests/ directory, commonly used setup logic can be used across integration tests, but there's a trick. You have to setup the utility module with the structure tests/utilities/mod.rs, where utilities/ can be whatever module name you prefer (e.g. common/). When you setup the files this way, Rust's test compiler doesn't include these files when compiling the integration tests. Of course, the module still has to be brought into scope in the integration tests themselves. For example,

use your_crate;

mod utilities;  // <-- This a helper module in utilities/mod.rs

#[test]
fn test_1() {
    utilities::init();
    assert_eq!(4, your_crate::give_four());
}

use your_crate;

mod utilities;  // <-- This a helper module in utilities/mod.rs

#[test]
fn test_1() {
    utilities::init();
    assert_eq!(4, your_crate::give_four());
}

Filtering Tests ​

You can select which tests to run by providing a test name or sub-string of a set of test names to run them. For example, if you have three tested named foo, bar, baz, then you could run only bar, and baz by issuing cargo test ba. The module becomes part of the name so entire test modules can be run independently using this method.

Using Mocks ​

The idea behind mocking is to enable the testing of systems that have intricate dependencies, such as network IO functions or those that depend on a database connection. Mocking involves replacing real dependencies with mocked objects that appear to be interacting with the dependency, but in reality are only emulating the behaviour. Mocking makes your tests significantly more reliable.

The mockall crate is a powerful solution for integrating mocks into your testing routines. There are two ways to use Mockall.

To create and work with mocks (from to mockall docs):

• Create a mock struct. The easiest is to use #[automock]. It can mock most traits, or structs that only have a single impl block. For things it can’t handle, there is mock!.
• In your test, instantiate the mock struct.
• Set expectations on the mock struct. Each expectation can have required argument matchers, a required call count, and a required position in a Sequence. Each expectation must also have a return value. - Expectations are set by calling .expect_ methods autogenerated on the mock struct. Each expectation declares how many times the function should be called, with what parameters, and
• Supply the mock object to the code that you’re testing. It will return the preprogrammed return values supplied in the previous step. Any accesses contrary to your expectations will cause a panic.

Useful Command Line Flags ​

• cargo test -- --test-threads=1: Limits the number of threads used by the test runner (by default each test uses its own thread).
• cargo test -- --show-output: Shows output from the tests (by defaul, any output to stdout or stderr is captured by the test runner and hidden).
• cargo test -- --include-ignored: Run all of the tests, including the ones tagged with #[ignore].
• cargo test --test integration_test1: Will run only the tests contained in the tests/integration_test1.rs file.

Additional Notes ​

• By nature of Rust, you can't make integration tests for binary crates (crates that only have a src/main.rs file and don't have a src/lib.rs file). This is why many crates will have a basic src/main.rs file, while the crate's functionality is exposed with use statements in its src/lib.rs. This way, the functionality can be tested in the test/ directory.

ADDITIONAL RESOURCES

• Rust Book Chapter 11
• Rust By Example Chapter 21