xd009642 / tarpaulin

Tarpaulin Tarpaulin is a code coverage reporting tool for the Cargo build system, named for a waterproof cloth used to cover cargo on a ship. Currently, Tarpaulin provides working line coverage, and while fairly reliable, may still contain minor inaccuracies in the results. A lot of work has been done to get it working on a wide range of projects, but unique combinations of packages and build features can cause issues, so please report anything you find that's wrong. Also, check out our roadmap for planned features. On Linux, Tarpaulin's default tracing backend is still Ptrace and will only work on x86\_64 processors. This can be changed to the llvm coverage instrumentation with . For Mac and Windows, this is the default collection method. It can also be run in Docker, which is useful for when you don't use Linux but want to run it locally, e.g. during development. See below for how to do that. Below is the help-text for a thorough explanation of the flags and features available: Note on tests using signals If your tests or application make use of unix signals they may not work with ptrace instrumentation in Tarpaulin. This is because Tarpaulin relies on the sigtrap signal to catch when the instrumentation points are hit. The option results in forwarding the signals from process stops not caused by SIGSTOP, SIGSEGV or SIGILL to the test binary. Nuances with LLVM Coverage Despite generally being far more accurate there are some nuances with the LLVM coverage instrumentation. • If a test has a non-zero exit code coverage data isn't returned • Some areas of thread unsafety • Unable to handle fork and similar syscalls (one process will overwrite another's profraw file) In these cases coverage results may differ a lot between ptrace and llvm and llvm coverage may be a worse choice. Things like doc tests with the attribute or won't report any coverage because of non-zero exit codes and if you use these and want coverage data from them you should avoid the llvm coverage backend. Features Below is a list of features currently implemented. As Tarpaulin loads binary files into memory and parses the debugging information, different setups could lead to coverage not working. In this instance, please raise an issue detailing your setup and an example project and I'll attempt to fix it (please link us to a repo and the commit containing your project and paste the verbose output). • Line coverage • Full compatibility with cargo test CLI arguments • Uploading coverage to or • HTML report generation and other coverage report types • Coverage of tests, doctests, benchmarks and examples possible • Excluding irrelevant files from coverage • Config file for mutually exclusive coverage settings (see section for details) Issues and Contributing Issues, feature requests and pull requests are always welcome! For a guide on how to approach bugs found in Tarpaulin and add features please check CONTRIBUTING. If you're having any troubles also look to our TROUBLESHOOTING Rust 1.23 introduced a regression in the compiler affecting Tarpaulin's accuracy. If you see missing lines or files, check your compiler version. Usage Installation Tarpaulin is a command-line program, you install it into your development environment with cargo install. A locked install is recommended to avoid being impacted by breaking changes in dependencies: When using the Nix package manager, the package can be used. This ensures that Tarpaulin will be built with the same rust version as the rest of your packages. You can also use cargo-binstall: Environment Variables When Tarpaulin runs your tests it strives to run them in the same environment as if they were run via cargo test. To achieve this it sets the following environment variables when executing the test binaries: • **RUST_BACKTRACE** - _Set to when --verbose flag is used unless it is already set_ • **CARGO_MANIFEST_DIR** - _Path to Cargo.toml From --root | --manifest-path or guessed from the current or parent directory_ • **CARGO_PKG_NAME** - _From Cargo.toml_ • **CARGO_PKG_AUTHORS** - _From Cargo.toml_ • **CARGO_PKG_VERSION** - _From Cargo.toml_ • **LLVM_PROFILE_FILE** - _Used for LLVM coverage_ Cargo Manifest For Tarpaulin to construct the Cargo environment correctly, Tarpaulin needs to find Cargo.toml by either: • Using *--root* or *--manifest-path* or • By invoking Cargo from the current working directory within the project holding Cargo.toml manifest or • By invoking Cargo from a sub-directory within the project If Cargo does not find any Cargo.toml from using either of the above methods the run will error "cargo metadata" and exit. Several RFCs are open in rust-lang to expose more of these directly in order to avoid the issues arising out of this. Command line To get detailed help on available arguments when running Tarpaulin call: Currently, no options are required, if no root directory is defined Tarpaulin will run in the current working directory. Below is a Tarpaulin run utilising one of our example projects. This is a relatively simple project to test and if you check the test, you can see the output correctly reports the lines the test hits. Tarpaulin can also report the change in coverage for each file between runs. If the tests were updated in the previous example to cover all the lines we would expect the following output. Hint: if using coveralls.io with travis-ci run with the options . The coveralls.io repo-token is mainly designed for private repos and it won't generate a badge for the coverage results submitted (although you can still see them on the coveralls web interface). For an example of a project using Tarpaulin, you can check out my crate keygraph-rs. Ignoring code in files Tarpaulin allows you to ignore modules or functions using attributes. Below is an example of ignoring the main function in a project: Unfortunately, due to the unexpected cfg warnings cargo now emits you will likely want to add the recommended lints to y…