In our blog, we’ve previously had a Tag1 Team Talk about Goose, by Tag1 CEO Jeremy Andrews. Goose is a Locust-inspired load testing tool In Rust. Goose has been effective in helping Tag1 support its clients by ensuring their websites hold up under stress.
Version 0.6.0 added Gaggles. A Gaggle is a distributed load test, made up of one Manager process and one or more Worker processes. With Goose, a single process can use all available CPU cores. This makes Goose very powerful and fast and more likely to bottleneck on available network speed, an important aspect of a full load test. You'll likely need multiple computers with Workers to scale beyond a 1G network interface, and sometimes it's useful to generate load from different places/clouds/etc. In all cases, the Workers can send statistics to the Manager, which aggregates it all together and summarizes at the end.
Gaggles are an optional, compile-time feature. For a more detailed discussion of Gaggles see Show me how this flock flies: A working Gaggle in the Goose Load Testing Framework.
Async enables the simulation of considerably more "users" by giving up the CPU when a task is blocked. For example, when a Goose "user" requests a web page, it's blocked until the server responds. Without Async, Goose blocks an entire CPU core while waiting for that response. With Async, Goose makes the request, then gives up the core (ie, sleeps) allowing another "user" to make a request with that same core, and again gives up the core (sleeps). When a response comes back, the appropriate "user" gets woken to complete the next task. This creates a 2x boost in performance.
Goose as a load test library
Tag1 Vice President of Software Engineering Fabian Franz, along with the help of the community, made it possible to pass a closure to
GooseTask::new. This change makes it possible to use Goose as a load test library within another application. Previously, load test task functions were standalone; now, another program can dynamically create task functions and pass them to the Goose load test process as needed.
Another way Goose was reworked as a library is that it also now passes errors up the stack instead of calling
exit(1). Goose generates valid errors which can be properly handled in applications, when previously it exited on errors. Rust uses a particular pattern for results:
Result<Ok(), Err()>. Jeremy explains the pattern in the README, in the section If you’re new to Rust.
If you're new to Rust,
main()'s return type of
Result<(), GooseError>may look strange. It essentially says that
mainwill return nothing (
()) on success, and will return a
GooseErroron failure. This is helpful as several of
GooseAttack's methods can fail, returning an error. In our example,
execute()each may fail. The
?that follows the method's name tells our program to return an error on failure, otherwise continue on.
The request throttling feature enables you to generate a standardized amount of load from different servers, even when those servers have different resources available, ensuring consistency of data. It is an API change that uses the
--throttle-requests option when running Goose to optionally limit the maximum requests per second.
For a full explanation of how throttling works, see this section of the Goose source code.
Goose includes per-task metrics in addition to per-request metrics. Previously, there was no way to know for certain whether all Goose Tasksets and Goose Tasks ran. This is a problem we've experienced with Locust, in which you can accidentally run an invalid load test by, for example, setting weights too high and running too few users.
When displaying per-task metrics, all Tasksets and Tasks are listed, even if they did not actually run. Disable this feature with
Improved metrics logging
Goose’s logging of metrics was significantly improved. Names, formats, and statistics are clearer and easier to understand.
metricsfor consistency and clarity. These are the finalized names:
--metrics-format=to switch between
--metrics-file=to optionally log all requests to file specified, enabling you to keep track of your metrics over time. This file can be downloaded and analyzed as needed
--no-metrics.)Goose defaults to collecting and displaying metrics, when originally we defaulted to not collecting and displaying metrics. Disable this with the --no-metrics option
- Default to resetting metrics, disable with
--no-reset-stats, display spawning metrics before resetting
For more information on test metrics, and to see an example, see Logging Load Test Metrics.
Goose now includes the
--debug-file option to write errors to the specified log file, and
--debug-format= option to switch between
json (default) and
raw formats. This feature solves a common problem when writing load tests, and helps in quickly understanding why things aren't working as you expect, and/or detecting when the load test is generating so much load your infrastructure is failing, and so on.
For more information, see Load Test Debug Logging in the Goose documentation.
GooseAttacks have been updated to give you more flexibility in configuring your testing runs. This means you can set all run-time options with custom defaults directly from your test and don't need to pass in command line options. Command line options however still can override the defaults. This change replaces
GooseAttack::set_host() with more generic
GooseAttack::set_default(), exposing new default settings. For a full list and a configuration example, see Defaults. The following list is a subset that may be most useful to customize:
- Users to start:
- Users to start per second:
- Number of seconds for test to run:
- Only print final summary metrics:
- Metrics log file name:
- Debug log file name:
- Maximum requests per second:
Goose has a number of other updates as well. Internal test coverage was increased.
Several options are new or updated, enabling additional command line configuration of your load test application and output. These changes and addition include:
- Some options were shortened to fit standard console width as part of the switch from
- Goose performs a checksum to confirm all Workers are running the same load test as the Manager, use
--no-hash-checkto ignore (not recommended).
--sticky-followmakes redirects of the GooseClient
base_urlsticky, affecting subsequent requests. When you make a request as part of a test, if that request redirects to a new URL (for example, from
foo.example.com), all subsequent requests will go to the redirected URL (f
oo.example.com). This feature is off by default.
set_client_builder, allowing load tests to build Reqwest clients with custom options. This exposes the full power of the Reqwest library to anyone writing a Goose load test.
Version 0.7 integrated httpmock into testing load tests: With this change we started doing proper functional, integration, load, and comment testing to validate every PR and and commit into the mainline branch. This improves the stability and reliability of Goose as development moves forward. Tag1 thanks Alexander Liesenfeld for his help in making this possible.
Finally, this release also includes updates to documentation, reworded errors, enforcement of
error.detail, increased precision in smaller metric values, and consistently built configurations from arguments.
All of these changes, in addition to the many other improvements made in the past year, have significantly improved the speed, usability, and performance of Goose and its ability to help you load test your website.