Features
Release 1.2 ("Eunomia") features
-
Multi-node test execution: the current release of NSTAT allows the execution of its basic components (controller, SouthBound generator, NorthBound generator) on different nodes. Every component can be built, started, executed and monitored on a node (e.g. physical or virtual machine) specified in the configuration file, which is assumed to be already up and running. NSTAT itself may run on a separate node from which it orchestrates and monitors all other components. To make this possible, nodes are assumed to support SSH connectivity and have already checked out an identical instance of NSTAT source tree.
-
NorthBound traffic generation handlers: in the current release, NorthBound traffic generation logic has been decoupled from the flow scalability test. Separate NorthBound handlers have been implemented and can be executed from a separate node ("NorthBound generator node"), as it happens with every other NSTAT component.
-
Fully automated, end-to-end testing: NSTAT is responsible for automating
and sequencing every single step of the testing procedure, including:
- component lifecycle management and orchestration: build, start, stop, etc., and synchronize the controller, generator, NB apps
- test dimensioning: iterate the test over a large space of experimental parameters
- online monitoring and sampling: additional threads are being launched in the background to monitor component progress and collect performance results and runtime statistics
- reporting: report generation and result plotting
-
Exhaustive stress testing: within a single run, NSTAT repeats the test over an extensive experimental space defined by the cartesian product of all test dimensions, e.g. the number of switches simulated by the SB generator, the number of simulated hosts, etc.. This allows the user to express and execute multiple test iterations into a single run, and to easily discover controller behavior at extreme cases.
-
Scalable traffic generation: NSTAT provides MT-Cbench, a multi-threaded implementation of Cbench, aiming at emulating large-scale SDN topologies with thousands of switches and bring controller performance to its limits.
-
Unification of stress tests: NSTAT unifies existing and new test cases and tools under a common environment. A summary of the supported tests is given here
-
Easy and rich configuration system: NSTAT allows the user to easily specify the test configuration parameters using a single JSON file. Nearly all test components and procedures from component orchestration to plotting are largely described by these configuration parameters.
-
Comprehensive reporting and flexible plotting system: NSTAT provides comprehensive output in a single HTML report, including all the information required to fully describe a test. It is also supported by a flexible plot generation methodology (within the configuration parameters JSON file) which allows the user to easily specify plot types and parameters.
-
Modular and extensible architecture: for most cases, NSTAT is not tightly coupled to a specific controller or generator implementation or version. Rather, it is pluggable to any controller/generator provided it adheres to certain conventions. Furthermore, extending the existing test dimensions and result statistics is rather easy and doesn't affect existing logic.
See the future releases page