MLPerf Inference Test Bench, or Mitten, is a framework by NVIDIA to run the MLPerf Inference benchmark.
This is an in-progress refactoring and extending of the framework used in NVIDIA's MLPerf Inference v3.0 and prior submissions.
Mitten, while more optimized for NVIDIA GPU-based systems, is a generic framework that supports arbitrary systems. Some of the things that Mitten handles are:
- System hardware detection
- Describing and running a benchmark as a pipeline
- Building TRT engines from various sources
- Executing C++ or other compiled executables as a pipeline operation inside Python
- Easier method of configuring pipelines
- Automatic debugging logs and artifacts
- Server-client system for benchmarking workloads over a network connection