diff --git a/docs/command_line_reference.rst b/docs/command_line_reference.rst index e997c167b..0dc14facd 100644 --- a/docs/command_line_reference.rst +++ b/docs/command_line_reference.rst @@ -886,8 +886,8 @@ Save the above responses as ``responses.json`` and execute a benchmark as follow This option controls how Rally behaves when a response error occurs. The following values are possible: -* ``continue``: (default) only records that an error has happened and will continue with the benchmark unless there is a fatal error. At the end of a race, errors show up in the "error rate" metric. -* ``abort``: aborts the benchmark on the first request error with a detailed error message. It is possible to permit *individual* tasks to ignore non-fatal errors using :ref:`ignore-response-error-level `. +* ``continue``: only records that an error has happened and will continue with the benchmark unless there is a fatal error. At the end of a race, errors show up in the "error rate" metric. +* ``abort``: (default) aborts the benchmark on the first request error with a detailed error message. It is possible to permit *individual* tasks to ignore non-fatal errors using :ref:`ignore-response-error-level `. * ``continue-on-network``: As with ``continue``, but also continues on network errors (such as connection refused). .. attention:: diff --git a/docs/migrate.rst b/docs/migrate.rst index 90ce710f7..62ca15a71 100644 --- a/docs/migrate.rst +++ b/docs/migrate.rst @@ -101,6 +101,11 @@ Once you are satisfied the data has been migrated, you can delete the old date-b Deleting old indices is irreversible. Ensure you have verified the reindex results and have backups before proceeding. +Default value of ``--on-error`` changed to ``abort`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Starting with Rally 2.14.0 the default value of ``--on-error`` has changed from ``continue`` to ``abort``. This means Rally will now abort the benchmark on the first request error by default. The previous behavior can be restored by explicitly passing ``--on-error=continue``. Individual tasks can still ignore non-fatal errors using the :ref:`ignore-response-error-level ` task parameter. + Migrating to Rally 2.13.0 ------------------------- diff --git a/docs/recipes.rst b/docs/recipes.rst index c40a5b2a6..eb6386512 100644 --- a/docs/recipes.rst +++ b/docs/recipes.rst @@ -209,11 +209,11 @@ Consider a simple example Rally operation:: This query requires the field ``source.geo.location`` to be mapped as a ``geo_point`` type. If incorrectly mapped, Elasticsearch will respond with an error. -Rally will not exit on errors (unless fatal e.g. `ECONNREFUSED `_) by default, instead reporting errors in the summary report via the :ref:`Error Rate ` statistic. This can potentially leading to misleading results. This behavior is by design and consistent with other load testing tools such as JMeter i.e. In most cases it is desirable that a large long running benchmark should not fail because of a single error response. +Rally will abort on the first request error by default. It is possible to permit *individual* tasks to ignore non-fatal errors using :ref:`ignore-response-error-level `. For a more lenient behavior, use ``--on-error=continue`` to only record errors in the summary report via the :ref:`Error Rate ` statistic instead of aborting. -This behavior can also be changed, by invoking Rally with the :ref:`--on-error ` switch e.g.:: +This behavior can be changed by invoking Rally with the :ref:`--on-error ` switch e.g.:: - esrally race --track=geonames --on-error=abort + esrally race --track=geonames --on-error=continue Checking Queries and Responses ------------------------------ diff --git a/esrally/rally.py b/esrally/rally.py index 61c87e247..bc120da4b 100644 --- a/esrally/rally.py +++ b/esrally/rally.py @@ -791,9 +791,9 @@ def add_track_source(subparser): "--on-error", type=OnErrorBehavior, choices=list(OnErrorBehavior), - help="Controls how Rally behaves on response errors (default: continue). 'continue-on-network' will retry on network errors" - "(e.g., connection refused).", - default=OnErrorBehavior.CONTINUE, + help="Controls how Rally behaves on response errors (default: abort). 'continue-on-network' will continue on network errors" + " (e.g., connection refused).", + default=OnErrorBehavior.ABORT, ) race_parser.add_argument( "--telemetry",