Default to abort for --on-error flag

[mmustafasenoglu]

This PR changes the default value of the --on-error flag from continue to abort.

Changes

• esrally/rally.py: Changed default from OnErrorBehavior.CONTINUE to OnErrorBehavior.ABORT and updated help text
• docs/command_line_reference.rst: Updated documentation to reflect new default
• docs/migrate.rst: Added migration note for 2.14.0
• docs/recipes.rst: Updated recipe documentation to reflect new default behavior

This is a breaking change for Rally 2.14.0. Users who want the old behavior can explicitly pass --on-error=continue.

Fixes: Reopens #2167 (original PR closed due to fork deletion)

Closes #1581

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[stevereads-elastic]

copilot review this

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

docs/migrate.rst:26

• This new "--on-error" migration section is inserted inside the "Migrate rally indices to data streams" section. Because it uses the same section underline level (^^^^), it likely breaks the document structure and causes the subsequent reindexing guidance to be grouped under the on-error section instead of the data-streams section. Consider moving this section to be a sibling of the other 2.14.0 migration subsections, or demoting it to a lower-level heading placed after the data-streams content.

Default value of ``--on-error`` changed to ``abort``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Starting with Rally 2.14.0 the default value of ``--on-error`` has changed from ``continue`` to ``abort``. This means Rally will now abort the benchmark on the first request error by default. The previous behavior can be restored by explicitly passing ``--on-error=continue``. Individual tasks can still ignore non-fatal errors using the :ref:`ignore-response-error-level <track_schedule>` task parameter.

docs/recipes.rst:214

• Minor grammar: the comma after "changed" is unnecessary here ("This behavior can be changed by invoking…").

This behavior can be changed, by invoking Rally with the :ref:`--on-error <command_line_reference_on_error>` switch e.g.::

esrally/rally.py:795

• The CLI help for --on-error says 'continue-on-network' will retry on network errors(e.g., …), but the code treats continue-on-network as continuing on connection errors (see esrally/driver/driver.py:2275-2277) and track docs also describe it as “continue”. Please update the help text to reflect the actual behavior (and add the missing space before (e.g., …)).

        help="Controls how Rally behaves on response errors (default: abort). 'continue-on-network' will retry on network errors"
        "(e.g., connection refused).",

[mmustafasenoglu]

Addressed the review feedback:

• migrate.rst: Moved the --on-error migration section after the data streams content to preserve the document heading hierarchy. Previously it was inserted in the middle of the data streams section, breaking the structure.
• recipes.rst: Removed unnecessary comma after "changed".
• rally.py: Fixed the --on-error help text — changed "retry" to "continue" to match the actual behavior in driver.py, and added the missing space before (e.g..

[inqueue]

Thank you for your patience @mmustafasenoglu. LGTM

[b-deam]

Buildkite test this