Skip to content

Prevent corrupting connections when cancelling futures #79

Open

Description

@pimeys

This has not bubbled up yet, but I highly suspect we're going to hit problems when cancelling futures reading or writing data with Tiberius.

This means, if we use select functionality between a Tiberius future with another, such as a timer, and if we're just reading or writing data to the wire, this operation will be stopped when the timer is ready. At this point the wire is in a dysfunctional state, where either it has partial data from the previous request still awaiting, or we wrote some of the data, but not all.

This means if implementing a service, that for example reads input from a field to perform a search, that is cancelled when the user presses more keys to replace the query with a new one, the connection probably ends into a state where we must reconnect for it to work again.

Some ways of fixing this:

  • Mark a connection as dirty until all data has been read and written
  • Implement the streams so they just peek the data until we know the headers, and only then reading the full packet
  • And then, somehow before every new query, check the wire status and if its dirty, find a way to clean it before triggering a new query.

Activity

  1. yoshuawuyts commented on Sep 15, 2020

    @yoshuawuyts
    Contributor

    Talked about this in-person with @pimeys today. This is crucial for the correct working of the driver. So far my hunch is that what we need to do is move the parsing layer from the future into the Client.

    That way if a future is cancelled midway through, the client knows which task was started on, and the next query can run the previous task to completion before starting the next task.


    Another thing we discussed is about TCP pooling: we need to signal that if the client errors out, the connection needs to be dropped and reopened. This prevents the TCP connection from existing in an invalid state.

  2. self-assigned this
    on Sep 15, 2020
  3. added and removed on Sep 16, 2020
  4. added this to the Backlog 2.8.0 milestone on Sep 16, 2020
  5. added and removed on Sep 16, 2020
  6. MattJackson commented on Sep 18, 2026

    @MattJackson
    Contributor

    Consolidating the cancellation threads here — #300 is the same root cause (an externally-dropped future) with a clean repro, so I'm closing it as a duplicate of this.

    Where this looks to stand after the recent work:

    Handled

    • Explicit cancellation — feat: modern SQL Server connectivity + TDS protocol + ergonomic APIs #442 (in review) adds Client::cancel_query(), which sends a TDS Attention signal and drains the acknowledgement: the protocol-correct way to abort an in-flight request, including from a timeout you drive yourself.
    • Write-side drop-safety — harden: malformed-input hardening, non-panicking encoders, unit coverage #443 (in review) adds connection poisoning. A multi-packet write interrupted by a dropped query/execute/bulk future (or select! losing the race) now leaves the connection marked unusable, so the next use fails fast with a clear error instead of appending after a half-sent message and silently desyncing the server. That's the write half of the "mark the connection dirty" idea in the original post.

    In the works

    • Read-side desync on a driver-driven timeout — a built-in command_timeout is being worked on: when it fires mid-response it marks the connection unusable rather than hanging on the half-read tail, which covers the read-side case for timeouts the driver itself drives.

    Still open (what this issue now tracks)

    • Read-side drop-safety for externally cancelled futures. A future dropped mid-response by a cancellation you wrap yourself — e.g. tokio::time::timeout or select! firing while rows are still streaming (the Cancel is not safety #300 repro: a timed-out WAITFOR, then the next query hangs) — doesn't appear to have a symmetric guard yet. The response tail is still on the wire, so the connection is out of sync but nothing flags it.

    The likely fix mirrors the write side: a guard on the response path that, if the stream isn't fully drained, marks the connection unusable so a pool discards and reopens it — turning today's hang into a clear error. Fuller recovery (send an Attention, drain to the DONE(ATTN), keep the connection) would be a heavier follow-on. @yoshuawuyts' earlier note — move the task/parsing state into the Client so the next query finishes or discards the previous one, and drop/reopen on error — still looks like the right shape. Leaving open to track the read-side piece.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions