Skip to content

fix(oauth): add thread-safe token refresh with double-checked locking#883

Merged
tolik0 merged 3 commits intomainfrom
devin/1769515185-thread-safe-oauth-refresh
Feb 3, 2026
Merged

fix(oauth): add thread-safe token refresh with double-checked locking#883
tolik0 merged 3 commits intomainfrom
devin/1769515185-thread-safe-oauth-refresh

Conversation

@devin-ai-integration
Copy link
Contributor

Summary

Fixes a race condition where multiple streams starting concurrently with an expired OAuth token would all attempt to refresh simultaneously. For single-use refresh tokens, this causes failures because the refresh token is invalidated after the first successful refresh.

The fix adds a class-level threading.Lock to AbstractOauth2Authenticator and implements double-checked locking in get_access_token():

  1. First check if token is expired (without lock - performance optimization)
  2. Acquire lock
  3. Re-check if token is still expired (another thread may have refreshed)
  4. If still expired, perform the refresh
  5. Release lock

The same pattern is applied to SingleUseRefreshTokenOauth2Authenticator.get_access_token() which overrides the base method.

Review & Testing Checklist for Human

  • Verify class-level lock scope is correct: The lock is shared across ALL authenticator instances. This is intentional because multiple streams share the same refresh token from the connector config. However, verify this won't cause issues if there are truly independent OAuth flows in the same process.
  • Test with a real connector: The unit tests mock refresh_access_token(). Test with an actual connector that has multiple streams and uses single-use refresh tokens (e.g., a connector where you've observed this issue).
  • Check for other code paths: Verify there are no other code paths that call refresh_access_token() directly, bypassing the lock protection.

Notes

  • The double-checked locking pattern is standard for this use case in Python
  • Unit tests verify that 5 concurrent threads result in exactly 1 refresh call
  • Lint and type checks pass locally

Requested by: gl_anatolii.yatsuk@airbyte.io

Link to Devin run: https://app.devin.ai/sessions/47c89308cb7f48b48d6bb424a393166e

When multiple streams start concurrently with an expired token, they can
all detect the token is expired and attempt to refresh simultaneously.
For single-use refresh tokens, this causes failures because the refresh
token is invalidated after the first successful refresh.

This fix adds a class-level lock to AbstractOauth2Authenticator that
serializes token refresh attempts. The double-checked locking pattern
ensures that:
1. Only one thread performs the actual refresh
2. Other threads wait and use the newly refreshed token
3. No redundant refresh attempts occur

Changes:
- Add _token_refresh_lock class-level threading.Lock to AbstractOauth2Authenticator
- Update get_access_token() in AbstractOauth2Authenticator to use double-checked locking
- Update get_access_token() in SingleUseRefreshTokenOauth2Authenticator with same pattern
- Add unit tests for concurrent token refresh scenarios

Co-Authored-By: gl_anatolii.yatsuk@airbyte.io <gl_anatolii.yatsuk@airbyte.io>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1769515185-thread-safe-oauth-refresh#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1769515185-thread-safe-oauth-refresh

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

PyTest Results (Fast)

3 855 tests  +2   3 843 ✅ +2   6m 34s ⏱️ +2s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit d806dc8. ± Comparison against base commit d3c9419.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Jan 27, 2026

PyTest Results (Full)

3 858 tests  +2   3 846 ✅ +2   10m 57s ⏱️ -5s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit d806dc8. ± Comparison against base commit d3c9419.

♻️ This comment has been updated with latest results.

@tolik0 tolik0 self-assigned this Jan 27, 2026
@tolik0 tolik0 marked this pull request as ready for review January 27, 2026 13:04
Copilot AI review requested due to automatic review settings January 27, 2026 13:04
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a race condition in OAuth token refresh by implementing thread-safe double-checked locking. When multiple streams start concurrently with an expired token, they would all attempt to refresh simultaneously, causing failures with single-use refresh tokens since the token is invalidated after the first successful refresh.

Changes:

  • Added a class-level threading.Lock to AbstractOauth2Authenticator to synchronize token refresh across all instances
  • Implemented double-checked locking pattern in both AbstractOauth2Authenticator.get_access_token() and SingleUseRefreshTokenOauth2Authenticator.get_access_token()
  • Added comprehensive unit tests verifying that 5 concurrent threads result in exactly 1 refresh call

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py Added class-level lock and double-checked locking in base authenticator
airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py Added double-checked locking to SingleUseRefreshTokenOauth2Authenticator
unit_tests/sources/streams/http/requests_native_auth/test_requests_native_auth.py Added concurrent token refresh tests for both authenticator types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@tolik0
Copy link
Contributor

tolik0 commented Jan 27, 2026

/prerelease

Prerelease Job Info

This job triggers the publish workflow with default arguments to create a prerelease.

Prerelease job started... Check job output.

✅ Prerelease workflow triggered successfully.

View the publish workflow run: https://github.com/airbytehq/airbyte-python-cdk/actions/runs/21398747886

Copy link
Contributor

@aldogonzalez8 aldogonzalez8 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVED
Just left a comment about PEP8.

Address PR review feedback from aldogonzalez8 to follow PEP8 import conventions.

Co-Authored-By: gl_anatolii.yatsuk@airbyte.io <gl_anatolii.yatsuk@airbyte.io>
…e conflicts

Co-Authored-By: gl_anatolii.yatsuk@airbyte.io <gl_anatolii.yatsuk@airbyte.io>
@tolik0 tolik0 merged commit 15542de into main Feb 3, 2026
28 checks passed
@tolik0 tolik0 deleted the devin/1769515185-thread-safe-oauth-refresh branch February 3, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants