fix: return real bucket/table IDs on dev branches (prefix-based model)

[padak]

Summary

On projects using the legacy prefix-based branch model (non-branched-storage), get_buckets, get_table, and get_tables were normalizing bucket/table IDs by stripping the branch prefix (e.g. out.c-15865-test became out.c-test). These normalized IDs don't physically exist in the Storage API and break all downstream tool calls that reference them.

Root cause: _combine_buckets, _get_table, and _list_tables in tools/storage/tools.py were replacing the real id field with prod_id (the stripped version) when returning results for dev-branch-only buckets/tables.

Fix: Stop overwriting id with prod_id in the output. The prod_id field is preserved for internal deduplication logic (dict key in _list_tables for merging prod+dev tables of the same logical table), but the id field returned to the caller is now always the real, physical Storage API ID.

Changes

• _combine_buckets (dev-only bucket case): removed 'id': dev_bucket.prod_id from model_copy update
• _get_table: removed 'id': table.prod_id from model_copy update
• _list_tables (dev bucket tables): removed 'id': table.prod_id from model_copy update
• Updated 7 test expected values to use real (prefixed) IDs instead of stripped ones

Fixes keboola/keboola_agent_cli#113

Test plan

• All 51 existing tests in tests/tools/storage/test_tools.py pass
• test_get_bucket dev-only case now expects real prefixed ID
• test_get_buckets dev branch case now expects real prefixed ID
• test_get_table dev branch cases (3 parametrizations) now expect real prefixed IDs
• test_get_tables dev branch case now expects real prefixed IDs for both emails and assets

[padak]

Update: Branched Storage Support Added

Added a second commit that extends the fix to also handle the branched storage model (projects where dev branches don't use prefixed bucket/table IDs).

What changed

1. _get_table now handles branched storage correctly

The old logic had a bug for branched storage:

1. _get_table('out.c-results.output') on a dev branch would find the table, see it has KBC.createdBy.branch.id metadata, and discard it ("not a prod table")
2. Then try the prefixed fallback out.c-9999999-results.output — which doesn't exist in branched storage
3. Result: table not found for a table that clearly exists

The fix mirrors how _find_buckets already works — when the first lookup returns a table whose branch_id matches client.branch_id, treat it as the dev table directly:

# Before (discards matching dev table):
if branch_id:
    prod_table = None  # gone forever

# After (saves matching dev table):
if branch_id:
    if branch_id == client.branch_id:
        dev_table = first_result  # branched storage: keep it

Bonus: this also eliminates a redundant API call when querying with a prefixed dev table ID (e.g., in.c-1246948-foo.emails) — previously it would call table_detail twice with the same ID.

2. Six new branched storage tests

Test	What it verifies
test_get_bucket_branched_storage (x2)	Dev-only bucket + prod bucket detail on branched storage branch
test_get_buckets_branched_storage	Listing all buckets, correct counts
test_get_table_branched_storage (x2)	Dev table with metadata (fails on main!) + prod table on branch
test_get_tables_branched_storage	Listing tables in dev-only bucket

Verified: fails on main, passes on fix

$ pytest -k "branched_storage" (on main branch)
PASSED  test_get_bucket_branched_storage[out.c-results]
PASSED  test_get_bucket_branched_storage[in.c-data]
PASSED  test_get_buckets_branched_storage
FAILED  test_get_table_branched_storage[out.c-results.output]  ← the key one
PASSED  test_get_table_branched_storage[in.c-data.customers]
PASSED  test_get_tables_branched_storage

$ pytest (on fix branch)
57 passed ✓
1139 passed (full suite) ✓

[padak]

Updated integration tests per @Matovidlo's feedback:

Now tests both branch storage models:

• [prefix-based] — project with storage-branches feature OFF (bucket IDs have c-{branch_id}- prefix)
• [branched-storage] — project with storage-branches feature ON (bucket IDs identical to production)

Feature detection: Each test run calls tokens/verify and checks owner.features for storage-branches. If the project doesn't match the expected mode, the test is skipped with a clear message.

10 tests total (5 per mode), all passing:

test_get_bucket_by_id_returns_same_id[prefix-based]       PASSED
test_get_bucket_by_id_returns_same_id[branched-storage]    PASSED
test_list_all_buckets_includes_dev_bucket[prefix-based]    PASSED
test_list_all_buckets_includes_dev_bucket[branched-storage] PASSED
test_returned_id_is_api_callable[prefix-based]             PASSED
test_returned_id_is_api_callable[branched-storage]          PASSED
test_get_tables_returns_real_id[prefix-based]               PASSED
test_get_tables_returns_real_id[branched-storage]            PASSED
test_returned_table_id_is_api_callable[prefix-based]        PASSED
test_returned_table_id_is_api_callable[branched-storage]    PASSED

Env vars:

TEST_KBC_TOKEN_PREFIX=xxx    # project with branched storage OFF
TEST_KBC_TOKEN_BRANCHED=yyy  # project with branched storage ON
TEST_KBC_URL=https://connection.keboola.com  # optional, defaults to this

Tests are skipped automatically when env vars are not set.

[padak]

I've addressed the feedback — moved tests to integtests/, adopted the INTEGTEST_STORAGE_TOKENS pool pattern, added feature detection via tokens/verify, and removed all skipping logic.

I just noticed the CLAUDE.md in the repo — should have read it earlier, apologies. I see the conventions now (Linear issue ID in branch/commits, version bump in every PR, tox for testing, PR template). Since I'm an external contributor without a Linear issue, some of these don't directly apply, but I understand the structure better now.

A few things I'm not sure how to handle from outside your team:

• Version bump: should I bump pyproject.toml or do you prefer to do that on merge?
• Linear ID: I don't have one for this fix — is there an existing ticket I should reference?
• tox: I tested locally with pytest against two real projects (prefix-based + branched-storage, 10/10 pass). Happy to run tox if you point me to the right venv setup.

The core fix is 3 one-line changes in tools.py (stop overwriting id with prod_id). Verified against real projects in both branch models. Let me know what else you need to get this across the line.

[Matovidlo]

LGTM