Repository navigation
fix: a select-only SQLRT session declines a non-select before running it - #10
TonyStack2026 wants to merge 2 commits into
Conversation
0651d78 to
fa00db0
Compare
M-B's B3: `SQLExecutor(INTERACTIVE)` reads a sub query through the instance
tunnel unless the client opts out, and that is the path JDBC's MaxQA mode
takes. Without it an interactive consumer silently re-runs its statement as an
offline instance, so the session plane could not be accepted end to end —
the previous increment had to set `useInstanceTunnel(false)` in its own
acceptance test to avoid being fooled by that fallback.
- `GET .../instances/{id}?data&cached&taskname=..[&queryid=N]` answers one sub
query as a Protobuf record stream. There is no download session behind it, so
the stream carries its own schema frame (field 1 JSON, then SCHEMA_END_TAG
33553920 with the CRC32C of the field number and payload); the records after
it are byte-identical to the ones the offline download path writes.
- `odps-tunnel-record-count` reports the rows still available from the read's
start offset, not the size of the result: `SessionRecordSetIterator` keeps the
first answer's count and pages `openRecordReader(offset+cursor, fetchSize)`
while the cursor is below it, so reporting the total sends a client reading
past the end.
- `rowrange` windows, `sizelimit` shortens a response by halving (at least one
row, so a reader always advances) and `instance_tunnel_limit_enabled` applies
READ_TABLE_MAX_ROW to that read only.
- A sub query that is still running makes the read wait — long polling is what
`cached` asks for — and a wait past the bound answers with the SDK's own
retryable tunnel timeout instead of an empty stream. Failed, cancelled,
unknown and dropped sub queries, and one-shot instances, each get their own
structured JSON error; a statement with no result set is refused with
`InstanceTypeNotSupported`, which is how the service sends the client to the
session API instead of an empty download.
- Sub queries now retain their typed result next to the CSV text, inside the
same retention window and the same instance byte budget, so both read paths of
one statement serve the same rows and the same types.
Evidence: 10 new Go cases (framing and CRC checked by decoding the response with
our own record decoder, paging, size limit, the 10k cap, non-select, structured
failures, long poll, idle-TTL activity, retention) — `go test -race ./internal/...`
green. Against a locally built binary `McqaSessionTest` is 6/6, including two new
SDK cases: the default fetch, and an offset read that pages the same result; both
fail on the previous build with `NoSuchObject: unsupported endpoint`, so they are
not passing by falling back. `EmulatorTest` 26/26 and `UdfMetadataTest` 4/4 stay
green, and the PyODPS contract probe stays at 30/0 on a fresh process.
Driving the published JDBC driver with interactiveMode=mcqa and the instance tunnel applied every write statement twice: the session executed the sub query, the tunnel read then answered InstanceTypeNotSupported, and the client's own reaction to that answer is to run the same statement offline again (SQLExecutorImpl NON_SELECT_QUERY -> runQueryInternal(OFFLINE, ..., true)). An insert of three rows landed six. The restriction is a session property, not an emulator opinion, and the client can lift it: odps.sql.session.select.only, in the session settings or per statement. The refusal is the pair the client already acts on - queryId -1, status ok, and a message carrying ODPS-185 plus the service's own sentence - so a declined submission becomes one offline run instead of an error. Multi-statement submissions are classified by every segment, because 'select 1; drop table t' must not pass for a select.
fa00db0 to
ca2e4dd
Compare
|
resolve conflict pls |
|
Closing rather than rebasing: #11 (merged as Post-merge exit regression against |
Stack note
Head branch is
tony/em-mcqa-instance-tunnel(PR #8, which is on top of #6) plus one commit,so three commits show here until #6/#8 land and this collapses to one.
The bug
Driving the published driver (
odps-jdbc3.10.13 from Maven Central) withinteractiveMode=mcqa&useInstanceTunnel=trueagainst the #6/#8 stack applied every writestatement twice. One
insert into ... values (1,'one'),(2,'two'),(3,'three')gavecount(*) = 6.The request sequence, captured with a logging proxy:
PUT .../instances/{session}?info(Key=query) — the session executes the DDL/DML, queryId 1.GET .../instances/{session}?data&cached&queryid=1— the tunnel read answers400 InstanceTypeNotSupported/Non select query not supported, which is what feat: download an MCQA sub query's result over the instance tunnel #8 added andwhat the service says.
POST /projects/{p}/instances— the driver reruns the same statement offline. That is thedocumented reaction, not a workaround:
SQLExecutorImpl#getSessionResultSetByInstanceTunnelcatches
TunnelRetryStatus.NON_SELECT_QUERYand callsrunQueryInternal(ExecuteMode.OFFLINE, retryInfo.errMsg, true).Step 2 is only a refusal of the download. By then the side effect has already happened in
step 1, so the rerun in step 3 lands on top of it. A session that executes a non-select and
then says it cannot hand back its result is the wrong order: the client can only ever be told
once, and it will act on that by running the statement itself.
The change
A SQLRT session declines a non-select before starting it, answering the way the client
already expects to be answered:
{"queryId": -1, "status": "ok", "result": "ODPS-1850001 Non select query not supported."}queryId == -1withstatus okis the branchSQLExecutorImpl#runInSessionWithRetryreads as"the session would not run this", and
ODPS-185inside the message is the flagFallbackPolicy#shouldFallbackmatches whenfallbackForUnsupportedFeatureis on — which iswhat turns the refusal into a single offline run instead of an error. Drop the code and keep
the sentence and clients with fallback disabled get a hard failure they cannot distinguish.
odps.sql.session.select.onlylifts it, in the session settings or per statement — the samekey the client sends in the sub-query settings block. Classification looks at every
statement of a submission, comments and quotes stripped, so
select 1; drop table tis not aselect.
Evidence
go test -race -count=1 ./internal/...,go vet,gofmt -lmcqa_select_only_test.goselect.only=falseat either level runs it; statement classification table incl.select ';' as xand multi-statementtests/javaagainst this binaryMcqaSessionTest6/6,EmulatorTest26/26,UdfMetadataTest4/4mcqa-tunnel-effecton #6+#8count after one insert statement=6count after one insert statement=3, stillinteractiveMode=INTERACTIVE,t1=BIGINTvia the tunnel readdisableFallback=trueODPS-1850001 Non select query not supported.— a client that turned fallback off is told, not silently reroutedTwo tests in #8 (
TestMCQADirectDownloadRefusesNonSelect,TestMCQADirectDownloadLimitedFlagCapsRows) now create their session withodps.sql.session.select.only=false: they need DDL/DML inside a session to reach the tunnellayer they pin, which is exactly what the opt-in is for. Their assertions are unchanged.
What this does not claim
queryId: -1+ODPS-185pair is derived from the client's handling of it (publicodps-sdk-core), not from a run against a real MCQA session — I have no interactive quotahere. The driver's double execution is measured; the shape of the answer is the one the
driver already branches on.
show/desc-style statements count as "not a select" under this classification and aredeclined. If any of them should stay in-session, that is a follow-up on the keyword rule.