Skip to content

fix: a select-only SQLRT session declines a non-select before running it - #10

Closed
TonyStack2026 wants to merge 2 commits into
dingxin-tech:mainfrom
TonyStack2026:tony/em-session-select-only
Closed

TonyStack2026 wants to merge 2 commits into
dingxin-tech:mainfrom
TonyStack2026:tony/em-session-select-only

Conversation

@TonyStack2026

Copy link
Copy Markdown
Contributor

Stack note

Head branch is tony/em-mcqa-instance-tunnel (PR #8, which is on top of #6) plus one commit,
so three commits show here until #6/#8 land and this collapses to one.

The bug

Driving the published driver (odps-jdbc 3.10.13 from Maven Central) with
interactiveMode=mcqa&useInstanceTunnel=true against the #6/#8 stack applied every write
statement twice
. One insert into ... values (1,'one'),(2,'two'),(3,'three') gave
count(*) = 6.

The request sequence, captured with a logging proxy:

  1. PUT .../instances/{session}?info (Key=query) — the session executes the DDL/DML, queryId 1.
  2. GET .../instances/{session}?data&cached&queryid=1 — the tunnel read answers
    400 InstanceTypeNotSupported / Non select query not supported, which is what feat: download an MCQA sub query's result over the instance tunnel #8 added and
    what the service says.
  3. POST /projects/{p}/instances — the driver reruns the same statement offline. That is the
    documented reaction, not a workaround: SQLExecutorImpl#getSessionResultSetByInstanceTunnel
    catches TunnelRetryStatus.NON_SELECT_QUERY and calls
    runQueryInternal(ExecuteMode.OFFLINE, retryInfo.errMsg, true).

Step 2 is only a refusal of the download. By then the side effect has already happened in
step 1, so the rerun in step 3 lands on top of it. A session that executes a non-select and
then says it cannot hand back its result is the wrong order: the client can only ever be told
once, and it will act on that by running the statement itself.

The change

A SQLRT session declines a non-select before starting it, answering the way the client
already expects to be answered:

{"queryId": -1, "status": "ok", "result": "ODPS-1850001 Non select query not supported."}

queryId == -1 with status ok is the branch SQLExecutorImpl#runInSessionWithRetry reads as
"the session would not run this", and ODPS-185 inside the message is the flag
FallbackPolicy#shouldFallback matches when fallbackForUnsupportedFeature is on — which is
what turns the refusal into a single offline run instead of an error. Drop the code and keep
the sentence and clients with fallback disabled get a hard failure they cannot distinguish.

odps.sql.session.select.only lifts it, in the session settings or per statement — the same
key the client sends in the sub-query settings block. Classification looks at every
statement of a submission, comments and quotes stripped, so select 1; drop table t is not a
select.

Evidence

Layer Check Result
Go go test -race -count=1 ./internal/..., go vet, gofmt -l green / clean
Go (new) mcqa_select_only_test.go refusal does not create the table; declines without consuming a sub-query id; select.only=false at either level runs it; statement classification table incl. select ';' as x and multi-statement
Consumer (unchanged behaviour) tests/java against this binary McqaSessionTest 6/6, EmulatorTest 26/26, UdfMetadataTest 4/4
Driver, before mcqa-tunnel-effect on #6+#8 count after one insert statement=6
Driver, after same probe on #6+#8+this+#9 count after one insert statement=3, still interactiveMode=INTERACTIVE, t1=BIGINT via the tunnel read
Driver, boundary same probe with disableFallback=true the refusal surfaces as ODPS-1850001 Non select query not supported. — a client that turned fallback off is told, not silently rerouted

Two tests in #8 (TestMCQADirectDownloadRefusesNonSelect,
TestMCQADirectDownloadLimitedFlagCapsRows) now create their session with
odps.sql.session.select.only=false: they need DDL/DML inside a session to reach the tunnel
layer they pin, which is exactly what the opt-in is for. Their assertions are unchanged.

What this does not claim

  • The queryId: -1 + ODPS-185 pair is derived from the client's handling of it (public
    odps-sdk-core), not from a run against a real MCQA session — I have no interactive quota
    here. The driver's double execution is measured; the shape of the answer is the one the
    driver already branches on.
  • show/desc-style statements count as "not a select" under this classification and are
    declined. If any of them should stay in-session, that is a follow-up on the keyword rule.
  • Session-scoped state isolation is untouched (still not simulated).

@TonyStack2026
TonyStack2026 force-pushed the tony/em-session-select-only branch from 0651d78 to fa00db0 Compare September 22, 2026 03:22
tonystack and others added 2 commits September 22, 2026 20:49
M-B's B3: `SQLExecutor(INTERACTIVE)` reads a sub query through the instance
tunnel unless the client opts out, and that is the path JDBC's MaxQA mode
takes. Without it an interactive consumer silently re-runs its statement as an
offline instance, so the session plane could not be accepted end to end —
the previous increment had to set `useInstanceTunnel(false)` in its own
acceptance test to avoid being fooled by that fallback.

- `GET .../instances/{id}?data&cached&taskname=..[&queryid=N]` answers one sub
  query as a Protobuf record stream. There is no download session behind it, so
  the stream carries its own schema frame (field 1 JSON, then SCHEMA_END_TAG
  33553920 with the CRC32C of the field number and payload); the records after
  it are byte-identical to the ones the offline download path writes.
- `odps-tunnel-record-count` reports the rows still available from the read's
  start offset, not the size of the result: `SessionRecordSetIterator` keeps the
  first answer's count and pages `openRecordReader(offset+cursor, fetchSize)`
  while the cursor is below it, so reporting the total sends a client reading
  past the end.
- `rowrange` windows, `sizelimit` shortens a response by halving (at least one
  row, so a reader always advances) and `instance_tunnel_limit_enabled` applies
  READ_TABLE_MAX_ROW to that read only.
- A sub query that is still running makes the read wait — long polling is what
  `cached` asks for — and a wait past the bound answers with the SDK's own
  retryable tunnel timeout instead of an empty stream. Failed, cancelled,
  unknown and dropped sub queries, and one-shot instances, each get their own
  structured JSON error; a statement with no result set is refused with
  `InstanceTypeNotSupported`, which is how the service sends the client to the
  session API instead of an empty download.
- Sub queries now retain their typed result next to the CSV text, inside the
  same retention window and the same instance byte budget, so both read paths of
  one statement serve the same rows and the same types.

Evidence: 10 new Go cases (framing and CRC checked by decoding the response with
our own record decoder, paging, size limit, the 10k cap, non-select, structured
failures, long poll, idle-TTL activity, retention) — `go test -race ./internal/...`
green. Against a locally built binary `McqaSessionTest` is 6/6, including two new
SDK cases: the default fetch, and an offset read that pages the same result; both
fail on the previous build with `NoSuchObject: unsupported endpoint`, so they are
not passing by falling back. `EmulatorTest` 26/26 and `UdfMetadataTest` 4/4 stay
green, and the PyODPS contract probe stays at 30/0 on a fresh process.
Driving the published JDBC driver with interactiveMode=mcqa and the instance
tunnel applied every write statement twice: the session executed the sub query,
the tunnel read then answered InstanceTypeNotSupported, and the client's own
reaction to that answer is to run the same statement offline again
(SQLExecutorImpl NON_SELECT_QUERY -> runQueryInternal(OFFLINE, ..., true)).
An insert of three rows landed six.

The restriction is a session property, not an emulator opinion, and the client
can lift it: odps.sql.session.select.only, in the session settings or per
statement. The refusal is the pair the client already acts on - queryId -1,
status ok, and a message carrying ODPS-185 plus the service's own sentence - so
a declined submission becomes one offline run instead of an error.

Multi-statement submissions are classified by every segment, because
'select 1; drop table t' must not pass for a select.
@TonyStack2026
TonyStack2026 force-pushed the tony/em-session-select-only branch from fa00db0 to ca2e4dd Compare September 22, 2026 12:50
@dingxin-tech

Copy link
Copy Markdown
Owner

resolve conflict pls

@TonyStack2026

Copy link
Copy Markdown
Contributor Author

Closing rather than rebasing: #11 (merged as 700181e) is the corrected replacement — it carries this branch's still-unmerged select-only implementation and additionally classifies writes hidden behind a WITH prefix. git diff origin/main tony/em-session-select-only now only removes #11's stronger mcqa_select_only.go and its tests, so a rebase here would regress main.

Post-merge exit regression against 700181e with the published driver (odps-jdbc:3.10.13): a WITH x AS (SELECT 42 AS id) INSERT … executed in one session run (count=4, the 42 row present exactly once, session still interactive), instance-tunnel reads stayed typed (t1=BIGINT), and tests/java (45/45, includes McqaSessionTest 6/6) and tests/jdbc (3/3) are green against the same Dockerfile-built image (mvn -B -f tests/java/pom.xml -Demulator.image=maxcompute-emulator:ci test against main). Nothing here is lost that #11 did not already cover.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants