Skip to content

docs: name the two divergences measured against the live service - #14

Merged
dingxin-tech merged 8 commits into
dingxin-tech:mainfrom
TonyStack2026:tony/2026-10-08-emulator-known-divergence-docs
Oct 13, 2026
Merged

dingxin-tech merged 8 commits into
dingxin-tech:mainfrom
TonyStack2026:tony/2026-10-08-emulator-known-divergence-docs

Conversation

@TonyStack2026

@TonyStack2026 TonyStack2026 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Two differences between this emulator and a live MaxCompute project were measured on the wire earlier
and never written down — docs/protocol.md in fact still said the service's HTTP status had not been
read back
. This documents them and corrects that sentence. No behavior changes.

What the docs now say

README gains Known divergences from the real service (plus its Chinese mirror), one section per gap,
each with the scenario, both readings, who observes it and what to assert instead:

  • A refused chunked merge comes back with different codes. Service: 500 InternalServerError with
    ODPS-0421213 … The merged file's signature does not match! when the assembled payload does not match
    the MD5 in the merge body, and 409 ObjectAlreadyExists with ODPS-0421121 when the target name
    already exists. Emulator: 400 InvalidParameter and 400 ResourceAlreadyExists. What agrees is worth
    depending on and is stated as such — which parts a refusal consumes or keeps, and 404 NoSuchObject
    for a manifest naming a part that was never uploaded.
  • A TABLE resource may point at a table that does not exist. Service: 404 NoSuchObject with
    ODPS-0422111: Table not found, nothing created. Emulator: 201 Created, the resource is listed and
    ?meta echoes x-odps-copy-table-source. A local run cannot catch the typo'd or not-yet-created name;
    the row says to create the table in the seed or assert it in fixture setup.

The section then lists what the readings do not cover, rather than letting them read as universal: one
project, one endpoint, plain http:// (no TLS termination or gateway in the path), Linux/amd64, and only
these refusal shapes — so another region, another service version, a public HTTPS path, a BSD/macOS native
build, and any third divergence are all explicitly unmeasured.

docs/protocol.md: the (HTTP 状态未回读) note is replaced by the measured codes plus the round and
conditions they came from, and the pointer to the new section. That row also attributed the emulator's
duplicate-target merge answer to 400 InvalidParameter; the emulator answers 400 ResourceAlreadyExists (internal/server/resources.go, asserted by resources_test.go), so the row now
names it correctly. The same stale sentence in a tests/python/run.py comment is fixed — comment only.

Verification

The commands the new text cites were run against this tree, not quoted from memory:

check result
go test ./internal/server -run TestResourceRESTContract -count=1 ok (0.118s)
python tests/python/run.py http://127.0.0.1:<port> test_project (PyODPS 0.13.2) 33 passed, 0 failed
POST /projects/test_project/resources with x-odps-resource-type: TABLE and x-odps-copy-table-source naming a table that was never created 201, ?meta → 200 with the reference echoed, payload download → 400 UnsupportedOperation, delete then ?meta → 404; nothing left behind
git diff --check clean; no .go file, no server behavior, no test assertion touched

The third row is why the doc can print 201 Created: the earlier measurement recorded that merge/create
call as "accepted" through a client that does not surface the status, so the number is now a reading.

Not in this PR, on purpose

No error-code or status-code change, no feature flag, no refactor, no new test case. Aligning either row
with the service is a contract decision — both codes are asserted by tests in this repository today — and
until it is taken, the section is the tracking list. Two honesty notes inside it: row 2 has no case in
this repo (the Go test registers a TABLE resource against a table that does exist), and the
declared-byte-count reading in the same docs/protocol.md cell ("4400 申报 / 304 实际") predates this
change and was not re-measured here, so this PR does not vouch for it beyond pointing at it.

Review request: my account cannot attach reviewers or assignees on this repo (pull: true, push: false, triage: false — re-checked at the time of opening), so dingxin-tech is not set here; the link is
sent directly instead.


Updates on this PR (kept as appends; the text above is unchanged)

f11ce23 - version scope. The Quick start pins the 1.1.0 image, and that image has no resource plane:
the PyODPS probe against a 1.1.0 build (2026-09-21) ended at 1 passed, 29 failed, those calls answered
as not found. So the two rows now say they describe main, and anyone needing the resource plane is pointed
at a source build or a post-1.1.0 tag. Docs only.

47d1220 - one claim had no archived log behind it. The ?rOpMerge row said the service "only validates
an upper limit" and accepted a 304-byte merge declared as 4400. Re-measured on 2026-10-08 against a live
project (single part, correct manifest MD5, header declaring 4400): accepted, and the published resource
is exactly those 304 bytes - so the acceptance half stands with a fresh reading, the upper-limit half does
not, and the row now stops where the readings do. The README's "what these readings do not cover" bullet also
names the third, opposite-direction difference instead of implying two rows is the whole list: the service
ignores a wrong declared total, this emulator refuses it (documented in docs/protocol.md, because it is the
emulator being stricter rather than missing a capability).

Verification for 47d1220: citation check 12 claims over 3 docs and 11 logs, 0 failures, with eight
counter-example directions (including two new ones aimed at the logs added in this commit) all rejected;
link/anchor check 0 broken; the reading is reproducible from the public request shape - post one part,
then send ?rOpMerge with the correct manifest MD5 and a declared total that does not match the bytes. go test ./internal/server -run TestResourceRESTContract (ok) and the PyODPS contract probe
(33 passed / 0 failed) were run on f6d9e78 - this commit changes documentation text only, and neither can
be moved by it, so they are not claimed here for 47d1220.

25f734d - the version-scope claim, checked against the artifact. The sentence above about 1.1.0 was
backed by a probe run against a 1.1.0 source build. What a reader actually pulls is the image, so it was
re-verified against maxcompute/maxcompute-emulator:1.1.0 (digest sha256:57d1d257…dee2d3) today:
GET /projects/<project>/resources → 404 UnsupportedOperation: unsupported endpoint, /capabilities
returns 11 keys and none of them is resources/functions/unsupported, and the current 33-case PyODPS
probe ends at 1 passed, 32 failed - the one case that passes there passes because the whole endpoint is
refused. The source-build run from 2026-09-21 stays as corroboration. README text only, both languages;
nothing else moves.

2b73069 - three more differences found while re-verifying row 1. Re-running the duplicate-target merge
today against a live project and against main confirmed the headline (409 ObjectAlreadyExists +
ODPS-0421121, so row 1 stands - not overturned) but exposed three further differences in the same
request, now documented inside row 1 instead of as new rows:

same ?rOpMerge service emulator
no x-odps-resource-merge-total-bytes 400 InvalidParameter + ODPS-0420051: Missing header... - header required header optional; answers for the target (400 ResourceAlreadyExists)
wrong declared total + target exists 409 - target check first, declared total never compared 400 on the declared total - emulator's own check first
the part a pre-read refusal keeps addressable by name, absent from every listing (exact-name prefix query included) addressable by name and listed

Two consequences stated in the README: the advice to "assert the part state" is now explicitly by name
(Resource.exists() / ?meta) because a list_resources(prefix=...)-based cleanup assertion reads clean in
the cloud while the object still exists; and the header-mandatory asymmetry means a client that omits it
passes locally and fails against the service - whether the emulator should require it too is a contract
decision, left open here (this PR changes documentation only).

Disclosure on how the readings came about: my first probe today omitted that required header, hit 400, and
briefly looked like it contradicted the documented 409. Running the same cell with the header correct, absent,
and wrong is what separated "the service changed" from "my request was different". Verification for this
commit: the citation check covers 16 claims over 3 docs and 16 logs with 0 failures, and 11 counter-example
directions all reject (two new ones aimed at these logs - a service-only 409 claimed as an emulator reading,
and a listing reading the emulator log never printed). Link/anchor check 0 broken. Docs only, so the Go test
and the PyODPS probe are not claimed as re-run on this head; CI runs the full suite on it.

3d6ca56 - second verification round, recorded. Both rows were re-measured today and neither was
overturned: row 1's 409 ObjectAlreadyExists + ODPS-0421121 reproduced, row 2's 404 NoSuchObject +
ODPS-0422111 reproduced for a table exist_table confirms missing. The provenance paragraph now names both
rounds with dates rather than one range, and states the trap that produced a false alarm this morning: the
first attempt at row 1 answered 400 InvalidParameter because the request omitted x-odps-resource-merge-total-bytes,
which the service requires (ODPS-0420051: Missing header in HTTP request) and which is checked before the
target exists - so a 400 there is a different failure, not a changed answer for the duplicate target.
Docs only; the emulator's answers are unchanged. Citation check: 18 claims over 3 docs and 18 logs, 0 failures,
12 counter-example directions all rejected (two new ones aimed at today's row-2 logs).

aad7b0a - self-review pass over the section. Six hand edits to the same section had left it
inconsistent with itself, which only shows when you read the rendered section rather than the diffs: the
Chinese README carried the "how to re-measure against a live project" recipe twice and the "stated as
unverified" lead-in twice; row 1 still said "two refusal shapes were compared" and "only the shape of the
refusal differs" after the section grew to five differences (one of them about listing visibility, not
shape); the closing line still asked for "two runs agreeing", which the second round had already done. All
corrected, plus the reproduction recipe now warns that a hand-built ?rOpMerge without
x-odps-resource-merge-total-bytes is answered 400 (ODPS-0420051) before the target is examined - the trap
this section hit during its own re-measurement today. Wording and provenance only; no claim, reading, status
code or test changed. Citation check unchanged at 18 claims / 18 logs, 0 failures, 12 counter-example
directions rejected; link check 0 broken.

README gains a "Known divergences from the real service" section (and its Chinese
mirror) for the two gaps that were measured on the wire instead of inferred:

  * a refused chunked merge - the service answers 500 InternalServerError with
    ODPS-0421213 for a digest mismatch and 409 ObjectAlreadyExists with
    ODPS-0421121 for a duplicate target, the emulator answers 400 in both;
  * a TABLE resource whose table was never created - the service refuses with
    404 NoSuchObject / ODPS-0422111, the emulator creates it (201, `?meta`
    readable), measured again on this commit's tree.

Each row states who can observe it and what to assert instead, and the section
lists what the readings do NOT cover: one project, `http://` endpoint, no TLS or
gateway path, no BSD/macOS native build, and no other merge refusal shapes.

docs/protocol.md said the service's HTTP status "had not been read back" and
attributed the emulator's duplicate-merge answer to InvalidParameter; both are
corrected with the round, the conditions and the pointer to the new section. The
same stale sentence in tests/python/run.py's comment is fixed. No behavior, no
status code, no assertion changed.
The Quick start pins `maxcompute/maxcompute-emulator:1.1.0`, and that tag has no resource plane at all -
running the PyODPS contract probe against a `1.1.0` build (2026-09-21) ended at `1 passed, 29 failed` with
those calls answered as not found. Read next to the Quick start, the new rows would have looked like
behavior to expect from the image you just pulled, when in fact there is nothing there to diverge from:
part + merge uploads and `TABLE` resources arrive after `1.1.0`.

So both READMEs now say which build the rows describe, keep the probe reading with its date (the case count
grows with the probe; the missing-capability answer is what stayed), and point anyone who needs the resource
plane at a source build or a post-1.1.0 tag. Documentation only - no behavior touched.
@TonyStack2026

Copy link
Copy Markdown
Contributor Author

Adding a second commit for a discoverability problem the first one had, on README only (no behavior touched).

The Quick start pins maxcompute/maxcompute-emulator:1.1.0, and that tag has no resource plane at all: running the PyODPS contract probe against a 1.1.0 build (2026-09-21) ended at 1 passed, 29 failed, those calls answered as not found. Sitting right under the Quick start, the two rows read like behavior to expect from the image a user just pulled - when on that tag there is nothing to diverge from.

f11ce23 says which build the rows describe (main), keeps the probe reading with its date, notes that the case count grows with the probe while the missing-capability answer is what stayed, and points anyone needing the resource plane at a source build or a post-1.1.0 tag.

Re-run against the tree as it now stands (04:4x): citation check - 10 claims over 3 docs and 9 logs, 0 failures, all six counter-example directions still REJECT; link/anchor check - 0 broken. The Go test (TestResourceRESTContract ok) and the PyODPS probe (33 passed / 0 failed) were run on f6d9e78; this commit changes README text only, so it cannot move those two, and I am not claiming them for f11ce23.

TonyStack2026 and others added 6 commits October 8, 2026 05:37
…to it

The `?rOpMerge` row in docs/protocol.md asserted that the service never compares
`x-odps-resource-merge-total-bytes` with what it assembled, "declaring 4400 bytes for a 304-byte merge is
accepted, content correct", plus "the service only validates an upper limit". Reproducing that from the
archived evidence: the acceptance reading is real, the upper-limit half is not something anyone here has a
log for. Rather than leave a stronger sentence than the readings, the row now states exactly what was
re-measured on 2026-10-08 (single part of 304 bytes, manifest MD5 correct, header declaring 4400 - accepted,
and the published resource is those 304 bytes) and marks the upper-limit behavior as not re-checked.

Same round, one pointer in README: the table lists two divergences, but there is a third known one that
runs the other way - the emulator refuses the mismatched declared total the service ignores. It belongs in
the protocol notes because it is the emulator being stricter, not a capability it lacks; the README's
"what these readings do not cover" bullet now says so instead of implying two rows is the whole list.

No behavior, status code, flag or test assertion touched: the emulator's refusal and its keep-the-parts
rule on that path are unchanged and still pinned by `go test ./internal/server` and the PyODPS probe.
…ust a source build

The previous commit said `1.1.0` has no resource plane and backed that with the
2026-09-21 probe run against a `1.1.0` **source build**. The sentence a user acts on is about the image they
`docker pull`, so it was verified one level removed from what it claims.

Re-checked against the published artifact today (`maxcompute/maxcompute-emulator:1.1.0`, digest
sha256:57d1d25757255a1f16d0a3c34ef8b99c24bf8615e1c14c13dfd5a41026dee2d3, linux/amd64 container on
127.0.0.1):

  * `GET /projects/<project>/resources` -> 404 with `UnsupportedOperation: unsupported endpoint`
  * `GET /capabilities` -> 11 keys, none of `resources` / `functions` / `unsupported`
  * the current PyODPS contract probe (33 cases) -> `1 passed, 32 failed`, and the one case that passes
    passes because the whole endpoint is refused

The source-build run stays as corroboration (same shape: `1 passed, 29 failed` over 30 cases on 2026-09-21),
with the sentence that the case count grows with the probe while the missing-capability answer does not.
README text only, in both languages; no behavior, endpoint, status code or assertion touched.
Row 1 said the interesting part is that the *shape* of a refused merge differs while the part lifecycle
agrees. Re-running that scenario today against a live project and against `main` turned up three more
differences in the same request, so they are documented in row 1 rather than as new rows:

  * `?rOpMerge` without `x-odps-resource-merge-total-bytes`: the service refuses with 400 InvalidParameter +
    `ODPS-0420051: Missing header in HTTP request` - the header is mandatory. The emulator treats it as
    optional and goes on to answer for the target (`400 ResourceAlreadyExists`). A client that omits it works
    locally and fails on the cloud.
  * declared total wrong **and** target already exists: the service reports 409 `ObjectAlreadyExists` (target
    check first; the declared total is never compared), the emulator reports 400 on the declared total - its
    own check runs first. Same two facts, opposite order.
  * the part a refusal-before-reading leaves behind: on the service it is addressable by name (`?meta` 200,
    delete works) but appears in **no listing**, not even an exact-name prefix query; on the emulator it is
    listed. So "assert the part state" now says *by name* - a sweep based on `list_resources(prefix=...)`
    reads clean in the cloud while the object still exists, and reads leaked locally where nothing leaked.

Row 1's headline claim was re-verified, not overturned: with the header present and correct, the duplicate
target still answers `409 ObjectAlreadyExists` + `ODPS-0421121` (measured 2026-10-08 07:35, previously
2026-10-02 and 2026-10-03).

Docs only - both README languages and the protocol notes. No endpoint, header, status code or assertion in
the server changed. One honest note on how these readings came about: my first probe of the day omitted the
required header, saw a 400 and briefly looked like a contradiction of the documented 409; running the same
cell with and without the header is what separated "the service changed" from "my request was different".
Round 2 happened today: row 1's `409 ObjectAlreadyExists` + `ODPS-0421121` reproduced (with the target-exists
shape run three ways), and row 2's `404 NoSuchObject` + `ODPS-0422111` reproduced for a table that
`exist_table` confirms is missing. Neither row was overturned, so the provenance now names both rounds and
their dates instead of a single date range.

One trap is stated on purpose: the first attempt at row 1 today answered `400 InvalidParameter`, which looks
like the service changed. It was our own request - the service requires `x-odps-resource-merge-total-bytes`
on `?rOpMerge` (missing header -> `ODPS-0420051: Missing header in HTTP request`), and that refusal happens
before the target check. A reader comparing notes should know a 400 there is a different failure, not a new
answer for the duplicate target.

Protocol notes carry the same third-round reading. Documentation only - no endpoint, header, status code,
flag or assertion changed; the emulator's answers are unchanged.
…tion and stale lead-ins)

Six edits by hand had left the section inconsistent with itself, which only shows up when you read the
rendered section end to end rather than the diffs:

  * the Chinese README carried the "how to re-measure against a live project" recipe twice - once as the
    current paragraph and once as the older wrapped one;
  * the "stated as unverified" lead-in appeared twice in Chinese;
  * row 1 still said "two refusal shapes were compared" and "only the *shape* of the refusal differs"
    after the section grew to five differences, one of which is about listing visibility, not shape;
  * the closing sentence asked for "two runs agreeing", which the second verification round had already
    satisfied - it now names the rounds each row has;
  * the last bullet said "unmeasured" twice in one sentence.

Also added where it matters: the reproduction recipe now warns that a hand-built `?rOpMerge` request
without `x-odps-resource-merge-total-bytes` is answered 400 (`ODPS-0420051: Missing header in HTTP request`)
before the target is examined. That is the trap this section hit on its own re-measurement today, so a
reader following the recipe should not mistake it for the duplicate-target answer.

Wording and provenance only - no measurement, claim, status code or test changed. Citation check still
covers 18 claims over 3 docs and 18 logs with 0 failures and 12 counter-example directions all rejected.
@dingxin-tech
dingxin-tech merged commit 6493c12 into dingxin-tech:main Oct 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants