Repository navigation
Commit 1b01580
Close stdin for the CI steps that run the engine
linux-aarch64-gnu failed on JooqIT.insertAndSelect with "Code: 48. Processing
async inserts with both inlined and external data (from stdin or infile) is
not supported", and passed on the other run of the same commit. Not flaky in
the usual sense -- deterministic given one input nobody was looking at.
chDB embeds clickhouse-local, and ClientBase asks three times whether the
process's own stdin holds external data for an INSERT:
!is_interactive && !stdin_is_a_tty && isStdinNotEmptyAndValid(*std_in)
A GitHub runner's stdin is not a TTY and sometimes has bytes in it. Then an
INSERT with inlined VALUES has, as far as the engine is concerned, two sources
of data, and it refuses. Reproducible on demand:
printf '(9999)\n' | java ... "INSERT INTO t VALUES (1), (2)" -> Code 48
java ... < /dev/null -> ok
So the CI steps that run the engine now close stdin. Four of them: the
integration suite, the packaged-JAR run and the consumer check (both run
QuickStart, which inserts), and UBSan over the whole suite. The AlmaLinux 8
step needs nothing -- `docker run` without -i already has no stdin.
This is CI hygiene, not a fix, and the comment in the workflow says so. The
hazard is real for any application embedding chDB whose stdin is a pipe with
data, and it is not ours to paper over in the driver.
Two things worth recording about why nothing else changed here.
async_insert defaulting to true is not chDB's doing. Upstream
clickhouse-server:26.7.3 reports async_insert=1 changed=0 as well -- the same
default, picked up correctly when chdb-core synced v26.3.9.8. What is
chDB-specific is only that a library embedded in someone else's process
inherits clickhouse-local's view of stdin. chdb-core already has an
is_background flag for the embedded case; these three checks do not consult it.
And passing --async_insert=0 from the driver, which is where I started, is
worse than the problem. It takes the same stdin and feeds it to the INSERT:
stdin "(9999)", async_insert=0: INSERT INTO t VALUES (1), (2) returns ok
and the table holds [1, 2, 9999]
Silent rows the application never wrote, against a loud refusal today. The
Code 48 error is the engine declining to guess between two data sources, and
it is protecting the caller. Measured for completeness before discarding the
idea: 300 sequential inserts 430ms against 423ms, 8 threads x 150 inserts
1123ms against 1047ms -- no throughput argument either way, because
wait_for_async_insert is true and the caller waits regardless.
Also corrected a stale comment in the sanitizer step: chdb_jni_test has not
parsed Arrow formats since that path was deleted.
178 unit + 176 integration green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent fc2fecc commit 1b01580
1 file changed
Lines changed: 19 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
178 | 178 | | |
179 | 179 | | |
180 | 180 | | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
181 | 191 | | |
182 | 192 | | |
183 | 193 | | |
| |||
237 | 247 | | |
238 | 248 | | |
239 | 249 | | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
240 | 253 | | |
241 | 254 | | |
242 | 255 | | |
| |||
259 | 272 | | |
260 | 273 | | |
261 | 274 | | |
262 | | - | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
263 | 278 | | |
264 | 279 | | |
265 | 280 | | |
| |||
417 | 432 | | |
418 | 433 | | |
419 | 434 | | |
420 | | - | |
421 | | - | |
| 435 | + | |
| 436 | + | |
422 | 437 | | |
423 | 438 | | |
424 | 439 | | |
425 | 440 | | |
426 | 441 | | |
427 | | - | |
| 442 | + | |
428 | 443 | | |
429 | 444 | | |
430 | 445 | | |
| |||
0 commit comments