Skip to content

Commit 1b01580

Browse files
Close stdin for the CI steps that run the engine
linux-aarch64-gnu failed on JooqIT.insertAndSelect with "Code: 48. Processing async inserts with both inlined and external data (from stdin or infile) is not supported", and passed on the other run of the same commit. Not flaky in the usual sense -- deterministic given one input nobody was looking at. chDB embeds clickhouse-local, and ClientBase asks three times whether the process's own stdin holds external data for an INSERT: !is_interactive && !stdin_is_a_tty && isStdinNotEmptyAndValid(*std_in) A GitHub runner's stdin is not a TTY and sometimes has bytes in it. Then an INSERT with inlined VALUES has, as far as the engine is concerned, two sources of data, and it refuses. Reproducible on demand: printf '(9999)\n' | java ... "INSERT INTO t VALUES (1), (2)" -> Code 48 java ... < /dev/null -> ok So the CI steps that run the engine now close stdin. Four of them: the integration suite, the packaged-JAR run and the consumer check (both run QuickStart, which inserts), and UBSan over the whole suite. The AlmaLinux 8 step needs nothing -- `docker run` without -i already has no stdin. This is CI hygiene, not a fix, and the comment in the workflow says so. The hazard is real for any application embedding chDB whose stdin is a pipe with data, and it is not ours to paper over in the driver. Two things worth recording about why nothing else changed here. async_insert defaulting to true is not chDB's doing. Upstream clickhouse-server:26.7.3 reports async_insert=1 changed=0 as well -- the same default, picked up correctly when chdb-core synced v26.3.9.8. What is chDB-specific is only that a library embedded in someone else's process inherits clickhouse-local's view of stdin. chdb-core already has an is_background flag for the embedded case; these three checks do not consult it. And passing --async_insert=0 from the driver, which is where I started, is worse than the problem. It takes the same stdin and feeds it to the INSERT: stdin "(9999)", async_insert=0: INSERT INTO t VALUES (1), (2) returns ok and the table holds [1, 2, 9999] Silent rows the application never wrote, against a loud refusal today. The Code 48 error is the engine declining to guess between two data sources, and it is protecting the caller. Measured for completeness before discarding the idea: 300 sequential inserts 430ms against 423ms, 8 threads x 150 inserts 1123ms against 1047ms -- no throughput argument either way, because wait_for_async_insert is true and the caller waits regardless. Also corrected a stale comment in the sanitizer step: chdb_jni_test has not parsed Arrow formats since that path was deleted. 178 unit + 176 integration green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent fc2fecc commit 1b01580

1 file changed

Lines changed: 19 additions & 4 deletions

File tree

‎.github/workflows/build.yml‎

Lines changed: 19 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,16 @@ jobs:
178178
- name: Integration tests on Java 11, 17, 21 and 25
179179
run: |
180180
set -eu
181+
# stdin closed for everything below. chDB embeds clickhouse-local, whose
182+
# ClientBase treats the process's own stdin as external data for an INSERT when it
183+
# is not a TTY and has bytes available (src/Client/ClientBase.cpp). A runner's stdin
184+
# sometimes does, and then "INSERT ... VALUES" fails with "Processing async inserts
185+
# with both inlined and external data". Reproducible on demand:
186+
# printf '(9999)\n' | java ... "INSERT INTO t VALUES (1)"
187+
# This closes it for our own test processes. It is not a fix -- the hazard is real
188+
# for any application embedding chDB whose stdin is a pipe with data.
189+
exec </dev/null
190+
181191
case "${{ matrix.platform }}" in
182192
macos-*) OS=macos ;;
183193
*) OS=linux ;;
@@ -237,6 +247,9 @@ jobs:
237247
# The production path: unpack into a content-addressed cache, verify the checksums,
238248
# load. Exercised separately because the integration tests use chdb.library.path.
239249
run: |
250+
# See the note on the integration-test step: chDB reads the process's stdin as
251+
# external data for an INSERT, and QuickStart inserts.
252+
exec </dev/null
240253
mvn $MAVEN_ARGS -pl chdb-examples -am compile
241254
# Globbed rather than spelled out: the driver JAR carries the project version, which
242255
# moves with the engine baseline, and a hardcoded name here goes stale silently.
@@ -259,7 +272,9 @@ jobs:
259272
# outside this checkout with its own empty local repository. Run here, on every platform,
260273
# so that the answer does not go stale between releases; the one thing it cannot cover is
261274
# a real network transfer, which only Central can.
262-
run: scripts/verify-consumer.sh ${{ matrix.platform }}
275+
# stdin closed: the consumer project runs QuickStart, which inserts. See the note on
276+
# the integration-test step.
277+
run: scripts/verify-consumer.sh ${{ matrix.platform }} </dev/null
263278

264279
- name: Archive debug symbols
265280
# Kept out of the runtime JAR (they are several times its size) but needed to read a
@@ -417,14 +432,14 @@ jobs:
417432
run: scripts/fetch-libchdb.sh ${{ matrix.platform }} target/engine/${{ matrix.platform }}
418433

419434
- name: ASan and UBSan over the shim's own logic
420-
# chdb_jni_test: Arrow format parsing, the handle registry and the signal guard, with
421-
# no JVM and no engine in the process, so ASan's allocator interception is safe.
435+
# chdb_jni_test: the handle registry and the signal guard, with no JVM and no engine
436+
# in the process, so ASan's allocator interception is safe.
422437
run: scripts/run-sanitizer-tests.sh ${{ matrix.platform }} address,undefined
423438

424439
- name: UBSan over the whole JDBC suite
425440
# In a real JVM against the real engine. UBSan instruments arithmetic rather than
426441
# intercepting allocation, so unlike ASan it coexists with both.
427-
run: scripts/run-sanitizer-tests.sh ${{ matrix.platform }} undefined
442+
run: scripts/run-sanitizer-tests.sh ${{ matrix.platform }} undefined </dev/null
428443

429444
- name: Archive sanitizer output
430445
if: failure()

0 commit comments

Comments
 (0)