feat: support sparse matrix write (NULL_RATIO) for iotdb-1.3 and iotd… by changxue2022 · Pull Request #561 · thulab/iot-benchmark

changxue2022 · 2026-09-02T03:30:11Z

…b-2.0

Adds NULL_RATIO config (default 0.0): each cell of every generated row is set to null with probability NULL_RATIO, independently per column, seeded by DATA_SEED for reproducibility.

core: NULL_RATIO config plumbing, null generation in GenerateDataWorkLoad, and startup guards in ConfigDescriptor.checkConfig (verification modes, non-iotdb-1.3/2.0 switches, and REST / SESSION_BY_RECORD(S) insert modes are rejected with a clear error)
iotdb-2.0: SessionStrategy.genTablet marks null cells via the measurement name addValue overload (tree and table model); JDBCStrategy emits SQL null literals
iotdb-1.3: IoTDBSessionBase.genTablet marks null cells in tablet BitMaps (BINARY/DATE slots get a non-null placeholder because the 1.3 session client reads those unconditionally); IoTDB.getInsertOneBatchSql emits SQL null literals
tests: null generation distribution/determinism, config guards, null serialize round-trip

…b-2.0 Adds NULL_RATIO config (default 0.0): each cell of every generated row is set to null with probability NULL_RATIO, independently per column, seeded by DATA_SEED for reproducibility. - core: NULL_RATIO config plumbing, null generation in GenerateDataWorkLoad, and startup guards in ConfigDescriptor.checkConfig (verification modes, non-iotdb-1.3/2.0 switches, and REST / SESSION_BY_RECORD(S) insert modes are rejected with a clear error) - iotdb-2.0: SessionStrategy.genTablet marks null cells via the measurement name addValue overload (tree and table model); JDBCStrategy emits SQL null literals - iotdb-1.3: IoTDBSessionBase.genTablet marks null cells in tablet BitMaps (BINARY/DATE slots get a non-null placeholder because the 1.3 session client reads those unconditionally); IoTDB.getInsertOneBatchSql emits SQL null literals - tests: null generation distribution/determinism, config guards, null serialize round-trip

The null-cell handling in SessionStrategy.genTablet already works for the table model: Tablet.addValue(measurementName, row, null) marks the cell in the per-column BitMap for column-category tablets, and the resolved client (iotdb-session 2.0.11-260701-SNAPSHOT, tsfile 2.3.2-260616-SNAPSHOT) plus the IoTDB server decode path both handle null cells via those BitMaps for tree and table model alike. Enable it by correcting the tree-only docs. - make SessionStrategy.genTablet static and take IoTDB so the null bitmap handling can be unit-tested without a live server connection - add SessionStrategyNullTabletTest asserting null cells are marked in the tablet BitMaps for both tree and table model - update config.properties / Config javadoc / JDBCStrategy comments

SpriCoder

Reviewed commit a36987b. The inline comments describe three locally reproduced functional issues and one test-coverage suggestion, with concrete examples and proposed fixes.

Validation: all 164 core tests passed on JDK 17. The IoTDB 1.3 module compiled successfully and has no unit tests. IoTDB 2.0 adapter testing could not be completed because dependency downloads stalled. No live database workload was run.

SpriCoder · 2026-09-08T10:08:35Z

+    if (config.getNULL_RATIO() > 0) {
+      for (int i = 0; i < values.size(); i++) {
+        if (probTool.returnTrueByProb(config.getNULL_RATIO(), nullRandom)) {
+          values.set(i, null);


[P1] Null cells are still counted as successfully written points

This change replaces values with null, but Batch.pointNum() and MultiDeviceBatch.pointNum() still calculate “number of FIELD columns × number of rows.” DBWrapper.measureOneBatch() uses that number for successful point counts and points/s.

For example, a batch with 10 rows and 200 columns contains no actual values when NULL_RATIO=1. In a local reproduction, the non-null count was 0, but pointNum() returned 2,000. If the write request succeeds, all 2,000 empty positions are therefore counted as successful points. With NULL_RATIO=0.9, the reported throughput would typically be about 10 times the non-null point throughput.

Please update both batch implementations to count non-null FIELD values and add tests for fully null and partially null batches. If throughput over all candidate cells is useful, report it as a separate, clearly named metric. This reproduction checks local counting; it does not involve a live database.

SpriCoder · 2026-09-08T10:08:36Z

+  // Random for sparse matrix write (NULL_RATIO), seeded by DATA_SEED so that the null pattern is
+  // deterministic and reproducible. All data clients share the same seed and thus generate the
+  // same null pattern.
+  private final Random nullRandom = new Random(config.getDATA_SEED());


[P2] The same seed does not reproduce null positions with concurrent writers

When IS_CLIENT_BIND=false, writer threads share one SingletonWorkDataWorkLoad, including this nullRandom. The seed fixes the random sequence, but thread scheduling determines which cell receives each random value. The same device, timestamp, and column can therefore be null in one run and populated in another.

I reproduced this by creating a fresh workload for each of two runs with the same configuration and seed, using eight threads. Both runs produced the same 8,000 row identifiers, but 7,949 rows had different null positions. This breaks the reproducibility promised in the PR and can affect comparisons between databases or configurations. The existing determinism test only covers sequential calls.

Please derive the null decision from DATA_SEED and the cell coordinates (device, row, and column), rather than the order in which threads consume random numbers. Add a concurrent test that compares null positions for matching row identifiers.

SpriCoder · 2026-09-08T10:08:36Z

+    if (workMode == BenchmarkMode.VERIFICATION_WRITE
+        || workMode == BenchmarkMode.VERIFICATION_QUERY) {
+      LOGGER.error(
+          "NULL_RATIO is not supported in {} mode. Please use testWithDefaultPath or generateDataMode.",


[P2] Sparse CSV files can be generated but cannot be read back correctly

This message recommends generateDataMode, but the CSV writer and reader do not yet agree on how to represent nulls. CSVDataWriter writes the text null, while CSVDataReader parses it directly as the column type: numeric/date values fail to parse, boolean nulls become false, and text nulls become the string "null".

In a local comparison, a dense batch wrote 10 rows and read back 10 rows. With NULL_RATIO=1, it wrote 10 rows but read back 0. Setting NULL_RATIO back to 0 before reading passes the configuration guard, but does not fix the nulls already stored in the file.

Please add matching CSV null encoding/decoding and round-trip tests across supported types. If sparse CSV support is outside this PR's scope, reject generateDataMode with a nonzero NULL_RATIO for now and update this message.

SpriCoder · 2026-09-08T10:08:36Z

+    config.setDEVICE_NUMBER(6000);
+    config.setSCHEMA_CLIENT_NUMBER(1);
+    config.setDATA_CLIENT_NUMBER(1);
+    config.setNULL_RATIO(0.9);


[P3] Add a test that loads NULL_RATIO from the configuration file

The new tests call setNULL_RATIO(0.9) directly, bypassing the configuration-file path that users actually use. They could therefore keep passing if the property-loading code were later removed or broken.

Please add a test that writes NULL_RATIO=0.9 to a temporary configuration file, loads it through the actual configuration entry point, and verifies that the effective value is 0.9. The loading code is currently wired up; this is a coverage suggestion, not a claim that the setting currently fails to load.

Copilot

🟡 Changes recommended

There are correctness and stability issues in NULL_RATIO validation and null generation that can lead to invalid configs being accepted and non-thread-safe behavior under concurrent workloads.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a “sparse matrix write” mode to IoT Benchmark by introducing a NULL_RATIO configuration that probabilistically turns generated cells into null (seeded by DATA_SEED for reproducibility), and plumbs null handling through supported IoTDB write paths.

Changes:

Introduces NULL_RATIO into core config, config loading, startup validation, and synthetic row generation.
Adds null-aware write support for IoTDB-2.0 (Session/Tablet + JDBC) and IoTDB-1.3 (Session/Tablet + JDBC SQL generation).
Adds tests covering null generation behavior, config guardrails, tablet null bitmap marking, and batch serialize/deserialize round-trips.

File summaries

File	Description
iotdb-2.0/src/test/java/cn/edu/tsinghua/iot/benchmark/iotdb200/SessionStrategyNullTabletTest.java	Adds unit tests asserting tablet BitMap null marking for tree/table dialects.
iotdb-2.0/src/main/java/cn/edu/tsinghua/iot/benchmark/iotdb200/DMLStrategy/SessionStrategy.java	Updates tablet generation to handle null cells via tablet APIs.
iotdb-2.0/src/main/java/cn/edu/tsinghua/iot/benchmark/iotdb200/DMLStrategy/JDBCStrategy.java	Emits SQL `null` literals for null values during insert SQL generation.
iotdb-1.3/src/main/java/cn/edu/tsinghua/iot/benchmark/iotdb130/IoTDBSessionBase.java	Initializes/marks tablet BitMaps and handles null placeholders for specific types.
iotdb-1.3/src/main/java/cn/edu/tsinghua/iot/benchmark/iotdb130/IoTDB.java	Emits SQL `null` literals for null values during insert SQL generation.
core/src/test/java/cn/edu/tsinghua/iot/benchmark/workload/GenerateDataWorkLoadNullTest.java	Adds tests for NULL_RATIO distribution, determinism, and buffer non-mutation.
core/src/test/java/cn/edu/tsinghua/iot/benchmark/serialize/BatchSerializeTest.java	Adds a serialization round-trip test ensuring null values persist.
core/src/test/java/cn/edu/tsinghua/iot/benchmark/conf/ConfigDescriptorTest.java	Adds validation tests for NULL_RATIO guardrails across modes/DB switches.
core/src/main/java/cn/edu/tsinghua/iot/benchmark/workload/GenerateDataWorkLoad.java	Implements per-cell nulling during synthetic row generation.
core/src/main/java/cn/edu/tsinghua/iot/benchmark/conf/ConfigDescriptor.java	Loads NULL_RATIO and adds config validation for supported combinations.
core/src/main/java/cn/edu/tsinghua/iot/benchmark/conf/Config.java	Adds NULL_RATIO field, accessors, and config reporting output.
configuration/conf/config.properties	Documents and defaults the new NULL_RATIO parameter.

Review details

Suppressed comments (1)

iotdb-2.0/src/test/java/cn/edu/tsinghua/iot/benchmark/iotdb200/SessionStrategyNullTabletTest.java:85

Since the test overrides the global "benchmark-conf" system property, it should be restored in tearDown() so it doesn't affect other test classes.

  public void tearDown() {
    CONFIG.setIoTDB_DIALECT_MODE(originalDialect);
    CONFIG.setDEVICE_NUM_PER_WRITE(originalDeviceNumPerWrite);
    CONFIG.setIS_DOUBLE_WRITE(originalDoubleWrite);
  }

Files reviewed: 12/12 changed files
Comments generated: 5
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    double nullRatio = config.getNULL_RATIO();
+    if (nullRatio < 0 || nullRatio > 1) {
+      LOGGER.error(
+          "Invalid parameter NULL_RATIO: {}, whose value range should be [0, 1]", nullRatio);
+      result = false;
+    }


+  // Random for sparse matrix write (NULL_RATIO), seeded by DATA_SEED so that the null pattern is
+  // deterministic and reproducible. All data clients share the same seed and thus generate the
+  // same null pattern.
+  private final Random nullRandom = new Random(config.getDATA_SEED());


+          tablet.bitMaps[sensorIndex].mark(recordIndex);
+          sensorIndex++;
+          continue;


+public class SessionStrategyNullTabletTest {
+  static {
+    System.setProperty(
+        "benchmark-conf",
+        Paths.get("..", "configuration", "conf").toAbsolutePath().normalize().toString());
+  }
+
+  private static final Config CONFIG = ConfigDescriptor.getInstance().getConfig();


+    if (config.getNULL_RATIO() > 0) {
+      for (int i = 0; i < values.size(); i++) {
+        if (probTool.returnTrueByProb(config.getNULL_RATIO(), nullRandom)) {
+          values.set(i, null);
+        }


jt2594838 · 2026-09-08T10:37:44Z

+          Object value = record.getRecordDataValue().get(recordValueIndex);
+          // Sparse matrix write (NULL_RATIO): the measurement-name addValue overload handles
+          // null natively (marks the cell in the tablet BitMaps and stores a typed null
+          // sentinel). A null reaching the typed switch below would NPE, so skip it.
+          if (value == null) {
+            tablet.addValue(sensors.get(sensorIndex).getName(), recordIndex, (Object) null);
+            sensorIndex++;
+            continue;
+          }


Not-set values are naturally nulls in a Tablet.
May skip tablet.addValue.

changxue2022 added 2 commits September 2, 2026 11:20

SpriCoder requested a lite review from Copilot September 8, 2026 10:07

Copilot started reviewing on behalf of SpriCoder September 8, 2026 10:07 View session

SpriCoder reviewed Sep 8, 2026

View reviewed changes

Copilot AI reviewed Sep 8, 2026

View reviewed changes

jt2594838 reviewed Sep 8, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat: support sparse matrix write (NULL_RATIO) for iotdb-1.3 and iotd… - #561

feat: support sparse matrix write (NULL_RATIO) for iotdb-1.3 and iotd…#561
changxue2022 wants to merge 2 commits into
thulab:masterfrom
changxue2022:master

changxue2022 commented Sep 2, 2026

Uh oh!

SpriCoder left a comment •

edited

Loading

Uh oh!

SpriCoder Sep 8, 2026 •

edited

Loading

Uh oh!

SpriCoder Sep 8, 2026 •

edited

Loading

Uh oh!

SpriCoder Sep 8, 2026 •

edited

Loading

Uh oh!

SpriCoder Sep 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

jt2594838 Sep 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

changxue2022 commented Sep 2, 2026

Uh oh!

SpriCoder left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SpriCoder Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SpriCoder Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SpriCoder Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SpriCoder Sep 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

jt2594838 Sep 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

SpriCoder left a comment •

edited

Loading

SpriCoder Sep 8, 2026 •

edited

Loading

SpriCoder Sep 8, 2026 •

edited

Loading

SpriCoder Sep 8, 2026 •

edited

Loading

SpriCoder Sep 8, 2026 •

edited

Loading