Skip to content

Bulk Improvements - #312

Closed
aersam wants to merge 7 commits into
tiberius-rs:mainfrom
aersam:bulk
Closed

aersam wants to merge 7 commits into
tiberius-rs:mainfrom
aersam:bulk

Conversation

@aersam

@aersam aersam commented Sep 21, 2023

Copy link
Copy Markdown
Contributor

No description provided.

@aersam
aersam marked this pull request as ready for review November 21, 2023 08:00
@aersam aersam mentioned this pull request Nov 21, 2023
Comment thread src/client.rs
if options.bits() > 0 || order_hints.len() > 0 {
let mut add_separator = false;
query.push_str(" WITH (");
if options.contains(SqlBulkCopyOptions::KeepNulls) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't you use bitflags's Flag::iter and join to simplify the whole separator business?

@ibotty

ibotty commented Jan 24, 2024

Copy link
Copy Markdown

It would be great to have this pull request merged.

@fnicastri

Copy link
Copy Markdown

@aersam @janpio any progress on this?

joelparkerhenderson added a commit to mssql-rust/mssql-rust that referenced this pull request Aug 29, 2026
Mirrors tiberius-rs/tiberius#312, reimplemented against this fork's already-
merged column_metadata()/bulk_insert_columns (#398/#359) rather than
patched, and fixing several issues along the way:

- New `Client::bulk_insert_with_options(table, columns, options,
  order_hints)`, mirroring .NET's SqlBulkCopy: `SqlBulkCopyOption` flags
  (CheckConstraints, TableLock, KeepNulls, FireTriggers, KeepIdentity)
  and an optional per-column `ORDER` hint. `bulk_insert` and
  `bulk_insert_columns` are now thin wrappers around it with no options.
  Uses `enumflags2` (already a dependency, used for `ColumnFlag`/`RpcParam`
  flags elsewhere) instead of adding a second bitflags crate as upstream's
  PR did.
- Fixed `KeepIdentity` to actually do something: upstream's PR defined
  the flag but never referenced it in the generated SQL, so it was a
  no-op. The TDS bulk-load protocol has no WITH-clause keyword for
  preserving identity values anyway (unlike CHECK_CONSTRAINTS/
  FIRE_TRIGGERS/KEEP_NULLS/TABLOCK, which are real INSERT BULK options) -
  SQL Server's own BULK INSERT/SqlBulkCopy preserve identity values by
  including the identity column in the column list instead. So
  `KeepIdentity` now overrides the existing Updateable-based column
  filter for identity columns specifically.
- Fixed 4 clippy-style issues present in the original PR: `len() > 0` ->
  `!is_empty()`, `push_str(")")` -> `push(')')`, a needless `return`, and
  uninlined format args in the improved "unrepresentable character"
  error message (now includes the offending char).
- Did not carry over the PR's unrelated deletion of a decode-completeness
  assertion in a column_data.rs test (`reader.read_u8().await
  .expect_err(...)`) - that assertion was never touched here since this
  was a from-scratch reimplementation rather than a patch apply, so
  there was nothing to restore.
- Live-tested and found (and fixed) a real bug of my own along the way:
  the initial `if !options.is_empty() || !order_hints.is_empty() { ...
  WITH (...) }` condition emitted an empty, syntactically invalid
  `WITH ()` clause when the only option set was `KeepIdentity` (which
  contributes no WITH-clause text) - now the clause is only emitted when
  there's an actual hint to put in it.

Added 12 unit tests (bulk_options.rs has none of its own logic beyond
plain enum/type definitions) plus 3 new live integration tests: KeepIdentity
preserving supplied identity values, confirming identity is still excluded
without it, and TableLock+CheckConstraints+FireTriggers+KeepNulls+ORDER
together.

Verified:
- cargo check: default, --features=rustls, --features=vendored-openssl,
  --no-default-features
- cargo clippy --all-targets -- -D warnings, for default, rustls, and
  vendored-openssl feature sets
- cargo fmt --check
- cargo test --lib (169 passed)
- cargo test --doc against a live server (27 passed, doctest included)
- Live integration tests over rustls: 300 passed (98 bulk + 202 query)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156Di1tRRLsJK8ctU1AmAJr
joelparkerhenderson added a commit to mssql-rust/mssql-rust that referenced this pull request Aug 30, 2026
Mirrors tiberius-rs/tiberius#312, reimplemented against this fork's already-
merged column_metadata()/bulk_insert_columns (#398/#359) rather than
patched, and fixing several issues along the way:

- New `Client::bulk_insert_with_options(table, columns, options,
  order_hints)`, mirroring .NET's SqlBulkCopy: `SqlBulkCopyOption` flags
  (CheckConstraints, TableLock, KeepNulls, FireTriggers, KeepIdentity)
  and an optional per-column `ORDER` hint. `bulk_insert` and
  `bulk_insert_columns` are now thin wrappers around it with no options.
  Uses `enumflags2` (already a dependency, used for `ColumnFlag`/`RpcParam`
  flags elsewhere) instead of adding a second bitflags crate as upstream's
  PR did.
- Fixed `KeepIdentity` to actually do something: upstream's PR defined
  the flag but never referenced it in the generated SQL, so it was a
  no-op. The TDS bulk-load protocol has no WITH-clause keyword for
  preserving identity values anyway (unlike CHECK_CONSTRAINTS/
  FIRE_TRIGGERS/KEEP_NULLS/TABLOCK, which are real INSERT BULK options) -
  SQL Server's own BULK INSERT/SqlBulkCopy preserve identity values by
  including the identity column in the column list instead. So
  `KeepIdentity` now overrides the existing Updateable-based column
  filter for identity columns specifically.
- Fixed 4 clippy-style issues present in the original PR: `len() > 0` ->
  `!is_empty()`, `push_str(")")` -> `push(')')`, a needless `return`, and
  uninlined format args in the improved "unrepresentable character"
  error message (now includes the offending char).
- Did not carry over the PR's unrelated deletion of a decode-completeness
  assertion in a column_data.rs test (`reader.read_u8().await
  .expect_err(...)`) - that assertion was never touched here since this
  was a from-scratch reimplementation rather than a patch apply, so
  there was nothing to restore.
- Live-tested and found (and fixed) a real bug of my own along the way:
  the initial `if !options.is_empty() || !order_hints.is_empty() { ...
  WITH (...) }` condition emitted an empty, syntactically invalid
  `WITH ()` clause when the only option set was `KeepIdentity` (which
  contributes no WITH-clause text) - now the clause is only emitted when
  there's an actual hint to put in it.

Added 12 unit tests (bulk_options.rs has none of its own logic beyond
plain enum/type definitions) plus 3 new live integration tests: KeepIdentity
preserving supplied identity values, confirming identity is still excluded
without it, and TableLock+CheckConstraints+FireTriggers+KeepNulls+ORDER
together.

Verified:
- cargo check: default, --features=rustls, --features=vendored-openssl,
  --no-default-features
- cargo clippy --all-targets -- -D warnings, for default, rustls, and
  vendored-openssl feature sets
- cargo fmt --check
- cargo test --lib (169 passed)
- cargo test --doc against a live server (27 passed, doctest included)
- Live integration tests over rustls: 300 passed (98 bulk + 202 query)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0156Di1tRRLsJK8ctU1AmAJr
@MattJackson

Copy link
Copy Markdown
Contributor

Bulk options/order hints landed in #451 — shipped in tiberius v0.13.0. Thanks for this! Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants