From 4aa4a9189902e5209ac24435a7ab1c578eedaede Mon Sep 17 00:00:00 2001 From: ChangRui-Ryan Date: Wed, 19 Aug 2026 16:13:18 +0800 Subject: [PATCH 1/3] tidb-cloud: update for Starter mode --- .../vector-search-auto-embedding-openai.md | 4 +- .../vector-search-auto-embedding-overview.md | 20 ++- .../vector-search-functions-and-operators.md | 20 +++ command-line-flags-for-tidb-configuration.md | 77 ++++++++++++ system-variable-reference.md | 49 ++++++++ system-variables.md | 80 +++++++++++- tidb-cloud/limited-sql-features.md | 4 +- tidb-configuration-file.md | 114 +++++++++++++++++- 8 files changed, 357 insertions(+), 11 deletions(-) diff --git a/ai/integrations/vector-search-auto-embedding-openai.md b/ai/integrations/vector-search-auto-embedding-openai.md index 79a8748dcfd87..d6d726c2bfd9c 100644 --- a/ai/integrations/vector-search-auto-embedding-openai.md +++ b/ai/integrations/vector-search-auto-embedding-openai.md @@ -266,7 +266,7 @@ SET @@GLOBAL.TIDB_EXP_EMBED_OPENAI_API_BASE = ''; > **Note:** > -> - For security reasons, you can only set the API base to an Azure OpenAI URL or the OpenAI URL. Arbitrary base URLs are not allowed. +> - For security reasons, you can only set the API base to an OpenAI, Azure OpenAI, or Alibaba Cloud DashScope URL. Arbitrary base URLs are not allowed. > - To use another OpenAI-compatible embedding service, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). ## Options @@ -294,4 +294,4 @@ For all available options, see [OpenAI Documentation](https://platform.openai.co - [Auto Embedding Overview](/ai/integrations/vector-search-auto-embedding-overview.md) - [Vector Search](/ai/concepts/vector-search-overview.md) - [Vector Functions and Operators](/ai/reference/vector-search-functions-and-operators.md) -- [Hybrid Search](/ai/guides/vector-search-hybrid-search.md) \ No newline at end of file +- [Hybrid Search](/ai/guides/vector-search-hybrid-search.md) diff --git a/ai/integrations/vector-search-auto-embedding-overview.md b/ai/integrations/vector-search-auto-embedding-overview.md index bbc12a988dc90..2fcb175852a13 100644 --- a/ai/integrations/vector-search-auto-embedding-overview.md +++ b/ai/integrations/vector-search-auto-embedding-overview.md @@ -139,13 +139,27 @@ Internally, [`VEC_EMBED_COSINE_DISTANCE()`](#vec_embed_cosine_distance) and [`VE ### `EMBED_TEXT()` -Converts text to vector embeddings: +Converts text to a vector embedding and returns a `VECTOR` value. This function is available only in Starter deployment mode. ```sql -EMBED_TEXT("model_name", text_content[, additional_json_options]) +EMBED_TEXT("provider/model", text_content[, additional_json_options]) ``` -Use this function in `GENERATED ALWAYS AS` clauses to automatically generate embeddings when inserting or updating text data. +The parameters are as follows: + +- `provider/model`: the embedding model name in the `/` format. Supported provider prefixes are `tidbcloud_free`, `jina_ai`, `openai`, `cohere`, `huggingface`, `nvidia_nim`, and `gemini`. The `tidbcloud_free` provider is available when the TiDB Cloud hosted embedding service is enabled. +- `text_content`: the text to convert into an embedding. +- `additional_json_options`: an optional JSON object passed to the embedding provider. An empty string or `NULL` means that no additional options are provided. For supported options, see the documentation of the corresponding provider in [Available text embedding models](#available-text-embedding-models). + +You can call `EMBED_TEXT()` directly in a query or use it as the direct expression of a stored generated column. When you use it in a generated column, the following restrictions apply: + +- The generated column must be `STORED`. `VIRTUAL` generated columns are not supported. +- `EMBED_TEXT()` must be the direct generated-column expression and cannot be nested in another expression. +- The model name and additional options must be string constants. +- You cannot use `ALTER TABLE ... ADD COLUMN` to add a generated column that uses `EMBED_TEXT()`. +- Another generated column cannot depend on a generated column that uses `EMBED_TEXT()`. + +For the complete SQL function reference, see [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text). ### `VEC_EMBED_COSINE_DISTANCE()` diff --git a/ai/reference/vector-search-functions-and-operators.md b/ai/reference/vector-search-functions-and-operators.md index 4ecd1e840b88e..d21c623c3fa3b 100644 --- a/ai/reference/vector-search-functions-and-operators.md +++ b/ai/reference/vector-search-functions-and-operators.md @@ -34,6 +34,7 @@ The following functions are designed specifically for [Vector data types](/ai/re | [`VEC_L2_NORM`](#vec_l2_norm) | Calculates the L2 norm (Euclidean norm) of a vector | | [`VEC_FROM_TEXT`](#vec_from_text) | Converts a string into a vector | | [`VEC_AS_TEXT`](#vec_as_text) | Converts a vector into a string | +| [`EMBED_TEXT`](#embed_text) | Converts text into a vector embedding | ## Extended built-in functions and operators @@ -100,6 +101,25 @@ For more information about how to use `CAST()`, see [Vector Data Type | Cast](/a ## Full references +### EMBED_TEXT + +```sql +EMBED_TEXT(model, text[, options]) +``` + +Uses an embedding provider to convert `text` into a `VECTOR` value. `model` must use the `/` format. The optional `options` argument must be a JSON object; an empty string or `NULL` means that no additional options are provided. + +`EMBED_TEXT()` is available only in Starter deployment mode. You can call it directly in a query or use it as the direct expression of a `STORED` generated column. Generated-column usage has additional DDL restrictions. For supported providers, examples, and the complete list of restrictions, see [Auto Embedding Overview](/ai/integrations/vector-search-auto-embedding-overview.md#embed_text). + +Example: + +```sql +SELECT EMBED_TEXT( + 'openai/text-embedding-3-small', + 'TiDB is a distributed SQL database.' +); +``` + ### VEC_L2_DISTANCE ```sql diff --git a/command-line-flags-for-tidb-configuration.md b/command-line-flags-for-tidb-configuration.md index 49daad9072c36..139c56cf2a3d5 100644 --- a/command-line-flags-for-tidb-configuration.md +++ b/command-line-flags-for-tidb-configuration.md @@ -14,6 +14,32 @@ When you start the TiDB cluster, you can use command-line options or environment - Default: `""` - This address must be accessible by the rest of the TiDB cluster and the user. +## `--activation-timeout` + +- Specifies the maximum time that TiDB waits for activation to complete after receiving an activation request in standby mode. +- Default: `0`, which means that there is no timeout. +- Unit: seconds + +## `--cluster-ca` + +- Specifies the path to the CA certificate used for TLS connections between TiDB and other cluster components. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.cluster-ssl-ca`](/tidb-configuration-file.md#cluster-ssl-ca) in the configuration file. + +## `--cluster-cert` + +- Specifies the path to the certificate used for TLS connections between TiDB and other cluster components. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.cluster-ssl-cert`](/tidb-configuration-file.md#cluster-ssl-cert) in the configuration file. +- You must specify `--cluster-cert` and [`--cluster-key`](#--cluster-key) together. If the effective cluster CA path is not empty, the effective certificate and private key paths must also be present. + +## `--cluster-key` + +- Specifies the path to the private key used for TLS connections between TiDB and other cluster components. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.cluster-ssl-key`](/tidb-configuration-file.md#cluster-ssl-key) in the configuration file. +- You must specify `--cluster-key` and [`--cluster-cert`](#--cluster-cert) together. + ## `--config` - The configuration file @@ -57,6 +83,12 @@ When you start the TiDB cluster, you can use command-line options or environment - The SQL script to be executed when the TiDB cluster is started for the first time. For details, see [configuration item `initialize-sql-file`](/tidb-configuration-file.md#initialize-sql-file-new-in-v660) - Default: `""` +## `--keyspace-activate` + +- Controls whether TiDB exits after activating the specified keyspace. +- Default: `false` +- This option is supported only in Starter deployment mode and cannot be enabled together with [`--standby`](#--standby). + ## `-L` - The log level @@ -99,6 +131,12 @@ When you start the TiDB cluster, you can use command-line options or environment - Default: `15s` - Setting the value to 0 stops the Prometheus client from pushing. +## `--max-idle-seconds` + +- Specifies the maximum idle time of an activated TiDB instance. When the instance remains idle for longer than this value and has no active transaction or query that prevents shutdown, TiDB exits. +- Default: `0`, which means that the idle timeout is disabled. +- Unit: seconds + ## `-P` - The monitoring port of TiDB services @@ -164,6 +202,45 @@ When you start the TiDB cluster, you can use command-line options or environment - Default: `""` - Use `/tmp/tidb.sock` to open the unix socket file. +## `--sql-ca` + +- Specifies the path to the CA certificate used for TLS connections from SQL clients. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.ssl-ca`](/tidb-configuration-file.md#ssl-ca) in the configuration file. + +## `--sql-cert` + +- Specifies the path to the certificate used for TLS connections from SQL clients. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.ssl-cert`](/tidb-configuration-file.md#ssl-cert) in the configuration file. +- You must specify `--sql-cert` and [`--sql-key`](#--sql-key) together. If the effective SQL CA path is not empty, the effective certificate and private key paths must also be present. + +## `--sql-key` + +- Specifies the path to the private key used for TLS connections from SQL clients. +- Default: `""` +- This option takes effect only in Starter deployment mode, where it overrides [`security.ssl-key`](/tidb-configuration-file.md#ssl-key) in the configuration file. +- You must specify `--sql-key` and [`--sql-cert`](#--sql-cert) together. + +In Starter deployment mode, you can also override the TLS file paths by setting the following environment variables: + +| Environment variable | Corresponding configuration item | +| --- | --- | +| `CLUSTER_CA` | `security.cluster-ssl-ca` | +| `CLUSTER_CERT` | `security.cluster-ssl-cert` | +| `CLUSTER_KEY` | `security.cluster-ssl-key` | +| `SQL_CA` | `security.ssl-ca` | +| `SQL_CERT` | `security.ssl-cert` | +| `SQL_KEY` | `security.ssl-key` | + +Non-empty environment variables take precedence over both the configuration file and the corresponding command-line options. `CLUSTER_CERT` and `CLUSTER_KEY` must be set together, as must `SQL_CERT` and `SQL_KEY`. If an effective CA path is not empty, the corresponding effective certificate and private key paths must also be present. + +## `--standby` + +- Controls whether TiDB starts in standby mode and waits for a keyspace activation request before starting SQL service. +- Default: `false` +- This option is supported only by NextGen TiDB. You cannot enable it together with [`--keyspace-activate`](#--keyspace-activate). + ## `--status` - The status report port for TiDB server diff --git a/system-variable-reference.md b/system-variable-reference.md index 3590ce3e5c8fd..37dd3235c1db5 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -2180,6 +2180,55 @@ Referenced in: - [TiDB 8.2.0 Release Notes](/releases/release-8.2.0.md) - [TiDB 5.0 Release Notes](/releases/release-5.0.0.md) +### tidb_exp_embed_cohere_api_key + +Referenced in: + +- [Cohere Embeddings](/ai/integrations/vector-search-auto-embedding-cohere.md) +- [System Variables](/system-variables.md#tidb_exp_embed_cohere_api_key) + +### tidb_exp_embed_gemini_api_key + +Referenced in: + +- [Gemini Embeddings](/ai/integrations/vector-search-auto-embedding-gemini.md) +- [System Variables](/system-variables.md#tidb_exp_embed_gemini_api_key) + +### tidb_exp_embed_huggingface_api_key + +Referenced in: + +- [Hugging Face Embeddings](/ai/integrations/vector-search-auto-embedding-huggingface.md) +- [System Variables](/system-variables.md#tidb_exp_embed_huggingface_api_key) + +### tidb_exp_embed_jina_ai_api_key + +Referenced in: + +- [Jina AI Embeddings](/ai/integrations/vector-search-auto-embedding-jina-ai.md) +- [System Variables](/system-variables.md#tidb_exp_embed_jina_ai_api_key) + +### tidb_exp_embed_nvidia_nim_api_key + +Referenced in: + +- [NVIDIA NIM Embeddings](/ai/integrations/vector-search-auto-embedding-nvidia-nim.md) +- [System Variables](/system-variables.md#tidb_exp_embed_nvidia_nim_api_key) + +### tidb_exp_embed_openai_api_base + +Referenced in: + +- [OpenAI Embeddings](/ai/integrations/vector-search-auto-embedding-openai.md) +- [System Variables](/system-variables.md#tidb_exp_embed_openai_api_base) + +### tidb_exp_embed_openai_api_key + +Referenced in: + +- [OpenAI Embeddings](/ai/integrations/vector-search-auto-embedding-openai.md) +- [System Variables](/system-variables.md#tidb_exp_embed_openai_api_key) + ### tidb_expensive_query_time_threshold Referenced in: diff --git a/system-variables.md b/system-variables.md index 1555a6ceb02e6..110fa52c7bc21 100644 --- a/system-variables.md +++ b/system-variables.md @@ -651,7 +651,7 @@ This variable is an alias for [`last_insert_id`](#last_insert_id). > **Note:** > -> This variable is read-only for [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential). +> This variable is read-only for [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential). Its effective value is controlled by TiDB Cloud. - Scope: SESSION | GLOBAL - Persists to cluster: Yes @@ -661,6 +661,7 @@ This variable is an alias for [`last_insert_id`](#last_insert_id). - The value should be an integer multiple of 1024. If the value is not divisible by 1024, a warning will be prompted and the value will be rounded down. For example, when the value is set to 1025, the actual value in TiDB is 1024. - The maximum packet size allowed by the server and the client in one transmission of packets. - In the `SESSION` scope, this variable is read-only. +- In Starter deployment mode, the effective value is configured by [`max-allowed-packet`](/tidb-configuration-file.md#max-allowed-packet-new-in-v900). TiDB uses the configured value when initializing sessions and when reporting the global `max_allowed_packet` value. - This variable is compatible with MySQL. ### max_connections @@ -889,6 +890,8 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'max_prepared_stmt_count'; > **Note:** > > Currently, this variable is not supported on [TiDB Cloud Dedicated](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-dedicated). DO **NOT** enable this variable for TiDB Cloud Dedicated clusters. Otherwise, you might get SQL client connection failures. This restriction is a temporary control measure and will be resolved in a future release. +> +> For [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential), this variable is read-only and always returns `ON`. - Scope: GLOBAL - Persists to cluster: Yes @@ -904,11 +907,11 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'max_prepared_stmt_count'; -- This variable ensures that all connections to TiDB are either on a local socket, or using TLS. +- For TiDB Cloud, secure connection behavior is managed by the service. For [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential), this variable is exposed as `ON` and cannot be changed by SQL statements. -- Setting this variable to `ON` requires you to connect to TiDB from a session that has TLS enabled. This helps prevent lock-out scenarios when TLS is not configured correctly. +- For TiDB Self-Managed, setting this variable to `ON` requires you to connect to TiDB from a session that has TLS enabled. This helps prevent lock-out scenarios when TLS is not configured correctly. - This setting was previously a `tidb.toml` option (`security.require-secure-transport`), but changed to a system variable starting from TiDB v6.1.0. - Starting from v6.5.6, v7.1.2, v7.5.1, and v8.0.0, when Security Enhanced Mode (SEM) is enabled, setting this variable to `ON` is prohibited to avoid potential connectivity issues for users. @@ -3155,6 +3158,77 @@ Since v5.0, you can still separately modify the system variables listed above (w For a system upgraded to v5.0 from an earlier version, if you have not modified any value of the variables listed above (which means that the `tidb_hash_join_concurrency` value is `5` and the values of the rest are `4`), the operator concurrency previously managed by these variables will automatically be managed by `tidb_executor_concurrency`. If you have modified any of these variables, the concurrency of the corresponding operators will still be controlled by the modified variables. +### tidb_exp_embed_cohere_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `cohere/` prefix in Starter deployment mode. For usage information, see [Cohere Embeddings](/ai/integrations/vector-search-auto-embedding-cohere.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + +### tidb_exp_embed_gemini_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `gemini/` prefix in Starter deployment mode. For usage information, see [Gemini Embeddings](/ai/integrations/vector-search-auto-embedding-gemini.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + +### tidb_exp_embed_huggingface_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `huggingface/` prefix in Starter deployment mode. For usage information, see [Hugging Face Embeddings](/ai/integrations/vector-search-auto-embedding-huggingface.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + +### tidb_exp_embed_jina_ai_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `jina_ai/` prefix in Starter deployment mode. For usage information, see [Jina AI Embeddings](/ai/integrations/vector-search-auto-embedding-jina-ai.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + +### tidb_exp_embed_nvidia_nim_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `nvidia_nim/` prefix in Starter deployment mode. For usage information, see [NVIDIA NIM Embeddings](/ai/integrations/vector-search-auto-embedding-nvidia-nim.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + +### tidb_exp_embed_openai_api_base + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `"https://api.openai.com/v1"` +- This variable specifies the base URL that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `openai/` prefix in Starter deployment mode. Setting the value to an empty string restores the default URL. +- The value must be an absolute HTTPS URL without query parameters or a fragment. The host must be OpenAI (`api.openai.com`), Azure OpenAI (`*.openai.azure.com`), or an Alibaba Cloud DashScope endpoint (`dashscope.aliyuncs.com`, `dashscope-intl.aliyuncs.com`, or `dashscope-us.aliyuncs.com`). +- For usage information, see [OpenAI Embeddings](/ai/integrations/vector-search-auto-embedding-openai.md). + +### tidb_exp_embed_openai_api_key + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: String +- Default value: `""` +- This variable specifies the API key that [`EMBED_TEXT()`](/ai/reference/vector-search-functions-and-operators.md#embed_text) uses for models with the `openai/` prefix in Starter deployment mode. For usage information, see [OpenAI Embeddings](/ai/integrations/vector-search-auto-embedding-openai.md). +- When you read this variable, TiDB returns a masked value. TiDB fully redacts the key value from `SET` statements in logs and the process list. + ### tidb_expensive_query_time_threshold > **Note:** diff --git a/tidb-cloud/limited-sql-features.md b/tidb-cloud/limited-sql-features.md index efa8101580cb6..d3fec85a2f6ec 100644 --- a/tidb-cloud/limited-sql-features.md +++ b/tidb-cloud/limited-sql-features.md @@ -125,10 +125,10 @@ TiDB Cloud works with almost all workloads that TiDB supports, but there are som |:-|:-|:-| | `datadir` | No limitation | Not supported [^1] | | `interactive_timeout` | No limitation | Read-only [^10] | -| `max_allowed_packet` | No limitation | Read-only [^10] | +| `max_allowed_packet` | No limitation | Read-only; controlled by TiDB Cloud [^10] | | `plugin_dir` | No limitation | Not supported [^8] | | `plugin_load` | No limitation | Not supported [^8] | -| `require_secure_transport` | Not supported [^12] | Read-only [^10] | +| `require_secure_transport` | Not supported [^12] | Read-only and always `ON` [^10] | | `skip_name_resolve` | No limitation | Read-only [^10] | | `sql_log_bin` | No limitation | Read-only [^10] | | `tidb_analyze_skip_column_types` | No limitation | Read-only [^10] | diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index 5e662a4140c57..e7c3f8de50e55 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -15,6 +15,37 @@ The TiDB configuration file supports more options than command-line parameters. > > If you need to adjust the value of a configuration item, refer to [Modify the configuration](/maintain-tidb-using-tiup.md#modify-the-configuration). +### `deploy-mode` + +- Specifies the deployment mode of a NextGen TiDB instance. +- Default value: `"premium"` +- Possible values: `"premium"`, `"premium_reserved"`, and `"starter"` +- This configuration item is initialized when TiDB starts and cannot be changed at runtime. +- This configuration item is supported only by NextGen TiDB. For Starter deployment mode, set the value to `"starter"`. + +### `keyspace-activate` + +- Controls whether TiDB exits after activating the specified keyspace. +- Default value: `false` +- This configuration item takes effect only in Starter deployment mode. +- You cannot enable `keyspace-activate` and [`standby.standby-mode`](#standby-mode) at the same time. + +### `error-msg-extension` + +- Appends a configured suffix to a SQL error when the error message matches a regular expression. +- Default value: `[]` +- This configuration item takes effect only in Starter deployment mode. +- Each array element contains a `pattern` field and a `suffix` field. TiDB removes trailing periods from the original error and the suffix, and then returns the extended error in the format `, .`. +- If multiple patterns match an error, TiDB applies only the first match after sorting patterns by length in descending order. To avoid matching unrelated errors, use anchored and specific patterns. + +For example: + +```toml +error-msg-extension = [ + { pattern = "^Feature '.+' is not supported$", suffix = "see the feature limitations for more details" }, +] +``` + ### `split-table` - Determines whether to create a separate Region for each table. @@ -43,6 +74,15 @@ The TiDB configuration file supports more options than command-line parameters. + Minimum value: `1` + Maximum value: `1048576` +### `max-allowed-packet` New in v9.0.0 + ++ Configures the effective value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) in Starter deployment mode. ++ Default value: `67108864` (64 MiB) ++ Minimum value: `1024` ++ Maximum value: `1073741824` ++ The value must be an integer multiple of `1024`. ++ This configuration item takes effect only in Starter deployment mode. In other deployment modes, configure the packet size by using the `max_allowed_packet` system variable. + ### `temp-dir` New in v6.3.0 + File system location used by TiDB to store temporary data. If a feature requires local storage in TiDB nodes, TiDB stores the corresponding temporary data in this location. @@ -522,7 +562,7 @@ Configuration items related to performance. - The size limit of a single key-value record in a transaction. If the size limit is exceeded, TiDB returns the `entry too large` error. The maximum value of this configuration item does not exceed `125829120` (120 MB). - Starting from v7.6.0, you can use the system variable [`tidb_txn_entry_size_limit`](/system-variables.md#tidb_txn_entry_size_limit-new-in-v760) to dynamically modify the value of this configuration item. - Note that TiKV has a similar limit. If the data size of a single write request exceeds [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size), which is 8 MB by default, TiKV refuses to process this request. When a table has a row of large size, you need to modify both configurations at the same time. -- The default value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) (the maximum size of a packet for the MySQL protocol) is 67108864 (64 MiB). If a row is larger than `max_allowed_packet`, the row gets truncated. +- The default value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) (the maximum size of a packet for the MySQL protocol) is 67108864 (64 MiB). In Starter deployment mode, its effective value is configured by [`max-allowed-packet`](#max-allowed-packet-new-in-v900). If a row is larger than `max_allowed_packet`, the row gets truncated. - The default value of [`txn-total-size-limit`](#txn-total-size-limit) (the size limit of a single transaction in TiDB) is 100 MiB. If you increase the `txn-entry-size-limit` value to be over 100 MiB, you need to increase the `txn-total-size-limit` value accordingly. ### `txn-total-size-limit` @@ -1076,3 +1116,75 @@ The `experimental` section, introduced in v3.1.0, describes the configurations r + Controls whether an expression index can be created. Since TiDB v5.2.0, if the function in an expression is safe, you can create an expression index directly based on this function without enabling this configuration. If you want to create an expression index based on other functions, you can enable this configuration, but correctness issues might exist. By querying the `tidb_allow_function_for_expression_index` variable, you can get the functions that are safe to be directly used for creating an expression. + Default value: `false` + +## standby + +Configuration items related to the standby and idle shutdown behavior of NextGen TiDB. + +### `standby-mode` + +- Controls whether TiDB starts in standby mode and waits for a keyspace activation request before starting SQL service. +- Default value: `false` +- You cannot enable `standby-mode` and [`keyspace-activate`](#keyspace-activate) at the same time. + +### `activation-timeout` + +- Specifies the maximum time that TiDB waits for activation to complete after receiving an activation request in standby mode. +- Default value: `0`, which means that there is no timeout. +- Unit: seconds + +### `max-idle-seconds` + +- Specifies the maximum idle time of an activated TiDB instance. When the instance remains idle for longer than this value and has no active transaction or query that prevents shutdown, TiDB exits. +- Default value: `0`, which means that the idle timeout is disabled. +- Unit: seconds + +### `enable-zero-backend` + +- Controls the idle shutdown behavior in Starter deployment mode. +- Default value: `false`. In Starter deployment mode, if you do not explicitly configure this item, TiDB changes its effective default value to `true`. +- When the value is `true`, TiDB does not wait for session migration during idle shutdown and does not treat interactive client connections as active work that prevents shutdown. Active queries and transactions still prevent idle shutdown. + +## starter-params + +Configuration items that take effect only in Starter deployment mode. + +### `bootstrap-file` + +- Specifies the path to a JSON manifest that initializes and upgrades Starter-specific SQL state. +- Default value: `""`, which disables the manifest. +- The manifest must contain exactly one JSON object and cannot contain unknown fields. The top-level `version` must be greater than `0`. +- The `bootstrap` array defines the complete state for a keyspace that does not have a recorded Starter bootstrap version. It must contain at least one statement when the manifest initializes such a keyspace. The `upgrades` array defines migrations for keyspaces with an earlier recorded version. +- Each upgrade version must be greater than `0`, no greater than the top-level `version`, and unique within the manifest. +- Each element in a SQL array must contain exactly one statement. `` is the only supported placeholder and is replaced with the current keyspace name. +- Statements in `bootstrap` must be `INSERT`, `REPLACE`, `UPDATE`, or `DELETE` statements and must create the `.root` account with host `%`. +- For a keyspace without a recorded Starter bootstrap version, TiDB executes `bootstrap` directly and records the top-level version without replaying `upgrades`. For an existing keyspace, TiDB applies upgrade entries in version order. +- Make every statement retry-safe and idempotent because TiDB might retry initialization or an upgrade after a startup failure. + +The following is an example manifest: + +```json +{ + "version": 2, + "bootstrap": [ + "INSERT INTO mysql.user (Host, User, authentication_string, plugin) VALUES ('%', '.root', '', 'mysql_native_password') ON DUPLICATE KEY UPDATE authentication_string = VALUES(authentication_string), plugin = VALUES(plugin)", + "INSERT INTO mysql.global_grants (User, Host, Priv) VALUES ('.root', '%', 'SYSTEM_VARIABLES_ADMIN') ON DUPLICATE KEY UPDATE WITH_GRANT_OPTION = 'N'" + ], + "upgrades": [ + { + "version": 2, + "sql": [ + "INSERT INTO mysql.global_grants (User, Host, Priv) VALUES ('.root', '%', 'SYSTEM_VARIABLES_ADMIN') ON DUPLICATE KEY UPDATE WITH_GRANT_OPTION = 'N'" + ] + } + ] +} +``` + +### `max-import-data-size` + +- Specifies the maximum total real size of source data that one [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md) statement can import. +- Default value: `"25GiB"` when this configuration item is not explicitly configured. +- To disable the data size limit, explicitly set this configuration item to `"0B"`. +- This configuration item accepts byte-size values such as `"1MiB"`. +- When the source data exceeds the limit, `IMPORT INTO` fails during the precheck phase before importing data. From cea08e8c9b9b6543fb9e6368d14541b3a75a871c Mon Sep 17 00:00:00 2001 From: ChangRui-Ryan Date: Wed, 19 Aug 2026 17:06:34 +0800 Subject: [PATCH 2/3] docs: clarify Starter import preparation --- sql-statements/sql-statement-import-into.md | 5 +++-- tidb-configuration-file.md | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/sql-statements/sql-statement-import-into.md b/sql-statements/sql-statement-import-into.md index da4ed9a2afc97..cd057bf98faa1 100644 --- a/sql-statements/sql-statement-import-into.md +++ b/sql-statements/sql-statement-import-into.md @@ -38,6 +38,7 @@ The `IMPORT INTO` statement lets you import data to TiDB via the [Physical Impor - For TiDB Self-Managed, each `IMPORT INTO` task supports importing data within 10 TiB. If you enable the [Global Sort](/tidb-global-sort.md) feature, each `IMPORT INTO` task supports importing data within 40 TiB. - For [TiDB Cloud Dedicated](https://docs.pingcap.com/tidbcloud/select-cluster-tier#tidb-cloud-dedicated), if your data to be imported exceeds 500 GiB, it is recommended to use TiDB nodes with at least 16 cores and enable the [Global Sort](/tidb-global-sort.md) feature, then each `IMPORT INTO` task supports importing data within 40 TiB. If your data to be imported is within 500 GiB or if the cores of your TiDB nodes are less than 16, it is not recommended to enable the [Global Sort](/tidb-global-sort.md) feature. - The execution of `IMPORT INTO ... FROM FILE` blocks the current connection until the import is completed. To execute the statement asynchronously, you can add the `DETACHED` option. +- On [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) instances, TiDB prepares source files and performs pre-checks synchronously before creating an import job. This behavior also applies when `DETACHED` is specified. The `DETACHED` option only makes the import job run asynchronously after it is created. If the source data exceeds the [`starter-params.max-import-data-size`](/tidb-configuration-file.md#max-import-data-size) limit, the statement returns an error and no import job is created. - Up to 16 `IMPORT INTO` tasks can run simultaneously on each cluster (see [TiDB Distributed eXecution Framework (DXF) usage limitations](/tidb-distributed-execution-framework.md#limitation)). When a cluster lacks sufficient resources or reaches the maximum number of tasks, newly submitted import tasks are queued for execution. - When the [Global Sort](/tidb-global-sort.md) feature is used for data import, the data size of a single row after encoding must not exceed 32 MiB. - All `IMPORT INTO` tasks that are created when [TiDB Distributed eXecution Framework (DXF)](/tidb-distributed-execution-framework.md) is not enabled run directly on the nodes where the tasks are submitted, and these tasks will not be scheduled for execution on other TiDB nodes even after DXF is enabled later. After DXF is enabled, only newly created `IMPORT INTO` tasks that import data from S3 or GCS are automatically scheduled or failed over to other TiDB nodes for execution. @@ -160,7 +161,7 @@ The supported options are described as follows: | `THREAD=` | All file formats and query results of `SELECT` | Specifies the concurrency for import. For `IMPORT INTO ... FROM FILE`, the default value of `THREAD` is 50% of the number of CPU cores on the TiDB node, the minimum value is `1`, and the maximum value is the number of CPU cores. For `IMPORT INTO ... FROM SELECT`, the default value of `THREAD` is `2`, the minimum value is `1`, and the maximum value is two times the number of CPU cores on the TiDB node. To import data into a new cluster without any data, it is recommended to increase this concurrency appropriately to improve import performance. If the target cluster is already used in a production environment, it is recommended to adjust this concurrency according to your application requirements. | | `MAX_WRITE_SPEED=''` | All file formats | Controls the write speed to a TiKV node. By default, there is no speed limit. For example, you can specify this option as `1MiB` to limit the write speed to 1 MiB/s. | | `CHECKSUM_TABLE=''` | All file formats | Configures whether to perform a checksum check on the target table after the import to validate the import integrity. The supported values include `"required"` (default), `"optional"`, and `"off"`. `"required"` means performing a checksum check after the import. If the checksum check fails, TiDB will return an error and the import will exit. `"optional"` means performing a checksum check after the import. If an error occurs, TiDB will return a warning and ignore the error. `"off"` means not performing a checksum check after the import. | -| `DETACHED` | All file formats | Controls whether to execute `IMPORT INTO` asynchronously. When this option is enabled, executing `IMPORT INTO` immediately returns the information of the import job (such as the `Job_ID`), and the job is executed asynchronously in the backend. | +| `DETACHED` | All file formats | Controls whether to execute `IMPORT INTO` asynchronously. When this option is enabled, `IMPORT INTO` returns the information of the import job (such as the `Job_ID`) after the job is created, and the job is executed asynchronously in the backend. | | `CLOUD_STORAGE_URI` | All file formats | Specifies the target address where encoded KV data for [Global Sort](/tidb-global-sort.md) is stored. When `CLOUD_STORAGE_URI` is not specified, `IMPORT INTO` determines whether to use Global Sort based on the value of the system variable [`tidb_cloud_storage_uri`](/system-variables.md#tidb_cloud_storage_uri-new-in-v740). If this system variable specifies a target storage address, `IMPORT INTO` uses this address for Global Sort. When `CLOUD_STORAGE_URI` is specified with a non-empty value, `IMPORT INTO` uses that value as the target storage address. When `CLOUD_STORAGE_URI` is specified with an empty value, local sorting is enforced. Currently, the target storage address only supports S3. For details about the URI configuration, see [Amazon S3 URI format](/external-storage-uri.md#amazon-s3-uri-format). When this feature is used, all TiDB nodes must have read and write access for the target S3 bucket, including at least these permissions: `s3:ListBucket`, `s3:GetObject`, `s3:DeleteObject`, `s3:PutObject`, `s3: AbortMultipartUpload`. | | `DISABLE_PRECHECK` | All file formats and query results of `SELECT` | Setting this option disables pre-checks of non-critical items, such as checking whether there are CDC or PITR tasks. | @@ -242,7 +243,7 @@ IMPORT INTO t FROM '/path/to/small.csv'; +--------+--------------------+--------------+----------+-------+----------+------------------+---------------+----------------+----------------------------+----------------------------+----------------------------+------------+ ``` -When the `DETACHED` mode is enabled, executing the `IMPORT INTO ... FROM FILE` statement will immediately return the job information in the output. From the output, you can see that the status of the job is `pending`, which means waiting for execution. +When the `DETACHED` mode is enabled, executing the `IMPORT INTO ... FROM FILE` statement returns the job information in the output after the job is created. From the output, you can see that the status of the job is `pending`, which means waiting for execution. ```sql IMPORT INTO t FROM '/path/to/small.csv' WITH DETACHED; diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index e7c3f8de50e55..d5dd4dce4b8a4 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -1187,4 +1187,4 @@ The following is an example manifest: - Default value: `"25GiB"` when this configuration item is not explicitly configured. - To disable the data size limit, explicitly set this configuration item to `"0B"`. - This configuration item accepts byte-size values such as `"1MiB"`. -- When the source data exceeds the limit, `IMPORT INTO` fails during the precheck phase before importing data. +- When the source data exceeds the limit, `IMPORT INTO` returns an error during the synchronous pre-check phase before creating an import job. From bc65ba154fb22b6bf66845240de6956ab05b52cf Mon Sep 17 00:00:00 2001 From: ChangRui-Ryan Date: Wed, 19 Aug 2026 20:22:41 +0800 Subject: [PATCH 3/3] docs: address Starter mode review feedback --- ai/integrations/vector-search-auto-embedding-openai.md | 2 +- system-variables.md | 6 +++--- tidb-configuration-file.md | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/ai/integrations/vector-search-auto-embedding-openai.md b/ai/integrations/vector-search-auto-embedding-openai.md index d6d726c2bfd9c..07fa56a6ac948 100644 --- a/ai/integrations/vector-search-auto-embedding-openai.md +++ b/ai/integrations/vector-search-auto-embedding-openai.md @@ -266,7 +266,7 @@ SET @@GLOBAL.TIDB_EXP_EMBED_OPENAI_API_BASE = ''; > **Note:** > -> - For security reasons, you can only set the API base to an OpenAI, Azure OpenAI, or Alibaba Cloud DashScope URL. Arbitrary base URLs are not allowed. +> - For security reasons, the API base must be an absolute HTTPS URL without query parameters or a fragment. Its host must be `api.openai.com`, `*.openai.azure.com`, `dashscope.aliyuncs.com`, `dashscope-intl.aliyuncs.com`, or `dashscope-us.aliyuncs.com`. > - To use another OpenAI-compatible embedding service, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md). ## Options diff --git a/system-variables.md b/system-variables.md index 110fa52c7bc21..d36e82dce1f28 100644 --- a/system-variables.md +++ b/system-variables.md @@ -651,7 +651,7 @@ This variable is an alias for [`last_insert_id`](#last_insert_id). > **Note:** > -> This variable is read-only for [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential). Its effective value is controlled by TiDB Cloud. +> This variable is read-only for [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential). TiDB Cloud controls its effective value. - Scope: SESSION | GLOBAL - Persists to cluster: Yes @@ -661,7 +661,7 @@ This variable is an alias for [`last_insert_id`](#last_insert_id). - The value should be an integer multiple of 1024. If the value is not divisible by 1024, a warning will be prompted and the value will be rounded down. For example, when the value is set to 1025, the actual value in TiDB is 1024. - The maximum packet size allowed by the server and the client in one transmission of packets. - In the `SESSION` scope, this variable is read-only. -- In Starter deployment mode, the effective value is configured by [`max-allowed-packet`](/tidb-configuration-file.md#max-allowed-packet-new-in-v900). TiDB uses the configured value when initializing sessions and when reporting the global `max_allowed_packet` value. +- In Starter deployment mode, [`max-allowed-packet`](/tidb-configuration-file.md#max-allowed-packet-new-in-v900) configures the effective value. TiDB uses this value when initializing sessions and when reporting the global `max_allowed_packet` value. - This variable is compatible with MySQL. ### max_connections @@ -907,7 +907,7 @@ mysql> SHOW GLOBAL VARIABLES LIKE 'max_prepared_stmt_count'; -- For TiDB Cloud, secure connection behavior is managed by the service. For [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential), this variable is exposed as `ON` and cannot be changed by SQL statements. +- For TiDB Cloud, the service manages secure connection behavior. For [{{{ .starter }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#starter) and [{{{ .essential }}}](https://docs.pingcap.com/tidbcloud/select-cluster-tier#essential), this variable is exposed as `ON` and cannot be changed by SQL statements. diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index d5dd4dce4b8a4..8daac9a9102ec 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -81,7 +81,7 @@ error-msg-extension = [ + Minimum value: `1024` + Maximum value: `1073741824` + The value must be an integer multiple of `1024`. -+ This configuration item takes effect only in Starter deployment mode. In other deployment modes, configure the packet size by using the `max_allowed_packet` system variable. ++ This configuration item takes effect only in Starter deployment mode. In other deployment modes, configure the packet size using the [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) system variable where it is writable. On TiDB Cloud Essential, the variable is read-only and controlled by TiDB Cloud. ### `temp-dir` New in v6.3.0 @@ -562,7 +562,7 @@ Configuration items related to performance. - The size limit of a single key-value record in a transaction. If the size limit is exceeded, TiDB returns the `entry too large` error. The maximum value of this configuration item does not exceed `125829120` (120 MB). - Starting from v7.6.0, you can use the system variable [`tidb_txn_entry_size_limit`](/system-variables.md#tidb_txn_entry_size_limit-new-in-v760) to dynamically modify the value of this configuration item. - Note that TiKV has a similar limit. If the data size of a single write request exceeds [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size), which is 8 MB by default, TiKV refuses to process this request. When a table has a row of large size, you need to modify both configurations at the same time. -- The default value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) (the maximum size of a packet for the MySQL protocol) is 67108864 (64 MiB). In Starter deployment mode, its effective value is configured by [`max-allowed-packet`](#max-allowed-packet-new-in-v900). If a row is larger than `max_allowed_packet`, the row gets truncated. +- The default value of [`max_allowed_packet`](/system-variables.md#max_allowed_packet-new-in-v610) (the maximum size of a packet for the MySQL protocol) is 67108864 (64 MiB). In Starter deployment mode, [`max-allowed-packet`](#max-allowed-packet-new-in-v900) configures its effective value. If a row is larger than `max_allowed_packet`, the row gets truncated. - The default value of [`txn-total-size-limit`](#txn-total-size-limit) (the size limit of a single transaction in TiDB) is 100 MiB. If you increase the `txn-entry-size-limit` value to be over 100 MiB, you need to increase the `txn-total-size-limit` value accordingly. ### `txn-total-size-limit` @@ -1157,7 +1157,7 @@ Configuration items that take effect only in Starter deployment mode. - The `bootstrap` array defines the complete state for a keyspace that does not have a recorded Starter bootstrap version. It must contain at least one statement when the manifest initializes such a keyspace. The `upgrades` array defines migrations for keyspaces with an earlier recorded version. - Each upgrade version must be greater than `0`, no greater than the top-level `version`, and unique within the manifest. - Each element in a SQL array must contain exactly one statement. `` is the only supported placeholder and is replaced with the current keyspace name. -- Statements in `bootstrap` must be `INSERT`, `REPLACE`, `UPDATE`, or `DELETE` statements and must create the `.root` account with host `%`. +- The statements in `bootstrap` must be `INSERT`, `REPLACE`, `UPDATE`, or `DELETE` statements. Together, they must create the `.root` account with host `%`. - For a keyspace without a recorded Starter bootstrap version, TiDB executes `bootstrap` directly and records the top-level version without replaying `upgrades`. For an existing keyspace, TiDB applies upgrade entries in version order. - Make every statement retry-safe and idempotent because TiDB might retry initialization or an upgrade after a startup failure.