lake: publish dbt integration docs - #23489
Conversation
ti-chi-bot
Bot
added
do-not-merge/work-in-progress
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan includes up to 4 reviews per rolling hour; 3 remain after this review. 📝 WalkthroughWalkthroughThe PR updates the TiDB Cloud Lake dbt guide. It renames the adapter, adds Snapshot support, updates installation and profile examples, and links the guide from the table of contents. ChangesTiDB Cloud Lake dbt documentation
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This PR updates the TiDB Cloud Lake dbt documentation and navigation without introducing product or runtime behavior changes; no actionable merge-blocking risk remains after normal checks and review. Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Skipping CI for Draft Pull Request. |
ti-chi-bot
Bot
added
missing-translation-status
ti-chi-bot
Bot
removed
the
missing-translation-status
ti-chi-bot
Bot
removed
the
do-not-merge/work-in-progress
ti-chi-bot
Bot
added
the
do-not-merge/work-in-progress
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: c871e6d5-f5d2-4e58-aac3-ac99e9130215
📒 Files selected for processing (2)
TOC-tidb-cloud-lake.mdtidb-cloud-lake/guides/load-with-dbt.md
|
|
||
| - Python 3.8 or later | ||
| - A {{{ .lake }}} account and warehouse | ||
| - The host, database, username, password, and warehouse name for your {{{ .lake }}} connection |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- relevant document sections ---'
sed -n '1,110p' tidb-cloud-lake/guides/load-with-dbt.md
printf '%s\n' '--- warehouse references ---'
rg -n -C 3 '\bwarehouse\b|tidbcloudlake-dbt|lake-dbt' tidb-cloud-lake .github resources 2>/dev/null || true
printf '%s\n' '--- candidate adapter files ---'
git ls-files | rg -i 'dbt|adapter|profile|lake'Repository: pingcap/docs
Length of output: 50371
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import json
import urllib.request
def get(url):
req = urllib.request.Request(url, headers={"User-Agent": "review-inspection"})
with urllib.request.urlopen(req, timeout=20) as response:
return json.load(response)
pypi = get("https://pypi.org/pypi/tidbcloudlake-dbt/json")
info = pypi["info"]
print("--- PyPI metadata ---")
for key in ("name", "version", "summary", "home_page", "project_urls", "requires_python", "requires_dist"):
print(f"{key}: {info.get(key)}")
print("--- release files ---")
for item in pypi["releases"].get(info["version"], []):
print(item.get("filename"), item.get("url"))
repo = get("https://api.github.com/repos/tidbcloud/lake-dbt")
print("--- GitHub repository ---")
for key in ("html_url", "default_branch", "visibility", "archived"):
print(f"{key}: {repo.get(key)}")
for path in ("profiles.yml", "dbt/adapters/tidbcloudlake/connections.py",
"dbt/adapters/tidbcloudlake/impl.py",
"dbt/adapters/tidbcloudlake/__init__.py"):
url = "https://api.github.com/repos/tidbcloud/lake-dbt/contents/" + path
try:
item = get(url)
except Exception as exc:
print(f"--- {path}: unavailable ({exc}) ---")
continue
print(f"--- {path} ---")
print(item.get("download_url"))
PYRepository: pingcap/docs
Length of output: 2469
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- PyPI project metadata ---'
curl -k -fsSL https://pypi.org/pypi/tidbcloudlake-dbt/json |
jq '{name: .info.name, version: .info.version, project_urls: .info.project_urls,
requires_python: .info.requires_python, requires_dist: .info.requires_dist,
release_urls: [.releases[.info.version][] | .url]}'
printf '%s\n' '--- upstream repository metadata ---'
curl -k -fsSL https://api.github.com/repos/tidbcloud/lake-dbt |
jq '{html_url, default_branch, visibility, archived}'
printf '%s\n' '--- upstream tree paths containing profile/connection/adapter ---'
curl -k -fsSL 'https://api.github.com/repos/tidbcloud/lake-dbt/git/trees/HEAD?recursive=1' |
jq -r '.tree[] | select(.path | test("(?i)(profile|connection|adapter|config)")) | .path'Repository: pingcap/docs
Length of output: 1133
🏁 Script executed:
#!/bin/bash
set -eu
wheel=/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl
curl -k -fsSL \
'https://files.pythonhosted.org/packages/a7/48/34e912b921839e80f6d722136e9624ab89fe899aa3aef6c781ec56c99a2e/tidbcloudlake_dbt-1.8.1-py3-none-any.whl' \
-o "$wheel"
python3 - "$wheel" <<'PY'
import re
import sys
import zipfile
wheel = sys.argv[1]
with zipfile.ZipFile(wheel) as z:
names = z.namelist()
print("--- package files ---")
for name in names:
if name.endswith((".py", ".yml", ".yaml", ".json", ".toml")):
print(name)
print("--- relevant source lines ---")
for name in names:
if not name.endswith(".py"):
continue
text = z.read(name).decode("utf-8", "replace")
if re.search(r"warehouse|Profile|Connection|credentials|required", text, re.I):
print(f"### {name}")
for number, line in enumerate(text.splitlines(), 1):
if re.search(r"warehouse|Profile|Connection|credentials|required", line, re.I):
print(f"{number}: {line}")
PY
printf '%s\n' '--- upstream repository metadata ---'
curl -k -fsSL https://api.github.com/repos/tidbcloud/lake-dbt |
jq '{html_url, default_branch, visibility, archived}'
printf '%s\n' '--- upstream tree paths ---'
curl -k -fsSL 'https://api.github.com/repos/tidbcloud/lake-dbt/git/trees/HEAD?recursive=1' |
jq -r '.tree[] | select(.path | test("(?i)(profile|connection|adapter|config)")) | .path'Repository: pingcap/docs
Length of output: 4497
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import zipfile
wheel = "/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl"
with zipfile.ZipFile(wheel) as z:
for name in (
"dbt/include/tidbcloudlake/profile_template.yml",
"dbt/adapters/tidbcloudlake/connections.py",
):
print(f"--- {name} ---")
for number, line in enumerate(z.read(name).decode().splitlines(), 1):
if name.endswith("profile_template.yml") or 30 <= number <= 170:
print(f"{number}: {line}")
source = z.read("dbt/adapters/tidbcloudlake/connections.py").decode()
namespace = {}
# Extract only the deterministic DSN-building logic for a read-only probe.
start = source.index(" def build_dsn")
end = source.index(" `@classmethod`", start)
print("--- build_dsn source slice ---")
print(source[start:end])
PYRepository: pingcap/docs
Length of output: 6419
🏁 Script executed:
#!/bin/bash
set -eu
python3 - <<'PY'
import zipfile
wheel = "/tmp/tidbcloudlake_dbt-1.8.1-py3-none-any.whl"
with zipfile.ZipFile(wheel) as z:
for name in (
"dbt/include/tidbcloudlake/profile_template.yml",
"dbt/adapters/tidbcloudlake/connections.py",
):
print(f"--- {name} ---")
for number, line in enumerate(z.read(name).decode().splitlines(), 1):
if name.endswith("profile_template.yml") or 30 <= number <= 170:
print(f"{number}: {line}")
source = z.read("dbt/adapters/tidbcloudlake/connections.py").decode()
start = source.index(" def build_dsn")
end = source.index(" `@classmethod`", start)
print("--- build_dsn source slice ---")
print(source[start:end])
PYRepository: pingcap/docs
Length of output: 6419
Treat warehouse as an optional profile field.
Keep the warehouse resource in the prerequisites, but do not list the optional warehouse profile value as required.
Committable suggestion
| - The host, database, username, password, and warehouse name for your {{{ .lake }}} connection | |
| - The host, database, username, and password for your {{{ .lake }}} connection; optionally, the warehouse name if you want to select a warehouse explicitly |
Source: Path instructions
|
|
220abb0 to
b1c3769
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
size/S
ti-chi-bot
Bot
added
size/M
[LGTM Timeline notifier]Timeline:
|
ti-chi-bot
Bot
removed
the
do-not-merge/work-in-progress
|
@sdojjy: adding LGTM is restricted to approvers and reviewers in OWNERS files. DetailsIn response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 participants
What is changed, added or deleted? (Required)
tidbcloudlake-dbtadapter while preserving the original document structure.Validation completed:
tidbcloudlake-dbt==1.8.1from public PyPI in a Python 3.11 Docker container and verified that dbt discovered thetidbcloudlakeplugin.tests/functionalsuite fromlake-dbtcommit88dda1438951d27371466c01bb679d45be45ad96against an isolated temporary database: 18 passed and 20 skipped.git diff --check.Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
AI agent involvement
Do your changes match any of the following descriptions?
Summary by CodeRabbit