Skip to content

Add weekly LinkML toolkit compatibility check#248

Open
matentzn wants to merge 1 commit intomainfrom
linkml-compatibility-gha
Open

Add weekly LinkML toolkit compatibility check#248
matentzn wants to merge 1 commit intomainfrom
linkml-compatibility-gha

Conversation

@matentzn
Copy link
Copy Markdown
Contributor

Adds a GitHub Actions workflow that runs every Monday to verify the linkml-model metamodel remains compatible with the latest linkml toolkit.

What it does

  1. Checks out linkml/linkml at HEAD and runs gen-project against meta.yaml
  2. Verifies that key outputs were generated (Python, JSON Schema, OWL, ShEx, GraphQL)
  3. On failure, automatically files a GitHub issue (or comments on an existing one) with the last 100 lines of output

The workflow runs on schedule (Mondays at 10:00 UTC, 1 hour after linkml's own metamodel check) and can be triggered manually via workflow_dispatch.

It intentionally tests against the latest linkml main rather than a pinned version — the goal is early detection of incompatibilities before they reach a release.

Adds a GitHub Actions workflow that runs every Monday to verify the linkml-model metamodel remains compatible with the latest linkml toolkit.
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an automated GitHub Actions workflow to continuously validate that linkml-model’s metamodel can still be processed by the latest linkml toolkit, with scheduled weekly runs and failure-to-issue reporting.

Changes:

  • Introduces a scheduled + manual workflow that checks out linkml/linkml and runs gen-project against linkml_model/model/schema/meta.yaml.
  • Adds a post-run verification step for expected generated artifacts (Python, JSON Schema, OWL, ShEx, GraphQL).
  • On failure, attempts to create/update a GitHub issue with the last 100 lines of captured output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +56 to +65
- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# Python files go in root, others in subdirectories (per GEN_MAP in projectgen.py)
for file in output/meta.py output/jsonschema/meta.schema.json output/owl/meta.owl.ttl output/shex/meta.shex output/graphql/meta.graphql; do
if [ ! -f "$file" ]; then
MISSING="${MISSING}${file}\n"
fi
done
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output checks hard-code output/meta.py, but in this repo the documented generation flow produces Python files via separate gen-python calls (see Makefile), and gen_project_config.yaml explicitly excludes the python generator. As written, this check can either (a) fail even when gen-project succeeded (if using the repo config), or (b) pass while diverging from the repo’s actual build artifacts. Adjust the verification list to match the outputs that are actually expected from the chosen generation command(s).

Copilot uses AI. Check for mistakes.
if: failure()
run: |
ISSUE_TITLE="LinkML toolkit compatibility test failure"
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The failure handler always does tail -n 100 genproject_output.txt, but that file is only created by the gen-project step. If the workflow fails earlier (e.g., checkout/uv sync/Graphviz), this tail will fail and prevent the issue from being created/updated. Consider guarding for a missing log file (fallback message), or narrowing the if: condition so this step runs only when the gen-project step actually executed.

Suggested change
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)
if [ -f genproject_output.txt ]; then
LOG_OUTPUT=$(tail -n 100 genproject_output.txt)
else
LOG_OUTPUT="genproject_output.txt was not created. The workflow likely failed before the gen-project step ran."
fi

Copilot uses AI. Check for mistakes.

jobs:
linkml-compat:
runs-on: ubuntu-latest
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job uses gh issue ... / gh label ... to create and update issues/labels, but the workflow doesn't declare any permissions:. In repositories with restricted default GITHUB_TOKEN permissions, these steps will fail (especially label creation and issue commenting). Consider explicitly setting job permissions (e.g., contents: read and issues: write) so scheduled/manual runs can reliably file/update the tracking issue.

Suggested change
runs-on: ubuntu-latest
runs-on: ubuntu-latest
permissions:
contents: read
issues: write

Copilot uses AI. Check for mistakes.
Comment on lines +23 to +38
uses: actions/checkout@v4.2.2

- name: Check out linkml toolkit
uses: actions/checkout@v4.2.2
with:
repository: linkml/linkml
path: linkml-toolkit

- name: Install uv
uses: astral-sh/setup-uv@v7.2.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true

- name: Set up Python
uses: actions/setup-python@v5.6.0
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow pins older action versions (actions/checkout@v4.2.2, actions/setup-python@v5.6.0, astral-sh/setup-uv@v7.2.0) while the rest of the repo’s workflows use newer pins (e.g., .github/workflows/main.yaml uses actions/checkout@v6.0.2, actions/setup-python@v6.2.0, setup-uv@v7.3.0). To keep maintenance and security updates consistent across CI, align these action versions with the versions already used elsewhere in this repository (or explain why this workflow must differ).

Suggested change
uses: actions/checkout@v4.2.2
- name: Check out linkml toolkit
uses: actions/checkout@v4.2.2
with:
repository: linkml/linkml
path: linkml-toolkit
- name: Install uv
uses: astral-sh/setup-uv@v7.2.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v5.6.0
uses: actions/checkout@v6.0.2
- name: Check out linkml toolkit
uses: actions/checkout@v6.0.2
with:
repository: linkml/linkml
path: linkml-toolkit
- name: Install uv
uses: astral-sh/setup-uv@v7.3.0
with:
version: ${{ env.UV_VERSION }}
enable-cache: true
- name: Set up Python
uses: actions/setup-python@v6.2.0

Copilot uses AI. Check for mistakes.
Comment on lines +54 to +60
uv run gen-project ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt

- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# Python files go in root, others in subdirectories (per GEN_MAP in projectgen.py)
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gen-project is invoked without the repo’s gen_project_config.yaml, but this repository’s Makefile consistently runs gen-project with that config (which also excludes some generators like python and markdown). Running without the config may introduce failures unrelated to compatibility (extra generators, different output layout) and makes the output verification brittle. Consider using the same config file as the repo’s normal generation path, and if you want to validate Python output too, run the repo’s gen-python --genmeta step separately rather than assuming gen-project will emit meta.py.

Suggested change
uv run gen-project ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt
- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# Python files go in root, others in subdirectories (per GEN_MAP in projectgen.py)
uv run gen-project --config-file ../gen_project_config.yaml ../linkml_model/model/schema/meta.yaml -d ../output 2>&1 | tee ../genproject_output.txt
- name: Run gen-python --genmeta against metamodel
id: run_genpython
working-directory: linkml-toolkit
run: |
uv run gen-python --genmeta ../linkml_model/model/schema/meta.yaml > ../output/meta.py
- name: Verify key outputs were generated
id: verify_outputs
run: |
MISSING=""
# meta.py is generated separately via gen-python --genmeta; the remaining files come from gen-project.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants