From 0ea33ec157355a055060693e39f142e0327a8454 Mon Sep 17 00:00:00 2001 From: kiranvuyurru Date: Thu, 11 Jun 2026 08:54:20 -0600 Subject: [PATCH 1/2] PMM-15135 updated test skill Updated the manual test skill to use mcp and execute the manual steps --- .agents/workflows/bugReport.md | 128 +++++---------------- .agents/workflows/mcpRules.md | 29 +++-- .agents/workflows/pmmLogin.md | 33 ++++-- .agents/workflows/report.md | 48 ++++---- .cursor/skills/pmm-manual-test/SKILL.md | 141 +++++++++++++++++++----- AGENTS.md | 6 + 6 files changed, 202 insertions(+), 183 deletions(-) diff --git a/.agents/workflows/bugReport.md b/.agents/workflows/bugReport.md index db5840768..c8940ff19 100644 --- a/.agents/workflows/bugReport.md +++ b/.agents/workflows/bugReport.md @@ -2,119 +2,49 @@ description: Evidence-first workflow for turning logs, screenshots, and brief context into a concise Jira-ready bug report --- -# Purpose +# Rules -Use this workflow to convert bug evidence into a compact report. - -Allowed input: -- user context -- screenshots -- logs -- trace snippets -- console errors -- network errors -- directly relevant code references - -Reporting only: -- Do not propose fixes or a test plan. -- Do not invent root cause, environment, or steps. - -# Core Rules - -- Report only confirmed facts from prompt, screenshots, logs, CLI, MCP, or code. -- Put anything likely but unproven in `Unverified`. -- Prefer concrete evidence over interpretation. -- Summarize noisy logs to the smallest useful line. -- Separate `Actual` from `Expected`. -- If details are missing, still write the report and note the gap in `Repro Notes`. -- Ask follow-up questions only if the report would otherwise be misleading. +- Report only confirmed facts (prompt, screenshots, logs, CLI, MCP, code). No fixes, test plans, or unproven root causes. +- Unproven-but-likely details → `Unverified`. Missing details → note in `Repro Notes`, don't block. +- Keep evidence minimal: exact error text, failed requests, stack lines, short log excerpts. +- Ask follow-ups only if the report would otherwise be misleading. # Analysis Order -1. Identify the affected feature, page, API, or component. -2. Classify the issue: UI, functional, backend/API, performance/stability, access, or install/upgrade/config. -3. Extract only proof: visible error text, broken UI state, timestamp, failed request, stack line, or log line. +1. Identify affected feature / page / API / component. +2. Classify: UI | functional | backend/API | performance/stability | access | install/upgrade/config. +3. Extract proof: error text, broken UI state, timestamp, failed request, stack/log line. 4. State user impact. -5. Infer the shortest credible repro steps from provided evidence only. +5. Infer shortest credible repro steps from evidence only. 6. State expected behavior from context and normal product behavior. -7. Set severity conservatively. - -# Input Rules - -## Screenshots - -- Capture visible page names, URLs, controls, and exact error text. -- Describe only what is visible. -- Mark an element as missing only when the intended element is clear. -- Reference the screenshot in `Evidence`. - -## Logs - -- Keep only the smallest lines that prove failure. -- Prefer explicit errors, status codes, timeouts, connection failures, and service names. -- Do not paste large log blocks. -- Move suspected root cause to `Unverified`. - -## Minimal Context +7. Set severity conservatively (lowest that matches confirmed impact). -- Do not block on missing details. -- Provide a constrained title, minimal steps, explicit evidence, and note gaps in `Repro Notes`. +# Title format -# Severity +`[Component] Short factual description` — no root-cause guesses. -Choose the lowest severity that matches confirmed impact. - -- `Critical`: data loss, security issue, complete outage, blocked install/upgrade, or unusable core workflow with no workaround -- `Urgent`: primary workflow broken, repeated hard failure, or severe impact with impractical workaround -- `High`: strong functional impact, incorrect results, partial workflow breakage, or major usability degradation -- `Medium`: limited functional issue, secondary workflow problem, or contained UI/behavior issue -- `Low`: cosmetic or low-impact polish issue - -Default to `Medium` if impact is unclear. - -# Title - -- Format: `[Component] Short factual description` -- Do not include root-cause guesses. - -Examples: - `[Dashboards] Filter panel stays blank after reload` - `[Backup] Restore action returns 500` -# Output Format - -Provide exactly these sections: +# Output - **Title**: `[Component] Short bug description` -- **Summary**: 2-3 short lines covering the defect and impact -- **Severity**: `Critical|Urgent|High|Medium|Low` -- **Steps to Reproduce**: - 1. Step one - 2. Step two -- **Actual**: Confirmed observed behavior -- **Expected**: Intended user-visible behavior -- **Evidence**: Short proof bullets -- **Unverified**: Only if needed -- **Repro Notes**: Missing details, consistency, timing, role, environment, or workaround notes - -# Evidence Rules - -Keep `Evidence` compact. - -Examples: -- `Screenshot: settings-page-error.png shows "Failed to load user"` -- `Log: pmm-managed timeout waiting for inventory sync` -- `API: GET /v1/inventory/nodes returned 500` -- `Console: TypeError on dashboard load` - -Avoid full stack traces, long logs, broad narratives, and unsupported root-cause claims. +- **Summary**: 2-3 lines — defect + impact +- **Severity**: `Critical|Urgent|High|Medium|Low`. Default `Medium` when impact is unclear. +- **Component**: affected PMM area +- **Steps to Reproduce**: numbered +- **Actual**: confirmed observed behavior (evidence-backed) +- **Expected**: intended user-visible behavior (not a fix) +- **Evidence**: short proof bullets +- **Unverified**: _(only if needed)_ +- **Repro Notes**: missing details, consistency, timing, role, environment, workaround # Final Check -- Title names the affected component. -- Severity matches confirmed impact. -- Steps are reproducible or clearly partial. -- `Actual` is supported by evidence. -- `Expected` describes behavior, not a fix. -- Uncertain points appear only in `Unverified`. -- Keep the report short enough to paste into Jira without cleanup. +- [ ] Title names the affected component +- [ ] Severity matches confirmed impact +- [ ] Steps are reproducible or clearly partial +- [ ] `Actual` is supported by evidence +- [ ] `Expected` describes behavior, not a fix +- [ ] Uncertain points appear only in `Unverified` +- [ ] Short enough to paste into Jira without cleanup diff --git a/.agents/workflows/mcpRules.md b/.agents/workflows/mcpRules.md index d05f337bd..d48220f6e 100644 --- a/.agents/workflows/mcpRules.md +++ b/.agents/workflows/mcpRules.md @@ -2,28 +2,27 @@ description: Execution, browser, and locator strategy rules for PMM Playwright tasks --- -# Execution & Context -- Execute directly; DO NOT output plans or reasoning unless blocked. -- Search first, read smallest possible files second (prefer nearby tests, POMs, helpers). -- NO broad repo scans. +# Execution +- Execute directly. No plans or reasoning unless blocked. +- Search first; read smallest possible files (prefer nearby tests, POMs, helpers). No broad repo scans. # API Setup -- Use PMM REST API to build test state; use UI ONLY to verify behavior. -- Check `apiIndex.md` first for routes. Open `pmmApi.json` ONLY for precise schema definitions. +- Use PMM REST API to build state; UI only to verify behavior. +- Check `apiIndex.md` first. Open `pmmApi.json` only for precise schema definitions. # Login -- Follow `pmmLogin.md`. -- Basic Auth headers ONLY. -- NEVER use the UI login form or `/graph/login` (except when debugging auth). +- Follow `pmmLogin.md`. Basic Auth headers only. +- Never use the UI login form or `/graph/login` (except when debugging auth). -# Browser interactions -- Batch multiple actions (fills, clicks, asserts) into one `mcp_playwright_browser_run_code` call. -- Use `browser_wait_for` or Playwright assertions. NO manual sleeps. +# Browser +- Batch multiple actions (fills, clicks, asserts) into one `browser_run_code` call. +- Use `browser_wait_for` or Playwright assertions. No manual sleeps. - Do not reload or re-authenticate unless state explicitly demands it. -- If blocked, `browser_snapshot` once, stop, and reassess. +- If blocked: `browser_snapshot` once, stop, reassess. +- After navigation: verify expected route and visible page-specific content/controls. HTTP `200` alone is not a valid assertion. # Locators & POM - Priority: `getByTestId` > `getByRole` > `getByLabel` > `getByPlaceholder`. - Reuse existing POM locators. -- If missing: do EXACTLY ONE DOM discovery pass, then update the POM. NEVER re-evaluate the same page's DOM. -- AVOID: `nth()`, `first()`, `last()`, dynamic XPath, and text-heavy selectors. +- If missing: one DOM discovery pass → update POM. Never re-evaluate the same page's DOM. +- Avoid: `nth()`, `first()`, `last()`, dynamic XPath, text-heavy selectors. diff --git a/.agents/workflows/pmmLogin.md b/.agents/workflows/pmmLogin.md index 8965bf6a5..b476a7829 100644 --- a/.agents/workflows/pmmLogin.md +++ b/.agents/workflows/pmmLogin.md @@ -2,27 +2,36 @@ description: PMM Login using basic Auth headers --- -- NEVER use UI login form. -- Use Basic Auth header via `mcp_playwright_browser_run_code`. -- DO NOT pass plain credentials in the URL string. +**Resolve credentials by context:** +- From `pmm-manual-test`: use user-provided PMM instance URL, `ADMIN_USERNAME=admin`, `ADMIN_PASSWORD=pmm3admin!` +- Standalone: use root `.env` — `PMM_UI_URL`, `ADMIN_USERNAME` (default `admin`), `ADMIN_PASSWORD` + +**``** = base64 of `ADMIN_USERNAME:ADMIN_PASSWORD` computed outside MCP (no `Buffer`, no `btoa`, no credentials in URL). ```javascript async (page) => { - const base = "https://127.0.0.1"; - const auth = Buffer.from("admin:admin").toString("base64"); + const base = ""; + const auth = ""; await page.context().setExtraHTTPHeaders({ Authorization: `Basic ${auth}` }); - - await page.route("**/api/user/auth-tokens/rotate", async (route) => { - await route.fulfill({ - body: "{}", + await page.route("**/api/user/auth-tokens/rotate", (route) => + route.fulfill({ body: "{}", contentType: "application/json", status: 200 }), + ); + await page.route("**/v1/users/me", (route) => + route.fulfill({ + body: JSON.stringify({ + alerting_tour_completed: true, + product_tour_completed: true, + snoozed_pmm_version: "", + user_id: 1, + }), contentType: "application/json", status: 200, - }); - }); + }), + ); await page.goto(`${base}/pmm-ui/help`); }; ``` -- Reply `Done` immediately after logging in. NO extra info. +Reply `Done`. diff --git a/.agents/workflows/report.md b/.agents/workflows/report.md index 9a010ca3b..4d1d0103b 100644 --- a/.agents/workflows/report.md +++ b/.agents/workflows/report.md @@ -2,60 +2,52 @@ description: Playwright handoff report --- -# Core Rules +# Rules -- Output ONLY a structured exploration report using a template. -- NO speculative plans, raw DOM dumps, reasoning, or Playwright code generation. -- If blocked execution: Stop immediately, report blocker. -- PREFER `[API]` for state creation. Use `[Auth]` only for session state. Use `[Nav]` for URL routing. Use `[UI]` inside pages. +- Output ONLY a structured report using the template below. +- No speculative plans, raw DOM dumps, reasoning, or Playwright code generation. +- If blocked: stop immediately, report blocker. +- Prefer `[API]` for state creation. Use `[Auth]` for session state. `[Nav]` for URL routing. `[UI]` inside pages. -# Output Format +# Output Template ## Steps -- Ordered execution trace (e.g. `1. [API] Stop session`, `2. [Nav] Admin path`). +Ordered execution trace — e.g. `1. [API] Stop session`, `2. [Nav] /pmm-ui/inventory` ## Key Locators -- Element | Purpose | Locator +| Element | Purpose | Locator | ## State (Before & After) - URL -- Values/Selections -- Visible Errors/Messages -- _(Log changed/relevant fields ONLY)_ +- Values / Selections +- Visible Errors / Messages + _(changed or relevant fields only)_ ## Assertions -- `Verify:` Behavior that happened. -- `Assert:` Playwright check needed. +- `Verify:` behavior that happened +- `Assert:` Playwright check needed -## Stability notes +## Stability Notes -- Log: dynamic IDs, long waits, flake risk, useful traces. +Dynamic IDs, long waits, flake risk, useful traces. --- -# Templates (Pick One) +# Templates ## POM Discovery -- **Steps:** `[Nav]`, `[UI]` -- **Locators:** New elements. -- **State:** Before/After visual limits. -- **Stability:** Selectors, loads. +Steps: `[Nav]`, `[UI]` · Locators: new elements · Stability: selectors, loads ## API Setup -- **Steps:** `[API]`, then `[UI]/[Nav]` checks. -- **Locators:** Verification elements. -- **Assertion:** End-state expectation. -- **Stability:** Race conditions. +Steps: `[API]` then `[UI]/[Nav]` checks · Locators: verification elements · Assertion: end-state · Stability: race conditions ## Bug Investigation -- **Steps:** `[Auth]`, `[Nav]`, `[UI]` -- **State:** Before/After (highlight explicit error!). -- **Stability:** Intermittent triggers. -- **IMPORTANT:** If a bug is confirmed, use `bugReport.md` (`/bugReport`) to format the report. +Steps: `[Auth]`, `[Nav]`, `[UI]` · State: before/after (highlight error) · Stability: intermittent triggers +→ If bug confirmed: use `bugReport.md` (`/bugReport`) to format the report. diff --git a/.cursor/skills/pmm-manual-test/SKILL.md b/.cursor/skills/pmm-manual-test/SKILL.md index 9b47e8f0b..9713bafed 100644 --- a/.cursor/skills/pmm-manual-test/SKILL.md +++ b/.cursor/skills/pmm-manual-test/SKILL.md @@ -13,12 +13,12 @@ Start when the user asks for help with **manual testing**. **Always ask for the PMM repos are expected as **siblings** under one parent directory (repos root). This skill ships in **pmm-qa**; resolve paths from the pmm-qa clone. -| Repo | Directory (under repos root) | Remote | -|------|------------------------------|--------| -| pmm-qa | `pmm-qa` | `percona/pmm-qa` | -| pmm | `pmm` | `percona/pmm` | -| grafana | `grafana` | `percona/grafana` | -| jenkins-pipelines | `jenkins-pipelines` | `Percona-Lab/jenkins-pipelines` | +| Repo | Directory (under repos root) | Remote | +| ----------------- | ---------------------------- | ------------------------------- | +| pmm-qa | `pmm-qa` | `percona/pmm-qa` | +| pmm | `pmm` | `percona/pmm` | +| grafana | `grafana` | `percona/grafana` | +| jenkins-pipelines | `jenkins-pipelines` | `Percona-Lab/jenkins-pipelines` | **pmm-submodules** — read PR comments and checks via `gh` only, never clone. @@ -36,13 +36,13 @@ PMM_QA_ROOT="$(git -C pmm-qa rev-parse --show-toplevel)" REPOS_ROOT="$(dirname "$PMM_QA_ROOT")" ``` -| Variable | Example path | -|----------|----------------| -| `$ReposRoot` / `$REPOS_ROOT` | `~/vscodeProjects/PMM` | -| pmm | `$ReposRoot/pmm` | -| grafana | `$ReposRoot/grafana` | -| jenkins-pipelines | `$ReposRoot/jenkins-pipelines` | -| pmm-qa | `$ReposRoot/pmm-qa` | +| Variable | Example path | +| ---------------------------- | ------------------------------ | +| `$ReposRoot` / `$REPOS_ROOT` | `~/vscodeProjects/PMM` | +| pmm | `$ReposRoot/pmm` | +| grafana | `$ReposRoot/grafana` | +| jenkins-pipelines | `$ReposRoot/jenkins-pipelines` | +| pmm-qa | `$ReposRoot/pmm-qa` | ### Clone if missing @@ -74,18 +74,20 @@ Then `git fetch` / `git pull --ff-only` in each repo that exists. ``` - [ ] 1. Get Jira ticket key from user -- [ ] 2. Read ticket via Atlassian MCP (summary, description, linked PRs, acceptance criteria) -- [ ] 3. Identify affected repos (pmm, grafana, or both) +- [ ] 2. Read ticket via Atlassian MCP (summary, description, comments, linked PRs, acceptance criteria) +- [ ] 3. Identify affected repos (pmm, grafana, or both) and read PR body, diff, issue comments, review summaries, and inline review comments - [ ] 4. Resolve repos root; clone missing siblings; update local repos (git pull) - [ ] 5. Find pmm-submodules PR and latest JNKPercona build comment — older comments are not valid - [ ] 6. Analyze FB Tests from latest FB build (`gh pr checks` on pmm-submodules PR) +- [ ] 6a. Compare ticket context vs PR context; if mismatch exists, inform the user before Jenkins setup - [ ] 7. Choose Jenkins job (pmm3-aws-staging-start vs pmm3-deploy-services) - [ ] 8. Determine required job parameters to setup instance based on ticket scope and PR diffs - [ ] 9. Build complete parambuild URL (all relevant params, not just DOCKER_VERSION + CLIENT_VERSION) - [ ] 10. Review PR diffs — do not trust "How to test" blindly - [ ] 11. Write test instructions in chat (adapt using FB test failures where relevant) - [ ] 12. Screenshot FB Tests Actions run (playwright-cli) and update Jira FB test screenshots field -- [ ] 13. Open Jenkins parambuild URL in browser +- [ ] 13. Open Jenkins parambuild URL in browser, then immediately ask the user to provide the PMM instance URL once Jenkins finishes provisioning +- [ ] 14. When the user provides the PMM instance URL, execute the generated manual test steps through MCP ``` ## Step 1: Get Jira ticket key @@ -97,10 +99,13 @@ Ask if missing. Proceed once you have the key (e.g. `PMM-14915`). Use **Atlassian MCP** (`jira_get_issue`, `jira_get_issue_development_info`): - Summary, description, QA notes, acceptance criteria +- Ticket comments, newest first when possible - Existing **How to test** (`customfield_10083`) and **FB test screenshots** (`customfield_10492`) - Development panel: linked GitHub PRs/branches - Environment or setup hints +Treat ticket comments as first-class testing input. Newer ticket comments may supersede stale description text, acceptance criteria, or existing **How to test** content. Extract late scope changes, retest requests, QA clarifications, feature flags, setup hints, known limitations, related links, and readiness/blocker notes. + **Setup is ticket-specific.** Pay attention to: - Which subsystems change (server, UI/grafana, clients, telemetry, advisors, etc.) @@ -108,7 +113,7 @@ Use **Atlassian MCP** (`jira_get_issue`, `jira_get_issue_development_info`): - Required **PMM server environment variables** (`DOCKER_ENV_VARIABLE`) — job defaults are often wrong for the test (e.g. default `PMM_ENABLE_TELEMETRY=0`) - PMM Settings changes the user must apply after provisioning -## Step 3: Identify affected repos +## Step 3: Identify affected repos and read PR discussion | Change area | Repo | |-------------|------| @@ -118,6 +123,29 @@ Use **Atlassian MCP** (`jira_get_issue`, `jira_get_issue_development_info`): Grafana-only tickets still need an FB **server** image from pmm-submodules. +For every linked or discovered PR in `percona/pmm`, `percona/grafana`, and `Percona-Lab/pmm-submodules`, read the PR body, implementation diff, issue comments, review summaries, and inline review comments. + +```powershell +gh pr view -R owner/repo --json title,body,url,state,mergeStateStatus,comments,reviews,reviewDecision,files +gh pr diff -R owner/repo +``` + +Use GitHub CLI for all PR context. Start with `gh pr view` and `gh pr diff`. +Use `gh api` only when `gh pr view` does not expose enough detail, especially for inline review comments with file path and line context or when complete discussion details are needed: + +```powershell +gh api repos/OWNER/REPO/issues//comments ` + --jq '[.[] | {created_at, author: .user.login, body}]' + +gh api repos/OWNER/REPO/pulls//comments ` + --jq '[.[] | {created_at, author: .user.login, path, line, body}]' + +gh api repos/OWNER/REPO/pulls//reviews ` + --jq '[.[] | {submitted_at, author: .user.login, state, body}]' +``` + +Use review comments to understand implementation constraints, unresolved concerns, setup requirements, and reviewer-requested behavior changes. + ## Step 4: Update local repos Resolve `$ReposRoot` (see above). Clone any missing sibling, then pull: @@ -191,6 +219,28 @@ gh pr checks -R Percona-Lab/pmm-submodules Full reference: [fb-tests.md](fb-tests.md) +## Pre-Jenkins mismatch gate + +Before choosing, generating, or opening any Jenkins job, compare ticket context against PR context: + +- Ticket context: summary, description, acceptance criteria, QA notes, existing **How to test**, and ticket comments +- PR context: PR body, implementation diff, issue comments, review summaries, inline review comments, and review state + +Check for mismatches in implemented behavior, setup needs, PMM server env vars, DB topology, feature flags, PMM Settings, readiness, and test scope. + +If any mismatch can affect setup or testing, stop and inform the user before proceeding: + +```markdown +Blocked before setup: ticket/PR mismatch + +- Ticket says: +- PR/review says or implements: +- Impact: +- Need: +``` + +If no mismatch exists, continue with Jenkins job selection and parameter planning. + ## Step 7: Choose Jenkins job | Scenario | Job | @@ -212,7 +262,7 @@ See [jenkins-jobs.md](jenkins-jobs.md) for all parameters. Groovy source: `$Repo ## Step 8: Determine required job parameters -Derive **every** param that differs from defaults. The user should not need to edit the Jenkins form. +Derive **every** param that differs from defaults. The user should not need to edit the Jenkins form. Use ticket comments and PR review comments as inputs, not just the ticket description and PR diff. Include **all** params in the `parambuild` URL for the chosen job — not only `DOCKER_VERSION` + `CLIENT_VERSION`. Walk through the full list below; keep defaults only when they are correct for this ticket. @@ -318,7 +368,7 @@ Do **not** trust PR "How to test" or Jira QA bullets without verifying code. 1. Read diffs in `percona/pmm` and/or `percona/grafana` 2. Map changes to concrete UI/API/network steps -3. Cross-check with FB test failures marked **relevant** +3. Cross-check with ticket comments, PR comments/reviews, and FB test failures marked **relevant** 4. Note post-provision steps (PMM Settings toggles, etc.) ## Step 11: Write test instructions @@ -327,9 +377,10 @@ Post in chat **before** Jira update and browser: 1. Job chosen and why 2. Params set and why -3. FB test summary (failures relevant to this ticket) -4. Provision → configure → exercise → expected result -5. DevTools / logs / PMM Settings checks +3. Ticket/PR mismatch result: either `No ticket/PR mismatch found` or the mismatch resolution used before setup +4. FB test summary (failures relevant to this ticket) +5. Provision → configure → exercise → expected result +6. DevTools / logs / PMM Settings checks ## Step 12: Screenshot FB Tests run → Jira @@ -365,6 +416,35 @@ Details: [fb-tests.md](fb-tests.md) Start-Process "" ``` +## Step 14: Execute generated manual test steps through MCP + +Do **not** wait for or poll Jenkins after opening the parambuild URL. Immediately ask the user: + +```text +Please provide the PMM instance URL once Jenkins finishes provisioning. +``` + +When the user provides the PMM instance URL, execute the generated manual test steps. Use default credentials: + +- Username: `admin` +- Password: `pmm3admin!` + +Do **not** generate an `Execution Handoff`. Execute the already generated manual test steps exactly as the test scope; do not invent or expand coverage beyond those steps unless the user explicitly asks. + +Before MCP execution, read and follow these workflow files. Do this quietly; do not narrate workflow-file reads, auth setup, or browser setup: + +1. Read/use `.agents/workflows/pmmLogin.md` first and immediately log in with Basic Auth using the user-provided PMM instance URL and default credentials. +2. After login, read/use `.agents/workflows/mcpRules.md` for execution behavior. +3. Use `.agents/workflows/apiIndex.md` only if API setup or API checks are needed. + +Prefer API setup first when possible, and use UI only for verification. When navigating between pages, verify the actual expected page content, route, or page-specific controls. Do not treat HTTP `200` as proof that the correct page loaded. + +Final MCP result must be short and use one of these formats: + +- `Fixed: ` when expected behavior passes. +- `Not fixed: ` when the issue reproduces. +- `Blocked: ` when execution cannot complete. + ## gh CLI quick reference | Task | Command | @@ -377,13 +457,16 @@ Start-Process "" ## Done criteria -1. Jira read; scope understood -2. Latest JNKPercona build comment used -3. FB Tests analyzed; relevant failures reflected in manual plan -4. Correct job + **complete** parambuild URL -5. Test instructions posted in chat -6. Jira `FB test screenshots` updated — screenshot only if all checks passed (when user approves) -7. Browser opened to parambuild URL +1. Jira read, including ticket comments; scope understood +2. PR body, diff, issue comments, review summaries, and inline review comments read +3. Ticket/PR mismatch gate completed before Jenkins setup; user informed if any mismatch exists +4. Latest JNKPercona build comment used +5. FB Tests analyzed; relevant failures reflected in manual plan +6. Correct job + **complete** parambuild URL +7. Test instructions posted in chat +8. Jira `FB test screenshots` updated — screenshot only if all checks passed (when user approves) +9. Browser opened to parambuild URL, then user asked to provide the PMM instance URL once Jenkins finishes provisioning +10. User-provided PMM instance URL collected, generated manual test steps executed through `.agents/workflows`, and final result reported as `Fixed:`, `Not fixed:`, or `Blocked:` ## Additional resources diff --git a/AGENTS.md b/AGENTS.md index 2c8b740e2..090834346 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,6 +12,12 @@ If any apply, update the relevant section of this file. Do **not** update for ro This file is the **single authoritative entry point** for AI agents working with pmm-qa. It owns the product overview, repository map, and a consolidated per-suite quick-reference. Detailed suite docs live next to the suite's code (`README.md` / config file) — links are in the [Repository Map](#repository-map). +## Chat Output Preference + +- Keep routine execution quiet to save tokens: do not narrate file reads, env parsing, token generation, browser setup, or command prep. +- For clear execution requests, run the task and reply only with the result, for example `Done`, `Blocked: `, or a compact summary when files changed. +- Speak up only for approval requests, blockers, destructive-risk warnings, or when the user explicitly asks for status/explanation. + --- ## Repository Overview From c643d6f786f95ba945bdc78f138c5a902407c0bc Mon Sep 17 00:00:00 2001 From: travagliad <215686151+travagliad@users.noreply.github.com> Date: Thu, 11 Jun 2026 18:49:35 +0100 Subject: [PATCH 2/2] PMM-15135: Fix FB test Jira field update instructions Document REST API + ADF for customfield_10492. Do not use Jira UI or wiki markup. --- .cursor/skills/pmm-manual-test/fb-tests.md | 55 +++++----------------- 1 file changed, 12 insertions(+), 43 deletions(-) diff --git a/.cursor/skills/pmm-manual-test/fb-tests.md b/.cursor/skills/pmm-manual-test/fb-tests.md index 8554f8be7..5e40e46b0 100644 --- a/.cursor/skills/pmm-manual-test/fb-tests.md +++ b/.cursor/skills/pmm-manual-test/fb-tests.md @@ -117,54 +117,23 @@ Save screenshots to a temp path the user can find (e.g. workspace root or `%TEMP | **FB test screenshots** | `customfield_10492` | FB test analysis summary + screenshot reference | | **How to test** | `customfield_10083` | Manual test steps (adapt using FB failures + PR diff) | -### Update with Atlassian MCP +### How to update `customfield_10492` (verified on PMM-15074) -**All checks passed** — attach screenshot and update fields in one call: +**Do not use:** Jira web UI / Playwright (agent browser is not logged in). **Do not** send wiki markdown (`!image.png|width=900!`) or plain text — Jira returns *Operation value must be an Atlassian Document*. +**Use:** Jira REST API v3, or Atlassian MCP only if it sends **ADF** for this field. Credentials: `JIRA_USERNAME`, `JIRA_API_TOKEN`, `JIRA_URL` (or `mcp-atlassian` env in `~/.cursor/mcp.json`). -```json -jira_update_issue( - issue_key: "PMM-14915", - fields: "{\"customfield_10492\": \"## FB Tests — PR-4376 (all green)\\n\\n**Run:** https://github.com/Percona-Lab/pmm-submodules/actions/runs/27009345670\\n\\n!fb-test-PMM-14915-checks.png|width=900!\"}", - attachments: "/path/to/fb-test-PMM-14915-checks.png" -) -``` - -After attachment upload, Jira may render images inline: `!fb-test-PMM-14915-checks.png|width=900!` - -**Any check failed** — text only, no attachment: - -```json -jira_update_issue( - issue_key: "PMM-14915", - fields: "{\"customfield_10492\": \"## FB Tests — PR-4376 (failures — no screenshot)\\n\\n**Run:** https://github.com/Percona-Lab/pmm-submodules/actions/runs/27009345670\\n\\n**Failed:** @rta UI tests, @pmm-ps-integration UI tests, CLI tests pmm-server container\\n\\n**Relevant to ticket:** none (flaky / out of scope)\\n\\n_Screenshot pending — waiting for all-green FB build._\"}" -) -``` - -Update **How to test** separately when manual steps are finalized: +1. Upload the screenshot: -```json -jira_update_issue( - issue_key: "PMM-14915", - fields: "{\"customfield_10083\": \"1. Provision FB staging (link in comment)...\\n2. DevTools: no /v1/ui-events/Store calls...\"}" -) +```bash +curl -sS -u "$JIRA_USERNAME:$JIRA_API_TOKEN" \ + -H "X-Atlassian-Token: no-check" \ + -F "file=@fb-test-PMM-XXXX-checks.png" \ + "$JIRA_URL/rest/api/3/issue/PMM-XXXX/attachments" ``` -**Ask the user before writing to Jira** if they did not explicitly request the update in this session. +Note the numeric `id` from the response. -### FB test screenshots field template +2. `PUT $JIRA_URL/rest/api/3/issue/PMM-XXXX` with `customfield_10492` as ADF (`type: doc`). Embed the image with `mediaSingle` → `media` → `type: external` → `url` = `$JIRA_URL/rest/api/3/attachment/content/`. -```markdown -## FB Tests — PR- - -**FB Tests run:** https://github.com/Percona-Lab/pmm-submodules/actions/runs/ - -### Failures (latest) -- `@suite` — [relevant|flaky|out of scope]: - -### Passed (ticket-relevant) -- list suites that cover the change area - -### Screenshot -!fb-test--checks.png|width=900! -``` +**Ask the user before writing to Jira** unless they explicitly requested the update in this session.