Enhance Quickbase documentation, validation, and query features

.vscode/launch.json

@@ -1,13 +1,16 @@
-// A launch configuration that launches the extension inside a new window
+// A launch configuration that launches only the extension under development.
 {
-    "version": "0.1.0",
+    "version": "0.2.0",
     "configurations": [
         {
             "name": "Launch Extension",
             "type": "extensionHost",
             "request": "launch",
             "runtimeExecutable": "${execPath}",
-            "args": ["--extensionDevelopmentPath=${workspaceRoot}" ]
+            "args": [
+                "--disable-extensions",
+                "--extensionDevelopmentPath=${workspaceFolder}"
+            ]
         }
     ]
-}
+}

CHANGELOG.md

@@ -5,4 +5,24 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
 ## [0.0.1] - 2017-06-23
-- Initial release
+- Initial release
+
+## [0.1.0] - 2026-03-18
+- Added highlighting for Quickbase formula-query functions such as `GetRecord()`, `GetRecords()`, `GetFieldValues()`, `Size()`, and `SumValues()`
+- Added support for Quickbase API query-string operators including `EX`, `GT`, `HAS`, `WC`, and related operators
+- Expanded formula syntax coverage for modern variable types and common operators like `=`, `!=`, `<>`, and `&`
+- Updated README and extension metadata to reflect formula and query-string support
+
+## [Unreleased]
+- Added live Quickbase diagnostics for duplicate variables, invalid variable names, malformed query strings, regex-pattern literal rules, structural expression errors, and baseline type/function validation
+- Query field IDs such as `'6'` or `6` inside API query blocks now use a variable-style scope instead of a field-name scope
+- Variable declarations now scope `var`, the declared data type, and the variable name independently for more consistent coloring, with `var` using a dedicated keyword scope
+- Bracketed field and table references such as `[_DBID_PROJECTS]` now use the same scope as variable names
+- Added PowerShell/Pester grammar regression tests for query field IDs and variable declarations
+- Added sample-based grammar snapshot tests that run against representative `.quickbase` fixtures
+- Added hover documentation for formula-query functions, query operators, variable declarations, variable types, and bracketed references
+- Added grammar and validation support for bare query values such as `today` and `_curuser_`
+
+
+
+

README.md

@@ -1,19 +1,63 @@
-# Syntax Highlighter for Quickbase Formulas
+# Quickbase Formula & Query Tools
 
-Highlights database fields, variables, operators, and built in functions.
+VS Code language support for Quickbase formulas and query expressions. This extension adds syntax highlighting, snippets, hover documentation, and live validation for `.quickbase` files so formula work is easier to read, write, and review.
 
-![screenshot](https://raw.githubusercontent.com/jdklub/vscode-quickbase-formula/master/images/screenshot.png)
+![screenshot](reference/example_image.png)
 
 ## Requirements
 
-Files must have a .quickbase extension
+Use the `.quickbase` file extension for Quickbase formulas, formula-query expressions, and standalone query text.
 
-## Known Issues
+## Features
 
-Very little testing has been done.
+- Syntax highlighting for Quickbase formulas, comments, operators, variables, variable declarations, bracketed field and table references, and modern Quickbase functions
+- Query-aware highlighting for API query strings and formula-query expressions, including operators like `EX`, `GT`, `TV`, `WC`, logical joiners like `AND` and `OR`, and special values such as `'today'` and `'_curuser_'`
+- Snippets for Quickbase functions, variables, and common authoring patterns
+- Live diagnostics for structural formula errors, malformed query expressions, duplicate variables, invalid variable names, unknown variable types, unknown functions, regex-pattern issues, and baseline argument/type mismatches
+- Hover documentation for formula-query functions, query operators, variable declarations, variable types, bracketed references, and special query values
+- Grammar, hover, and validation regression coverage through PowerShell/Pester tests and committed tokenization snapshots
 
-## Release Notes
+## Examples
 
-### 0.0.1
+```quickbase
+var RecordList openProjects = GetRecords("{'6'.EX.'In Progress'}", [_DBID_PROJECTS]);
+var TextList owners = GetFieldValues($openProjects, 8);
 
-Initial version. Good luck!
+Join($owners, "; ")
+```
+
+```quickbase
+var Text todayQuery = "{'13'.EX.'today'}AND{'20'.TV.'_curuser_'}";
+```
+
+## Current Scope
+
+The extension is intentionally lightweight and editor-focused. It does not execute formulas or connect to Quickbase metadata, so field-level inference and exhaustive function semantics are still conservative.
+
+## Development
+
+Reference notes gathered from current Quickbase documentation live in `docs/quickbase-documentation-notes.md`.
+
+Run the full test suite from PowerShell:
+
+```powershell
+powershell -ExecutionPolicy Bypass -Command "Invoke-Pester .\tests"
+```
+
+Refresh committed tokenization snapshots after an intentional grammar change:
+
+```powershell
+powershell -ExecutionPolicy Bypass -File .\tests\Invoke-QuickbaseTokenization.ps1 update
+```
+
+## Attribution
+
+This project builds on the original `vscode-quickbase-formula` extension and its contributors.
+
+- Justin Klubnik created the original extension, published the initial Quickbase syntax support, and remains the named copyright holder in `LICENSE.txt`
+- Chris Pliakas contributed later grammar, snippet, scope, and branding updates that expanded the extension's Quickbase coverage
+- Derek Banker led the current modernization work, including updated formula and query support, diagnostics, hover documentation, and automated regression testing
+
+## License
+
+MIT-style license. See `LICENSE.txt`.

docs/quickbase-documentation-notes.md

@@ -0,0 +1,140 @@
+# Quickbase Documentation Notes
+
+Updated: 2026-03-19
+
+Purpose: capture the current official Quickbase formula and query documentation we should use as the source of truth for syntax, validation, and future completion/hover work in this extension.
+
+## Official sources reviewed
+
+- Formula components: https://help.quickbase.com/docs/formula-components
+- Formula variables: https://help.quickbase.com/docs/formula-variables
+- Creating and using application variables: https://help.quickbase.com/variables
+- What are formula queries?: https://help.quickbase.com/docs/what-are-formula-queries
+- Build queries for your formulas: https://help.quickbase.com/docs/build-queries-for-your-formulas
+- Formula queries: field types, type conversions, and variables: https://help.quickbase.com/docs/formula-queries-field-types-type-conversions-and-variables
+- Formula queries and performance: https://help.quickbase.com/docs/formula-queries-and-performance
+- Find field, record IDs, and table aliases: https://help.quickbase.com/docs/find-field-record-ids-and-table-aliases
+- API_DoQuery: https://help.quickbase.com/docs/api-doquery
+- Regex in formulas: https://help.quickbase.com/hc/en-us/articles/32800503697044-Regex-in-formulas
+- Quickbase December 2022 release notes: https://help.quickbase.com/docs/quickbase-december-2022-release-notes
+- Quickbase January 2025 release notes: https://help.quickbase.com/docs/quickbase-january-2025-release-notes
+- Quickbase April 2025 release notes: https://help.quickbase.com/hc/en-us/articles/36386041144852-Quickbase-April-2025-Release-Notes
+- Quickbase May 2025 release notes: https://help.quickbase.com/docs/quickbase-may-2025-release-notes
+- Secure links: https://help.quickbase.com/hc/en-us/articles/29353956195220-Secure-links
+
+## Local seed files reviewed
+
+- `C:\CFS - Derek\Reference Documents\QuickBase Formula Documentation.csv`
+- `C:\CFS - Derek\Refrence Documents\QuickBase Formula Examples.csv`
+
+The documentation CSV still looks useful as a seed list of functions and signatures, but it appears to predate several newer formula-query additions and newer text/query helpers.
+
+## Stable syntax rules to support
+
+### Formula variables
+
+- Declaration format: `var <type> <name> = <formula snippet>;`
+- Official declared types currently documented:
+  - `bool`
+  - `number`
+  - `text`
+  - `textlist`
+  - `date`
+  - `datetime`
+  - `duration`
+  - `timeofday`
+  - `workdate`
+  - `user`
+  - `recordlist`
+- Variables are referenced with `$name`.
+- Variable names must contain letters only. No numbers, spaces, or special characters are documented.
+- `var recordlist` is specifically called out for formula queries.
+
+### Field references and application variables
+
+- Field references use square brackets, for example `[Manager]`.
+- Application variables also use bracket syntax, for example `[Project Start Date]`.
+- Application variables are always treated as text unless they are converted with a formula function such as `ToDate()`.
+
+### Formula queries
+
+- Official formula-query functions:
+  - `GetRecord()`
+  - `GetRecordByUniqueField()`
+  - `GetRecords()`
+  - `GetFieldValues()`
+  - `SumValues()`
+  - `Size()`
+- Query strings in formula-query functions should be enclosed in double quotes.
+- A query block uses `{field.operator.value}` structure.
+- Single quotes around the field ID are optional in formula queries.
+- Comparison operators must be uppercase.
+- Multiple query blocks can be combined with `AND` or `OR`.
+- To inject a field value into a query string, concatenate with `&`, for example:
+  - `"{'5'.EX.'" & [Manager name] & "'}"`
+- For user-field queries, prefer `TV` and wrap the field reference in `UserToID()` or `UserToEmail()`.
+- When querying another table, Quickbase recommends using the table alias instead of the raw DBID.
+
+## Query operators confirmed from API_DoQuery
+
+- `CT`: contains
+- `XCT`: does not contain
+- `WC`: wildcard search with `*` and `?`
+- `HAS`: contains values in List - User or Multi-select Text fields
+- `XHAS`: does not contain values in List - User or Multi-select Text fields
+- `EX`: equals
+- `TV`: true value comparison, including user fields and relationship keys
+- `XTV`: not equal using true-value comparison
+- `XEX`: not equal
+- `SW`: starts with
+- `XSW`: does not start with
+- `BF`: before
+- `OBF`: on or before
+- `AF`: after
+- `OAF`: on or after
+- `IR`: is during a relative date range
+- `XIR`: is not during a relative date range
+- `LT`: less than
+- `LTE`: less than or equal
+- `GT`: greater than
+- `GTE`: greater than or equal
+
+## Type and structure guidance from the docs
+
+- `GetFieldValues()` can be used directly in Formula - Multi-select text fields.
+- `SumValues()` and `Size()` can be used directly in Formula - Numeric fields.
+- To use formula-query results in other field types, Quickbase expects either:
+  - a conversion function such as `ToText()`, or
+  - additional formula logic around the query result.
+- `recordlist` values are intermediate values. They are normally consumed by `GetFieldValues()`, `SumValues()`, `Size()`, or converted to text.
+- Regex support is documented as:
+  - `RegexExtract()`
+  - `RegexReplace()`
+  - `RegexMatch()`
+- Regex patterns must be written directly in the formula call and cannot come from a field or variable.
+
+## Performance guidance worth validating against later
+
+- Prefer table relationships when a simple relationship or summary field already solves the problem.
+- Avoid filtering, sorting, or grouping reports on formula-query fields when possible.
+- Avoid making formula-query fields searchable unless needed.
+- Start queries with the most selective comparison when possible.
+- Prefer exact matches over broader operators like contains or wildcard matching when performance matters.
+
+## Gaps between the local CSV and current docs
+
+The local CSV is still a strong starting point, but the official docs confirm newer capabilities that should drive future validator coverage:
+
+- `GetRecordByUniqueField()` is officially documented in the Quickbase December 2022 release notes and formula-query docs, but it is not present in the local documentation CSV.
+- `RegexExtract()`, `RegexReplace()`, and `RegexMatch()` are officially documented in current help content and January 2025 release notes, but they are not present in the local documentation CSV.
+- `Join()` and `Median()` are called out in the April 2025 release notes and should be treated as current syntax.
+- Aggregate functions on formula-query results such as `Avg()`, `Min()`, `Max()`, and `Median()` are called out in the May 2025 release notes.
+- `SHA256()` and `ToUnixTime()` are evidenced in the Secure links help article and should be treated as live formula syntax.
+- `GetAccessKey()` appears in the snippet catalog, but I did not confirm a current official help-center page for it during this pass, so it should stay marked as "needs source confirmation."
+
+## Useful follow-up work
+
+- Normalize the local CSV into structured JSON so validator signatures can be generated instead of hand-maintained.
+- Add a curated metadata layer for formula-query functions and query operators from the official docs.
+- Tighten variable-name validation to match the documented "letters only" rule.
+- Add diagnostics for unsupported use of formula-query return types in incompatible field contexts once field metadata is available.