Run the trace suite through canonicalisation by jviotti · Pull Request #698 · sourcemeta/blaze

jviotti · 2026-04-10T16:42:07Z

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

github-actions

Benchmark (linux/llvm)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`E2E_Compiler_adaptivecard`	`64302042.44444818` ns/iter	`63624628.39999807` ns/iter	`1.01`
`E2E_Compiler_ansible_meta`	`29180730.249999937` ns/iter	`29578961.374999344` ns/iter	`0.99`
`E2E_Compiler_aws_cdk`	`373609.0501868763` ns/iter	`381169.7147523152` ns/iter	`0.98`
`E2E_Compiler_babelrc`	`2564611.2426469657` ns/iter	`2596322.4444444254` ns/iter	`0.99`
`E2E_Compiler_clang_format`	`18380208.97368378` ns/iter	`18638481.52631638` ns/iter	`0.99`
`E2E_Compiler_cmake_presets`	`25388170.32142861` ns/iter	`25685592.999999933` ns/iter	`0.99`
`E2E_Compiler_code_climate`	`2582441.7919708523` ns/iter	`2583495.18450182` ns/iter	`1.00`
`E2E_Compiler_cql2`	`12664459.509091392` ns/iter	`12828755.381818527` ns/iter	`0.99`
`E2E_Compiler_cspell`	`22621365.09677343` ns/iter	`22800157.806450956` ns/iter	`0.99`
`E2E_Compiler_cypress`	`3877092.6277777557` ns/iter	`3881247.3944444843` ns/iter	`1.00`
`E2E_Compiler_deno`	`7727873.999999826` ns/iter	`7806212.733333951` ns/iter	`0.99`
`E2E_Compiler_dependabot`	`3854171.2333334824` ns/iter	`3919101.401129988` ns/iter	`0.98`
`E2E_Compiler_draft_04`	`2662069.0229009585` ns/iter	`2712041.606177613` ns/iter	`0.98`
`E2E_Compiler_fabric_mod`	`4705006.838926101` ns/iter	`4772867.6870748` ns/iter	`0.99`
`E2E_Compiler_geojson`	`21716516.562499065` ns/iter	`22044185.531250093` ns/iter	`0.99`
`E2E_Compiler_gitpod_configuration`	`4872024.138889072` ns/iter	`4946069.4751774715` ns/iter	`0.99`
`E2E_Compiler_helm_chart_lock`	`704375.922457161` ns/iter	`725563.1821862487` ns/iter	`0.97`
`E2E_Compiler_importmap`	`283505.95091277413` ns/iter	`290627.3796680567` ns/iter	`0.98`
`E2E_Compiler_jasmine`	`1352012.7007721888` ns/iter	`1388594.170923321` ns/iter	`0.97`
`E2E_Compiler_jshintrc`	`4309055.537037112` ns/iter	`4383176.387499788` ns/iter	`0.98`
`E2E_Compiler_jsconfig`	`21485561.125000887` ns/iter	`21692771.37500103` ns/iter	`0.99`
`E2E_Compiler_krakend`	`136509617.1999994` ns/iter	`139644314.20000666` ns/iter	`0.98`
`E2E_Compiler_lazygit`	`32958363.285715785` ns/iter	`33629482.857144296` ns/iter	`0.98`
`E2E_Compiler_lerna`	`1578853.5294117378` ns/iter	`1603123.525229424` ns/iter	`0.98`
`E2E_Compiler_nest_cli`	`7348649.778947746` ns/iter	`7459876.819148895` ns/iter	`0.99`
`E2E_Compiler_omc`	`150009944.99998796` ns/iter	`153434727.00000405` ns/iter	`0.98`
`E2E_Compiler_omnisharp`	`6647998.028571627` ns/iter	`6729016.336538398` ns/iter	`0.99`
`E2E_Compiler_openapi`	`24816186.64285585` ns/iter	`25164731.50000057` ns/iter	`0.99`
`E2E_Compiler_pre_commit_hooks`	`5262938.150375998` ns/iter	`5351217.816793736` ns/iter	`0.98`
`E2E_Compiler_pulumi`	`4029086.200000295` ns/iter	`4059523.2906976566` ns/iter	`0.99`
`E2E_Compiler_semantic_release`	`1565372.1696427932` ns/iter	`1585092.900677127` ns/iter	`0.99`
`E2E_Compiler_stale`	`1540297.416299696` ns/iter	`1560817.5346756326` ns/iter	`0.99`
`E2E_Compiler_stylecop`	`6164921.522123721` ns/iter	`6218012.398229665` ns/iter	`0.99`
`E2E_Compiler_tmuxinator`	`1920192.7863013512` ns/iter	`1944242.3222222438` ns/iter	`0.99`
`E2E_Compiler_ui5`	`46668866.06667049` ns/iter	`46890189.06666812` ns/iter	`1.00`
`E2E_Compiler_ui5_manifest`	`306711013.5000348` ns/iter	`319427967.49999046` ns/iter	`0.96`
`E2E_Compiler_unreal_engine_uproject`	`4552543.077922222` ns/iter	`4612716.3684207965` ns/iter	`0.99`
`E2E_Compiler_vercel`	`12330005.614035543` ns/iter	`12386222.280701786` ns/iter	`1.00`
`E2E_Compiler_yamllint`	`345217.5470851888` ns/iter	`354976.13164556865` ns/iter	`0.97`
`E2E_Evaluator_adaptivecard`	`24947.465423631045` ns/iter	`24752.489027040694` ns/iter	`1.01`
`E2E_Evaluator_ansible_meta`	`226879.75327160305` ns/iter	`226291.00580645408` ns/iter	`1.00`
`E2E_Evaluator_aws_cdk`	`50778.89254227181` ns/iter	`51268.43794339295` ns/iter	`0.99`
`E2E_Evaluator_babelrc`	`114241.80657831354` ns/iter	`116981.25097719574` ns/iter	`0.98`
`E2E_Evaluator_cerebrum_criminal_case`	`1192805.598639577` ns/iter	`1201439.0121317967` ns/iter	`0.99`
`E2E_Evaluator_clang_format`	`123347.88806103192` ns/iter	`123154.71009657954` ns/iter	`1.00`
`E2E_Evaluator_cmake_presets`	`4241327.375757464` ns/iter	`4268980.220858919` ns/iter	`0.99`
`E2E_Evaluator_code_climate`	`173071.11970947505` ns/iter	`175568.91729512066` ns/iter	`0.99`
`E2E_Evaluator_cql2`	`210453.358352813` ns/iter	`231047.25570469355` ns/iter	`0.91`
`E2E_Evaluator_cspell`	`554153.4077287719` ns/iter	`559117.5343573204` ns/iter	`0.99`
`E2E_Evaluator_cypress`	`191853.84319607465` ns/iter	`191972.01608506453` ns/iter	`1.00`
`E2E_Evaluator_deno`	`310319.96597435133` ns/iter	`310159.02141900104` ns/iter	`1.00`
`E2E_Evaluator_dependabot`	`411150.8151408502` ns/iter	`417083.8920522048` ns/iter	`0.99`
`E2E_Evaluator_draft_04`	`7198724.316326985` ns/iter	`7502816.869564709` ns/iter	`0.96`
`E2E_Evaluator_fabric_mod`	`705205.6167176729` ns/iter	`753826.6439394143` ns/iter	`0.94`
`E2E_Evaluator_geojson`	`15298629.297872333` ns/iter	`15353177.934782559` ns/iter	`1.00`
`E2E_Evaluator_gitpod_configuration`	`229493.45817122055` ns/iter	`225756.7529258908` ns/iter	`1.02`
`E2E_Evaluator_helm_chart_lock`	`353071.6576440632` ns/iter	`351751.078934122` ns/iter	`1.00`
`E2E_Evaluator_importmap`	`47167.459516507544` ns/iter	`47683.88812587709` ns/iter	`0.99`
`E2E_Evaluator_jasmine`	`121693.98614718596` ns/iter	`122023.3471045887` ns/iter	`1.00`
`E2E_Evaluator_jshintrc`	`1200907.77873064` ns/iter	`1206749.463121736` ns/iter	`1.00`
`E2E_Evaluator_jsconfig`	`424399.61473431485` ns/iter	`425921.0641646735` ns/iter	`1.00`
`E2E_Evaluator_krakend`	`201951.86535113072` ns/iter	`200059.40361965887` ns/iter	`1.01`
`E2E_Evaluator_lazygit`	`145672.01941748933` ns/iter	`146074.68008386128` ns/iter	`1.00`
`E2E_Evaluator_lerna`	`143764.37454399647` ns/iter	`144553.82095979687` ns/iter	`0.99`
`E2E_Evaluator_nest_cli`	`180390.67141745109` ns/iter	`183923.46480329172` ns/iter	`0.98`
`E2E_Evaluator_omc`	`20048.80393758356` ns/iter	`19879.589216943867` ns/iter	`1.01`
`E2E_Evaluator_omnisharp`	`609506.7295652133` ns/iter	`615884.0645161093` ns/iter	`0.99`
`E2E_Evaluator_openapi`	`12602566.615385057` ns/iter	`14525268.565216225` ns/iter	`0.87`
`E2E_Evaluator_pre_commit_hooks`	`469741.75286192715` ns/iter	`468945.7489986825` ns/iter	`1.00`
`E2E_Evaluator_pulumi`	`703644.2098889772` ns/iter	`724625.5900621` ns/iter	`0.97`
`E2E_Evaluator_semantic_release`	`100826.63371429135` ns/iter	`100267.46845178313` ns/iter	`1.01`
`E2E_Evaluator_stale`	`150335.0010745665` ns/iter	`149659.76205651418` ns/iter	`1.00`
`E2E_Evaluator_stylecop`	`285284.0511890102` ns/iter	`281272.5732637415` ns/iter	`1.01`
`E2E_Evaluator_tmuxinator`	`96230.50850919286` ns/iter	`96929.59638553474` ns/iter	`0.99`
`E2E_Evaluator_ui5`	`500918.1295373533` ns/iter	`497198.1005061243` ns/iter	`1.01`
`E2E_Evaluator_ui5_manifest`	`2469259.880701844` ns/iter	`2611434.4531833073` ns/iter	`0.95`
`E2E_Evaluator_unreal_engine_uproject`	`428147.3548585348` ns/iter	`429581.5076266181` ns/iter	`1.00`
`E2E_Evaluator_vercel`	`316804.43260474503` ns/iter	`317040.44621693884` ns/iter	`1.00`
`E2E_Evaluator_yamllint`	`8741.651060048245` ns/iter	`8545.473780293787` ns/iter	`1.02`
`Micro_Draft4_Meta_1_No_Callback`	`195.20145686781976` ns/iter	`194.88618144997744` ns/iter	`1.00`
`Micro_Draft4_Required_Properties`	`797.746929635522` ns/iter	`795.8698595130137` ns/iter	`1.00`
`Micro_Draft4_Many_Optional_Properties_Minimal_Match`	`22.869477666879945` ns/iter	`22.722319741171436` ns/iter	`1.01`
`Micro_Draft4_Few_Optional_Properties_Minimal_Match`	`10.90852323036998` ns/iter	`10.899474673241329` ns/iter	`1.00`
`Micro_Draft4_Items_Schema`	`278.5465345517991` ns/iter	`278.49861644561565` ns/iter	`1.00`
`Micro_Draft4_Nested_Object`	`1.8132573541158383` ns/iter	`1.7634701599314` ns/iter	`1.03`
`Micro_Draft4_Properties_Triad_Optional`	`318.63058583359805` ns/iter	`318.07135761360666` ns/iter	`1.00`
`Micro_Draft4_Properties_Triad_Closed`	`262.839612954171` ns/iter	`262.84981637897795` ns/iter	`1.00`
`Micro_Draft4_Properties_Triad_Required`	`329.06854358335664` ns/iter	`329.205871998956` ns/iter	`1.00`
`Micro_Draft4_Properties_Closed`	`53.97274115844278` ns/iter	`53.940591965718596` ns/iter	`1.00`
`Micro_Draft4_Non_Recursive_Ref`	`11.655301577763792` ns/iter	`11.884404051178869` ns/iter	`0.98`
`Micro_Draft4_Pattern_Properties_True`	`158.47822917067302` ns/iter	`158.51117314956153` ns/iter	`1.00`
`Micro_Draft4_Ref_To_Single_Property`	`10.907595230681373` ns/iter	`10.904896570339385` ns/iter	`1.00`
`Micro_Draft4_Additional_Properties_Type`	`21.473305521613483` ns/iter	`21.3047345131397` ns/iter	`1.01`
`Micro_Draft4_Nested_Oneof`	`57.292527173277044` ns/iter	`57.53734914037533` ns/iter	`1.00`
`Micro_Draft4_Short_Enum`	`11.834943684837892` ns/iter	`14.11880335120495` ns/iter	`0.84`
`Micro_Draft4_Long_Enum`	`13.776070020092288` ns/iter	`14.025368460819363` ns/iter	`0.98`
`Micro_Draft4_Long_Enum_Short_Strings`	`11.836709414828086` ns/iter	`14.076804564257777` ns/iter	`0.84`
`Micro_Draft4_Type_Object`	`3.4398959993678693` ns/iter	`3.4262093489149845` ns/iter	`1.00`
`Micro_Draft4_Ref_Single_100`	`2384038.0979728205` ns/iter	`2386175.4115647012` ns/iter	`1.00`
`Micro_Draft4_Compile_Ref_Many_Nested`	`1713338.0763548727` ns/iter	`1729590.0419755112` ns/iter	`0.99`
`Micro_Draft4_Compile_Wrap`	`2350045.385906274` ns/iter	`2388792.217687228` ns/iter	`0.98`
`Micro_Draft6_Property_Names`	`119.02034705678432` ns/iter	`122.6088303437834` ns/iter	`0.97`
`Micro_Draft6_Compile_FHIR`	`13667666181.999949` ns/iter	`13723755017.00003` ns/iter	`1.00`
`Micro_Draft7_If_Then_Else`	`22.713981198100537` ns/iter	`22.72824611259822` ns/iter	`1.00`
`Micro_2019_09_Unevaluated_Properties`	`29.033042390444745` ns/iter	`29.83504029198717` ns/iter	`0.97`
`Micro_2019_09_Compile_Wrap`	`285312180.99999475` ns/iter	`314136828.5000112` ns/iter	`0.91`
`Micro_2020_12_Dynamic_Ref`	`229.54302625468281` ns/iter	`229.88884487196654` ns/iter	`1.00`
`Micro_2020_12_Dynamic_Ref_Single`	`3.4238865288467983` ns/iter	`3.4293495082211924` ns/iter	`1.00`
`Micro_2020_12_Simple_Output_Mask`	`66055.92479238917` ns/iter	`66054.48487668505` ns/iter	`1.00`
`Micro_2020_12_Simple_Output_Annotations`	`102517.72728600718` ns/iter	`101993.72025022475` ns/iter	`1.01`
`Micro_2020_12_Compile_NonCircular_Shared_Refs`	`1090905.9269052418` ns/iter	`1110974.0748407273` ns/iter	`0.98`
`Micro_2020_12_Exhaustive_Deep_Numeric`	`8103.841436188225` ns/iter	`7975.512037682365` ns/iter	`1.02`
`Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput`	`175105.66013396953` ns/iter	`174311.25129853925` ns/iter	`1.00`
`Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput`	`22798.476003451313` ns/iter	`22193.637509701` ns/iter	`1.03`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail`	`141.6446257437348` ns/iter	`138.67982234896144` ns/iter	`1.02`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput`	`4247.17427056491` ns/iter	`4314.5650067732195` ns/iter	`0.98`
`Alterschema_Check_Readibility_ISO_Language_Set_3`	`335234592.5000009` ns/iter	`348324498.5000056` ns/iter	`0.96`
`Alterschema_Check_Readibility_OMC`	`83011825.50000875` ns/iter	`83706814.6250033` ns/iter	`0.99`
`Alterschema_Check_Readibility_KrakenD`	`1165440820.0000718` ns/iter	`1169466032.9999352` ns/iter	`1.00`
`Alterschema_Apply_Readibility_KrakenD`	`15384974225.99994` ns/iter	`15471092557.00002` ns/iter	`0.99`
`Alterschema_Check_Invalid_External_Refs`	`292124338.0000078` ns/iter	`297249782.49998915` ns/iter	`0.98`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

JavaScript (linux/llvm)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`E2E_Evaluator_adaptivecard`	`117479` ns	`118655` ns	`0.99`
`E2E_Evaluator_ansible-meta`	`592097` ns	`620776` ns	`0.95`
`E2E_Evaluator_aws-cdk`	`139467` ns	`138218` ns	`1.01`
`E2E_Evaluator_babelrc`	`329884` ns	`326990` ns	`1.01`
`E2E_Evaluator_cerebrum-criminal-case`	`2608080` ns	`2625680` ns	`0.99`
`E2E_Evaluator_clang-format`	`281164` ns	`293998` ns	`0.96`
`E2E_Evaluator_cmake-presets`	`19312272` ns	`19847584` ns	`0.97`
`E2E_Evaluator_code-climate`	`441288` ns	`437313` ns	`1.01`
`E2E_Evaluator_cql2`	`1167086` ns	`1191379` ns	`0.98`
`E2E_Evaluator_cspell`	`1563062` ns	`1579876` ns	`0.99`
`E2E_Evaluator_cypress`	`863637` ns	`877804` ns	`0.98`
`E2E_Evaluator_deno`	`1127566` ns	`1124974` ns	`1.00`
`E2E_Evaluator_dependabot`	`1087593` ns	`1092568` ns	`1.00`
`E2E_Evaluator_draft-04`	`20353360` ns	`22469957` ns	`0.91`
`E2E_Evaluator_fabric-mod`	`2518796` ns	`2572297` ns	`0.98`
`E2E_Evaluator_geojson`	`21997595` ns	`22096749` ns	`1.00`
`E2E_Evaluator_gitpod-configuration`	`764099` ns	`760038` ns	`1.01`
`E2E_Evaluator_helm-chart-lock`	`969592` ns	`1031088` ns	`0.94`
`E2E_Evaluator_importmap`	`282349` ns	`277518` ns	`1.02`
`E2E_Evaluator_jasmine`	`276899` ns	`295290` ns	`0.94`
`E2E_Evaluator_jsconfig`	`1333746` ns	`1373320` ns	`0.97`
`E2E_Evaluator_jshintrc`	`3158132` ns	`3135917` ns	`1.01`
`E2E_Evaluator_krakend`	`769679` ns	`783570` ns	`0.98`
`E2E_Evaluator_lazygit`	`579208` ns	`585378` ns	`0.99`
`E2E_Evaluator_lerna`	`474123` ns	`483574` ns	`0.98`
`E2E_Evaluator_nest-cli`	`632736` ns	`632212` ns	`1.00`
`E2E_Evaluator_omc`	`84511` ns	`85337` ns	`0.99`
`E2E_Evaluator_omnisharp`	`1588872` ns	`1602680` ns	`0.99`
`E2E_Evaluator_openapi`	`37199366` ns	`38009553` ns	`0.98`
`E2E_Evaluator_pre-commit-hooks`	`2030732` ns	`1986990` ns	`1.02`
`E2E_Evaluator_pulumi`	`1953153` ns	`1967627` ns	`0.99`
`E2E_Evaluator_semantic-release`	`271403` ns	`262559` ns	`1.03`
`E2E_Evaluator_stale`	`438680` ns	`428002` ns	`1.02`
`E2E_Evaluator_stylecop`	`1204285` ns	`1235070` ns	`0.98`
`E2E_Evaluator_tmuxinator`	`225053` ns	`226623` ns	`0.99`
`E2E_Evaluator_ui5`	`2249098` ns	`2293790` ns	`0.98`
`E2E_Evaluator_ui5-manifest`	`10833326` ns	`11135559` ns	`0.97`
`E2E_Evaluator_unreal-engine-uproject`	`1525485` ns	`1520267` ns	`1.00`
`E2E_Evaluator_vercel`	`1047272` ns	`1054605` ns	`0.99`
`E2E_Evaluator_yamllint`	`25142` ns	`24869` ns	`1.01`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

Benchmark (linux/gcc)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`Alterschema_Check_Readibility_ISO_Language_Set_3`	`344285962.4999528` ns/iter	`349744019.5000081` ns/iter	`0.98`
`Alterschema_Check_Readibility_OMC`	`99008582.14286603` ns/iter	`95632564.28571418` ns/iter	`1.04`
`Alterschema_Check_Readibility_KrakenD`	`1402066723.0000753` ns/iter	`1316515046.999939` ns/iter	`1.06`
`Alterschema_Apply_Readibility_KrakenD`	`17590835810.99992` ns/iter	`17597419393.99987` ns/iter	`1.00`
`Alterschema_Check_Invalid_External_Refs`	`303226289.5000031` ns/iter	`342061899.99997604` ns/iter	`0.89`
`Micro_2020_12_Dynamic_Ref`	`260.5730183072501` ns/iter	`267.120571500253` ns/iter	`0.98`
`Micro_2020_12_Dynamic_Ref_Single`	`4.574899190574423` ns/iter	`4.99561662967429` ns/iter	`0.92`
`Micro_2020_12_Simple_Output_Mask`	`74169.12423872843` ns/iter	`75246.1179793263` ns/iter	`0.99`
`Micro_2020_12_Simple_Output_Annotations`	`102233.59538327517` ns/iter	`113105.32122587004` ns/iter	`0.90`
`Micro_2020_12_Compile_NonCircular_Shared_Refs`	`1198843.311532009` ns/iter	`1220857.8952879535` ns/iter	`0.98`
`Micro_2020_12_Exhaustive_Deep_Numeric`	`8135.130753667423` ns/iter	`8405.373964955528` ns/iter	`0.97`
`Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput`	`170444.69539125753` ns/iter	`170277.189934666` ns/iter	`1.00`
`Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput`	`26536.448988082215` ns/iter	`25365.677301763717` ns/iter	`1.05`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail`	`136.32073077094282` ns/iter	`134.75819347407983` ns/iter	`1.01`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput`	`4395.439493195485` ns/iter	`4694.536744134641` ns/iter	`0.94`
`Micro_2019_09_Unevaluated_Properties`	`29.273727566618664` ns/iter	`29.916029397891528` ns/iter	`0.98`
`Micro_2019_09_Compile_Wrap`	`322936330.4999993` ns/iter	`321987031.5000435` ns/iter	`1.00`
`Micro_Draft7_If_Then_Else`	`25.13885896816035` ns/iter	`24.704699366257962` ns/iter	`1.02`
`Micro_Draft6_Property_Names`	`142.79730525410116` ns/iter	`142.40253496208385` ns/iter	`1.00`
`Micro_Draft6_Compile_FHIR`	`14605514975.999994` ns/iter	`14274288408.000074` ns/iter	`1.02`
`Micro_Draft4_Meta_1_No_Callback`	`172.10155409834945` ns/iter	`166.65843922373602` ns/iter	`1.03`
`Micro_Draft4_Required_Properties`	`799.8271661543195` ns/iter	`802.4857732370153` ns/iter	`1.00`
`Micro_Draft4_Many_Optional_Properties_Minimal_Match`	`21.02295510448132` ns/iter	`21.237796974224526` ns/iter	`0.99`
`Micro_Draft4_Few_Optional_Properties_Minimal_Match`	`13.029644222568221` ns/iter	`13.715948437201531` ns/iter	`0.95`
`Micro_Draft4_Items_Schema`	`317.55171442553467` ns/iter	`339.2831149348946` ns/iter	`0.94`
`Micro_Draft4_Nested_Object`	`2.113850288205624` ns/iter	`1.8689383762665122` ns/iter	`1.13`
`Micro_Draft4_Properties_Triad_Optional`	`433.07542970128134` ns/iter	`376.6311150890972` ns/iter	`1.15`
`Micro_Draft4_Properties_Triad_Closed`	`386.0921156708148` ns/iter	`324.7077743943816` ns/iter	`1.19`
`Micro_Draft4_Properties_Triad_Required`	`453.9311695987688` ns/iter	`436.6106933680796` ns/iter	`1.04`
`Micro_Draft4_Properties_Closed`	`62.49382286004256` ns/iter	`63.990735571816835` ns/iter	`0.98`
`Micro_Draft4_Non_Recursive_Ref`	`8.821251796023057` ns/iter	`10.299258619128695` ns/iter	`0.86`
`Micro_Draft4_Pattern_Properties_True`	`209.3407416899914` ns/iter	`221.94697737793638` ns/iter	`0.94`
`Micro_Draft4_Ref_To_Single_Property`	`12.681291630846188` ns/iter	`13.71638935304092` ns/iter	`0.92`
`Micro_Draft4_Additional_Properties_Type`	`14.92069652897018` ns/iter	`14.93510754737344` ns/iter	`1.00`
`Micro_Draft4_Nested_Oneof`	`65.80593632809091` ns/iter	`65.7314344659433` ns/iter	`1.00`
`Micro_Draft4_Short_Enum`	`14.78811837661285` ns/iter	`14.696748024445496` ns/iter	`1.01`
`Micro_Draft4_Long_Enum`	`14.089328259727846` ns/iter	`14.655106720087328` ns/iter	`0.96`
`Micro_Draft4_Long_Enum_Short_Strings`	`14.803863279728894` ns/iter	`14.751482638346541` ns/iter	`1.00`
`Micro_Draft4_Type_Object`	`4.573927500820239` ns/iter	`4.979329052087536` ns/iter	`0.92`
`Micro_Draft4_Ref_Single_100`	`2511759.5340503533` ns/iter	`2496640.4928574907` ns/iter	`1.01`
`Micro_Draft4_Compile_Ref_Many_Nested`	`1816255.6684075736` ns/iter	`1820316.2441561478` ns/iter	`1.00`
`Micro_Draft4_Compile_Wrap`	`2588574.4797049626` ns/iter	`2582092.768382354` ns/iter	`1.00`
`E2E_Compiler_adaptivecard`	`65927716.81818538` ns/iter	`66287170.099985816` ns/iter	`0.99`
`E2E_Compiler_ansible_meta`	`31317980.681819793` ns/iter	`31003607.608691588` ns/iter	`1.01`
`E2E_Compiler_aws_cdk`	`401488.21781611274` ns/iter	`417409.52212917234` ns/iter	`0.96`
`E2E_Compiler_babelrc`	`2828784.5384612717` ns/iter	`2791777.108870424` ns/iter	`1.01`
`E2E_Compiler_clang_format`	`20121383.74285704` ns/iter	`19655567.583337564` ns/iter	`1.02`
`E2E_Compiler_cmake_presets`	`27739092.7200031` ns/iter	`26994925.88461593` ns/iter	`1.03`
`E2E_Compiler_code_climate`	`2864111.2540990654` ns/iter	`2786549.7499997295` ns/iter	`1.03`
`E2E_Compiler_cql2`	`14018223.93999737` ns/iter	`14022363.399999449` ns/iter	`1.00`
`E2E_Compiler_cspell`	`24357872.6206908` ns/iter	`23876880.72414268` ns/iter	`1.02`
`E2E_Compiler_cypress`	`4217638.072288471` ns/iter	`4125605.705882449` ns/iter	`1.02`
`E2E_Compiler_deno`	`8430400.048193559` ns/iter	`8299985.083334874` ns/iter	`1.02`
`E2E_Compiler_dependabot`	`4331843.950310612` ns/iter	`4200947.77844301` ns/iter	`1.03`
`E2E_Compiler_draft_04`	`2956052.3459913996` ns/iter	`2970355.1779654566` ns/iter	`1.00`
`E2E_Compiler_fabric_mod`	`5237712.283581038` ns/iter	`5152844.683822997` ns/iter	`1.02`
`E2E_Compiler_geojson`	`24773827.464287024` ns/iter	`23949269.75862604` ns/iter	`1.03`
`E2E_Compiler_gitpod_configuration`	`5415322.395349924` ns/iter	`5339207.469698219` ns/iter	`1.01`
`E2E_Compiler_helm_chart_lock`	`740578.5708202185` ns/iter	`759175.4793479496` ns/iter	`0.98`
`E2E_Compiler_importmap`	`303621.1744936812` ns/iter	`312565.0031531247` ns/iter	`0.97`
`E2E_Compiler_jasmine`	`1477214.9642105529` ns/iter	`1480840.8135596146` ns/iter	`1.00`
`E2E_Compiler_jshintrc`	`4733674.2229725355` ns/iter	`4709832.711408877` ns/iter	`1.01`
`E2E_Compiler_jsconfig`	`24038241.931034934` ns/iter	`23556785.999994643` ns/iter	`1.02`
`E2E_Compiler_krakend`	`143910363.59997997` ns/iter	`142731336.59999076` ns/iter	`1.01`
`E2E_Compiler_lazygit`	`36672884.15789173` ns/iter	`35537150.199991174` ns/iter	`1.03`
`E2E_Compiler_lerna`	`1733348.3970224073` ns/iter	`1742264.6343283895` ns/iter	`0.99`
`E2E_Compiler_nest_cli`	`8138359.267440835` ns/iter	`8014440.909092131` ns/iter	`1.02`
`E2E_Compiler_omc`	`170358754.50000048` ns/iter	`163328716.74995884` ns/iter	`1.04`
`E2E_Compiler_omnisharp`	`7363587.684209817` ns/iter	`7205493.474226181` ns/iter	`1.02`
`E2E_Compiler_openapi`	`27344102.923078794` ns/iter	`27123333.461531732` ns/iter	`1.01`
`E2E_Compiler_pre_commit_hooks`	`5834908.158334429` ns/iter	`5742815.93442526` ns/iter	`1.02`
`E2E_Compiler_pulumi`	`4447194.038216569` ns/iter	`4384045.462499841` ns/iter	`1.01`
`E2E_Compiler_semantic_release`	`1717533.995085591` ns/iter	`1714330.1299020273` ns/iter	`1.00`
`E2E_Compiler_stale`	`1689102.4178746461` ns/iter	`1682338.1373495683` ns/iter	`1.00`
`E2E_Compiler_stylecop`	`6588371.792452322` ns/iter	`6348575.454545756` ns/iter	`1.04`
`E2E_Compiler_tmuxinator`	`2118066.0332325315` ns/iter	`2127699.5498490664` ns/iter	`1.00`
`E2E_Compiler_ui5`	`51057787.857140675` ns/iter	`48362999.214288846` ns/iter	`1.06`
`E2E_Compiler_ui5_manifest`	`321357691.00000286` ns/iter	`322244002.4999287` ns/iter	`1.00`
`E2E_Compiler_unreal_engine_uproject`	`5023466.568345874` ns/iter	`4961090.056737013` ns/iter	`1.01`
`E2E_Compiler_vercel`	`13330989.65384439` ns/iter	`13162426.490564615` ns/iter	`1.01`
`E2E_Compiler_yamllint`	`368453.6961558289` ns/iter	`385619.15795270615` ns/iter	`0.96`
`E2E_Evaluator_adaptivecard`	`23113.47570509911` ns/iter	`23200.40941340863` ns/iter	`1.00`
`E2E_Evaluator_ansible_meta`	`223957.17479810448` ns/iter	`247202.56990397023` ns/iter	`0.91`
`E2E_Evaluator_aws_cdk`	`43806.27167303309` ns/iter	`52474.320362885446` ns/iter	`0.83`
`E2E_Evaluator_babelrc`	`83773.56259010249` ns/iter	`106791.57319836594` ns/iter	`0.78`
`E2E_Evaluator_cerebrum_criminal_case`	`1327669.3434533542` ns/iter	`1271401.3836363268` ns/iter	`1.04`
`E2E_Evaluator_clang_format`	`101345.9769119777` ns/iter	`111158.41875994085` ns/iter	`0.91`
`E2E_Evaluator_cmake_presets`	`4533174.266232702` ns/iter	`4502649.987096213` ns/iter	`1.01`
`E2E_Evaluator_code_climate`	`158712.19827192606` ns/iter	`187447.0919509778` ns/iter	`0.85`
`E2E_Evaluator_cql2`	`199039.41948541877` ns/iter	`216429.53486926478` ns/iter	`0.92`
`E2E_Evaluator_cspell`	`506254.7558996935` ns/iter	`545346.1610429058` ns/iter	`0.93`
`E2E_Evaluator_cypress`	`142957.5478385398` ns/iter	`176305.59772725854` ns/iter	`0.81`
`E2E_Evaluator_deno`	`273719.30629209476` ns/iter	`335563.8444872418` ns/iter	`0.82`
`E2E_Evaluator_dependabot`	`444919.0816847589` ns/iter	`468468.15379536484` ns/iter	`0.95`
`E2E_Evaluator_draft_04`	`7959113.8686866015` ns/iter	`7072946.223300198` ns/iter	`1.13`
`E2E_Evaluator_fabric_mod`	`711489.5645161727` ns/iter	`735808.1949154` ns/iter	`0.97`
`E2E_Evaluator_geojson`	`16980184.731709216` ns/iter	`16851846.409090076` ns/iter	`1.01`
`E2E_Evaluator_gitpod_configuration`	`220629.7834334362` ns/iter	`260081.77802202723` ns/iter	`0.85`
`E2E_Evaluator_helm_chart_lock`	`214241.54254007392` ns/iter	`325336.14754103625` ns/iter	`0.66`
`E2E_Evaluator_importmap`	`39478.171763769176` ns/iter	`49124.61427578145` ns/iter	`0.80`
`E2E_Evaluator_jasmine`	`103979.858264142` ns/iter	`118547.90020788736` ns/iter	`0.88`
`E2E_Evaluator_jshintrc`	`1020423.9666182322` ns/iter	`1057849.0030119123` ns/iter	`0.96`
`E2E_Evaluator_jsconfig`	`371373.69696967176` ns/iter	`404559.9282407495` ns/iter	`0.92`
`E2E_Evaluator_krakend`	`186165.19125828668` ns/iter	`212820.06128049534` ns/iter	`0.87`
`E2E_Evaluator_lazygit`	`125615.02768290538` ns/iter	`137431.0807920782` ns/iter	`0.91`
`E2E_Evaluator_lerna`	`122061.24825418425` ns/iter	`162100.28699654416` ns/iter	`0.75`
`E2E_Evaluator_nest_cli`	`180039.93703607089` ns/iter	`195506.34722223648` ns/iter	`0.92`
`E2E_Evaluator_omc`	`18840.54501612383` ns/iter	`19146.181263694092` ns/iter	`0.98`
`E2E_Evaluator_omnisharp`	`535226.8050075386` ns/iter	`581168.1626555469` ns/iter	`0.92`
`E2E_Evaluator_openapi`	`13163253.173073687` ns/iter	`13294615.720001275` ns/iter	`0.99`
`E2E_Evaluator_pre_commit_hooks`	`466864.55645157053` ns/iter	`526734.3735031141` ns/iter	`0.89`
`E2E_Evaluator_pulumi`	`720930.4953172387` ns/iter	`717669.5528795652` ns/iter	`1.00`
`E2E_Evaluator_semantic_release`	`84997.53223828018` ns/iter	`102915.51166100061` ns/iter	`0.83`
`E2E_Evaluator_stale`	`151047.831472749` ns/iter	`161176.1067467309` ns/iter	`0.94`
`E2E_Evaluator_stylecop`	`292055.9759235536` ns/iter	`302545.7968951353` ns/iter	`0.97`
`E2E_Evaluator_tmuxinator`	`86448.4155795544` ns/iter	`102219.12106849838` ns/iter	`0.85`
`E2E_Evaluator_ui5`	`491669.026722853` ns/iter	`531586.4538462078` ns/iter	`0.92`
`E2E_Evaluator_ui5_manifest`	`2649572.342205632` ns/iter	`2787185.665338945` ns/iter	`0.95`
`E2E_Evaluator_unreal_engine_uproject`	`430618.6552154846` ns/iter	`468604.20485371357` ns/iter	`0.92`
`E2E_Evaluator_vercel`	`267372.3706797777` ns/iter	`315397.3092642567` ns/iter	`0.85`
`E2E_Evaluator_yamllint`	`9824.514120739208` ns/iter	`11126.609676199736` ns/iter	`0.88`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

JavaScript (linux/gcc)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`E2E_Evaluator_adaptivecard`	`117901` ns	`117864` ns	`1.00`
`E2E_Evaluator_ansible-meta`	`637645` ns	`603679` ns	`1.06`
`E2E_Evaluator_aws-cdk`	`131937` ns	`135120` ns	`0.98`
`E2E_Evaluator_babelrc`	`320553` ns	`323906` ns	`0.99`
`E2E_Evaluator_cerebrum-criminal-case`	`2909018` ns	`2587560` ns	`1.12`
`E2E_Evaluator_clang-format`	`270272` ns	`280544` ns	`0.96`
`E2E_Evaluator_cmake-presets`	`20170746` ns	`19869597` ns	`1.02`
`E2E_Evaluator_code-climate`	`431990` ns	`434632` ns	`0.99`
`E2E_Evaluator_cql2`	`1186799` ns	`1172878` ns	`1.01`
`E2E_Evaluator_cspell`	`1735425` ns	`1602864` ns	`1.08`
`E2E_Evaluator_cypress`	`917301` ns	`861399` ns	`1.06`
`E2E_Evaluator_deno`	`1118396` ns	`1088435` ns	`1.03`
`E2E_Evaluator_dependabot`	`1133189` ns	`1098228` ns	`1.03`
`E2E_Evaluator_draft-04`	`21081167` ns	`19696781` ns	`1.07`
`E2E_Evaluator_fabric-mod`	`2737179` ns	`2506739` ns	`1.09`
`E2E_Evaluator_geojson`	`22374276` ns	`21801258` ns	`1.03`
`E2E_Evaluator_gitpod-configuration`	`815889` ns	`756418` ns	`1.08`
`E2E_Evaluator_helm-chart-lock`	`1058940` ns	`1077113` ns	`0.98`
`E2E_Evaluator_importmap`	`282374` ns	`274560` ns	`1.03`
`E2E_Evaluator_jasmine`	`333112` ns	`300322` ns	`1.11`
`E2E_Evaluator_jsconfig`	`1512346` ns	`1373709` ns	`1.10`
`E2E_Evaluator_jshintrc`	`3312096` ns	`3266732` ns	`1.01`
`E2E_Evaluator_krakend`	`776041` ns	`773290` ns	`1.00`
`E2E_Evaluator_lazygit`	`497571` ns	`581473` ns	`0.86`
`E2E_Evaluator_lerna`	`470615` ns	`468614` ns	`1.00`
`E2E_Evaluator_nest-cli`	`674988` ns	`638589` ns	`1.06`
`E2E_Evaluator_omc`	`87187` ns	`85552` ns	`1.02`
`E2E_Evaluator_omnisharp`	`1923694` ns	`1629340` ns	`1.18`
`E2E_Evaluator_openapi`	`36194555` ns	`36862143` ns	`0.98`
`E2E_Evaluator_pre-commit-hooks`	`2224520` ns	`2051824` ns	`1.08`
`E2E_Evaluator_pulumi`	`2172378` ns	`2011009` ns	`1.08`
`E2E_Evaluator_semantic-release`	`249412` ns	`263794` ns	`0.95`
`E2E_Evaluator_stale`	`431742` ns	`438379` ns	`0.98`
`E2E_Evaluator_stylecop`	`1241374` ns	`1227335` ns	`1.01`
`E2E_Evaluator_tmuxinator`	`231111` ns	`222704` ns	`1.04`
`E2E_Evaluator_ui5`	`2243724` ns	`2249152` ns	`1.00`
`E2E_Evaluator_ui5-manifest`	`10124339` ns	`11118158` ns	`0.91`
`E2E_Evaluator_unreal-engine-uproject`	`1690758` ns	`1490135` ns	`1.13`
`E2E_Evaluator_vercel`	`1112162` ns	`1072382` ns	`1.04`
`E2E_Evaluator_yamllint`	`24582` ns	`24884` ns	`0.99`

This comment was automatically generated by workflow using github-action-benchmark.

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

cubic-dev-ai

No issues found across 2 files

augmentcode · 2026-04-10T20:43:25Z

🤖 Augment PR Summary

Summary: This PR adds a new evaluator test binary that runs the existing JSON-driven evaluator suites after first canonicalising the schema.

Changes:

Introduces a new GoogleTest target evaluator_trace_suite_canonical and wires it into test/evaluator/CMakeLists.txt
Adds test/evaluator/tracesuite_canonical.cc to load the same evaluator JSON files and register per-case tests
For each test case, applies AlterSchemaMode::Canonicalizer via SchemaTransformer, then compiles and validates the instance
Runs both FastValidation and Exhaustive modes for each case

Technical Notes: Links the new unit to sourcemeta::blaze::alterschema and passes the evaluator test directory via TRACE_SUITE_CANONICAL_PATH.

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 2 suggestions posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-04-10T20:43:26Z

test/evaluator/tracesuite_canonical.cc

+    __ASSERT_TEMPLATE_JSON_SERIALISATION(compiled_schema);
+
+    sourcemeta::blaze::Evaluator evaluator;
+    const auto result{evaluator.validate(compiled_schema, instance)};


test/evaluator/tracesuite_canonical.cc:46: This “trace suite” only asserts the final validate() boolean and never checks the expected trace output (unlike tracesuite.cc which uses EVALUATE_WITH_TRACE/trace expectations). If the goal is to run the trace suite through canonicalisation, this won’t catch regressions where canonicalisation changes the produced trace while preserving validity.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

augmentcode · 2026-04-10T20:43:26Z

test/evaluator/tracesuite_canonical.cc

+                     sourcemeta::core::schema_resolver,
+                     [](const auto &, const auto &, const auto &, const auto &,
+                        const auto &) {})};
+    EXPECT_TRUE(canonicalize_result.first);


test/evaluator/tracesuite_canonical.cc:37: If bundle.apply() reports failure, the test continues to compile/validate anyway, which can hide the root cause behind later failures or exceptions. A fatal failure here would make canonicalisation problems surface more clearly.

Severity: low

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

github-actions

Benchmark (macos/llvm)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`E2E_Compiler_adaptivecard`	`97180822.87500352` ns/iter	`60983212.09091492` ns/iter	`1.59`
`E2E_Compiler_ansible_meta`	`37409308.00000055` ns/iter	`26127627.964285694` ns/iter	`1.43`
`E2E_Compiler_aws_cdk`	`520549.436939641` ns/iter	`319525.856255197` ns/iter	`1.63`
`E2E_Compiler_babelrc`	`3434249.0000003637` ns/iter	`2547284.87777769` ns/iter	`1.35`
`E2E_Compiler_clang_format`	`22614150.321430773` ns/iter	`15904836.95652082` ns/iter	`1.42`
`E2E_Compiler_cmake_presets`	`42448905.11111205` ns/iter	`37614249.99999882` ns/iter	`1.13`
`E2E_Compiler_code_climate`	`2978356.2583729057` ns/iter	`2643680.554054295` ns/iter	`1.13`
`E2E_Compiler_cql2`	`15798623.212765265` ns/iter	`12402891.807018055` ns/iter	`1.27`
`E2E_Compiler_cspell`	`26294467.285716955` ns/iter	`23350704.647060305` ns/iter	`1.13`
`E2E_Compiler_cypress`	`5424070.013605098` ns/iter	`4900027.637305531` ns/iter	`1.11`
`E2E_Compiler_deno`	`8752629.905882929` ns/iter	`7865173.6111118635` ns/iter	`1.11`
`E2E_Compiler_dependabot`	`5129349.159999493` ns/iter	`3763401.561402974` ns/iter	`1.36`
`E2E_Compiler_draft_04`	`3149925.429184891` ns/iter	`2277088.714714688` ns/iter	`1.38`
`E2E_Compiler_fabric_mod`	`5738833.641791997` ns/iter	`4340686.126373749` ns/iter	`1.32`
`E2E_Compiler_geojson`	`40848172.94999538` ns/iter	`24647448.275859147` ns/iter	`1.66`
`E2E_Compiler_gitpod_configuration`	`7135340.270833031` ns/iter	`4298751.268292768` ns/iter	`1.66`
`E2E_Compiler_helm_chart_lock`	`976170.7028796729` ns/iter	`620991.2246490057` ns/iter	`1.57`
`E2E_Compiler_importmap`	`351499.4913529551` ns/iter	`218028.50887573982` ns/iter	`1.61`
`E2E_Compiler_jasmine`	`1737316.460055172` ns/iter	`1226937.9534160583` ns/iter	`1.42`
`E2E_Compiler_jshintrc`	`5530758.1041669585` ns/iter	`4094661.1128202467` ns/iter	`1.35`
`E2E_Compiler_jsconfig`	`28259145.000001807` ns/iter	`20275617.13513527` ns/iter	`1.39`
`E2E_Compiler_krakend`	`196603562.50000176` ns/iter	`139178608.40001595` ns/iter	`1.41`
`E2E_Compiler_lazygit`	`49829230.76923166` ns/iter	`38345760.40000002` ns/iter	`1.30`
`E2E_Compiler_lerna`	`2107681.840336446` ns/iter	`1389019.6984435439` ns/iter	`1.52`
`E2E_Compiler_nest_cli`	`9030499.12631583` ns/iter	`7933426.278845865` ns/iter	`1.14`
`E2E_Compiler_omc`	`249558110.99994206` ns/iter	`181925041.74999157` ns/iter	`1.37`
`E2E_Compiler_omnisharp`	`10758197.11538537` ns/iter	`6182725.89256246` ns/iter	`1.74`
`E2E_Compiler_openapi`	`40645462.73684755` ns/iter	`24852157.562502697` ns/iter	`1.64`
`E2E_Compiler_pre_commit_hooks`	`5741553.217819841` ns/iter	`4557881.900662275` ns/iter	`1.26`
`E2E_Compiler_pulumi`	`5430332.500000076` ns/iter	`3661094.1874997136` ns/iter	`1.48`
`E2E_Compiler_semantic_release`	`1915267.491357902` ns/iter	`1433757.2848723873` ns/iter	`1.34`
`E2E_Compiler_stale`	`2063160.7142861572` ns/iter	`1310966.6660000128` ns/iter	`1.57`
`E2E_Compiler_stylecop`	`8246542.938775524` ns/iter	`5837948.562499306` ns/iter	`1.41`
`E2E_Compiler_tmuxinator`	`2515779.8245616728` ns/iter	`1703303.4266667832` ns/iter	`1.48`
`E2E_Compiler_ui5`	`89561151.12499674` ns/iter	`65746170.08333424` ns/iter	`1.36`
`E2E_Compiler_ui5_manifest`	`532313584.0001214` ns/iter	`344293312.4999854` ns/iter	`1.55`
`E2E_Compiler_unreal_engine_uproject`	`5388017.079999371` ns/iter	`4153401.2602746207` ns/iter	`1.30`
`E2E_Compiler_vercel`	`17150375.976192273` ns/iter	`12855762.931035863` ns/iter	`1.33`
`E2E_Compiler_yamllint`	`584658.7110494543` ns/iter	`324707.7218348758` ns/iter	`1.80`
`E2E_Evaluator_adaptivecard`	`27851.6221361001` ns/iter	`19146.654870443283` ns/iter	`1.45`
`E2E_Evaluator_ansible_meta`	`248014.74829573985` ns/iter	`165010.33584372923` ns/iter	`1.50`
`E2E_Evaluator_aws_cdk`	`51485.9000000115` ns/iter	`33867.716130925495` ns/iter	`1.52`
`E2E_Evaluator_babelrc`	`114361.78963109826` ns/iter	`81529.61626353196` ns/iter	`1.40`
`E2E_Evaluator_cerebrum_criminal_case`	`1082384.2267136297` ns/iter	`908014.919437309` ns/iter	`1.19`
`E2E_Evaluator_clang_format`	`132786.14321203888` ns/iter	`101255.58019996736` ns/iter	`1.31`
`E2E_Evaluator_cmake_presets`	`5951124.579999032` ns/iter	`3763022.465686087` ns/iter	`1.58`
`E2E_Evaluator_code_climate`	`163946.99170121795` ns/iter	`133639.97220729064` ns/iter	`1.23`
`E2E_Evaluator_cql2`	`194895.08316255684` ns/iter	`178995.51570489164` ns/iter	`1.09`
`E2E_Evaluator_cspell`	`642419.2879347701` ns/iter	`413544.9141069611` ns/iter	`1.55`
`E2E_Evaluator_cypress`	`221471.40954093722` ns/iter	`140476.22499999762` ns/iter	`1.58`
`E2E_Evaluator_deno`	`341635.7729700561` ns/iter	`245296.91313869844` ns/iter	`1.39`
`E2E_Evaluator_dependabot`	`550564.0410001433` ns/iter	`321893.45059471537` ns/iter	`1.71`
`E2E_Evaluator_draft_04`	`7053793.623529546` ns/iter	`5477842.174241921` ns/iter	`1.29`
`E2E_Evaluator_fabric_mod`	`746951.9576515809` ns/iter	`479182.51599448913` ns/iter	`1.56`
`E2E_Evaluator_geojson`	`19026209.341463096` ns/iter	`16633594.096777327` ns/iter	`1.14`
`E2E_Evaluator_gitpod_configuration`	`240501.16317716477` ns/iter	`242097.3259762344` ns/iter	`0.99`
`E2E_Evaluator_helm_chart_lock`	`260618.28025663056` ns/iter	`242113.5557792569` ns/iter	`1.08`
`E2E_Evaluator_importmap`	`33955.8329274512` ns/iter	`40506.47007252971` ns/iter	`0.84`
`E2E_Evaluator_jasmine`	`88128.73425118932` ns/iter	`109696.94497998837` ns/iter	`0.80`
`E2E_Evaluator_jshintrc`	`1379861.691813849` ns/iter	`1083960.1753048794` ns/iter	`1.27`
`E2E_Evaluator_jsconfig`	`366695.3478527746` ns/iter	`347691.47567822994` ns/iter	`1.05`
`E2E_Evaluator_krakend`	`137695.0126990643` ns/iter	`220744.45734797276` ns/iter	`0.62`
`E2E_Evaluator_lazygit`	`107085.28794149541` ns/iter	`121577.01910025334` ns/iter	`0.88`
`E2E_Evaluator_lerna`	`147014.7552293492` ns/iter	`137718.90125426528` ns/iter	`1.07`
`E2E_Evaluator_nest_cli`	`140328.53393588564` ns/iter	`172832.96220816992` ns/iter	`0.81`
`E2E_Evaluator_omc`	`18434.192185025662` ns/iter	`23031.933345527395` ns/iter	`0.80`
`E2E_Evaluator_omnisharp`	`493861.18899854197` ns/iter	`464074.0598031018` ns/iter	`1.06`
`E2E_Evaluator_openapi`	`9649549.40000098` ns/iter	`9744756.499998175` ns/iter	`0.99`
`E2E_Evaluator_pre_commit_hooks`	`458474.8697596001` ns/iter	`434668.692265123` ns/iter	`1.05`
`E2E_Evaluator_pulumi`	`531152.2148916252` ns/iter	`580242.9351211343` ns/iter	`0.92`
`E2E_Evaluator_semantic_release`	`67172.09474590704` ns/iter	`78730.73127802281` ns/iter	`0.85`
`E2E_Evaluator_stale`	`108754.17942073903` ns/iter	`138307.69840993476` ns/iter	`0.79`
`E2E_Evaluator_stylecop`	`204925.95796210677` ns/iter	`213953.01302932488` ns/iter	`0.96`
`E2E_Evaluator_tmuxinator`	`75171.29884799897` ns/iter	`78316.99722530834` ns/iter	`0.96`
`E2E_Evaluator_ui5`	`505625.5829999827` ns/iter	`554614.849573722` ns/iter	`0.91`
`E2E_Evaluator_ui5_manifest`	`2067199.4245280728` ns/iter	`2036778.6366196142` ns/iter	`1.01`
`E2E_Evaluator_unreal_engine_uproject`	`311900.01119077497` ns/iter	`405142.52735985257` ns/iter	`0.77`
`E2E_Evaluator_vercel`	`227212.37416106416` ns/iter	`272072.08030430163` ns/iter	`0.84`
`E2E_Evaluator_yamllint`	`5933.369061873973` ns/iter	`7119.736842105635` ns/iter	`0.83`
`Micro_Draft4_Meta_1_No_Callback`	`131.68603491288854` ns/iter	`145.73686766221198` ns/iter	`0.90`
`Micro_Draft4_Required_Properties`	`655.5063115252681` ns/iter	`787.2573338190274` ns/iter	`0.83`
`Micro_Draft4_Many_Optional_Properties_Minimal_Match`	`16.036098963949268` ns/iter	`19.423037665762514` ns/iter	`0.83`
`Micro_Draft4_Few_Optional_Properties_Minimal_Match`	`7.334032890997835` ns/iter	`14.464055019242709` ns/iter	`0.51`
`Micro_Draft4_Items_Schema`	`252.79178814932166` ns/iter	`341.88707478341314` ns/iter	`0.74`
`Micro_Draft4_Nested_Object`	`0.698643577116635` ns/iter	`0.9208896147368825` ns/iter	`0.76`
`Micro_Draft4_Properties_Triad_Optional`	`200.2647027345069` ns/iter	`250.03603558857571` ns/iter	`0.80`
`Micro_Draft4_Properties_Triad_Closed`	`155.73610953720592` ns/iter	`184.8054029253811` ns/iter	`0.84`
`Micro_Draft4_Properties_Triad_Required`	`202.925496391626` ns/iter	`260.44985373600764` ns/iter	`0.78`
`Micro_Draft4_Properties_Closed`	`36.28426402124923` ns/iter	`40.99813592558531` ns/iter	`0.89`
`Micro_Draft4_Non_Recursive_Ref`	`12.600102127704547` ns/iter	`15.06227128605031` ns/iter	`0.84`
`Micro_Draft4_Pattern_Properties_True`	`129.75895201333498` ns/iter	`175.66234148645094` ns/iter	`0.74`
`Micro_Draft4_Ref_To_Single_Property`	`7.108950484099159` ns/iter	`8.124863953809928` ns/iter	`0.87`
`Micro_Draft4_Additional_Properties_Type`	`23.780867717977564` ns/iter	`30.530741989247666` ns/iter	`0.78`
`Micro_Draft4_Nested_Oneof`	`37.93313678659439` ns/iter	`47.900999051560895` ns/iter	`0.79`
`Micro_Draft4_Short_Enum`	`4.614699655107114` ns/iter	`9.805879558766497` ns/iter	`0.47`
`Micro_Draft4_Long_Enum`	`9.207543292360468` ns/iter	`14.159982876718246` ns/iter	`0.65`
`Micro_Draft4_Long_Enum_Short_Strings`	`5.872383293137167` ns/iter	`7.378939674318188` ns/iter	`0.80`
`Micro_Draft4_Type_Object`	`2.0304259856225277` ns/iter	`2.744276520537535` ns/iter	`0.74`
`Micro_Draft4_Ref_Single_100`	`1851253.5128802035` ns/iter	`2602619.946745743` ns/iter	`0.71`
`Micro_Draft4_Compile_Ref_Many_Nested`	`1208110.1841154073` ns/iter	`1643865.50990106` ns/iter	`0.73`
`Micro_Draft4_Compile_Wrap`	`1931349.233983639` ns/iter	`2681810.167910417` ns/iter	`0.72`
`Micro_Draft6_Property_Names`	`87.06451705952487` ns/iter	`107.22437773100195` ns/iter	`0.81`
`Micro_Draft6_Compile_FHIR`	`7466527541.999994` ns/iter	`16895108666.00017` ns/iter	`0.44`
`Micro_Draft7_If_Then_Else`	`17.467712510466217` ns/iter	`20.142114084208302` ns/iter	`0.87`
`Micro_2019_09_Unevaluated_Properties`	`26.206654194748758` ns/iter	`33.038860377516215` ns/iter	`0.79`
`Micro_2019_09_Compile_Wrap`	`291397166.500019` ns/iter	`439551937.50002766` ns/iter	`0.66`
`Micro_2020_12_Dynamic_Ref`	`184.3454558207817` ns/iter	`226.27877253698307` ns/iter	`0.81`
`Micro_2020_12_Dynamic_Ref_Single`	`2.2839547538818654` ns/iter	`3.0641394447529606` ns/iter	`0.75`
`Micro_2020_12_Simple_Output_Mask`	`63240.29444068383` ns/iter	`113546.44619178714` ns/iter	`0.56`
`Micro_2020_12_Simple_Output_Annotations`	`86387.86739338854` ns/iter	`122706.47037204563` ns/iter	`0.70`
`Micro_2020_12_Compile_NonCircular_Shared_Refs`	`791433.729638124` ns/iter	`1088780.3703702013` ns/iter	`0.73`
`Micro_2020_12_Exhaustive_Deep_Numeric`	`5089.995830001044` ns/iter	`6037.918087049043` ns/iter	`0.84`
`Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput`	`111736.5512597003` ns/iter	`143386.04530173697` ns/iter	`0.78`
`Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput`	`14784.360201564548` ns/iter	`18610.712662962545` ns/iter	`0.79`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail`	`98.2712057897857` ns/iter	`115.87184799961632` ns/iter	`0.85`
`Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput`	`5166.979170001014` ns/iter	`5901.166047651346` ns/iter	`0.88`
`Alterschema_Check_Readibility_ISO_Language_Set_3`	`241514249.999985` ns/iter	`322679229.5000678` ns/iter	`0.75`
`Alterschema_Check_Readibility_OMC`	`58162291.66667124` ns/iter	`91472523.85715936` ns/iter	`0.64`
`Alterschema_Check_Readibility_KrakenD`	`573702457.9998434` ns/iter	`982544416.0001098` ns/iter	`0.58`
`Alterschema_Apply_Readibility_KrakenD`	`8401673083.999867` ns/iter	`12758383042.000105` ns/iter	`0.66`
`Alterschema_Check_Invalid_External_Refs`	`186571739.49998197` ns/iter	`188575645.75000653` ns/iter	`0.99`

This comment was automatically generated by workflow using github-action-benchmark.

github-actions

JavaScript (macos/llvm)

Details

Benchmark suite	Current: `7f47d6f`	Previous: `f01e314`	Ratio
`E2E_Evaluator_adaptivecard`	`108956` ns	`122630` ns	`0.89`
`E2E_Evaluator_ansible-meta`	`609339` ns	`710457` ns	`0.86`
`E2E_Evaluator_aws-cdk`	`150350` ns	`140184` ns	`1.07`
`E2E_Evaluator_babelrc`	`428103` ns	`440170` ns	`0.97`
`E2E_Evaluator_cerebrum-criminal-case`	`2669982` ns	`3454978` ns	`0.77`
`E2E_Evaluator_clang-format`	`264784` ns	`381604` ns	`0.69`
`E2E_Evaluator_cmake-presets`	`18212112` ns	`21719520` ns	`0.84`
`E2E_Evaluator_code-climate`	`423106` ns	`497049` ns	`0.85`
`E2E_Evaluator_cql2`	`986814` ns	`1101542` ns	`0.90`
`E2E_Evaluator_cspell`	`1404299` ns	`2044860` ns	`0.69`
`E2E_Evaluator_cypress`	`766473` ns	`1157257` ns	`0.66`
`E2E_Evaluator_deno`	`744668` ns	`2084657` ns	`0.36`
`E2E_Evaluator_dependabot`	`888220` ns	`1525301` ns	`0.58`
`E2E_Evaluator_draft-04`	`15432262` ns	`22166865` ns	`0.70`
`E2E_Evaluator_fabric-mod`	`2028093` ns	`2726575` ns	`0.74`
`E2E_Evaluator_geojson`	`21666809` ns	`28261866` ns	`0.77`
`E2E_Evaluator_gitpod-configuration`	`702515` ns	`948478` ns	`0.74`
`E2E_Evaluator_helm-chart-lock`	`1052473` ns	`1300587` ns	`0.81`
`E2E_Evaluator_importmap`	`240068` ns	`322417` ns	`0.74`
`E2E_Evaluator_jasmine`	`281742` ns	`430048` ns	`0.66`
`E2E_Evaluator_jsconfig`	`1290115` ns	`1874350` ns	`0.69`
`E2E_Evaluator_jshintrc`	`2577757` ns	`3778239` ns	`0.68`
`E2E_Evaluator_krakend`	`462475` ns	`745248` ns	`0.62`
`E2E_Evaluator_lazygit`	`403137` ns	`669804` ns	`0.60`
`E2E_Evaluator_lerna`	`379178` ns	`499633` ns	`0.76`
`E2E_Evaluator_nest-cli`	`550607` ns	`839760` ns	`0.66`
`E2E_Evaluator_omc`	`73101` ns	`105294` ns	`0.69`
`E2E_Evaluator_omnisharp`	`1374022` ns	`1768162` ns	`0.78`
`E2E_Evaluator_openapi`	`24711415` ns	`35578524` ns	`0.69`
`E2E_Evaluator_pre-commit-hooks`	`1695046` ns	`2242275` ns	`0.76`
`E2E_Evaluator_pulumi`	`1874393` ns	`2437809` ns	`0.77`
`E2E_Evaluator_semantic-release`	`247675` ns	`320910` ns	`0.77`
`E2E_Evaluator_stale`	`377281` ns	`584860` ns	`0.65`
`E2E_Evaluator_stylecop`	`1002405` ns	`1366340` ns	`0.73`
`E2E_Evaluator_tmuxinator`	`261379` ns	`347858` ns	`0.75`
`E2E_Evaluator_ui5`	`1668168` ns	`2439331` ns	`0.68`
`E2E_Evaluator_ui5-manifest`	`7481843` ns	`12857669` ns	`0.58`
`E2E_Evaluator_unreal-engine-uproject`	`1458057` ns	`1856386` ns	`0.79`
`E2E_Evaluator_vercel`	`1041433` ns	`1184121` ns	`0.88`
`E2E_Evaluator_yamllint`	`28914` ns	`34022` ns	`0.85`

This comment was automatically generated by workflow using github-action-benchmark.

jviotti force-pushed the trace-canonicalize branch 4 times, most recently from 04df6f6 to c6444b3 Compare April 10, 2026 19:56

github-actions bot reviewed Apr 10, 2026

View reviewed changes

jviotti added 4 commits April 10, 2026 16:36

[WIP] Run the trace suite through canonicalisation

0ae6223

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

Fix names

f5f04bd

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

FIX

4c59013

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

Simpler

7f47d6f

Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>

jviotti force-pushed the trace-canonicalize branch from 2614035 to 7f47d6f Compare April 10, 2026 20:37

jviotti marked this pull request as ready for review April 10, 2026 20:39

jviotti changed the title ~~[WIP] Run the trace suite through canonicalisation~~ Run the trace suite through canonicalisation Apr 10, 2026

cubic-dev-ai bot reviewed Apr 10, 2026

View reviewed changes

augmentcode bot reviewed Apr 10, 2026

View reviewed changes

github-actions bot reviewed Apr 10, 2026

View reviewed changes

jviotti merged commit 61eee69 into main Apr 10, 2026
14 checks passed

jviotti deleted the trace-canonicalize branch April 10, 2026 20:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Run the trace suite through canonicalisation#698

Run the trace suite through canonicalisation#698
jviotti merged 4 commits intomainfrom
trace-canonicalize

jviotti commented Apr 10, 2026

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

cubic-dev-ai bot left a comment

Uh oh!

augmentcode bot commented Apr 10, 2026

Uh oh!

augmentcode bot left a comment

Uh oh!

augmentcode bot Apr 10, 2026

Uh oh!

augmentcode bot Apr 10, 2026

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

jviotti commented Apr 10, 2026

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark (linux/llvm)

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

JavaScript (linux/llvm)

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark (linux/gcc)

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

JavaScript (linux/gcc)

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode bot commented Apr 10, 2026

Uh oh!

augmentcode bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode bot Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

augmentcode bot Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Benchmark (macos/llvm)

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

JavaScript (macos/llvm)

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading

github-actions bot left a comment •

edited

Loading