loop-benchmarking

Controlled experiments across agentic coding configurations. Same task, one variable, what actually works.
git clone https://git.shiptheloop.com/loop-benchmarking.git
Log | Files | Refs | README

commit 932df6b569f5de093993e826d9e5d3e5b2ba5b6d
parent 476b885bb47439b71b3967aaed0a7f15c66699d8
Author: Brian Graham <brian@buildingbetterteams.de>
Date:   Mon,  6 Apr 2026 19:13:35 +0200

Add zai-smoke profile, fix provider in profiles

zai-smoke: 2 GLM models (glm-4.5-air, glm-4.7) x 2 prompt styles = 4 cells.
All profiles now include both providers so --provider flag filters correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Diffstat:
Mgrid.yaml | 36++++++++++++++++++++++++++++++++----
1 file changed, 32 insertions(+), 4 deletions(-)

diff --git a/grid.yaml b/grid.yaml @@ -113,7 +113,35 @@ profiles: error_checking: ["none"] context_noise: ["clean"] renderer: ["none"] - provider: ["anthropic"] + provider: ["anthropic", "zai"] + runs_per_cell: 1 + + zai-smoke: + description: "Quick validation for Z.AI GLM models" + axes: + model: [haiku, sonnet] + effort: [high] + prompt_style: [simple, detailed] + language: [typescript] + human_language: [en] + tool_read: ["on"] + tool_write: ["on"] + tool_edit: ["on"] + tool_glob: ["on"] + tool_grep: ["on"] + linter: ["off"] + playwright: ["off"] + context_file: [none] + web_search: ["off"] + max_budget: [low] + tests_provided: ["none"] + strategy: ["none"] + design_guidance: ["none"] + architecture: ["none"] + error_checking: ["none"] + context_noise: ["clean"] + renderer: ["none"] + provider: ["zai"] runs_per_cell: 1 core: @@ -141,7 +169,7 @@ profiles: error_checking: ["none"] context_noise: ["clean"] renderer: ["none"] - provider: ["anthropic"] + provider: ["anthropic", "zai"] runs_per_cell: 3 all-on: @@ -169,7 +197,7 @@ profiles: error_checking: ["self_verify"] context_noise: ["clean"] renderer: ["canvas"] - provider: ["anthropic"] + provider: ["anthropic", "zai"] runs_per_cell: 3 all-off: @@ -197,7 +225,7 @@ profiles: error_checking: ["none"] context_noise: ["clean"] renderer: ["none"] - provider: ["anthropic"] + provider: ["anthropic", "zai"] runs_per_cell: 3 full:

Impressum · Datenschutz