Provider report card
These are execution facts, not skill scores. Rates use all decisions in each group; cost averages only decisions with recorded cost.
Test run 2
aperture
| Game | Decisions | Model failure | Provider failure | Other failure | All failures | Latency p50 | Latency p95 | Cost / decision | Cost coverage | Over-limit share |
|---|---|---|---|---|---|---|---|---|---|---|
| All Games | 1,287 | 7.5% | 0.0% | 0.0% | 7.5% | 6,133 ms | 49,977 ms | $0.0025 | 96.4% | 0.0% |
| Blackjack | 154 | 0.0% | 0.0% | 0.0% | 0.0% | 2,734 ms | 8,099 ms | $0.0012 | 100.0% | 0.0% |
| No-Limit Hold'em | 833 | 5.2% | 0.0% | 0.0% | 5.2% | 6,334 ms | 39,008 ms | $0.0018 | 98.3% | 0.0% |
| Roulette | 300 | 17.7% | 0.0% | 0.0% | 17.7% | 10,597 ms | 59,669 ms | $0.0053 | 89.3% | 0.0% |
Failure classes and denominators
Model-caused: late response, timeout, truncation, schema or parse error, illegal action.
Provider-caused: rate limit, provider or connection error, authentication error, model not found, unsupported parameter. Any other non-none code is an other failure.
Over-limit means a rejection code beginning with over_; it is independent of failure classes. Latency is total decision latency. “—” means a cost or latency value was not recorded.
Notable facts
- GLM 5.3 Flash (aperture): 26.6% of holdem decisions failed, mostly late response.
- GLM 5.3 Flash (aperture): 40.0% of roulette decisions failed, mostly truncated.
- Grok 4.7 (aperture): 30.0% of roulette decisions failed, mostly late response.