Testing and Quality
The test suite, the sweeps, the hardening record and CI
The framework is developed test-first in the literal sense: every subsystem has tests, and whole-surface sweeps check universal invariants across every component type. This page explains what is covered, how to run it, and what the tests have caught.
Table of Contents
Running the Tests
dotnet test Beep.Skia.Tests/Beep.Skia.Tests.csproj # 501 tests
dotnet test --filter "FullyQualifiedName~HardeningTests" # just the hardening suitesTests target net10.0-windows10.0.19041.0 and run serialized: several suites exercise process-wide state
(themes, static caches), so parallel class execution would make them flaky.
What Is Covered
- Every library — all 26 projects, including the loader integration.
- Feature coverage — each family's algorithms (codegen, schema diffing, CPM, XMI/BPMN export, PageRank, DREAD, ERC, LAS grouping) and each subsystem (automation, extensions, marketplace, collaboration, assistance).
- Rendering — smoke tests that assert ink is produced, image-diff checks for edits, and pixel checks for migrated drawing code.
- End-to-end — one scenario drives the whole story: author → save → reload → execute → export → collaborate → extend → generate.
Whole-Surface Sweeps
Rather than trusting per-type tests, sweeps instantiate every public component type (293 of them) and assert universal invariants:
| Invariant | Why it matters |
|---|---|
| Renders without throwing | A component that throws breaks the whole canvas |
| Updates without throwing | Update runs every frame, outside drawing |
| Survives a save/load round-trip | Silent data loss is worse than a crash |
| Accepts its own property values | This is exactly what loading a diagram does |
| Renders at 0.1× to 8× zoom | Zoom-scaled drawing is easy to get wrong |
| Survives the input pipeline | Click, drag, pan, wheel and keys run on every component |
| Supports clipboard and undo/redo | Editing must work for every type |
Hardening Record
Adversarial tests feed malformed, hostile and extreme input to every entry point: diagram DTOs, DDL, expressions, the flowchart and mind-map DSLs, JSON/XML flattening, BPMN XML, LAS files, packages and vaults, plus concurrency and resource-lifecycle checks. They found and fixed 24 defects, including:
- Six crash paths — unbounded recursion in the expression parser (nested parentheses, unary chains, nested calls), the expression evaluator (very long flat chains), the mind-map layout, and a component that drew a path it had already disposed.
- Three silent save/load data losses — automation connections, automation configuration, and ML connections were dropped on reload.
- Three data races — collaboration state, the credential vault, and the package catalog were not thread-safe.
- A decompression bomb and an authorization bypass (role state handed out by reference) in the marketplace and collaboration services.
Continuous Integration
.github/workflows/ci.yml builds Beep.Skia.CI.slnf, runs the suite, and packs packages on
tags. It treats warnings as errors and the solution is warning-free with no suppressions, so the
quality bar is enforced rather than aspirational. Versions come from git tags through MinVer.