Regression Testing: Proving Your Code Doesn't Lose Data
· 3 min read
Building a working prototype for yourself is fairly easy. Pushing that rogue software into the official processes of a massive corporation is much harder. We finished developing the core logic and had to defend the project in front of management. People were used to the old generator. It spent hours building documentation while remaining predictable and providing strict control over code coverage.
The Wall of Distrust
Our engine relied on a completely different architecture. We dreamed of writing our own lightweight parsers. Tight deadlines forced us to grab Doxygen instead. It runs slowly and barely caches anything, but it consistently produces stable XML. Our code ingests this format, processes it into a custom data model, and renders clean Markdown.
Management looked at this layered setup with deep skepticism. They asked us directly for guarantees that the third-party parser wouldn’t skip half the classes in our complex C++ codebase. Promises were useless. We needed hard numbers.
The Head-to-Head Collision
We wrote a regression testing script to directly compare the old corporate standard against our new Flude generator. We ran these tests across 36 combinations of SDKs and programming languages, including C++, C#, and Python.
The approach involved extracting the exact count of classes, methods, and enumerations from the legacy documentation. We then cross-checked those numbers against the output of our engine. All the data flowed into a single dashboard. We monitored the baseline metrics, our findings, and the final match percentage. The dashboard also listed missing entities and accidentally included garbage.
Battling the Discrepancies
The initial runs produced tons of inconsistencies. Doxygen choked on complex C++ template constructs and generated empty files instead of actual code. Internal autogenerator classes leaked out in the C# and Python wrappers—classes which the old tool had silently hidden.
We started the tedious grind of writing on-the-fly preprocessors. We stripped out problematic code blocks before parsing even started. We invented merge rules for anonymous enumerations and scrubbed utility trails. Every tweak plugged another hole in the metrics. We methodically adjusted the results until the match percentage reached 99.9-100% across all 36 combinations.
The final numbers proved that the Flude and Doxygen combination found the exact same tens of thousands of entities as the old standard. Nobody gave us the green light immediately. We simply validated the concept for ourselves. Our documentation includes detailed Developer Guides alongside the API references. The next article covers how we integrated those guides into the new generator.