System Weakness Analysis and Improvement Report

So far, every test has asked "is the system working?" This report does the opposite: "where does the system break?" Six weakness hypotheses were formed, each tested separately, the most serious one was fixed, and the fix was confirmed to have also improved a second issue.

Diagnostic Summary — Six Hypotheses

Three real weaknesses (H2, H3, H6), one moderate issue (H1), two clean areas (H4, H5).

H2 — Fragile Party Exclusion (most serious finding)

Test: 8 bosses × all parties, Normal win rate for parties with heal vs. without heal.

Finding (before fix):

Three-quarters of fragile parties couldn't practically win even on Normal. This shows the system is overconcentrated on a single axis — durability/healing. The imposition of "build the right party (bring a healer) or die" restricts player freedom.

Root cause: Boss damage flows "flat" — a similar amount every turn. The party with healing absorbs this every turn (heal was especially strong at T3/T4, e.g. 5d8+8 / 6d8+10); the party without healing dies on a single bad turn. The presence or absence of a safety net was determining everything.

H3 — The Key System's Hidden Ineffectiveness

Test: Among durable parties, the win-rate difference between having the Key vs. not.

Finding (before fix): The average Key advantage was only +10 points. ±2 on most bosses. Meaningful (+37) only on burst-heavy bosses like the Red Dragon.

Interpretation: This is surprising and troubling — because the Key system is the heart of this project. But for durable parties it was almost irrelevant: they had healers and won 94-96% even without the Key. The Key only made a difference for parties that were already in a tough spot.

Connection to H2: The two are two faces of the same root cause. When heal is too strong, it both excludes the fragile (H2) and overshadows the Key (H3). The healing monopoly was crushing both party diversity and Key-puzzle gameplay.

H4 & H5 — Solid Areas

H4 (Telegraph): Tested on three different bosses, works consistently on all of them. Phase-transition fairness is preserved. No issue.

H5 (Robustness): When boss HP was varied by ±10%, win-rate sensitivity was 0.1-0.9 points/1% HP. In other words, the system is stable against small parameter changes — small tweaks when tuning a boss give predictable results, no chaotic jumps. This confirms the tool's reliability.

H6 — The Battery Doesn't Represent the "Average Party"

Test: The current 6 archetypes are always extremes (full-tank or pure-nova). Real tables are usually in between: semi-durable, single healer, partial control. What happens when such a party is added?

Finding: The semi-durable middle party scores 94% on Fire Giant Normal — meaning the party built as "average" actually performs at the same level as the durable group. Because even a single healer is enough to place it in the "durable" category.

Interpretation: The battery's binary structure (fully-durable vs. purely-fragile) doesn't fully reflect reality. There's a spectrum in the real world. But since H5 (robustness) is solid, this doesn't distort the results — it just means the battery could be richer. Action: middle archetypes could be added (optional improvement).

H1 — Tier Consistency (known moderate issue)

The tier-spread for the Hard and Deadly tiers remains high:

Normal is very consistent, but on Hard/Deadly the difference between bosses is large (one boss at 4% on Hard, another at 51%). This was also seen in report 16. Within acceptable bounds but improvable — Hard/Deadly multipliers could scale more intelligently relative to the boss's Normal difficulty.

Fix Applied — "Fragile Escape + Heal Reduction"

Since H2 and H3 stemmed from the same root cause, I targeted both with a single fix.

Two changes (new Rule 5)

1. Heal power reduced to ×0.72 — softened the durable parties' "full heal every turn" monopoly.
2. Fragile parties get a constant 18% damage reduction (escape allowance) — parties without healing represent surviving through maneuvering/positioning.

Rationale: A party without healing doesn't die stupidly at a real table — it flees, takes position, uses control. The model was missing this with its "flat damage" assumption. The escape allowance compensates for that.

Result — balanced point after parameter sweep

Two gains at once: The gap was cut in half (H2) and once the heal monopoly broke, the Key advantage doubled (H3) — a single fix improved two issues at once. And all bosses still PASS 4/4 on Normal.

Why not full equality?

Fragile 24%, durable 64% — a gap remains but it's deliberate. Durability should be rewarded (bringing a healer is a smart choice), it just shouldn't be a monopoly. 24% means "a fragile party can win if played well"; 0% meant "never." The goal wasn't equality, it was a fair chance.

Remaining Directions for Development (in priority order)

1. Hard/Deadly consistency (H1) — moderate priority

Multipliers scale by a fixed tier rather than by the boss's Normal difficulty. A boss-based scale (a boss at 55% Normal and one at 78% Normal shouldn't get the same Hard multiplier) could reduce the spread.

2. Battery enrichment (H6) — moderate priority

To better represent real tables, "middle" archetypes: semi-healer, dual-key, hybrid. The current binary (tank/nova) structure doesn't distort results (H5 is solid) but could give finer resolution.

3. Further narrowing the fragile gap (H2 remainder) — low priority

A 40% gap is acceptable, but the escape allowance could be increased slightly more if desired. Caution: if increased too much, the sign reverses (fragile > durable), which testing showed.

4. Strengthening the Key on non-burst bosses (H3 remainder) — low priority

The Key is now +23 but still stronger on burst bosses like the Red Dragon. Making the Key more decisive on control/soldier bosses would increase gameplay variety.

Assessment

Does the system have room for improvement? — Yes, and the most important one was fixed

The stress test found a real structural weakness beneath a system that appeared to be "working": the durability/healing monopoly was crushing both party diversity (H2) and overshadowing the Key system that is the heart of the project (H3). This wasn't visible in normal tests because the averages were balanced — the problem was beneath the distribution.

Was the fix successful?

Yes, but partially: the gap dropped from 81→40 points (halved), the Key advantage rose from +10→+23 (doubled), and the full checklist was preserved. Full equality wasn't the goal (durability should be rewarded); a fair chance was the goal, and it was achieved.

The most valuable lesson

"Balanced on average" ≠ "system healthy." While the average of six battles looked fine, beneath it was an 81-point archetype gap. The system needs to be audited not just with aggregate metrics but with subgroup distributions. This lesson can be added to the checklist as a permanent test (Test 5: archetype gap).

Methodology: Six hypotheses were tested separately, broken down by subgroup (fragile/durable, keyed/unkeyed); parameter perturbation (HP ±10%) was used for robustness. Fix parameters (escape ×0.82, heal ×0.72) were calibrated with a full-battery sweep. 800-1,500 repetitions per party, preserved dice variance. Absolute percentages are model-sensitive; what's reliable is the subgroup differences and the before/after comparison of the fix. Updated engine.py in the output. This report is an extension of documents 10, 12-17.

License: Original work. Rule terms are based on the SRD 5.2 (Wizards of the Coast LLC, CC-BY-4.0). This work includes material from the SRD 5.2 by Wizards of the Coast LLC (https://www.dndbeyond.com/srd), licensed under CC-BY-4.0.