[PUSHPAK-JAISWAL]

What & why

Bridges the gap from "a test failed" to "I know why and what to do", delivering zero-configuration, 100% local, deterministic Failure Intelligence that works out of the box on the first run with no AI or API key required.

Background & Problem

Prior to this PR:

1. Classification was limited to fine-grained technical categories (FailureCategory: LOCATOR_NOT_FOUND, TIMEOUT, STALE_ELEMENT, etc.) without high-level triage categorization (Product bug, Test bug, Environment, Flaky).
2. No prescriptive remediation advice ("What to do") was generated without an external AI API key configured.
3. When opt-in AI failure analysis was enabled (ai.failureAnalysis: true), prompts contained freeform stack traces and unstructured instructions without baseline deterministic classifications, yielding unstructured free text.
4. Flakiness status (PASSED_ON_RETRY, INTERMITTENT) was tracked in metrics history but not factored into high-level failure triage.

Changes Made

1. High-Level Failure Classification (FailureType):

   • Introduced FailureType enum: PRODUCT_BUG, TEST_BUG, ENVIRONMENT, FLAKY, and UNKNOWN.
   • Enhanced FailureCategory with deterministic mapping via defaultType() and initial remediation via defaultAction():
     • ASSERTION_MISMATCH $\rightarrow$ PRODUCT_BUG
     • LOCATOR_NOT_FOUND, ELEMENT_NOT_INTERACTABLE, STALE_ELEMENT $\rightarrow$ TEST_BUG
     • NAVIGATION_ERROR, BROWSER_CRASH $\rightarrow$ ENVIRONMENT
     • TIMEOUT, UNKNOWN $\rightarrow$ UNKNOWN
   • Added status-aware classification in FailureClassifier.classify(TestTiming, TestStatus):
     • Tests with status PASSED_ON_RETRY or INTERMITTENT are promoted to FLAKY.
     • Tests with NEWLY_FAILING maintain their underlying failure type with an explicit regression note.

2. Context-Aware Prescriptive Guidance ("Suggested Action"):

   • FailureClassifier synthesizes concrete next steps without AI:
     • For LOCATOR_NOT_FOUND / ELEMENT_NOT_INTERACTABLE: extracts the failing selector and advises verifying DOM state and auto-waiting conditions.
     • For console errors with HTTP 5xx responses: detects backend service failures and shifts triage from locator fixes to backend endpoint health.
     • For NAVIGATION_ERROR / BROWSER_CRASH: advises checking network reachability, DNS, or WebDriver/browser binary compatibility.

3. HTML Report & Export Integration:

   • HTML Dashboard: Rendered color-coded failure-type-badge badges and prominent failure-action callouts in failure detail rows and same-cause group panels.
   • Metrics JSON: Added failureType and suggestedAction under each test entry alongside failureCategory and failureEvidence.
   • JUnit XML: Enriched failure tag bodies with [FailureType] and Suggested Action: for direct triage in CI build logs.

4. Structured AI Analyzer Prompt:

   • Updated AiFailureAnalyzer to pass the local rule-based FailureType and suggested action into Claude/OpenAI/Gemini prompts as a baseline, prompting for structured output:
     • Classification: <PRODUCT_BUG | TEST_BUG | ENVIRONMENT | FLAKY>
     • Root Cause: <1-2 sentences>
     • Suggested Fix: <concrete next step>

5. Cucumber Lifecycle Guard:

   • Guarded CucumberHooks.checkCucumberQuarantine with SeleniumBootContext.isInitialized() so unbootstrapped unit tests in CucumberQuarantineTest don't silently swallow quarantine exceptions.

6. Documentation & Changelog:

   • Added comprehensive guide docs-site/docs/guides/failure-intelligence.md covering the four pillars, taxonomy, remediation, root-cause grouping, and opt-in AI enrichment.
   • Updated docs-site/docs/reporting/html-report.md and ROADMAP.md.
   • Documented under ### Unreleased in CHANGELOG.md and docs-site/docs/changelog.md.

Closes #138

Type of change

• Bug fix (guarded uninitialized context in CucumberHooks)
• New feature (opt-in, aligned with the framework philosophy)
• Documentation
• Internal refactor / test coverage

Checklist

• All existing tests pass (mvn test / mvn clean verify — 748 tests run, 0 failures, 0 skipped)
• New behaviour is covered by unit tests (FailureClassifierTest, FailureDetailTest, FailureDetailReportTest, FailureDetailExportTest, AiFailureAnalyzerPromptTest)
• No new external dependencies introduced without prior discussion
• Code follows existing conventions (no new frameworks, minimal abstraction, zero network overhead by default)
• No breaking changes to @SeleniumBootApi public APIs (all previous FailureClassifier.classify / Result constructors preserved via backward-compatible overloads)
• Docs updated if user-facing behaviour changed (docs-site/docs/guides/failure-intelligence.md, docs-site/docs/reporting/html-report.md, ROADMAP.md, CHANGELOG.md)

Notes for reviewers

• 100% Offline / Local Default: Failure Intelligence sends no data over the network and requires zero API keys. Every classification shows the underlying evidence and leaves raw exception traces one click away.
• Backward Compatibility: FailureClassifier.Result records and previous single-argument/two-argument classify(...) signatures remain fully functional, supplying sensible defaults for type and suggestedAction.
• Ambiguity Disambiguation: In FailureClassifierTest, calls to FailureClassifier.classify(null, null) were disambiguated to classify((String) null, (String) null) to avoid compiler ambiguity with the new (TestTiming, TestStatus) overload.

[PUSHPAK-JAISWAL]

@shossain786 i know this is a large change feel free to send it back if required please take your time in reviving this. If Any changes please let me know. Also i have tested it on my side and works ok.

[shossain786]

@PUSHPAK-JAISWAL Thanks for picking up #138.
I ran this PR in selenium-boot-test with a few deliberately failing tests. The Product bug and Environment badges, the "Suggested action" banners, and failureType / suggestedAction in the metrics JSON all work as described.

Before we merge, could you make these changes?

1. Wrong classification for a missing element. findElement(By.id("does-not-exist")) is tagged Environment (Navigation error) instead of Test bug. Selenium's error message includes a capabilities dump containing pageLoadStrategy, and that trips the navigation text match. Please make the exception class chain win over message text, so a NoSuchElementException or LocatorException is never classified as Navigation error from the message alone. Add a test for this case.

2. PR description and docs don't match the code.

   • The description says TIMEOUT maps to UNKNOWN. The code and the new guide map it to ENVIRONMENT. Please make them agree. I'd lean towards a timeout being Unknown, since a timeout can be a slow page or a bad wait and doesn't clearly point to the environment.
   • ROADMAP.md marks design: Failure Intelligence — know why a test failed, with no API key #138 as completed, but design: Failure Intelligence — know why a test failed, with no API key #138 is still open. Please remove that line. feat(failure-intel): rule-based failure classification #139–feat(failure-intel): detail record per failure, reused by AI analysis #142 are already merged.

3. Please keep the classifier readable. The deeply nested if/else block in FailureClassifier.classify is hard to review. Please restore a flat early-return or helper structure, and keep the diff to what's new (FailureType, suggested action, status-aware override).

4. Please confirm two design choices:

   • A test that passed on retry is shown as Flaky, which hides the underlying cause (e.g. Test bug). Could the cause stay visible as well, such as "Flaky (Locator not found)"?
   • Please check that NEWLY_FAILING with an assertion error is always a Product bug. A changed test expectation would also produce that.

Everything else looks good, and CI is green. Please push the changes and I'll re-check.

[PUSHPAK-JAISWAL]

@PUSHPAK-JAISWAL I will start on the changes in some time.

[PUSHPAK-JAISWAL]

@shossain786

1. Missing element classification fix: The exception class chain now strictly takes precedence over message text matching. Element exceptions (NoSuchElementException, LocatorException, ElementNotInteractableException, StaleElementReferenceException) are matched before error message text searches. We also removed pageloadstrategy from the navigation needle list so Selenium capabilities dumps never trip navigation matching. Added unit test missingElement_withCapabilitiesDump_isLocatorNotFoundNotNavigationError verifying this.
2. TIMEOUT mapping & docs: Aligned code, docs, and tests so TIMEOUT maps to UNKNOWN.
3. ROADMAP.md: Removed the line marking tracking issue design: Failure Intelligence — know why a test failed, with no API key #138 as completed.
4. Classifier readability: Flattened FailureClassifier into clean, early-returning helper logic without deep nesting.
5. Flaky underlying cause: When a test passes on retry or is intermittent, the badge and Result.displayLabel() now display Flaky (<Category>) (e.g. Flaky (Locator not found)), keeping the underlying cause visible while indicating non-deterministic execution.
6. Newly failing assertions: Updated the remediation guidance to explicitly prompt checking whether application behavior regressed or if requirements/mock data changed and test expectations need updating.
7. Synced with master: Cleanly merged upstream/master (including PR fix(report): open on Dashboard, collapse failures by default, stop Failures tab emptying #178 tab fixes) and resolved changelog conflicts.

All 752 tests are passing green.