Credibility Check Audit

Credibility Check Audit Workflow

Goal: When an external reasoning tool (Deep Think, Deep Research, or any other source) returns results that identify issues the original credibility analysis missed, this workflow forces the original analyst to defend their work against those findings using a structured adversarial exchange (Auditor's Challenge → Analyst's Defense). The output drives a v2 revision of the memos and synthesis.

Output: A file named adversarial_debrief_[source].md (e.g., adversarial_debrief_deep_research.md) saved in the credibility check's session folder. The file ends with explicit incorporation recommendations that drive the v2 memo revisions.

I. Prerequisites

The caller (the credibility-check workflow, Phase 5) must provide:

1. The Source Name — a short label for the external tool (e.g., deep_think, deep_research). This determines the output filename.
2. The External Results File — the saved output from the external tool. (If the user provides results via clipboard or chat, save the raw text to [source]_raw_results.md in the session folder first.)
3. The Session Folder — the credibility check's numbered session directory.
4. The Original Work Product — the memos (01_claim_is_credible.md, 02_claim_is_not_credible.md, 03_other.md, 04_synthesis_and_verdict.md) and the README.md claim breakdown.

II. Step-by-Step Execution

1. Ingest Phase: Read the External Results File thoroughly. Read the original memos and synthesis. Identify every finding in the external results that is NOT already present in the original work product. Discard duplicates.

2. File Creation: Create adversarial_debrief_[source].md inside the session folder. Include at the top:

   • The source name and file path
   • A brief list of the new findings being audited
   • Which findings were already covered (if this isn't the first audit)

3. Auditor's Challenge: Assume the persona of a rigorous External Auditor. For each genuinely new finding, write a pointed question directed at the original analyst. Write a maximum of 7 questions. Save under the heading ## Auditor's Challenge. Calibrate hostility by cause:

   • (a) Analytical errors or overlooked data the analyst should have found → hostile ("why did you miss this? the data was publicly available").
   • (b) Foreseeable gaps the analyst should have flagged as uncertain → critical but fair ("why didn't you flag this as an open question?").
   • (c) Genuinely new information unavailable when memos were drafted → no blame; just ask how the new facts alter the analysis.

4. Persona Pivot: After writing the Auditor's Challenge, output the following line verbatim: ### ⚔ PERSONA SWITCH: I am now discarding the Auditor role and fully adopting the ORIGINAL ANALYST persona. I will defend the work without reflexively apologizing, conceding only where genuinely warranted. ⚔ ### Then immediately proceed to Step 5.

5. Analyst's Defense: Respond IN CHARACTER to each Auditor question. Do not reflexively apologize — explain WHY choices made sense under the constraints at the time. Concede only where genuinely warranted. For each answer, end with one of: Full concession, Partial concession, or No concession. Append under ## Analyst's Defense. Then append THREE mandatory sections:

   • ## Debrief Summary Table — columns: Question, Verdict, Impact on Analysis.
   • ## Incorporation Recommendations — for EACH finding, explicitly state YES incorporate or NO do not incorporate, with reasoning. Include a subsection ### Impact on Monte Carlo Histograms estimating the net shift in each interpretation's distribution.
   • ## Process Improvements — any workflow changes surfaced by the debrief.

III. After Completion

The adversarial debrief file now contains the Incorporation Recommendations. This drives the v2 revision:

1. Create v2 memos: For each memo that needs updating per the Incorporation Recommendations, create a new file with the _v2 suffix (e.g., 01_claim_is_credible_v2.md). Do NOT overwrite the originals — they serve as the audit trail.
2. Create v2 synthesis: Create 04_synthesis_and_verdict_v2.md with updated Monte Carlo distributions, updated evidence summaries, and a revised Executive Verdict reflecting all incorporated findings.
3. Wait for all audits: Do not create v2 memos until ALL external source audits for this session are complete. The v2 should incorporate findings from all sources at once.