Independent pilot integrity review

Opportunity: Obtain blinded, independent construct and probe labels for ET-004-P01 and ET-014-P01.

Background and unknowns: Designer calibration cannot establish construct validity. It is unknown whether independent experts agree that the tasks are answerable, representative, leakage-resistant, and correctly graded.

Origin and evidence: research/evaluation/NEXT-AGENT-START-HERE.md (“Current Blocker”); 13-independent-review-protocol.md; 07-results.md (“Scope Warning”, “Failed/Unexecuted Tests”); EV-E014.

Dependencies: External reviewers with no role in task design.

Suggested REP/method: Cycle 004, preregister reviewer materials, collect labels independently, measure agreement, adjudicate only after capture, then validate/revise/quarantine each pilot.

Outputs and success: Immutable reviews, agreement table, adjudication record, and pilot disposition. Success requires two independent reviews per pilot and no concealed disagreement.

Recommended agent: Research coordinator plus independent domain reviewers. Effort: small. Expected gain: converts the main validity assumption into evidence and gates all baseline work.