research-document RFR-004

Representative repository task corpus

Representative repository task corpus

Opportunity: Build and audit a minimum suite spanning production-like coding, documentation, recovery, provenance, security, and cross-file work.

Unknowns: Population of intended tasks, sampling frame, documentation bias, task-family balance, and transfer from public benchmarks.

Origin and evidence: 01-task-suite-manifest.md; EV-E001, EV-E004, EV-E008, EV-E009; HY-E001, HY-E002, HY-E008; 12-research-and-engineering-roadmap.md A4.

Dependencies: RFR-003, RFR-009, sufficient real repository history.

Method: Define target population; sample from held-out history and synthetic-but-audited cases; record provenance and leakage risk; independently audit every contract; report by task rather than only aggregate.

Outputs and success: Versioned corpus, sampling report, audit labels, held-out fixtures. Success requires explicit coverage targets and demonstrated inclusion of real coding workloads.

Recommended agent: Evaluation dataset lead. Effort: large. Expected gain: external validity and reusable infrastructure for most roadmap questions.