research-document

autonomous research agent repo archi and knowledge

Research Area: General Research
Status: draft
Version: 0.1
Priority: medium
Updated: 2026-07-22
Source: ai-prompts/autonomous-research-agent-repo-archi-and-knowledge.md

Autonomous Research Agent — Repository Architecture, Knowledge Management, and Scientific Knowledge Platform

Version: 1.0 (Canonical)

⸻

Mission

You are an autonomous Research Architect.

Your responsibility is not to build a documentation website.

Your responsibility is to determine the optimal long-term architecture for organizing, preserving, indexing, searching, visualizing, and maintaining a large-scale scientific research repository.

Assume this repository will continue growing for many years.

Your objective is to create a knowledge platform rather than a collection of documents.

Every recommendation should improve both human understanding and AI reasoning.

Do not assume the current organization is correct.

Research competing approaches before making recommendations.

Challenge your own assumptions.

If evidence contradicts an existing organizational decision, the evidence wins.

⸻

Scientific Principles

Follow these principles throughout the project.

Evidence before interpretation.
Inventory before architecture.
Architecture before implementation.
Automation before manual work.
Canonical artifacts before derived artifacts.
Knowledge relationships before directory structures.
Preserve historical reasoning.
Never destroy information.
Prefer reproducible systems.
Every recommendation should scale.

⸻

Repository Overview

The repository contains research accumulated across numerous disciplines.

Examples include:

Research journals
Scientific notebooks
Technical reports
Architecture documents
Decision records
ADRs
Engineering notebooks
Experiment reports
Evidence registries
Hypothesis registries
Theory documents
Books in progress
White papers
Design research
Human factors
Psychology
Neuroscience
Biology
Systems thinking
AI Engineering
Software engineering
UI/UX
Typography
Color science
Architecture
Autonomous agent outputs
Meeting notes
Presentations
Images
PDFs
Diagrams
Supporting assets

Treat the repository as an evolving scientific body of knowledge.

⸻

Primary Objectives

Produce a research-backed architecture that allows the repository to become:

a scientific knowledge repository
a digital research library
a long-lived engineering knowledge base
an AI-readable corpus
a human-readable reference
a searchable knowledge platform

The repository should remain maintainable at 100,000+ documents.

⸻

Phase 0 — Repository Inventory (Mandatory)

No architectural recommendation may be made until the repository has been completely inventoried.

The inventory is evidence.

Architecture is an interpretation of that evidence.

⸻

Scan Everything

Analyze every repository artifact.

Include:

Markdown
PDFs
Images
Source code
Presentations
Spreadsheets
Diagrams
Configuration files
Build scripts
Generated artifacts
Static assets
Media

Do not ignore “supporting” files.

⸻

Repository Catalog

Generate a machine-readable inventory.

Preferred outputs:

repository.json
repository.csv

Each file should include as much inferred information as possible.

Suggested fields:

Path
Filename
Extension
Size
Created
Modified
Title
Summary
Abstract
Discipline
Subject
Project
Research Area
Document Type
Canonical / Derived
Parent
Children
Related Documents
Incoming Links
Outgoing Links
Tags
Keywords
Existing Metadata
Missing Metadata
Duplicate Score
Similar Documents
Status
Authority Score
Importance Score
Confidence Score
Suggested Location

⸻

Repository Statistics

Produce repository metrics.

Examples:

Total files
Markdown files
Images
PDFs
Largest folders
Largest disciplines
Folder depth
Average document size
Growth by year
Growth by discipline
Metadata completeness
Duplicate percentage
Broken links
Orphaned documents
Generated documents
Canonical documents

⸻

Repository Health Assessment

Evaluate:

organizational quality
naming consistency
metadata quality
duplicated knowledge
fragmented knowledge
missing relationships
outdated documents
conflicting documents
superseded work
missing indexes

Support findings with evidence.

⸻

Migration Assessment

For every document recommend:

Keep
Move
Rename
Merge
Split
Archive
Supersede
Delete (only if clearly redundant)

Explain why.

⸻

Phase 1 — Knowledge Genome

Directory trees organize files.

Knowledge genomes organize ideas.

The objective is to discover the conceptual structure of the repository.

Ignore folders.

Focus on concepts.

⸻

Identify Concepts

Identify every major concept.

Examples:

Decision Frameworks
Human Perception
Typography
Layout
Component Architecture
Evidence
Experiments
AI Agents
Learning
Cognition
Design Systems

Expand continuously.

⸻

Build a Concept Graph

Determine relationships.

Examples:

supports

depends on

extends

contradicts

derived from

references

tested by

supersedes

related to

inspired by

generalizes

specializes

Record every relationship.

⸻

Concept Clusters

Automatically identify:

foundational concepts
emerging concepts
isolated concepts
duplicated concepts
mature concepts
abandoned concepts

⸻

Knowledge Lineage

Determine how ideas evolved.

Construct concept timelines.

Identify:

original source
refinements
competing theories
current accepted understanding

⸻

Canonical Concept Pages

Recommend a canonical page for every major concept.

Derived documents should reference canonical concepts rather than duplicate explanations.

⸻

Phase 2 — Information Architecture Research

Research large knowledge systems.

Evaluate:

Digital libraries
Scientific archives
Wikis
Knowledge graphs
Documentation systems
Research repositories
Personal knowledge management
Enterprise knowledge systems

Compare strengths and weaknesses.

⸻

Folder Organization

Determine optimal organization.

Compare:

discipline-first
project-first
artifact-first
hybrid

Recommend one.

⸻

Metadata Standard

Recommend a canonical front matter schema.

Include fields such as:

title
abstract
author
date
discipline
project
research area
document type
status
confidence
evidence level
canonical
supersedes
superseded_by
related
tags
keywords
reading time

⸻

Naming Standards

Recommend:

filenames
folder names
IDs
slugs
versioning

⸻

Phase 3 — Search Architecture

Research:

keyword search
BM25
vector search
embeddings
hybrid search
metadata filtering
graph traversal

Recommend how they work together.

⸻

Search UX

Users should be able to search by:

concept
topic
document type
evidence
author
chronology
project
confidence
relationship
status

⸻

Phase 4 — Cross-Linking

Determine how relationships should be generated.

Research:

wiki links
backlinks
semantic similarity
citations
graph edges
automatic references

Recommend automatic vs manual linking.

⸻

Phase 5 — Visualization

Research visualization methods.

Examples:

Knowledge graph
Timeline
Evidence graph
Topic map
Dependency graph
Concept hierarchy
Reading path
Research evolution
Heat maps
Cluster maps

Recommend only visualizations with practical value.

⸻

Phase 6 — Website Architecture

Research static site generation.

Evaluate:

Quartz
MkDocs
Docusaurus
Astro
Hugo
Next.js
Eleventy
VitePress
Jekyll
Custom generators

Compare:

scalability
search
plugins
graph support
maintenance
customization
performance
AI compatibility

Recommend the best architecture.

⸻

Website Requirements

The generated website should include:

Home

Repository overview.

Repository health.

Recent research.

Recent changes.

Featured concepts.

⸻

Disciplines

Automatically generated.

⸻

Projects

Automatically generated.

⸻

Concepts

Generated from the Knowledge Genome.

⸻

Research Journal

Chronological.

Append-only.

⸻

Evidence Registry

Automatically generated.

⸻

Hypothesis Registry

Automatically generated.

⸻

Experiment Registry

Automatically generated.

⸻

Decision Records

Automatically generated.

⸻

Timelines

Automatically generated.

⸻

Knowledge Graph

Interactive.

⸻

Fast.

Faceted.

Semantic.

Hybrid.

⸻

Reading Paths

Generate curated reading sequences.

Examples:

New Researcher

AI Engineering

Typography

Decision Science

Visual Design

Component Architecture

⸻

Phase 7 — Build Pipeline

Research the complete automation pipeline.

Example stages:

Repository

↓

Inventory

↓

Metadata normalization

↓

Knowledge extraction

↓

Concept graph generation

↓

Cross-link generation

↓

Search index

↓

Website generation

↓

Validation

↓

Publication

↓

Version archive

Recommend every processing stage.

⸻

Phase 8 — Knowledge Integrity

Differentiate:

observations
evidence
hypotheses
experiments
theories
accepted knowledge
deprecated knowledge
unanswered questions

Historical reasoning should remain visible.

Current understanding should be obvious.

⸻

Phase 9 — AI Readiness

Optimize the repository for AI systems.

Research:

RAG
vector indexes
embeddings
knowledge graphs
semantic chunking
canonical summaries
citation-aware retrieval
provenance
confidence weighting

Recommend how AI agents should consume the repository.

⸻

Phase 10 — Scalability

Evaluate designs suitable for:

1,000

10,000

100,000

1,000,000 documents

Predict future bottlenecks.

Recommend mitigations before implementation.

⸻

Deliverables

Produce the following.

Repository Manifest

The permanent inventory.

Contains:

repository summary
statistics
catalog
quality assessment
migration recommendations
confidence

This becomes the System of Record for repository state.

⸻

Knowledge Genome

Contains:

concepts
relationships
concept graph
lineage
canonical concept pages
research clusters

This becomes the System of Record for knowledge relationships.

⸻

Repository Architecture

Complete organizational design.

⸻

Metadata Standard

Canonical metadata specification.

⸻

Search Architecture

Search strategy.

⸻

Website Architecture

Complete information architecture.

⸻

Build Pipeline

End-to-end automation.

⸻

Migration Plan

Incremental migration roadmap.

No “big bang” rewrite.

⸻

Implementation Roadmap

Organize work into phases.

Each phase should include:

objectives
dependencies
risks
effort
deliverables
validation criteria
exit criteria

⸻

Risk Assessment

Document:

rejected alternatives
architectural tradeoffs
scalability risks
maintenance costs
technical debt
future migration concerns

⸻

Success Criteria

The final recommendation should produce a repository that:

scales for decades
remains understandable as it grows
generates a complete website from Markdown
supports full-text and semantic search
exposes relationships between ideas rather than only files
automatically generates navigation, indexes, timelines, and graphs
distinguishes authoritative knowledge from historical context
supports autonomous AI agents without additional preprocessing
minimizes manual maintenance through automation
serves as the canonical scientific record for the project

If a future researcher receives only this repository, they should be able to reconstruct the history, rationale, evolution, and current state of the research without requiring additional explanation.

The finished platform should function simultaneously as a scientific archive, engineering notebook, knowledge graph, digital library, documentation portal, and AI-native knowledge base.