FirstRun

Verify the newcomer onboarding and first-run experience of any software repository.

Have you ever discovered an exciting open-source repository, followed the README step-by-step, only to hit an unhandled dependency error, a missing .env template, a broken setup link, or undocumented commands?

FirstRun inspects a repository from the perspective of a brand-new contributor or user. It identifies configuration gaps, broken documentation, and missing onboarding guardrails before your users do.

---

Why FirstRun?

Most repository linters focus on code style (flake8, ESLint) or security vulnerabilities (Dependabot, Snyk). FirstRun focuses on the human onboarding journey:

• Can a newcomer immediately understand what this project is?
• Are setup and test instructions present and unambiguous?
• Do relative documentation links and anchors actually point to real files?
• Is there an environment variable template (.env.example), or does the project risk breaking silently?
• Are dependencies pinned deterministically with lockfiles?
• Are configuration files (JSON, YAML, TOML) syntactically valid?
• Can setup and test commands be reliably extracted and verified?

---

Quickstart & Installation

Requirements

• Python 3.11+

Local Installation

Clone and install FirstRun locally:

git clone https://github.com/Navaneethan-P/FirstRun.git
cd FirstRun

# Create a virtual environment
python -m venv .venv
source .venv/bin/activate  # Or on Windows: .\.venv\Scripts\Activate.ps1

# Install FirstRun in editable development mode
pip install -e ".[dev]"

Verify the installation:

firstrun --version

---

Running FirstRun

Inspect any local repository by pointing FirstRun to its path:

# Check the current directory
firstrun check .

# Check another local repository
firstrun check ../another-project

# Output machine-readable JSON
firstrun check . --format json -o firstrun_report.json

# Generate a GitHub-flavored Markdown report
firstrun check . --format markdown -o report.md

# Fail on warnings (ideal for CI pipelines)
firstrun check . --strict

Example Terminal Output

FirstRun Onboarding & Diagnostic Report
Target: . (Python)

Findings:
[PASS] DOC001   README Present               Found README.md (7000+ characters).
[PASS] DOC002   Setup Instructions Present   Found documented setup section.
[PASS] DOC003   Test Instructions Present    Found testing guidance.
[PASS] LNK001   Relative Documentation Links All relative doc links point to existing files.
[PASS] ENV001   Environment Configuration    No environment configuration required.
[PASS] CMD001   Documented Commands          Extracted 13 documented setup/test commands.
[PASS] CFG001   Configuration Syntax Valid   All JSON, YAML, and TOML configs are valid.
[PASS] REP001   License File Present         Found MIT LICENSE.
[PASS] REP002   Contributor Guide Present    Found CONTRIBUTING.md.
[PASS] REP003   CI Configuration Present     Found 1 automated CI workflow.
[PASS] REP004   Test Suite Present           Detected automated tests directory.

[OK] Excellent! Repository passed all onboarding checks with no blockers or warnings.

Pre-generated full report examples are available in sample_report.md and sample_report.json.

---

Running the Automated Test Suite

FirstRun includes a comprehensive test suite with realistic test repository fixtures:

pytest

To run test coverage analysis:

pytest --cov=firstrun --cov-report=term-missing

---

Initial Verification Checks

Check ID	Category	Name	What It Checks
DOC001	documentation	README Presence	Verifies that a non-empty, informative README.md exists.
DOC002	documentation	Setup Instructions	Ensures installation or quickstart sections are clearly documented.
DOC003	documentation	Testing Instructions	Checks if the documentation instructs newcomers how to run tests.
LNK001	doc_links	Relative Documentation Links	Detects missing or broken local relative links without flagging external URLs.
ENV001	environment	Environment Template	Checks if the project requires env vars and verifies a safe .env.example template is present.
LCK001	lockfiles	Dependency Determinism	Verifies presence of lockfiles (package-lock.json, poetry.lock, etc.) to prevent dependency drift.
CMD001	commands	Documented Commands	Reliably extracts setup/test commands, flagging uncertain or complex shell pipelines for manual review.
CFG001	configuration	Config File Syntax	Validates syntax of JSON, YAML, and TOML configuration files.
REP001	repository	Open-Source License	Confirms an open-source LICENSE file is present.
REP002	repository	Contributor Guidelines	Checks for CONTRIBUTING.md guidelines.
REP003	repository	Continuous Integration	Verifies presence of automated CI workflows (.github/workflows/, etc.).
REP004	repository	Test Suite Presence	Checks for test directories and automated test files.

Verified Test Results & Real-World Audit

FirstRun has been audited against automated regression test suites and real-world multi-ecosystem repositories:

• Automated Test Suite: 50 tests passing, 87% statement coverage across Linux, macOS, and Windows.
• Tested Repositories:
  • FirstRun (Python CLI tool — passed with zero warnings or blockers)
  • verra (Node.js/Next.js/TypeScript — detected conflicting lockfiles package-lock.json + pnpm-lock.yaml, and committed .env files in subdirectories)
  • vibeDeploy (Python backend + Node.js frontend — identified standard command syntax vs complex commands requiring review)
  • synapse-docs (Multi-service repository — verified URL-encoded %20 path resolution and cloud configuration validation)
  • student_resource (Python data science repo — verified filtering of ASCII directory trees and mathematical formulas from command extraction)

---

Known Limitations

1. Static Analysis Only: FirstRun inspects repository files, configurations, and documentation. It does not execute repository code. A passed check confirms that documented commands and configurations are syntactically sound and structurally complete, not that third-party upstream packages will succeed on every machine.
2. Detection vs. Execution Support: Detecting language manifests (e.g. pom.xml, go.mod, Cargo.toml) confirms project structure, but sandboxed execution verification is a separate capability planned for future isolated runner phases.
3. Complex Shell Commands: Multi-stage pipes (curl ... | bash) or dynamic environment expansions in documentation are intentionally flagged as Requires Review rather than guessed.

---

Safety & Security Principles

FirstRun is designed to inspect untrusted repositories safely:

1. Zero Host Execution in Static Mode: FirstRun never executes repository code, scripts, or build steps during static analysis. Verified by automated security canary tests (test_static_analysis_never_executes_repository_code).
2. Safe Path Resolution: Resolves canonical paths and prevents path traversal (../../) or symlink escapes outside the target repository (is_path_contained).
3. No Secret Exposure: Identifies and redacts potential API tokens, credentials, or keys from reports (mask_secrets).
4. Offline & Private: Runs 100% locally on your machine with zero telemetry or data uploads.

See SECURITY.md for full details.

---

Contributing

We welcome contributions! Please see CONTRIBUTING.md for instructions on adding new checks, language detectors, and formatters.

---

License

This project is licensed under the MIT License.