tuneloop./docs
GitHub

GitHub

Tuneloop connects to GitHub via a GitHub App that you create in your organization. This provides PR lifecycle tracking, merge events, review data, and line-level code attribution.

Create the GitHub App

  1. Go to your GitHub organization's Settings > Developer settings > GitHub Apps > New GitHub App
  2. Configure the app:
    • Name: Tuneloop (or any name you choose)
    • Homepage URL: Your Tuneloop server URL
    • Webhook URL: https://your-tuneloop-server/api/webhooks/github
    • Webhook secret: Generate a strong secret
  3. Set permissions (read-only):
    • Contents: Read
    • Metadata: Read
    • Pull requests: Read
    • Issues: Read
  4. Subscribe to events:
    • pull_request
    • pull_request_review
  5. Create the app, then generate a private key (.pem file)
  6. Install the app on your organization's repositories

Configure Tuneloop

Add the GitHub App credentials to your deployment:

GITHUB_APP_CLIENT_ID=Iv1.abc123
GITHUB_APP_PRIVATE_KEY_PATH=/run/secrets/github-app.pem
GITHUB_APP_INSTALLATION_ID=12345678
GITHUB_WEBHOOK_SECRET=your-webhook-secret

Mount the private key file into both the web and worker containers.

If Tuneloop is deployed behind a firewall, allow inbound traffic from GitHub's webhook IPs so that PR events can reach your server. For GitHub.com, the current IP ranges are published at github.com/meta under the hooks key. Not required for GitHub Enterprise Server on your own network.

What It Provides

PR Lifecycle Tracking

GitHub webhooks feed PR state changes into Tuneloop in real time: opened, merged, closed, reopened, reviewed. This lets Tuneloop link coding agent sessions with appropriate outcomes such as pr_merged, along with tracking metrics such as time to merge.

Line-Level Attribution

On PR merge, Tuneloop matches the PR diff against its index of agent-authored lines to compute AI% — what fraction of the merged code was written by an AI agent, per PR. Line-level attribution also powers metrics like Code Churn Rate and AI Defect Rate, allowing Tuneloop to attribute bug fixes to AI sessions that originated bugs or tickets.

PR Enrichment

Tuneloop annotates each PR with complexity (trivial, routine, substantial, open-ended), work type, and agent autonomy level. This lets you track agent success rate by complexity — understanding not just how often agents succeed, but on what difficulty of work.

Benchmark Creation

Tuneloop mines merged PRs to automatically generate benchmarks — real-world coding tasks derived from your own codebase. Each benchmark captures the issue description, the repo state before the fix, and the expected outcome. Use these to evaluate new models, agent configurations, or prompt strategies against tasks that reflect your actual engineering work.

Custom HarnessesJira