RL Environments & Coding Benchmarks for AI Labs

Private coding tasks with graders models can't game

Request a Sample Task

Why Coding Tasks Break During RL Training

A task can look fine, pass its tests, and still teach the model the wrong thing. Most problems come from four places:

• Reward hacking: the model finds a shortcut, such as editing tests or hard-coding outputs, and the grader rewards it.

• Toy repositories: clean, tiny repos teach clean, tiny skills. Real work has legacy code, odd dependencies and unclear issues.

• Contamination: tasks built from public repos or benchmarks may already be in the model's training data.

• Flaky runs: unpinned dependencies make a task pass once and fail the next time, which adds noise to the reward.

Fixing these takes careful engineering on every task, which is slow to do in-house.

What We Deliver

• Containerised repo tasks: a realistic codebase, a task statement and a pinned environment, ready to run.

• Graders and verifiers: scoring that checks behaviour, not just output format, with partial credit where you need it.

• Hidden test splits: tests the agent never sees, kept apart from any visible tests.

• Private coding benchmarks: held-out task sets to measure model progress, never published.

• Reference solutions: a working solution for every task, run against its grader.

• Task documentation: setup notes, expected behaviour and known ways the task could be gamed.

How We Stop Reward Hacking

1. Scope — task types, languages, difficulty and grading rules agreed with your team

2. Author — an engineer matched to the language and domain writes the task and its reference solution

3. Grade — visible and hidden tests are written and run against the reference solution

4. Break — a second engineer tries to pass the task without solving it; any shortcut found is fixed or the task is dropped

5. Reproduce — the task is run from a clean container to confirm it behaves the same each time

6. Review — a senior engineer checks the task against your guidelines before delivery

Example Task Format

An illustrative task spec. The final format follows your harness.

{
  "task_id": "env_0031",
  "language": "python",
  "repo": "inventory-service (containerised, dependencies pinned)",
  "issue": "Bulk import fails silently when a CSV row has a missing SKU.",
  "visible_tests": ["tests/test_import_basic.py"],
  "hidden_tests": ["held out, run by grader only"],
  "grader": {
    "type": "test_based",
    "checks": ["hidden tests pass", "visible tests pass", "test files unchanged"]
  },
  "reference_solution": "patch included, passes grader",
  "red_team_review": "passed: no shortcut found"
}

Coverage

Training Environments vs. Private Benchmarks

How It Works

1. Scope — we agree task types, languages, format and grading rules

2. Sample — we deliver a sample task so you can judge quality on your own criteria

3. Pilot — a 2-week pilot at small volume to check quality and throughput

4. Scale — a dedicated team delivers batches on an agreed weekly schedule

Engagement Models

• Per accepted task: a fixed price for each task that passes your review. Best for well-defined batches.

• Dedicated team: reserved engineers billed monthly, working in your tools and guidelines.

• Supply partner for environment platforms: coding capacity for companies that sell environments to labs, working inside your platform and rubrics.

Confidentiality & Ownership

• An NDA with every client, and confidentiality terms for every engineer

• All tasks, graders and benchmarks are assigned to the client

• Tasks are never published or reused across clients

• Work in your environment or an access-restricted workspace

Related Services

• AI Training Data: original coding tasks, reasoning traces and RLHF preference ratings

• LLM Evaluation & Benchmark Engineering: eval suites for AI products

• AI Red Teaming: adversarial testing before launch

• Forward Deployed Engineers: engineers embedded in your team monthly

• Frontier AI Labs: custom R&D and engineering for advanced AI work

FAQ

What is an RL environment for coding?

A coding task packaged so a model can attempt it and be scored automatically: a containerised repository, a task statement, a grader with tests, and setup files so every run behaves the same.

How do you stop models from gaming the grader?

Tests the agent cannot see or edit, graders that check behaviour rather than output format, and a separate engineer who tries to pass each task without solving it. Tasks that can be passed this way are fixed or dropped before delivery.

Are the tasks original?

Yes. Every task is written for your project and never published, so it cannot appear in public training data.

What formats do you deliver in?

Docker images, JSONL task files, issue-patch-test style tasks, or your own harness and spec. We agree the format during scoping.

Who owns the environments and benchmarks?

You do. Everything delivered is assigned to the client, under NDA.

How do we start?

Request a sample task. Once you approve it, we run a 2-week pilot at small volume, then scale on an agreed weekly schedule.

Start With a Sample Task

Tell us your task types, languages and harness. We'll build one matching task with its grader so you can test it yourself.

Request a Sample Task