Refresh

Overview

Refresh builds simulated software worlds for coding and computer-use agents. Its public offering describes environments with verifiable rewards for both evaluation and training. The company identifies frontier labs and enterprises as its intended users.

Three published environment categories are connected software suites, terminal-style engineering tasks, and extended tool-based workflows. Examples include business and health-record applications, debugging failed CI pipelines, and tasks requiring many dependent tool calls. Refresh says it works with expert engineers to identify real failures, reproduce them, and turn them into tasks with checks.

The company describes both deterministic checks and rubric-based verification. That distinction matters: a test suite can establish one kind of correctness, while a broader artifact may require criteria that involve judgment. The reliability of the reward should be assessed rather than assumed from the label verifiable.

Where the approach fits

Our assessment is that failure-driven environment design can be useful when a team already knows which workflows its agent cannot complete. In an illustrative engineering case, an agent may repeatedly fix the visible symptom of a broken pipeline while missing a dependency problem. Reproducing that failure in a controlled setting makes it easier to compare interventions.

However, training only on known failures can narrow the task distribution. A strong program needs variation and separate tests that check whether the model learned the underlying skill. It should also verify that the environment remains representative as software and agent capabilities change.

Evaluating fit

Ask for an example linking the original failure to the resulting task, reference outcome, and grader. Inspect whether the environment contains the state and tools needed for a legitimate solution. Test several approaches, including plausible shortcuts, to understand the verifier's coverage. A pilot should compare performance on fresh task variants and report both completion and unintended side effects. Clarify whether the engagement supplies existing environments, a bespoke dataset, or custom workflow development. Define maintenance, hosting, and licensing responsibilities explicitly. The practical value is a repeatable way to improve a specific capability, with evidence that higher reward corresponds to better work outside the training examples.