Metaphi

Overview

Metaphi develops reinforcement-learning environments, datasets, and agent evaluations. Its public site emphasizes the relationship between agents and real-world environments, with research covering extended enterprise work and software engineering. Products include environment engagements, evaluation leaderboards, and off-the-shelf datasets.

Named research projects include EnterpriseSWE, which addresses long-horizon enterprise software tasks; COBOLBench, which evaluates agents on enterprise COBOL; and LH-Bench, which studies subjective enterprise tasks. These examples give Metaphi a particular focus on work that extends beyond short modern-code exercises.

The company also discusses delayed rewards and the challenge of evaluating domains where verification is expensive. Those are research themes rather than proof that every difficulty has been solved. Buyers should connect a proposed product to a specific task family and delivery format.

Why enterprise software is distinctive

Our assessment is that enterprise engineering often involves understanding constraints accumulated over time. In an illustrative maintenance task, an agent might need to modify an older system while preserving file formats, interfaces, and downstream behavior. A locally correct change can still be unsuitable if it breaks a dependency outside the immediate module.

Established languages and systems can also expose gaps in the distribution of publicly available training data. The useful question is not simply whether a model recognizes syntax, but whether it can reason about the larger workflow and produce a change that fits the operating context. This makes realistic source material and outcome checks particularly important.

Evaluating fit

Choose a benchmark or dataset based on the actual systems the buyer wants to support. Inspect task duration, source context, tool availability, and grading criteria. For subjective work, ask how expert judgments are converted into a consistent rubric and how alternative valid results are treated. For coding tasks, test whether evaluation covers compatibility and unintended side effects. A pilot should include unfamiliar repositories or business contexts and separate training data from evaluation material. Clarify whether environments are hosted or delivered for the buyer to operate, and who maintains dependencies. The strongest evidence is reliable completion of representative enterprise work, with enough trace detail to explain both progress and remaining failures.