Ulam

Overview

Ulam develops mathematical reasoning datasets, private evaluations, and reinforcement-learning environments. Its public offering is grounded in research-level mathematics and emphasizes the quality of reasoning rather than only final answers. The team describes expert-reviewed proof attempts, human-in-the-loop traces, and data about partial progress.

The product set includes curated problem collections, custom data, private benchmark programs, and stateful RL gyms. Ulam names ErdősBench and SimoBench as starting points for pilot evaluations, with subsequent work guided by the failures observed. Its datasets can include critiques, repairs, unsuccessful approaches, and checker-supported artifacts.

This makes Ulam a specialist in reasoning and research supervision rather than a general-purpose math-question vendor. The public description does not imply that every problem has a known complete solution or that every proof attempt is formally verified. Buyers should inspect the exact verification method used for each asset.

Why partial progress matters

Our assessment is that mathematical research often produces useful information before a complete solution exists. In an illustrative problem, an agent might prove a special case, identify a counterexample to an overly broad claim, or locate a gap in an argument. Those outcomes can teach valuable reasoning even when they do not resolve the entire question.

The challenge is labeling the contribution accurately. A fluent but invalid proof should not receive the same credit as a limited result with a sound justification. Expert review and explicit checks can help distinguish genuine progress from persuasive presentation.

Evaluating fit

Ask how problems are selected, how difficulty is assessed, and what evidence supports each label. Inspect the relationship between informal reasoning, formal artifacts, and expert judgment. For traces, check whether revisions and failed approaches are preserved with enough context to understand them. A pilot should evaluate unfamiliar problem families and keep private tests separate from training material. Clarify whether the intended use is model training, critic training, diagnosis, or an interactive research environment. Buyers should also establish rights to source problems and derived work, required compute, and maintenance of verification dependencies. The useful outcome is more reliable reasoning with inspectable evidence, rather than a higher rate of confident final answers.