Training Reasoning Models to Generate Diverse Scientific Ideas

We propose training with a set-level RL objective. The model is instructed to generate K idea cards in one generation, and each idea's goodness (or quality) is weighted by its distance from the ideas generated before it. This gives explicit control to the model and trains it to steer each new idea away from the ones it has already written. The objective builds on Creative Utility, the metric from our earlier work CREATE (Wadhwa et al., 2026).

Our 8B model beats Claude Opus 4.8 and GPT-6 Astra on creative utility, which measures quality and diversity together. In a pilot study with four experts, its sets are judged as promising as Claude Opus 4.8's and contain slightly more unique ideas. When a coding agent runs proof-of-concept experiments on the ideas, our success@k is comparable to Claude Opus 4.8's, so the added diversity does not cost executability.

Why sets of ideas?

As researchers, when we generate ideas and brainstorm, we hold multiple hypothesis from the same literature and then pick among them (Platt, 1964; Klahr & Dunbar, 1988). We generally do this pre-execution, based our experiment and intuitive. This process of brainstorming can also be thought of as interleaved diversget thinking followed by convergent thinking (Guilford, 1967; Boden, 2004), which is why we chose to use the creative utility formulation as the training/evaluation objective. Independent LLM samples share no context to explore deliberately and no pool to converge over, so any diversity among them is incidental (Puri et al., 2026; Bahlous-Boldi et al., 2026).

Method: a set-level reward

The model writes a set U = {u₁, …, u_K} after one shared reasoning prefix, and GRPO (Shao et al., 2024) scores the whole set:

Each idea's quality is multiplied by its distance to the closest idea written before it, so a redundant idea earns almost nothing, however good it is.

Quality. We measure long-term scientific impact, judged pairwise against the target paper's own idea: 1 if the generated idea is better, 0.5 for a tie, 0 otherwise. Anchoring to the gold idea means the judgment reflects what the field found worth pursuing. We use SciJudge-30B (Tong et al., 2026).

Distance. Two ideas can share a problem and still differ completely in method, so we score distance separately over the problem, claim and method fields and average them, judged by gpt-oss-120b (OpenAI, 2025).

Training. We start from Qwen3-8B (Team, 2025) with a light SFT stage that teaches the idea-card format and a reasoning pattern. The traces come from hindsight prompting: a teacher model sees its own ideas and writes the exploration that leads to them. We then run GRPO with K = 3, with and without thinking.

Experimental setup

The input is summaries of the papers a target paper builds on. The model writes K idea cards, each with a problem gap, a core claim, a method, and the source papers it draws on. The target's own idea is held out as the gold idea.

We train on 7,172 instances built from GiantsBench (He-Yueya et al., 2026) and test on a contamination-free set of 1,740 papers from 2026 across 16 domains. We also evaluate on the GiantsBench test split and on IdeaSeed (Chen et al., 2026).

Baselines: Claude Opus 4.8, GPT-6 Astra, Nemotron 120B (NVIDIA, 2025), DARLING (Li et al., 2025), impact-only RL, our reward without SFT, and the base and SFT checkpoints. The main metric is Creative Utility (CU) from CREATE (Wadhwa et al., 2026).

Results

Our no-think model reaches the highest creative utility: 0.64, up from 0.03 for the base model and 0.21 after SFT. Claude Opus 4.8 reaches 0.38 and GPT-6 Astra 0.28.

Training for quality alone gives a 43% win rate but collapses the set: average distance falls to 0.38, as the model settles on one good idea instead of several. DARLING, which rewards diversity across separate single-idea rollouts, reaches 0.52 CU with lower distance (0.67 vs. 0.88). Rewarding diversity within the set works better.

The SFT warm-start keeps the model grounded. Skipping it still gets 0.51 CU, but grounding drops to 0.19, because the model gets its diversity by drifting from its sources. Thinking also helps grounding: the think variant has lower utility (0.51 vs. 0.64) but higher grounding (0.81 vs. 0.61).

We talk about more results in the paper, including results from Giants-4B, distribution of idea types across the population and how grounding and quality trade off!

Do the ideas hold up?

Four NLP researchers, each with a PhD and at least four top-venue papers, judged 60 idea sets blind. For each set they marked which ideas are worth pursuing and the largest subset that pursues different directions.

We also gave each idea card to a coding agent in a fixed sandbox (90 minutes, $30, no GPU), which designs and runs a proof-of-concept and reports whether the result supports the claim. Our success@k matches Claude Opus 4.8 and rises with k, so a set of ideas is more useful than a single one.

BibTeX

@article{wadhwa2026training,
  title   = {Training Reasoning Models to Generate Diverse Scientific Ideas},
  author  = {Wadhwa, Manya and Venkit, Pranav Narayanan and Hayashi, Hiroaki and
             Li, Yu and Chakrabarty, Tuhin and Wu, Chien-Sheng},
  journal = {arXiv preprint arXiv:2610.22324},
  year    = {2026}
}