Can a general-purpose LLM search crystal compositions?
In a closed computational benchmark, GPT-5.4 recovered 95.65% of 3,740 low-energy Elpasolite targets within 5,000 proposals. Iterative evaluator feedback drove the result; no new material was synthesised, and the approximate energy labels are not proof of stability or usefulness.
The short version
- What happened
- Across three runs of 5,000 proposals, GPT-5.4 recovered 3,577 plus or minus 9 of the 3,740 benchmark targets, or 95.65% plus or minus 0.24%.
- Who it affects
- Materials researchers may gain a flexible search assistant, but remain responsible for physical constraints, validation and laboratory safety.
- What remains unproven
- The main task used a known, pre-tabulated composition space and an approximate learned formation-energy evaluator rather than prospective experiments.
Keep in mind
The main task used a known, pre-tabulated composition space and an approximate learned formation-energy evaluator rather than prospective experiments.
The model proposed compositions within fixed prototypes, not complete crystal structures or synthesis routes.
No candidate was synthesised, and a low predicted formation energy does not prove stability, manufacturability, safety or useful performance.
The LLM received iterative feedback while headline specialist baselines were one-shot generators, making the comparison architecturally asymmetric.
Prompt history grows with the number of proposals, raising context-window and cost limits for larger or less structured spaces.
Source analysis · 2 linked sources. Primary analysed source date: 9 October 2026.
Read the full report · 7 min The evidence, methods, implications and limitations
The direct answer: yes in this closed search task
A general-purpose language model searched this constrained computational benchmark unusually efficiently. Across three independent runs of 5,000 proposed compositions, GPT-5.4 recovered an average of 3,577 of the 3,740 Elpasolite compositions labelled below the target formation-energy threshold. That is 95.65%, with a spread of plus or minus 0.24 percentage points. The first repeated proposal occurred after 297 plus or minus 178 attempts, suggesting that the model initially explored broadly rather than immediately cycling through the same candidates.
This is a result about navigating a known mathematical search space, not discovering a usable material. The candidates were compositions conforming to a fixed ABC2D6 prototype and were evaluated against a pre-tabulated energy landscape. The researchers did not generate complete atomic structures, run an experimental synthesis, measure a physical property or establish that a candidate is stable under real conditions. 'Recovered a target label' is therefore the accurate claim; 'invented 3,577 materials' would be misleading.[1][2]
The benchmark contained a known answer set
The Elpasolite benchmark covers almost two million possible compositions formed by assigning chemical elements to four crystallographic sites. Formation energies were available from a kernel-ridge-regression model trained on roughly 10,000 density-functional-theory calculations. The target was the 3,740 compositions with predicted formation energy below minus 2.26 electronvolts per atom—less than 0.2% of the full space. Because the entire landscape was known, the team could count recall, diversity and duplication rather than judging only a few apparently good outputs.
That completeness is a strength for benchmarking but also a boundary. The score is only as meaningful as the reference evaluator. The paper notes chemically suspicious features: 262 target compositions contain noble gases, and 688 composition pairs created by swapping two sites straddle the threshold. Those cases expose uncertainty in a learned energy model and the underlying representation. A target classification may be useful for comparing search behaviour without being reliable evidence that a chemist should attempt synthesis.[1]
Feedback, not a single prompt, was the central mechanism
At every step, the model proposed one composition in the required formula. A validator checked compositional consistency, retrieved the precomputed formation energy and returned that value along with the growing history of previous proposals. The model then used the enlarged context to make its next choice. No task-specific weights were trained. In effect, the prompt became an active-learning notebook in which the model could infer which regions of the periodic table led toward the target.
The feedback is crucial. When the model generated a large batch without frequent scores, performance deteriorated. Batches of 10 to 50 retained much of the sequential mode's performance with fewer model calls, while larger batches degraded progressively. Medium reasoning effort recovered about 96% within 5,000 proposals; low and minimal settings recovered about 88%, and removing reasoning produced a sharper fall. More reasoning also increased cost because output tokens are more expensive than input tokens, so recovery alone is not a complete efficiency measure.[1]
The comparison with specialist models is informative but asymmetric
Earlier task-specific generative adversarial, variational-autoencoder and reinforcement-learning models recovered about 40% to 46% of the same 3,740 targets in their first 5,000 proposals, rising to 75% to 94% after 250,000. GPT-5.4's 95.65% within 5,000 is therefore a large benchmark advantage, achieved without task-specific fine-tuning. The authors also report performance nearly four times that of a Bayesian-optimisation comparison in their supplementary analysis.
But the language model saw an evaluator score after every proposal, while the older generative baselines operated in a one-shot manner. The authors explicitly call this an architectural advantage and say it must be considered in comparison. A fair operational contest would include strong iterative optimisers, equal evaluator access, matched compute and monetary cost, and multiple modern foundation models. The experiment demonstrates that an LLM can implement a flexible feedback-driven strategy; it does not establish that language modelling is intrinsically superior to every materials-search algorithm.[1]
Tests against memorisation strengthen—but do not settle—the case
One concern is that a general model may have encountered Elpasolite data during pre-training. The researchers tested an abstracted version in which element identities were anonymised and the target unit removed; the model still adapted through feedback. They also applied the workflow to a structurally different AB2X4 spinel dataset whose target property values were released after the model's training cutoff. Those tests make simple recall of the complete target landscape less plausible.
They do not eliminate every pre-training advantage. Chemical regularities and token representations learned from public literature can still help a model reason about element combinations. Indeed, a realistic starting composition improved early performance compared with an anonymous formula. That is not misconduct—it is part of what a general model offers—but it complicates attribution. Repetition with independently held-out simulators, unpublished landscapes and controlled starting information would clarify how much comes from prior chemical knowledge versus feedback adaptation.[1]
What this could change for materials scientists
The promising use is as a configurable proposal engine around a trusted scientific evaluator. A researcher could specify a formula family, return computed scores, inspect diversity and redirect the search without building a new neural generator for each property. Code and outputs are publicly linked, which should support independent reproduction. For small teams, that flexibility may reduce setup time and make exploratory inverse-design workflows easier to prototype.
Human scientific control remains essential. Experts must define physically meaningful constraints, inspect why the evaluator favours a candidate, check charge balance and structural feasibility, and decide whether higher-fidelity simulation or laboratory work is justified. Safety review cannot be delegated to a model optimising a number. A system may exploit defects in the scorer just as readily as it learns chemistry. The practical gain should be measured as validated candidates per unit of compute, researcher time and experimental cost—not as the count of strings below an approximate threshold.[1][2]
Funding, disclosure and what would change the assessment
The authors are based at the Fritz Haber Institute of the Max Planck Society. They report using Max Planck Computing and Data Facility resources; open-access funding was organised through Projekt DEAL. The paper declares no competing interests. It provides a Zenodo dataset with model outputs and a public GitLab workflow, important foundations for checking sensitivity to prompts, model versions and evaluator settings.
The assessment would strengthen if independent groups reproduce the result with held-out property landscapes, modern iterative baselines and full accounting of tokens, compute and failed proposals. The scientific bar is higher still: selected candidates should survive higher-fidelity calculations, uncertainty analysis and prospective laboratory synthesis, then deliver a relevant property under realistic conditions. Until that chain is demonstrated, the paper supports a precise conclusion—general-purpose LLMs can be strong adaptive search components in constrained computational composition spaces—not autonomous materials discovery.[1][2]