Name: Frontier Score for "Generating interesting scientific ideas using knowledge graphs and LLMs: Evaluations with 100 research group leaders"
Rating: 73
Author: Frontier

What it is

The growth of the scientific literature makes it harder for researchers to spot novel and impactful ideas, especially across disciplines, and AI offers a new way to generate them, but how compelling the results are is unclear. The authors introduce SciMuse, which generates personalized research ideas from a knowledge graph of 58 million papers and a large language model. To find out how interesting the ideas are, they ran a large-scale evaluation in which more than 100 research group leaders, spanning the natural sciences to the humanities, rated over 4,400 personalized ideas for their level of interest. Expert ratings were modest: a mean of 2.40 on a 5-point scale, with the most common rating being 1, while 24.9% of ideas were rated 4 or 5. Supplying concept pairs chosen with the knowledge graph did not improve expert-rated interest over a titles-only GPT baseline, and pairs predicted to be high-citation even showed a weak tendency (1.94 sigma) towards lower interest than random pairs. Graph features can still be used to control properties of the ideas, and the authors show that interest can be predicted, both by a supervised neural network built on graph features and by a zero-shot ranking with an LLM.

Why it matters

Experts rated most of these AI-generated ideas low. The knowledge-graph step meant to be the method's advantage did not beat the same GPT prompt given only the two researchers' recent paper titles and no concept pair. The authors frame the contribution as a methodology plus a large interdisciplinary expert evaluation, opening the way to study and improve hard-to-measure qualities such as expert-perceived interestingness, and in the preprint their interest predictors reach an area under the curve of 63.2% for the supervised network and 67.4% for a zero-shot GPT-4o ranking, where a coin flip scores 50; the authors call this a weak, but non-negligible signal.

How to read this

Interest is a judgement, not an outcome: the preprint is explicit that a high rating 'should therefore not be interpreted to mean that they would necessarily pursue it immediately', and the study never followed up on whether any idea was taken up. The raters are a narrow slice of science. In the preprint, all were Max Planck Society group leaders, 104 of 110 from natural science institutes and the remaining 6 from social science institutes, with no humanities participant reported; 41.2% of those invited took part, and the authors say they cannot exclude self-selection bias and that different researchers likely used the scale with different thresholds. A second, small test in the preprint set the AI ideas against 148 written by seven PhD students for six physics group leaders, and the sources scored alike (29.1% against 25.0% rated 4 or 5, a difference the authors say cannot distinguish them). Even there the humans worked from the supplied material alone while the models could draw on their pretraining, an asymmetry the preprint says potentially favors the AI systems and does not constitute a fully matched head-to-head comparison. Whether selecting ideas with these predictors actually improves what is generated 'remains to be tested'.

Underlined numbers link to their source. Every metric and quoted figure is listed under Sources and data below.

Filed underTopic Modeling, Recommender Systems and Techniques, scientometrics and bibliometrics research

Scored 2026-11-12How scoring works

The Frontier Score, in full

How this entry's score is built, term by term. Open any pillar to see each sub-metric's raw value, how it maps to 0 to 100, and a link to its source (OpenAlex for most sub-metrics that combine several sources, NIH iCite where its ratio is part of field-normalized impact).

The three pillars as one shape: a result strong on every axis fills the triangle.

How the Frontier Score is calculated for this breakthrough
Term	Score	Weight	Points
EvidenceHow established and verifiable the result is.	88	× 0.25	22.0
ImpactHow much the result matters, judged field-relative and by current momentum.	63	× 0.25	15.6
NoveltyHow genuinely new the result is, and how fast-moving.	72	× 0.50	35.9
Weighted pillars			73.5
Frontier recency0 months since publication		× 1.00
IntegrityNot retracted		× 1
Frontier Score			73

Confidence: Early signalEarly signal: little citation history yet, so impact is an estimate and may shift as the work is taken up.

Every sub-metric, traced to its source

EvidenceHow established and verifiable the result is.88

Evidence sub-metrics
Sub-metric	Raw value	0 to 100	Weight	Points	Source
Peer review & venueWhether the work has cleared peer review, and how selective its venue is. Peer review is necessary but not sufficient: before citations accrue, a fresh result in one of the five general flagships (Nature, Science, Cell, NEJM, the Lancet) is a stronger evidence signal than one in a selective specialist or society journal, which in turn outranks one in a legitimate but very high-volume mega-journal, and any of them outranks a preprint that has not been reviewed at all.Maps to 0 to 100: Peer-reviewed article graded by venue tier: flagship journal (Nature, Science, Cell, NEJM, the Lancet) 100; selective journal (every other journal the index admits: the specialist journals of those families, JAMA, PNAS, Physical Review Letters and X) 90; high-volume mega-journal (Nature Communications, Science Advances) 78; venue not recorded 88. Other records: review 90, book 70, dataset 60, preprint 45, an article that has not been peer reviewed 45, other 55.	Peer-reviewed, selective journal	90	0.30	27.0	OpenAlex type / primary_location / source
CorroborationIndependent corroboration that the result is on the record and reaching beyond a single source, read as the BREADTH of independent records rather than the size of the citation pile. Counts how many of the five independent citation indices (OpenAlex, Crossref, Semantic Scholar, OpenCitations, Europe PMC) report a citation count for the work at all, zero included, plus orthogonal, non-citation lines of corroboration: citation in Wikipedia, registered clinical trials or clinical citations, released datasets or software or a repository copy, public funding on record, and technical-community discussion. Citation MAGNITUDE is deliberately judged under Impact, not here, so Evidence and Impact measure genuinely different things instead of both rewarding the same citation pile twice.Maps to 0 to 100: Starts at 28 and rises by 9 for each of the five independent citation indices that returns a citation count for the work (zero counts; OpenCitations returns zero even for a DOI it does not hold) and by 6 for each orthogonal non-citation line (Wikipedia, clinical trials or clinical citations, open datasets, software or a repository copy, public funding, community discussion), up to the top of the 0-100 scale.	4 of 5 citation indices report a count, 1 independent non-citation line	70	0.30	21.0	OpenAlex, Crossref, Semantic Scholar, OpenCitations, Europe PMC, DataCite, NIH RePORTER, Wikipedia, Hacker News count of indices that report a count for the work + orthogonal reach
VerifiabilityHow openly the result can be read, reused, and reproduced. Rewards open access, a Creative Commons BY or CC0 license, a green repository copy anyone can archive, openly minable full text, and released datasets or software a reader can actually run. Cross-checked across OpenAlex, Unpaywall, Europe PMC, and DataCite.Maps to 0 to 100: Closed access 45; open access 88; any Creative Commons BY license (including the NC, ND and SA variants) or CC0 100. A repository copy raises the floor to 85, openly minable full text to 90, and released data or software to 92.	Open access (hybrid), CC-BY, repository copy	100	0.20	20.0	OpenAlex, Unpaywall, Europe PMC, DataCite is_oa / license / has_repository_copy / open datasets
IntegrityRetraction and post-publication correction status. A retracted result is not evidence of anything and collapses the whole Frontier Score; a correction or expression of concern is a softer flag. A correction the publisher labels as its own error (a Publisher Correction) is not a flag on the work. Read across OpenAlex and Crossref (both update-to and updated-by notices).Maps to 0 to 100: Clean 100 (also when the only notice is a Publisher Correction); a correction or expression of concern 75; retracted 0. Flagged if OpenAlex OR Crossref records the notice.	No retraction or concern on record	100	0.20	20.0	OpenAlex, Crossref is_retracted / update-to / updated-by

ImpactHow much the result matters, judged field-relative and by current momentum.63

Impact sub-metrics
Sub-metric	Raw value	0 to 100	Weight	Points	Source
Field-normalized impactImpact relative to the world average for the same field, year, and type, so a small field and a large field are judged fairly. Triangulates three independent field-normalized measures: OpenAlex's Field-Weighted Citation Impact, its citation percentile within the exact field-and-year cohort, and the NIH iCite Relative Citation Ratio. 1.0x is average; the percentile is the share of same-field, same-year work it out-cites.Maps to 0 to 100: Each ratio (1.0 = field average) maps 1.0 to 50, 3.0 to 75, 9.0 to 90; the field+year percentile maps directly (top 1% -> ~99). The available measures are averaged. When none is available yet (very recent work), this sub-metric is left out and Impact is computed from the others.	39.5x FWCI, top 0.2% of field-year vs field	99	0.45	44.4	OpenAlex, NIH iCite fwci / citation_normalized_percentile / relative_citation_ratio
Influential citationsCitations that genuinely build on the work rather than mention it in passing (Semantic Scholar's influential-citation measure). A sharper signal of real impact than a raw count.Maps to 0 to 100: Saturating: 15 influential citations maps to 50, and none maps to 0. When Semantic Scholar has no measure for the work, this sub-metric is left out and Impact is computed from the others.	0 influential	0	0.20	0.0	Semantic Scholar influentialCitationCount
Citation velocityCitations received in the trailing twelve months (estimated from yearly citation counts: this year's so far plus the share of last year's that falls inside the window), judged as a rate rather than a lifetime total. Momentum is what a frontier index cares about: a result being taken up fast right now is landing, whether its lifetime pile is large or still small. Where an expected field citation rate is known, the momentum is also read relative to it, so a fast-moving result in a quiet field is not overlooked.Maps to 0 to 100: Saturating: 30 citations in the last twelve months maps to 50. When NIH iCite provides the field's expected rate, this is blended evenly with the ratio of observed to expected momentum.	32 in last 12 months	52	0.35	18.1	OpenAlex, NIH iCite counts_by_year / field_citation_rate

NoveltyHow genuinely new the result is, and how fast-moving.72

Novelty sub-metrics
Sub-metric	Raw value	0 to 100	Weight	Points	Source
Conceptual noveltyHow genuinely new the contribution is, measured at publication from the work's OWN content, not its date: how atypical its combination of research areas is versus all prior work, and whether it builds on a broad, deep base rather than only the newest papers in a fast-moving area. This is what separates a real advance from a recent-but-commodity result, and it needs no forward citations, so it is stable for brand-new work. An honestly noisy proxy (even state-of-the-art novelty measures agree only moderately with expert judgment), so it carries bounded weight and never drives the ranking alone.Maps to 0 to 100: Two publication-time signals, each 0-100. The primary one is atypical combination: how unusual the work's median pairing of research topics is versus the prior literature, by pointwise mutual information (a pairing made far more often than chance scores low). The second, foundational reach (the median age of the work's references and the share older than five years), can only lift it: when reach is higher, the score rises by 35% of the gap, and it is never lowered. When only one signal is measurable it is used alone; when neither is (a work too thin in topics or references), this sub-metric is left out and Novelty is computed from the other three. A review is damped to 60%.	Median topic pairing Recommender Systems and Techniques x scientometrics and bibliometrics research (2,074 prior works made this pairing); builds on refs a median 3.3y deep	46	0.45	20.8	OpenAlex topics + referenced_works (publication-time)
RecencyHow recently the work appeared. The frontier is now, so newer scores higher.Maps to 0 to 100: Exponential decay with a 12-month half-life from the publication date.	This month	99	0.30	29.8	OpenAlex publication_date
Attention accelerationWhether attention is rising: citations in the latest full year versus the year before. Accelerating interest signals an active, opening frontier.Maps to 0 to 100: Latest/prior-year ratio: 1x maps to 50, 2x to 75, 4x to 100. The prior year counts as at least 3 citations, so a jump from one citation to a few is not read as explosive growth. With no citations in the latest full year it reads as unknown, and unknown is neutral (50).	3.1x rising	91	0.10	9.1	OpenAlex counts_by_year
Edge of reviewPreprints sit ahead of the peer-review process and brand-new papers have only just cleared it. Both earn novelty here (while a preprint is discounted under Evidence), because that is where the frontier forms first.Maps to 0 to 100: Preprint 95; any other record 80 when under 6 months old, 65 under 18 months, 45 after that.	Peer-reviewed record	80	0.15	12.0	OpenAlex type / publication_date

ProvenanceAll sources

Sources and data

The source figures behind this entry, grouped by the source that holds them: up to five citation indices cross-checked, field-normalized impact, then real-world reach (Wikipedia, community discussion, clinical trials, released data and public funding) where it exists. Each value links to its record.

OpenAlex

Primary record

Total citations: 65
Field-weighted citation impact: 39.5x field average
Citation percentile: Top 0.2% of its field-year
Citations, recent 12 months: 32
Publication date: 2026-11-09
Access: Open access (hybrid)

Crossref

Citations: 0
Works it builds on: 91
Funders on record: 3

Semantic Scholar

Citations, all versions: 31
Influential citations: 0

OpenCitations

Citations: 0

Unpaywall

Reuse license: CC-BY

OpenAlex record Read it in Proceedings of the National Academy of Sciences

Figures quoted in the write-up

Figure	Value	Source
Papers in the knowledge graph (Abstract: SciMuse generates personalized research ideas from the graph and a large language model. The preprint says the graph's 123,128 concept vertices came from about 2.44 million arXiv, bioRxiv, ChemRxiv and medRxiv articles, and its edges from the 58 million papers that contain at least two of those concepts)	58 million papers	Proceedings of the National Academy of Sciences
Researchers who rated the ideas (Abstract: spanning the natural sciences to the humanities)	more than 100 research group leaders	Proceedings of the National Academy of Sciences
Share rated 4 or 5 in the human-versus-AI comparison (Preprint: 148 ideas written by seven experienced PhD students for six physics-related group leaders, rated under the same protocol; the authors say the results cannot distinguish the quality of human and AI-generated ideas at a statistically significant level)	(29.1 ± 3.7)% for human-generated ideas, (25.0 ± 4.4)% for GPT-4	arXiv: the authors' preprint
Ideas rated (Abstract: rated according to their level of interest)	over 4,400 personalized ideas	Proceedings of the National Academy of Sciences
Mean expert rating (Abstract: the preprint gives the scale as 1 (not interesting) to 5 (very interesting), with 1,632 ideas (36.7%) rated 1)	mean 2.40 on a 5-point scale, most common rating 1	Proceedings of the National Academy of Sciences
Share of ideas rated 4 or 5 (Abstract)	24.9%	Proceedings of the National Academy of Sciences
Weak tendency of pairs predicted to be high-impact towards lower interest (Abstract: "High-citation-predicted pairs even showed a weak tendency (1.94 σ) toward lower interest than random pairs". The prediction is of whether a concept pair's papers would pass a citation threshold within 3 years, not a count of citations)	1.94 σ	Proceedings of the National Academy of Sciences
Interest prediction by the supervised network (Preprint: a small neural network with 25 selected input features; 49.2% of its top-5 suggestions were rated highly interesting)	an AUC of (63.20 ± 0.31)%	arXiv: the authors' preprint
Interest prediction by zero-shot LLM ranking (Preprint: GPT-4o ranking 2,996 suggestions from most to least interesting)	(67.44 ± 0.01)% for GPT-4o; (60.39 ± 0.02)% for GPT-3.5	arXiv: the authors' preprint
Raters from natural science institutes (Preprint: among the participating group leaders, the other 6 are from social science institutes)	104 of 110	arXiv: the authors' preprint
Share of invited group leaders who took part (Preprint: of 267 invited; the authors say they cannot exclude self-selection bias)	110 participated (41.2%)	arXiv: the authors' preprint

For the authors

Show this score where your work lives

A badge for your lab page, your CV or the paper's code repository. It links readers to this entry, and it always carries the entry's current score.

Live preview

<a href="https://frontierresearch.co/breakthroughs/generating-interesting-scientific-ideas-using-knowledge-w4399116274?utm_source=badge&amp;utm_medium=badge&amp;utm_campaign=author-kit"><img src="https://frontierresearch.co/api/badge/W4399116274.svg" alt="Frontier Score for this paper" height="20"></a>

For a lab or personal website, a CV page, or anywhere that takes HTML.

The number is live. The badge is drawn from the same signals as this page, so it never shows a score the entry does not. Every entry is re-checked weekly against the same public sources, and a result's standing eases as it ages, so the active edge of the field stays on top. How scoring works

Your other papers. The free paper check shows any paper's field-weighted citation impact, its citations in five open indexes and any retraction on record, whether or not it is in the index, and the researcher check does it for your whole record from your ORCID iD.

Something wrong or out of date? Write to hello@frontierresearch.co and the entry will be corrected.

AI & ComputingThe whole field

AI & Computing · Notable

An open vision-language model for diverse medical applications

Open medical AI models built on Gemma 3 beat their general-purpose base across medical images and text

E 91I 96N 64Oct 1, 2026

AI & Computing · Notable

Reimagining research papers as interactive and reliable AI agents

E 94I 68N 76Sep 16, 2026

AI & Computing · Notable

Intrinsically motivated humans and agents in open-world exploration

Children, adults and AI agents in a Minecraft-like game seem to seek novelty first, then control

E 91I 57N 71Nov 4, 2026

Back to the Frontier Index, or browse every breakthrough by date.

What it is

Why it matters

The Frontier Score, in full

Sources and data

OpenAlex

Crossref

Semantic Scholar

OpenCitations

Unpaywall

Show this score where your work lives

Related on the frontier

An open vision-language model for diverse medical applications

Reimagining research papers as interactive and reliable AI agents

Intrinsically motivated humans and agents in open-world exploration

The week's AI & Computing, scored.