logo
ProjectForthcoming

GenEval: Linking Generation and Evaluation for Reliable NLG Assessment

Forthcoming · Spain

Source: EU Funding & Tenders Portal

"Large Language Models (LLMs) are increasingly leveraged as evaluators of machine-generated text, a paradigm known as ""LLMs-as-judges."" While this approach offers flexibility and typically strong performance, its reliability remains inconsistent and poorly understood. Strong generative performance does not guarantee reliable evaluation, and the mechanisms linking these two capabilities remain op

Status
Prospective
Sector
Education
Location
Spain
Updated
19 Nov 2026
Source
EU Funding & Tenders Portal

ConstructChat

Ask ConstructChat about GenEval: Linking Generation and Evaluation for Reliable NLG Assessment

Pick a question to edit it, or write your own.

First answer free · no account

I am a

Project Information FAQ

Project Information

3 Q
The project “GenEval: Linking Generation and Evaluation for Reliable NLG Assessment” is an infrastructure initiative in the Education sector, located in Spain. Taiyo aggregates data on it from EU Funding & Tenders Portal.

Participants

Not available for this record.

Status

Original status

forthcoming

Taiyo status

Prospective

Taiyo last update

2026-11-19

Contact

Not available for this record.

Description

Description

"Large Language Models (LLMs) are increasingly leveraged as evaluators of machine-generated text, a paradigm known as ""LLMs-as-judges."" While this approach offers flexibility and typically strong performance, its reliability remains inconsistent and poorly understood. Strong generative performance does not guarantee reliable evaluation, and the mechanisms linking these two capabilities remain opaque. Without systematic validation, current evaluation practices risk being blind to misleading and factually incorrect content and misrepresenting system capabilities. GenEval addresses this challenge by investigating the fundamental relationship between generation and evaluation in LLMs, developing novel representation-based metrics, and predicting LLMs' evaluation reliability across tasks and models. We will analyze LLMs at a mechanistic level, identifying circuits and representations that underlie generative and evaluative behaviors. This understanding will enable the design of new evaluation metrics that directly exploit LLMs' internal representations, providing interpretable, efficient, and robust alternatives to existing approaches. Finally, GenEval will develop predictive tools to estimate when an LLM is likely to be a reliable evaluator even in the absence of human judgment data, supporting informed model selection and human-in-the-loop evaluation. By integrating mechanistic insights with practical evaluation methods, GenEval will deliver both theoretical advances and applied tools. This action will be possible thanks to the integration of the scientific expertise of Prof. Horacio Saggion, an internationally recognized expert in Natural Language Generation, and that of the researcher, who has a strong background in evaluation, Natural Language Processing, and Machine Learning. The action will develop impacting technology, and provide the researcher with the necessary training to become independent and strengthen her academic profile."

Location

Country

Spain

Source

Source

EU Funding & Tenders Portal

URL

More Details

Record type

Project

Sector

Education

Article published date

2026-09-01

More on this project

Research

Free account

View all details

You are reading the public part. A free account opens the rest:

  • Every detail on this project
  • News and media coverage
  • Technical research
View all details

Free account · no card · lands you back on this project