Could a language model pass a job interview for a specialist position in a particle accelerator? That is the underlying question behind the IEEE CIS Technical Challenge on Domain-Specific Question Answering with LLMs, an international competition organised by the DaSCI (University of Granada) in collaboration with IFMIF-DONES, and funded by the IEEE Computational Intelligence Society with up to $20,000 in prizes.

The challenge launched on 28 September and will remain open for two months: teams can register until 1 November and submit their solutions until 30 November. In its first few days, more than twenty participants have already signed up on the competition platform.

Large language models (LLMs) commonly used today can write emails or summarise reports with ease, but their reliability and performance decline when dealing with highly specialised technical knowledge. Moreover, the most powerful solutions rely on large commercial models running in the cloud—an option that is not always viable for critical scientific infrastructures, research centres or companies. Technical documentation is often confidential and should not be sent to external servers, and access to large computing resources is not always available.

“We have fantastic models, but when they need to answer very specific, domain-level technical questions, they can fall short—especially when computational resources are limited, which is often the case in real-world scenarios,” explains Isaac Triguero, Director of DaSCI and coordinator of the competition. “It is not always possible to rely on a cloud provider, either due to cost, lack of resources, or simply because data privacy cannot be compromised.”

For this reason, the competition replicates a realistic scenario. All teams must work with the same open, moderately sized model: OLMo, developed by the Allen Institute for AI, which can run on a single GPU. Solutions are executed on servers at the University of Granada, without internet access and under limited computing time, ensuring that neither the questions nor the reference documents leave the evaluation environment.

The challenge is therefore not about using more computing power, but about adapting the model intelligently. Techniques from computational intelligence—such as efficient fine-tuning, prompt engineering, knowledge graphs or fuzzy modelling—allow specialisation without large-scale retraining or excessive resource consumption.

Solutions are evaluated using a benchmark of 100 questions and answers developed by experts in two domains: the IFMIF-DONES facility—the particle accelerator under construction in Escúzar (Granada) to test materials for future fusion reactors—and specialised scientific literature in artificial intelligence. The questions range from retrieving specific information to comparative, causal and predictive reasoning. These questions are not made public; participants only have access to a sample of 25 examples. Experts from IFMIF-DONES and researchers from DaSCI spent nearly a year preparing and annotating this benchmark.

The collaboration with IFMIF-DONES is part of the DONES-AXIA project, a multi-agent system designed to improve the operation, safety and energy efficiency of the facility, funded under the INNTERCONECTA-STEP 2025 call by the CDTI with ERDF co-funding, and led by the company Plain Concepts. Within this project, the University of Granada develops specialised, lightweight and secure language models for IFMIF-DONES—the same type of solution the competition aims to promote. The initiative is also linked to the University of Granada’s DONITOS programme, which supports doctoral training in collaboration with IFMIF-DONES. On behalf of the consortium, Director Moisés Weber and Patricia Martín, Head of the Operation and Maintenance Department, are involved.

The competition is structured in two phases. In the first, fully automated phase, each team’s code is executed on hidden questions and ranked on a public leaderboard. “Optimising a single metric is not enough—we are looking for the science behind the solutions,” says Triguero. “We are not only seeking the best numerical result, but also the most innovative and highest-quality approach.”

Approximately twenty top teams will advance to the second phase, where expert panels will assess the quality of the answers and a scientific committee will evaluate technical innovation. Final results will be announced on 14 December, with prizes of $9,000 for first place, $5,000 for second, $3,000 for third, and $1,500 for fourth and fifth places.

The UGR organising team includes Isaac Triguero, researchers Jaime Uclés and Carlos Peláez-González (PhD candidates), and Ignacio Aguilera Martos (PhD in Computer Science). The challenge committee also includes Xingyu Wu (Hong Kong Polytechnic University), Christian Wagner (University of Nottingham), and Katherine Malan (University of South Africa). The scientific committee includes Javier Del Ser (University of the Basque Country) and Daniel Molina (University of Granada). The University of Granada also provides the computing infrastructure used to evaluate the solutions.