Starting date: 01/09/2026 Job Reference 361_26_AII_LM_RE3 Position Senior Pre-training Engineer (RE3) Closing Date Monday, 31 August, **** Reference 361_26_AII_LM_RE3 Job title Senior Pre-training Engineer (RE3) About BSC The Barcelona Supercomputing Center - Centro Nacional de Supercomputación (BSC-CNS) is the leading supercomputing center in Spain.
It houses MareNostrum, one of the most powerful supercomputers in Europe, was a founding and hosting member of the former European HPC infrastructure PRACE (Partnership for Advanced Computing in Europe), and is now hosting entity for EuroHPC JU, the Joint Undertaking that leads large-scale investments and HPC provision in Europe.
The mission of BSC is to research, develop and manage information technologies in order to facilitate scientific progress.
BSC combines HPC service provision and R&D; into both computer and computational science (life, earth and engineering sciences) under one roof, and currently has over **** staff from 60 countries.
CNS YouTube Channel Let's stay connected with BSC Folks!
We are particularly interested for this role in the strengths and lived experiences of women and underrepresented groups to help us avoid perpetuating biases and oversights in science and IT research.
It has been entrusted by the Spanish and the Catalan governments with the mission to develop fundamental open?source resources and technologies for Spanish and Catalan.
In connection with this, the LM team is currently in charge of two flagship projects at the national and regional level: the ALIA project, funded by the Spanish Secretariat of Digitalisation and Artificial Intelligence, and the AINA project, aimed at developing AI resources for Catalan, funded by the Catalan Digitalisation Department.
In addition,



the team is a key contributor to Open Euro LLM, the European initiative to build open, multilingual foundation models trained on EuroHPC infrastructure, in collaboration with leading European research centers.
training operations and infrastructure.
The successful candidate will work in a highly sophisticated HPC environment, have access to state?of?the?art systems and computational infrastructures, and establish collaborations with experts in different areas at the local and international levels.
training pipelines for multilingual foundation models on multi?Design and implement distributed training strategies (data, tensor, pipeline, and sequence parallelism, ZeRO/FSDP) to scale training efficiently across hundreds to thousands of GPUs.
optimal decisions on model size, dataset size, and training budget.
training at scale.
Optimize data pipelines, tokenization and curriculum design for multilingual, multi?node pre?training.
Mentor junior team members on best practices for distributed pre?training and performance bottleneck identification.
Requirements Education PhD or Master's degree in Computer Science, Artificial Intelligence or a related field.
training large language models, including multi?GPU distributed training.
Proven expertise with distributed training frameworks such as Megatron?LM, DeepSpeed, PyTorch FSDP or NeMo.
Expert knowledge of Python and Deep Learning libraries including PyTorch and DeepSpeed.




optimal training strategies.
scale distributed training and high?performance computing (HPC) environments.
Excellent level of English (written and spoken).
Fluency in Spanish is highly valued.
official languages, including Catalan, Basque and Galician is highly desirable.
Technical leadership skills to establish best practices in training within the group.
solving skills focused on performance bottlenecks and scalability.
Conditions The position will be located at BSC within the Directors Department We offer a full?time contract (35h/week), a good working environment, a highly stimulating environment with state?of?the?art infrastructure, versátil working hours, extensive training plan, restaurant tickets, private health insurance, support to the relocation procedures Duration: Open?ended contract due to technical and scientific activities linked to the project and budget duration Holidays: 22 days of holidays + 6 personal days + 24th and 31st of December per our collective agreement Salary: we offer a competitive salary commensurate with the qualifications and experience of the candidate and according to the cost of living in Barcelona Starting date: 01/09/2026
The recruitment panel will be composed of at least three people, ensuring at least 25% representation of women.
After reviewing the content of the applications, the panel will begin the interviews, with at least one technical and one administrative interview.
We are pleased to consider all qualified applicants for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability or any other basis protected by applicable state or local law.
#

📌 Lead Distributed Pre-Training Engineer For Multilingual Llms (Barcelona)
🏢 Barcelona Supercomputing Center
📍 Barcelona