ppAt Johnson Johnson,we believe health is everything. Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented, treated, and cured,where treatments are smarter and less invasive, andsolutions are personal.Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity.Learn more at jnj.com. As guided by Our Credo, Johnson Johnson is responsible to our employees who work with us throughout the world. We provide an inclusive work environment where each person is considered as an individual. At Johnson Johnson, we respect the diversity and dignity of our employees and recognize their merit. /ph3Job Function: Career Programs /h3h3Job Sub Function: Post Doc – Data Analytics Computational Sciences /h3h3Job Category: Career Program /h3h3All Job Posting Locations: /h3ulliBarcelona, Spain /liliBeerse, Antwerp, Belgium /liliMadrid, Spain /liliRaritan, New Jersey, United States of America /liliTitusville, New Jersey, United States of America /li /ulpJob Description: Job description At JJ we are developing Generative AI solutions to support pharmaceutical RD, including literature review, evidence synthesis, document QA, therapeutic area knowledge search, translational science workflows, and RD decision support. These systems need to be tested before teams use them in scientific workflows. In pharmaceutical RD, a useful AI response depends on the question, user, source material, therapeutic area, and risk of error. We are looking for a postdoctoral researcher to help design methods that test whether GenAI tools produce answers that are accurate, evidence-grounded, traceable, usable, and appropriate for the intended task. The role reports to the Associate Director, Generative AI Evaluation Quality Standards. The team defines how JJ Innovative Medicine evaluates GenAI systems before use and helps determine when they are ready for release, expansion, or improvement. /ph3Key responsibilities /h3ulliDesign evaluation frameworks, rubrics, and criteria for GenAI tools used across pharmaceutical RD. /liliDevelop therapeutic-area-specific criteria with business and scientific teams to reflect domain and use-case quality needs. /liliBuild benchmark datasets, reference answer sets, annotation guides, and evaluation datasets. /liliRun expert reviews with scientific, clinical, regulatory, medical, data science,



and engineering teams. /liliTest LLM, RAG, and agent performance, including accuracy, source grounding, retrieval quality, citation fidelity, task completion, robustness, safety, and usability. /liliAnalyze failure patterns such as unsupported claims, incorrect reasoning, poor evidence use, missing uncertainty, weak traceability, or failure to follow instructions. /liliTranslate evaluation findings into improvements in prompts, retrieval methods, agent workflows, tools, and user experience. /liliHelp define release criteria for systems moving from prototype to limited release, expanded use, or product support. /liliReview emerging evaluation methods and adapt useful approaches for pharmaceutical RD. /liliDocument methods, findings, and recommendations so teams can apply consistent evaluation practices. /liliDesign and develop agentic judge methods to evaluate GenAI outputs against defined criteria, flag evidence gaps or unsupported claims, and support expert review workflows. /li /ulh3Qualifications /h3ulliEducation PhD or equivalent research experience in biomedical science, computational biology, bioinformatics, AI/ML, data science, clinical research, regulatory science, biostatistics, pharmaceutical sciences, or a related field. /liliExperience and skills Required Understanding of biomedical science, pharmaceutical RD, therapeutic area science, translational science, clinical development, regulatory science, biomedical informatics, data science, or related areas. /liliExperience translating expert judgment into criteria, rubrics, datasets, protocols, or measurable outcomes. /liliExperience designing or applying evaluation methods, benchmark datasets, annotation protocols, validation studies, quality reviews, or assessment frameworks. /liliInterest in testing GenAI systems, including LLMs, RAG, and AI agents. /liliProficiency in Python and common data science or machine learning tools. /liliAbility to analyze model outputs, compare performance, identify failure patterns, and recommend improvements. /liliClear written and verbal communication skills. /liliPreferred Experience with LLM APIs, embeddings,



vector databases, prompt engineering, agent frameworks, or AI evaluation tools. /liliExperience evaluating retrieval quality, generated answers, multi-step workflows, tool use, scientific reasoning, citation quality, or evidence-grounded outputs. /liliExperience designing expert review workflows, annotation instructions, adjudication processes, or inter-rater reliability analyses. /liliDomain knowledge in one or more biomedical or therapeutic areas. /liliFamiliarity with biomedical data standards, structured scientific or clinical data, ontologies, knowledge graphs, CDISC, FHIR, or related frameworks. /liliPublications or applied research in AI evaluation, NLP, biomedical informatics, machine learning, data science, computational biology, bioinformatics, or a related field. /liliRequired Skills: GenAI evaluation, Python, data science, benchmarking, rubric design, biomedical research, technical communication. /liliPreferred Skills: RAG evaluation, agent evaluation, biomedical informatics, expert review, annotation protocols, therapeutic area expertise, responsible AI. /liliRequired Skills: Preferred Skills: /li /ulpThe anticipated base pay range for this position is: €43,600.00 - €70,150.00 /ph3Benefits /h3ullian annual bonus with set target (% of pay) depending on pay grade / location, where the vigente amount is based on the employees’ and companies’ performance of the previous calendar year, or sales commissions. /lilivacation days, parental leave for a minimum of 12 weeks, bereavement leave, caregiver leave, volunteer leave, well-being reimbursement, programs for financial, physical and mental health. /liliservice anniversary and recognition awards, and subject to the terms of their respective plans, employees - and in some location’s eligible dependents - can participate in several insurance plans. /li /ulpAt Johnson Johnson,we believe health is everything. Our strength in healthcare innovation empowers us to build aworld where complex diseases are prevented, treated, and cured,where treatments are smarter and less invasive, andsolutions are personal.Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity.Learn more at /ppDo Not Sell or Share My Personal Information /ppLimit the Use of My Personal Information /p /p #J-18808-Ljbffr

📌 Postdoctoral Scholar, AI Evaluation & Standards (Madrid)
🏢 7300-Janssen-Cilag S.A. Legal Entity
📍 Madrid