ph3Job Description /h3 pAt Johnson Johnson, we believe health is everything. Our strength in healthcare innovation empowers us to build a world where complex diseases are prevented, treated, and cured, where treatments are smarter and less invasive, and solutions are personal. Through our expertise in Innovative Medicine and MedTech, we are uniquely positioned to innovate across the full spectrum of healthcare solutions today to deliver the breakthroughs of tomorrow, and profoundly impact health for humanity. Learn more at jnj.com. /p pAs guided by Our Credo, Johnson Johnson is responsible to our employees who work with us throughout the world. We provide an inclusive work environment where each person is considered as an individual. At Johnson Johnson, we respect the diversity and dignity of our employees and recognize their merit. /p h3Job Function /h3 pCareer Programs /p h3Job Sub Function /h3 pPost Doc – Data Analytics Computational Sciences /p h3Job Category /h3 pCareer Program /p h3All Job Posting Locations: /h3 pBarcelona, Spain, Beerse, Antwerp, Belgium, Madrid, Spain, Raritan, New Jersey, United States of America, Titusville, New Jersey, United States of America /p h3Job Description /h3 pAt JJ we are developing Generative AI solutions to support pharmaceutical RD, including literature review, evidence synthesis, document QA, therapeutic area knowledge search, translational science workflows, and RD decision support. These systems need to be tested before teams use them in scientific workflows. In pharmaceutical RD, a useful AI response depends on the question, user, source material, therapeutic area, and risk of error. We are looking for a postdoctoral researcher to help design methods that test whether GenAI tools produce answers that are accurate, evidence-grounded, traceable, usable, and appropriate for the intended task. The role reports to the Associate Director, Generative AI Evaluation Quality Standards. The team defines how JJ Innovative Medicine evaluates GenAI systems before use and helps determine when they are ready for release, expansion, or improvement. /p h3Key Responsibilities /h3 ul liDesign evaluation frameworks, rubrics, and criteria for GenAI tools used across pharmaceutical RD. /li liDevelop therapeutic-area-specific criteria with business and scientific teams to reflect domain and use-case quality needs. /li liBuild benchmark datasets, reference answer sets,



annotation guides, and evaluation datasets. /li liRun expert reviews with scientific, clinical, regulatory, medical, data science, and engineering teams. /li liTest LLM, RAG, and agent performance, including accuracy, source grounding, retrieval quality, citation fidelity, task completion, robustness, safety, and usability. /li liAnalyze failure patterns such as unsupported claims, incorrect reasoning, poor evidence use, missing uncertainty, weak traceability, or failure to follow instructions. /li liTranslate evaluation findings into improvements in prompts, retrieval methods, agent workflows, tools, and user experience. /li liHelp define release criteria for systems moving from prototype to limited release, expanded use, or product support. /li liReview emerging evaluation methods and adapt useful approaches for pharmaceutical RD. /li liDocument methods, findings, and recommendations so teams can apply consistent evaluation practices. /li liDesign and develop agentic judge methods to evaluate GenAI outputs against defined criteria, flag evidence gaps or unsupported claims, and support expert review workflows. /li /ul h3Qualifications /h3 h3Education /h3 ul liPhD or equivalent research experience in biomedical science, computational biology, bioinformatics, AI/ML, data science, clinical research, regulatory science, biostatistics, pharmaceutical sciences, or a related field. /li /ul h3Required /h3 h3Experience and skills /h3 ul liUnderstanding of biomedical science, pharmaceutical RD, therapeutic area science, translational science, clinical development, regulatory science, biomedical informatics, data science, or related areas. /li liExperience translating expert judgment into criteria, rubrics, datasets, protocols, or measurable outcomes. /li liExperience designing or applying evaluation methods, benchmark datasets, annotation protocols, validation studies, quality reviews, or assessment frameworks. /li liInterest in testing GenAI systems, including LLMs, RAG, and AI agents.



/li liProficiency in Python and common data science or machine learning tools. /li liAbility to analyze model outputs, compare performance, identify failure patterns, and recommend improvements. /li liClear written and verbal communication skills. /li /ul h3Preferred /h3 ul liExperience with LLM APIs, embeddings, vector databases, prompt engineering, agent frameworks, or AI evaluation tools. /li liExperience evaluating retrieval quality, generated answers, multi-step workflows, tool use, scientific reasoning, citation quality, or evidence-grounded outputs. /li liExperience designing expert review workflows, annotation instructions, adjudication processes, or inter-rater reliability analyses. /li liDomain knowledge in one or more biomedical or therapeutic areas. /li liFamiliarity with biomedical data standards, structured scientific or clinical data, ontologies, knowledge graphs, CDISC, FHIR, or related frameworks. /li liPublications or applied research in AI evaluation, NLP, biomedical informatics, machine learning, data science, computational biology, bioinformatics, or a related field. /li /ul h3Required Skills: /h3 pGenAI evaluation, Python, data science, benchmarking, rubric design, biomedical research, technical communication. /p h3Preferred Skills: /h3 pRAG evaluation, agent evaluation, biomedical informatics, expert review, annotation protocols, therapeutic area expertise, responsible AI. /p h3The Anticipated Base Pay Range For This Position Is /h3 p€43,600.00 - €70,150.00 /p h3Benefits /h3 ul liannual bonus with set target (% of pay) depending on pay grade / location, where the actual amount is based on the employees’ and companies’ performance of the previous calendar year, or sales commissions. /li livacation days /li liparental leave for a minimum of 12 weeks /li libereavement leave /li licaregiver leave /li livolunteer leave /li liwell‑being reimbursement /li liprograms for financial, physical and mental health /li liservice anniversary and recognition awards /li liparticipation in several insurance plans, subject to the terms of their respective plans, employees - and in some location’s eligible dependents - can participate in several insurance plans. /li liThis is for informative purposes only. Amounts and presente benefits may vary by location and are subject to change. /li /ul /p #J-18808-Ljbffr

📌 Postdoctoral Scholar, AI Evaluation & Standards (Madrid)
🏢 Johnson & Johnson Innovative Medicine
📍 Madrid