Applied Scientist I

Are you passionate about advancing the science of evaluating large language models and intelligent agents? Join Thomson Reuters Labs, where we experiment, build, and deliver cutting-edge AI systems that empower professionals worldwide.

Our flagship AI assistant, CoCounsel, helps legal, tax, and business professionals work smarter. We’re expanding our LLM Evaluation team, focused on developing automated, scalable, and trustworthy evaluation frameworks that measure model reasoning, reliability, and alignment.

At Thomson Reuters Labs, we blend applied research with real-world impact. Our scientists work on projects spanning LLM reasoning, benchmarking, grounding, and agentic behavior—all aimed at ensuring our AI systems are effective, explainable, and robust.

We believe that rigorous evaluation is the foundation of responsible AI. This role offers the opportunity to push the boundaries of auto-evaluation, LLM-as-a-judge, and agentic evaluation methodologies, influencing how AI systems are measured and improved at scale.

As an Applied Scientist I, you will:

• Design and Conduct Evaluations: Develop and execute evaluation pipelines for LLMs and agentic systems, assessing reasoning, factual accuracy, and alignment.

• Automate and Scale: Build tools and frameworks for automatic evaluation, including synthetic dataset creation, LLM-as-a-judge workflows, and continuous benchmarking systems.

• Collaborate and Translate: Partner with applied scientists, ML engineers, and product managers to translate evaluation results into model improvements and product insights.

• Research and Experiment: Prototype new evaluation metrics, contribute to internal reports, and support publications or presentations on evaluation methods.

• Champion Best Practices: Promote reproducibility, transparency, and ethical AI evaluation within the team and broader organization.

You’re a great fit for this role if your background includes:

• PhD in Computer Science, Artificial Intelligence, Machine Learning, or a related field (exceptional Master’s candidates with equivalent experience will be considered).

• Research or hands-on experience with large language models, NLP evaluation, or agent-based AI systems.

• Strong understanding of LLM performance measurement, prompt evaluation, and reliability testing.

• Proficiency in Python and familiarity with ML libraries such as PyTorch, Transformers, and LangChain.

• Comfort with experimental design, data analysis, and communicating technical findings clearly.

• Experience with LLM evaluation frameworks (e.g., OpenAI Evals, HELM, LM Harness, or custom auto-eval tools).

• Familiarity with retrieval-augmented generation (RAG), tool-using agents, or agentic evaluation methodologies.

• Experience in cloud-based ML development (AWS, Azure, or GCP).

• Record of publications or preprints in top-tier venues (e.g., NeurIPS, ACL, EMNLP, ICLR) or equivalent research contributions.

• Interest in Responsible AI, fairness, and interpretability research.

#LI-AB3

About Us

Thomson Reuters informs the way forward by bringing together the trusted content and technology that people and organizations need to make the right decisions. We serve professionals across legal, tax, accounting, compliance, government, and media. Our products combine highly specialized software and insights to empower professionals with the data, intelligence, and solutions needed to make informed decisions, and to help institutions in their pursuit of justice, truth, and transparency. Reuters, part of Thomson Reuters, is a world leading provider of trusted journalism and news.

We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments. At a time when objectivity, accuracy, fairness, and transparency are under attack, we consider it our duty to pursue them. Sound exciting? Join us and help shape the industries that move society forward.

As a global business, we rely on the unique backgrounds, perspectives, and experiences of all employees to deliver on our business goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity Employer providing a drug-free workplace.

We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law. More information on requesting an accommodation here.

Learn more on how to protect yourself from fraudulent job postings here.

More information about Thomson Reuters can be found on thomsonreuters.com.