Research Engineer, Frontier Safety Risk Assessment

We are seeking 2 Research Engineers for the Frontier Safety Risk Assessment team within the AGI Safety and Alignment Team. 

In this role, you will contribute novel research towards our ability to measure and assess risk from frontier models. This might include:

• Identifying new risk pathways within current areas (loss of control, ML R&D, cyber, CBRN, harmful manipulation) or in new ones;
• Conceiving of, designing, and developing new ways to measure pre-mitigation and post-mitigation risk.
• Forecasting and scenario planning for future risks which are not yet material.

Your work will involve complex conceptual thinking as well as engineering. You should be comfortable with research that is uncertain, under-constrained, and which does not have an achievable “right answer”. You should also be skilled at engineering, especially using Python, and able to rapidly familiarise yourself with internal and external codebases. Lastly, you should be able to adapt to pragmatic constraints around compute and researcher time that require us to prioritise effort based on the value of information.

Although this job description is written for a Research Engineer, all members of this team are better thought of as members of technical staff. We expect everyone to contribute to the research as well as the engineering and to be strong in both areas.

The role will mostly depend on your general ability to assess and manage future risks, rather than from specialist knowledge within the risk domains, but insofar as specialist knowledge is helpful, knowledge in ML R&D and loss of control as risk domains are likely the most valuable.

• You are adept at generating ideas and designing experiments, and implementing these in Python with real AI systems.
• You are keen to address risks from foundation models, and have thought about how to do so. You plan for your research to impact production systems on a timescale between “immediately” and “a few years”.
• You are excited to work with strong contributors to make progress towards a shared ambitious goal.
• With strong, clear communication skills, you are confident engaging technical stakeholders to share research insights tailored to their background.

• Experience in areas such as frontier risk assessment and/or mitigations, safety, and alignment.
• Engineering experience with LLM training and inference.
• PhD in Computer Science or Machine Learning related field.
• A track record of publications at venues such as NeurIPS, ICLR, ICML, RL/DL, EMNLP, AAAI and UAI.
• Experience with collaborating or leading an applied research project.