Research Assistant / Research Associate in Safe Reinforcement Learning through Formal Methods

This post of Research Assistant (pre-doctoral) or Research Associate (post-doctoral) is to conduct world-leading research on safe reinforcement learning through formal methods, under the direction of Dr Francesco Belardinelli, within the EPSRC New Investigator Award An Abstraction-based Technique for Safe Reinforcement Learning.

Autonomous agents learning to act in unknown environments have been attracting research interest due to their wider implications for AI, as well as for their applications in key domains, including robotics, network optimisation, resource allocation. Currently, one of the most successful approaches is reinforcement learning (RL). However, to learn how to act, agents are required to explore the environment, which in safety-critical scenarios means that they might take dangerous actions, possibly harming themselves or even putting human lives at risk. Consequently, reinforcement learning is still rarely used in real-world applications, where multiple safety-critical constraints need to be satisfied simultaneously.

The main goal of this project is to develop Safe through Abstraction (multi-agent) Reinforcement learning (StAR), a framework to formally guarantee the safe behaviour of agents learning to act in unknown environments, through the satisfaction of safety constraints by the policies synthesized through RL, both at training and test time. We aim at combining RL and formal methods to ensure the satisfaction of constraints expressed in (probabilistic) temporal logic (PTL) in multi-agent environments.

The post is based in the Department of Computing at Imperial College London. This is a leading department of Computer Science among UK Universities. The department has achieved top results in each of the research assessment exercises undertaken by the Higher Education Funding Council for England. The successful applicant will join the Formal Methods in AI (FMAI) research group, led by Dr Belardinelli. FMAI is a friendly, vibrant, multi-national team working on various aspects of safe and trustworthy RL (for further information on the group and related projects, see: https://www.doc.ic.ac.uk/~fbelard/).

Duties and responsibilities

The position offers an exciting opportunity for conducting internationally leading and impactful research in safe reinforcement learning. The Research Assistant/Associate will be responsible for researching and delivering abstraction-based methods to guarantee the safe and trustworthy behaviour of autonomous agents based on the most widely-used RL algorithms. They will also be expected to submit publications to top-tier conferences and journals in AI.

Essential requirements

To apply for this position, you must have a strong computer science background with a focus on AI, have experience, including a proven publication track-record, in at least two of the following areas, as well as ability and willingness to become familiar with the other: Logic-based languages and formal methods; Formal verification, including model checking; (safe) Reinforcement. You should also have:

• A PhD degree (or close to complettion) in computer science or a related area.
• Familiarity with standard reinforcement learning libraries/data analysis.
• Excellent communication skills and ability to work with others.
• Ability to organise your own work and set priorities to meet deadlines.

• Candidates who have not yet been officially awarded their PhD but are in the process of obtaining one will be appointed as Research Assistant within the salary range £38,194 to £41,388 per annum.

Further information

In addition to completing the online application, candidates should attach:

• A full CV, with a list of all publications
• A 2-page research statement indicating what you see are interesting research issues relating to the above post and why your expertise is relevant.

Informal enquiries related to the position should be directed to Dr. Francesco Belardinelli: francesco.belardinelli@imperial.ac.uk.

For queries regarding the application process contact Jamie Perrins: j.perrins@imperial.ac.uk