Distinguished Speaker Series from CUIDS - Topic: Victims of Circumstance: How Environment Manipulation Shapes Reinforcement-Learning Behaviours. - Abstract:
Machine learning algorithms have been subjected to a range of attacks, both to thwart and to subvert their learning. It is particularly easy to do with Reinforcement Learning algorithms that heavily depend on their perceptions being reliable, their attempted actions correctly executed, and the rewards they reap indicative of the progress towards their goal. Control any one of those aspects, and you can make an RL agent fail or, worse, learn a bad behaviour. But what if perceptions come with error correcting codes, actions are verifiable, and the reward is strictly intrinsic to the agent? Are our RL agents safe from manipulation, then? Turns out no. It is possible, by the process of environment poisoning (i.e., changing how the environment behaves in response to agent actions), to manipulate an RL agent into learning a target (bad) behaviour. In this talk, I will show how it can be done, discuss how flexible the approach is, and what the future expects of it.