Craig Sherstan

AI Research Scientist with 10+ years of experience specializing in Reinforcement Learning (RL) and predictive modeling. Proven track record of applying RL to gaming and robotics, including leading contributions to GT Sophy, an AI agent that outperformed human champions in Gran Turismo. Published extensively in top conferences such as NeurIPS, AAAI, and Nature. Passionate about solving real-world problems through cutting-edge AI technologies.

I’m a research scientist working in the field of Reinforcement Learning. That means I make AI agents that learn to solve tasks from interactions with their world. For example, I was part of a team that (trained an AI agent to outrace the top human racers in the racing game Gran Turismo)[https://ai.sony/blog/blog-016/]. Our agent learned to race by playing against other AI agents, receiving rewards or penalties, and adjusting its behavior to get a higher score.

I’ve been involved in reinforcement learning since 2013 when I started my MSc at the University of Alberta under the supervision of Richard Sutton and Patrick Pilarski. I received the Vanier Scholarship which enabled me to do a PhD under the supervision of Patrick Pilarski. My graduate research was on learning predictive representations of an agent’s interactions with its world using a particular approach from reinforcement learning known as general value functions. During my PhD I had internships with 3 different companies doing RL related work: Cogitai (now part of Sony AI), Borealis AI, and Huawei. I’ve been working as a research scientist at Sony AI since 2020. In that time I’ve managed to get both a Nature paper and a NeurIPS paper published! In 2022 I moved to Tokyo, Japan to work on applying RL to robot control.

In addition to my recent career as an AI researcher I worked for a bit less than a decade in software development with various roles including development, architect, project management, and business development.