• Erik Talvitie
Assistant Professor of Computer Science

717-358-5869

revx.gnyivgvr@snaqz.rqh

Office: STA232

Spring 2016 Office Hours

Monday 10:30-12:00 Wednesday 2:00-3:30 Thursday 3:35-5:00
  or by appointment  

Education

Ph.D. in Computer Science, University of Michigan (2010)

M.S. in Computer Science, University of Michigan (2007)

B.A. in Computer Science and Mathematics, Oberlin College (2004)

Research

My primary research interests focus on the core artificial intelligence ambition of creating artifical autonomous agents that can behave flexibly and competently in rich, complex environments. I tend to approach this problem through my background in machine learning, and specifically reinforcement learning. Broadly speaking, then, I focus on the learning problem faced by an agent that is placed in an unknown environment and would like to learn, from its own experience, how to make good decisions (where "good" is defined by some specified reward signal). This raises interesting and challenging questions about how such an agent should represent and maintain knowledge in order to make both learning and planning tractible, even when the world is very complex. 

In 2016 I was fortunate enough to receive an NSF CAREER grant. The five year project will focus on the challenge of applying model-based reinforcement learning (MBRL) in complex environments. In MBRL the agent learns to make predictions about future events based on its own decisions (model learning). Then it uses those predictions to make decisions (planning). In sufficiently complicated environments, the model the agent learns is necessarily flawed, which can lead to catastrophic failures in planning. This project seeks to develop MBRL methods that are aware of and robust to flaws in the agent's model and planner, hopefully making it possible to learn approximate models and make suboptimal plans that nevertheless result in reasonable behavior.

Publications

 

Conference Papers

  • State of the Art Control of Atari Games Using Shallow Reinforcement Learning. Yitao Liang, Marlos C. Machado, Erik Talvitie, Michael Bowling. In 'Proceedings of the Fifteenth International Conference on Automous Agents and Multiagent Systems (AAMAS).' 2016. (pdf) (bibtex) (arXiv)
  • Agnostic System Identification for Monte Carlo Planning. Erik Talvitie. In 'Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI),' 2015. (pdf) (bibtex)
  • Improving Exploration in UCT Using Local Manifolds. Sriram Srinivasan, Erik Talvitie, and Michael Bowling. In 'Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI),' 2015. (pdf) (bibtex)
  • Policy Tree: Adaptive Representation for Policy Gradient. Ujjwal Das Gupta, Erik Talvitie, and Michael Bowling. In 'Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI),' 2015. (pdf) (bibtex)
  • Model Regularization for Stable Sample Rollouts. Erik Talvitie. In 'Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence (UAI),' 2014. (pdf) (bibtex)
  • Skip Context Tree Switching. Marc Bellemare, Joel Veness, and Erik Talvitie. In 'Proceedings of the Thirty-First International Conference on Machine Learning (ICML),' 2014. (pdf) (bibtex)
  • Learning Partially Observable Models Using Temporally Abstract Decision Trees. Erik Talvitie. In 'Advances in Neural Information Processing Systems 25 (NIPS),' 2012. (pdf) (bibtex)
  • Maintaining Predictions Over Time Without a Model. Erik Talvitie and Satinder Singh. In 'Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI),' 2009. (pdf) (bibtex)
  • Simple Local Models for Complex Dynamical Systems. Erik Talvitie and Satinder Singh. In 'Advances in Neural Information Processing Systems 21 (NIPS),' 2008. (pdf) (bibtex)
  • An Experts Algorithm for Transfer Learning. Erik Talvitie and Satinder Singh. In 'Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI),' 2007. (pdf) (bibtex)

 

Journal Papers

 

  • Learning to Make Predictions In Partially Observable Environments Without a Generative Model. Erik Talvitie and Satinder Singh. Journal of AI Research, Volume 42 (2011), pages 353-392. (pdf) (bibtex)

 

Workshop and Symposium Papers

 

  • Pairwise Relative Offset Features for Atari 2600 Games. Erik Talvitie and Michael Bowling. In 'Proceedings of the AAAI Workshop on Learning for General Competency in Video Games,' 2015. (pdf)
  • Building Incomplete but Accurate Models. Erik Talvitie, Britton Wolfe, and Satinder Singh. Tenth International Symposium on Artificial Intelligence and Mathematics (ISAIM), 2008. (pdf) (bibtex)

 

Dissertation

 

  • Simple Partial Models for Complex Dynamical Systems. Erik Talvitie. Ph.D. Thesis, University of Michigan, 2010. (pdf)

 

Grants and Awards 
  • NSF Award #1552533. CAREER:  Using Imperfect Predictions to Make Good Decisions, Principle Investigator. July 2016-June 2021.

Student Collaborations

Relative Offset Features in Atari 2600 Games

  • Yitao Liang ('16) (Summer 2015 - Spring 2016) 

Adaptive Representation for Policy Gradient

State Generalization in UCT with Local Manifolds

Combining Learning and Search in Atari 2600 Games

  • George Gallo ('14) (Summer 2012 - Fall 2012) 
    • Hackman Scholar and Independent Study
  • William Tran ('14) (Summer 2012 - Spring 2013)
    • Hackman Scholar and Independent Study

Course Information

Fall 2016:

  • CPS 222: Computer Science III
  • CPS/MAT 237: Discrete Mathematics

Spring 2016:

  • CPS 112: Computer Science II -- Algorithms and Data Structures
  • CPS 112 Lab
  • CPS 337: Theoretical Foundations of Computer Science

Fall 2015:

  • CPS 222: Computer Science III

Spring 2015:

  • CPS 337: Theoretical Foundations of Computer Science
  • CPS 371: Artificial Intelligence

Fall 2014:

  • CPS 111: Computer Science I -- Introduction to Computational Thinking
  • CPS 111 Lab
  • CPS 222: Computer Science III

Fall 2013/Spring 2014: Junior Faculty Leave

Spring 2013:

  • CPS 337: Theoretical Foundations of Computer Science

Fall 2012:

  • CPS 275: Advanced Data Structures
  • CPS 370: Machine Learning

Spring 2012:

  • CPS 371: Artificial Intelligence
  • CPS 374: Theoretical Foundations of Computer Science

Fall 2011:

  • CPS 111: Computer Science I -- Introduction to Computational Thinking
  • CPS 111 Lab
  • CPS 275: Advanced Data Structures

Spring 2011:

  • CPS 112: Computer Science II -- Algorithms and Data Structures
  • CPS 274: Theoretical Foundations of Computer Science

Fall 2010:

  • CPS 170: Computer Science I -- Introduction to Computational Thinking
  • CPS 170 Lab