bounded optimality and control

Our work on bounded optimality and bounded rationality provides a new way to model and explain human behavior by leveraging its adaptive nature. The basic idea is that human behavior—from the lowest levels of motor control to multi-tasking strategies to complex decision making—can be understood as an adaptation to the joint constraints (resources and bounds) of the human processing system, the external probabilistic environment, and an internal reward function. This idea has been around in some form in psychology for many decades, but advances in our understanding of the processing constraints, advances in computational algorithms for adaptive control, and advances in raw computing power make it possible now to more fully explore its implications. In artificial intelligence, the idea of bounded optimality was given an elegant formal definition by Stuart Russell and Devika Subramanian in 1995.

More specifically, with collaborators at Birmingham (Andrew Howes) and NASA Ames (Alonso Vera), we have developed an approach to cognitive theory and modeling called cognitively bounded rational analysis that derives adaptive strategies for the control of behavior given a specific set of processing constraints (e.g. on short-term memory or perceptual processes) a probabilistic task environment, and a quantitative utility function (e.g., specifying a precise speed-accuracy tradeoff).

The approach has strong ties to reinforcement learning, and our most recent computational modeling developments adopt the reinforcement learning approach to derive adaptive behavior. This work (in collaboration with both Andrew Howes and Satinder Singh, has now reached a level of maturity that it is possible to apply it to many challenging domains, including psycholinguistics, judgement and decision making, and applied human performance modeling.

For more on this work, read the papers below, and visit Andrew Howes' website at Birminghamr.

key overview publications

Attend to the copyright notice.

Lewis, R. L., Shvartsman, M., and Singh, S. (to appear). The adaptive nature of eye-movements in linguistic tasks: How payoff and architecture shape speed-accuracy tradeoffs. Topics in Cognitive Science. [ PDF ]

Bratman, J., Shvartsman, M., Lewis, R. L., and Singh, S. (2010). A new approach to exploring language emergence as boundedly optimal control in the face of environmental and cognitive constraints. In Salvucci, D. and Gunzelmann, G., editors, Proceedings of the 10th International Conference on Cognitive Modeling. To appear. [ PDF ]

Singh, S., Lewis, R. L., Barto, A. G., and Sorg, J. (2010). Instrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transactions on Autonomous Mental Development. To appear. [ PDF ]

Howes, A., Lewis, R. L., and Vera, A. H. (2009). Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review, 116(4):717-751. [ PDF ]

Howes, A., Lewis, R. L., and Vera, A. (2007). Bounding rational analysis: Constraints on asymptotic performance. In Gray, W. D., editor, Integrated Models of Cognitive Systems. Oxford University Press, New York. [ PDF ]

These references were generated by bibtex2html 1.97.

 

other relevant publications

Sorg, J., Singh, S., and Lewis, R. L. (2010). Internal rewards mitigate agent boundedness. In International Conference on Machine Learning, Haifa, Israel. [ PDF ]

Singh, S., Lewis, R. L., and Barto, A. G. (2009). Where do rewards come from? In Proceedings of the Annual Conference of the Cognitive Science Society, pages 2601-2606, Amsterdam. [ PDF ]

Smith, M. R., Lewis, R. L., Howes, A., Chu, A., Green, C., and Vera, A. (2008). More than 8,192 ways to skin a cat: Modeling behavior in multidimensional strategy spaces. In Love, B. C., McRae, K., and Sloutsky, V. M., editors, Proceedings of the 30th Annual Conference of the Cognitive Science Society, pages 1441-1446, Austin, TX. [ PDF ]

Chu, A., Lewis, R. L., and Howes, A. (2007). Evaluating the performance of optimizing constraint satisfaction techniques for cognitive constraint modeling. In Lewis, R., Polk, T., and .Laird, J., editors, The Proceedings of the 8th International Conference on Cognitive Modeling, pages 26-31, Ann Arbor, Michigan. Psychology Press/Taylor & Francis. [ PDF ]

Eng, K., Lewis, R. L., Tollinger, I., Chu, A., and Howes, A. (2006). Generating automated predictions of behavior strategically adapted to specific performance objectives. In Proceedings of the Computer-Human Interaction Conference, CHI 2006. Best paper nomination: top 5 per cent of submissions. [ PDF ]

Howes, A., Lewis, R. L., Vera, A., and Richardson, J. (2005). Information-requirements grammar: A theory of the structure of competence for interaction. In Proceedings of the Cognitive Science Society, Stresa, Italy. [ PDF ]

Tollinger, I., Lewis, R. L., McCurdy, M., Tollinger, P., Vera, A., Howes, A., and Pelton, L. (2005). Supporting efficient development of cognitive models at multiple skill levels: Exploring recent advances in constraint-based modeling. In Proceedings of the Computer-Human Interaction Conference, Portland, Ore. Best paper nomination: top 5 per cent of submissions. [ PDF ]

Vera, A. H., Howes, A., Lewis, R. L., Tollinger, I., Eng, K., and Richardson, J. (2005a). A constraint-based approach to understanding the composition of skill. In Proceedings of the Human-Computer Interaction 2005 Symposium,, Las Vegas.

Vera, A. H., Tollinger, I., Eng, K., Lewis, R. L., and Howes, A. (2005b). Architectural building blocks as the locus of adaptive behavior selection. In Proceedings of the Cognitive Science Society, Stresa, Italy. [ PDF ]

Howes, A., Vera, A., Lewis, R. L., and McCurdy, M. (2004). Cognitive constraint modeling: A formal approach to supporting reasoning about behavior. In Proceedings of the Cognitive Science Society, Chicago. [ PDF ]

Lewis, R. L., Vera, A., and Howes, A. (2004). A constraint-based approach to understanding the composition of skill. In Proceedings of the International Conference on Cognitive Modeling. [ PDF ]

Vera, A., Howes, A., McCurdy, M., and Lewis, R. L. (2004). A constraint-satisfaction approach to predicting skilled interactive cognition. In Proceedings of the Computer-Human Interaction Conferece CHI-2004, Vienna, Austria. [ PDF ]

These references were generated by bibtex2html 1.97.