Microfoundations of rationality in the age of AI: On emotions, bodies and intelligence

Abstract

Seventy years from the birth of AI, organizations are adopting AI-based agents and tools at an unprecedented pace. The coming of age of AI in organizations signifies the emergence of human-AI configurations promising to surpass the inherited bounded rational nature of humans. These promises are built on two underlying assumptions that 1) there is nothing else involved in thinking beyond information processing and, hence, 2) computers can successfully take over or assist humans in knowledge work. Yet, studies on human-AI collaboration show that humans and AI often have trouble finding the “optimal grip” and appropriate reliance for working together. We suggest that what makes appropriate reliance in human-AI collaborations difficult are the fundamental differences in human and computational cognition which challenge the underlying assumptions of bounded rationality. First, we show that with advancements in AI, both human and computational cognition now involve emotions, however, the necessary conditions for human cognition involve both architectural and communicative aspects of emotion while computational cognition involves only the latter. Second, we show that with advancements in robotics, both human and computational cognition now involve physicality, however the necessary conditions for human cognition involve physical sensing, experiencing and performing, whereas computational cognition involves only physical sensing and performing. Our arguments imply that (a) while machine cognition is evolving, it is still bound to information processing, whereas human cognition is not, and (b) only focusing on what human and machine cognition have in common (e.g., communication of emotion) is not sufficient to ensure appropriate reliance in human-AI collaboration. We end our paper by discussing the implications for research, education, and policy, and proposing a research agenda.