Awarded 2 Million SEK from Digital Futures for Multimodal GenAI in Engineering Education

I’m excited to announce that Olga Viberg and I have been awarded a 2 million SEK grant from Digital Futures for our project “Transforming Engineering Education with Multimodal GenAI and Intelligent Agents for Improving Students’ Problem-Solving Skills.”

Project Overview

Large Multimodal Models (LμMs) have the potential to transform engineering education by supporting hands-on, experiential learning. LμMs can process images, audio, video, and other data types, making them ideal for supporting physical engineering design tasks. However, these tools must be carefully designed to align with educational theories and support, rather than hinder, student learning.

This project aims to develop and evaluate a pedagogically-aligned virtual teaching assistant (μTA) powered by LμMs to support problem-solving with physical systems in real-world settings for engineering education. We’re addressing the challenges students face when dealing with complex, ill-defined problems in engineering design courses and other experiential learning contexts.

Research Focus

While generative AI tools like large language models have revolutionized education, they remain largely confined to screen-based, text-centric tasks such as programming and writing. Recent advancements in Large Multimodal Models enable processing of diverse inputs, such as text, images, and videos, offering opportunities to extend AI’s benefits to experiential learning environments like workshops and labs.

Little is known about how LμMs can support hands-on, ill-defined problem-solving tasks central to engineering education. This project pioneers the integration of LμMs into these settings, co-designing tools with students and educators to foster skills critical for engineering innovation.

Cross-Disciplinary Collaboration

This Research Pair grant brings together expertise from two departments at KTH Royal Institute of Technology. The collaboration integrates Olga’s expertise in the design and evaluation of educational technologies—with a strong focus on AI adoption in STEM education and participatory design methods—with my experience in designing AI-driven tools for experiential learning, integrating multimodal systems, and advancing pedagogical alignment for generative AI technologies.

The two-year project runs from January 2025 through December 2026.

For more details about the project, visit the full announcement on the Digital Futures website.