publications | Richard Lee Davis

2025

SIGCSE

Jupyter Analytics: A Toolkit for Collecting, Analyzing, and Visualizing Distributed Student Activity in Jupyter Notebooks

Zhenyu Cai, Richard Lee Davis, Raphaël Mariétan, and 2 more authors

In Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1, Feb 2025

Abs DOI PDF Code

Jupyter is a web-based, interactive computing environment that supports many commonly-used programming languages. It has been widely adopted in the CS education community and is now rapidly expanding to other STEM disciplines due to the growing integration of programming in STEM education. However, unlike other educational platforms, there is currently no integrated way to capture, analyze, and visualize student interaction data in Jupyter notebooks. This means that teachers have limited to no visibility into student activity, preventing them from drawing insights from these data and providing timely interventions on the fly. In this paper, we present Jupyter Analytics, an end-to-end solution for teachers to collect, analyze, and visualize both synchronous and asynchronous learning activities in Jupyter. The Jupyter Analytics system consists of two JupyterLab extensions connected via a cloud-based backend. On the student side, we introduce the Jupyter Analytics Telemetry extension to anonymously capture students’ interaction activity with more structure and higher granularity than log data. On the teacher side, we introduce the Jupyter Analytics Dashboard extension, which visualizes real-time student data directly in the notebook interface. The Jupyter Analytics system was developed through an iterative co-design process with university instructors and teaching assistants, and has been implemented and tested in several university STEM courses. We report two use cases where Jupyter Analytics impacted teaching and learning in the context of exercise sessions, and discuss the potential value of our tools for CS education.
CSCL

“Jupyter-Notebook-as-Script”: Investigating the Nature and Impact of Implicit Collaboration Scripts in Computational Notebooks

Zhenyu Cai, Richard Lee Davis, Roland Tormey, and 1 more author

In Proceedings of the 18th International Conference on Computer-Supported Collaborative Learning-CSCL 2025, Pp. 223-231, Feb 2025

Naomi Miyake Outstanding Student Paper Award PDF

Won the Naomi Miyake Outstanding Student Paper Award, Nominated for the Best Full Paper Award
CHI

SketchAI: A "Sketch-First" Approach to Incorporating Generative AI into Fashion Design

Richard Lee Davis, Kevin Fred Mwaita, Livia Müller, and 4 more authors

In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Apr 2025

Abs DOI PDF

Generative AI technologies are increasingly being incorporated into creativity support tools. However, most generative AI tools rely on text-based prompting, requiring users to translate visual ideas into linguistic descriptions. This approach is misaligned with the sketch-driven workflows of creative professionals. To address this gap, we introduce SketchAI, a novel sketch-first interface for diffusion models that allows practitioners to use real-time sketching on a tablet computer to guide model outputs. Through a qualitative study with 29 fashion design apprentices, we explored the interface’s potential impacts on creative workflows. While some participants identified use cases where SketchAI streamlined routine tasks, others expressed concerns about its potential to undermine creative agency and exploration. These findings unearthed hidden complexity: while generative AI can support some aspects of creativity, its core capabilities may challenge the central identity of creative practitioners. While SketchAI does not resolve this problem, it does take a meaningful step towards reconciliation.
CHI

SpatiaLearn: Exploring XR Learning Environments for Reflective Writing

Jinqiao Li, Seyed Parsa Neshaei, Livia Müller, and 3 more authors

In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Apr 2025

Abs DOI PDF

Reflective writing promotes deeper learning by enhancing metacognitive awareness and critical thinking, but learners often struggle with structuring their reflections and maintaining focus. Generative AI and advances in spatial computing offer promising solutions. Extended reality (XR) environments create immersive, distraction-free settings, while conversational agents use dialog-based scaffolding guides to structure learners’ thoughts. However, research on combining dialog-based scaffolding with XR for reflective writing remains limited. To address this, we introduce SpatiaLearn, an adaptive XR tool that enhances reflective writing through conversational guidance in both traditional and immersive environments. A within-subjects study (N = 19) compared participants’ performance in traditional laptop and XR environments. Qualitative analysis shows the spatial interface enhances engagement but raises challenges like unfamiliar interactions and health concerns, requiring task adaptation for XR. This study advances the design of immersive tools for reflective writing, highlighting both the opportunities and challenges of spatial interfaces.
BJET

Metacognition Meets AI: Empowering Reflective Writing with Large Language Models

Seyed Parsa Neshaei, Paola Mejia-Domenzain, Richard Lee Davis, and 1 more author

British Journal of Educational Technology, Apr 2025

Abs DOI PDF

Reflective writing is known as a useful method in learning sciences to improve the metacognitive skills of students. However, students struggle to structure their reflections properly, limiting the possible learning gains. Previous works in educational technologies literature have explored the paradigms of learning from worked and modelling examples, but (a) their application to the domain of reflective writing is rare, (b) such methods might not scale properly to large-scale classrooms, and (c) they do not necessarily take the learning needs of each student into account. In this work, we suggest two approaches of integrating AI-enabled support in digital systems designed around learning from worked and modelling examples paradigms, to provide personalized learning and feedback to students using large language models (LLMs). We evaluate Reflectium, our reflective writing assistant, show benefits of integrating AI support into the learning from examples modalities and compare the perception of the users and their interaction behaviour when using each version of our tool. Our work sheds light on the applicability of generative LLMs to different types of providing support using the learning from examples paradigm, in the domain of reflective writing. Practitioner notes What is already known about this topic Reflective writing fosters metacognitive skills and improves learning gains and personal growth. The learning from worked and modelling examples paradigms is effective for skill acquisition and applying the acquired knowledge. Existing reflective writing assistants usually lack dynamic, AI-driven feedback or interactivity, limiting personalization and adaptability to each user’s own needs in the learning process. What this paper adds It introduces Reflectium, an AI-enabled reflective writing assistant, integrating intelligent and interactive writing support for both the learning from worked and modelling examples paradigms. It demonstrates the use of a fine-tuned large language model (LLM) for providing feedback in the learning from worked examples version, and an LLM-powered conversational agent simulating instructor interactions for the learning from modelling examples version. It reports findings from a user study comparing the positive impact of artificial intelligence (AI) support on learners’ performance, interaction behaviour and learning experience. Implications for practice and/or policy Digital tutoring systems for teaching reflective writing using the learning from worked examples paradigm should incorporate adaptive AI feedback to enhance learning gains. Conversational agents simulating peers/instructors and powered by LLMs can provide scalable, interactive support for learning from modelling examples, notably in large-scale educational settings. Reflective writing tools should be evaluated for their impact on different aspects of the learning process, such as task performance, interaction behaviour and user experience, to guide future improvements. Educators and policymakers should consider the integration of AI-driven reflective writing tools into teaching curricula to enhance reflective practices and metacognitive skill development.
AIED

One Code to Predict Them All: Universal Encoding for Inquiry Modeling

Jade Mai Cock, Valentine Delevaux, Ido Roll, and <span class="more-authors" title="click to view 2 more authors" onclick=" var element = $(this); element.attr('title', ''); var more_authors_text = element.text() == '2 more authors' ? 'Richard Davis, Tanja K"aser' : '2 more authors'; var cursorPosition = 0; var textAdder = setInterval(function(){ element.html(more_authors_text.substring(0, cursorPosition + 1)); if (++cursorPosition == more_authors_text.length){ clearInterval(textAdder); } }, '10'); " >2 more authors</span>

In International Conference on Artificial Intelligence in Education, Apr 2025
EDM

Using Large Multimodal Models to Extract Knowledge Components for Knowledge Tracing from Multimedia Question Information

Hyeongdon Moon, Richard Lee Davis, Seyed Parsa Neshaei, and 1 more author

In Proceedings of the 18th International Conference on Educational Data Mining, Apr 2025

DOI
EDM

Bridging the Data Gap: Using LLMs to Augment Datasets for Text Classification

Seyed Parsa Neshaei, Richard Lee Davis, Paola Mejia-Domenzain, and 2 more authors

In Proceedings of the 18th International Conference on Educational Data Mining, Apr 2025

DOI

2024

SIGCSE

A Comparative Analysis of Tools & Task Types for Measuring Computational Problem-Solving

Engin Bumbacher, Jérôme Brender, and Richard Lee Davis

In Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 2, Mar 2024

DOI PDF
ECTEL

Learning Analytics Beyond Traditional Classrooms: Addressing the Tensions of Cognitive and Meta-Cognitive Goals in Exercise Sessions

Zhenyu Cai, Richard Davis, Roland Tormey, and 1 more author

In Technology Enhanced Learning for Inclusive and Equitable Quality Education, Mar 2024

Abs DOI PDF

A range of learning analytics (LA) tools have been designed and integrated into university classes to facilitate teaching and learning. However, exercise sessions, the educational setting that complements lectures with practical activities, are commonly overlooked by LA researchers and designers. Little work has focused on involving the key stakeholders, teaching assistants (TAs), and incorporating human-centered design approaches in this context. To address this gap, we conducted a qualitative study to understand TAs’ common approaches and challenges of teaching in exercise sessions, and to explore their visions for LA dashboards that could be adapted into their current practices. Our results indicated that TAs in exercise sessions held two sets of goals in supporting students’ cognitive and meta-cognitive activities, and while LA tools were seen as offering numerous potential benefits, they were also seen as introducing tensions threatened to disrupt the delicate balance of both goals.
EDM

Investigation of Behavioral Differences: Uncovering Behavioral Sources of Demographic Bias in Educational Algorithms

Jade Mai Cock, Hugues Saltini, Haoyu Sheng, and 3 more authors

In Proceedings of the 17th International Conference on Educational Data Mining, Mar 2024

DOI PDF
CHI

Fashioning Creative Expertise with Generative AI: Graphical Interfaces for Design Space Exploration Better Support Ideation Than Text Prompts

Richard Lee Davis, Thiemo Wambsganss, Wei Jiang, and 3 more authors

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, May 2024

Abs DOI PDF

This paper investigates the potential impact of deep generative models on the work of creative professionals. We argue that current generative modeling tools lack critical features that would make them useful creativity support tools, and introduce our own tool, generative.fashion1, which was designed with theoretical principles of design space exploration in mind. Through qualitative studies with fashion design apprentices, we demonstrate how generative.fashion supported both divergent and convergent thinking, and compare it with a state-of-the-art text-based interface using Stable Diffusion. In general, the apprentices preferred generative.fashion, citing the features explicitly designed to support ideation. In two follow-up studies, we provide quantitative results that support and expand on these insights. We conclude that text-only prompts in existing models restrict creative exploration, especially for novices. Our work demonstrates that interfaces which are theoretically aligned with principles of design space exploration are essential for unlocking the full creative potential of generative AI.
ETRD

Hands-on Tasks Make Learning Visible: A Learning Analytics Lens on the Development of Mechanistic Problem-Solving Expertise in Makerspaces

Richard Lee Davis, Bertrand Schneider, Leah F. Rosenbaum, and 1 more author

Education Tech Research Dev, Feb 2024

Abs DOI PDF

This study investigated the impact of participating in a year-long digital-fabrication course on high-school seniors’ problem-solving skills, with a focus on problems involving mechanistic systems. The research questions centered on whether working in a makerspace impacted students’ abilities to solve such problems and whether the process data generated during problem-solving activities could be used to identify the different problem-solving approaches taken by the participants. A novel set of hands-on, mechanistic problems were created to answer these questions, and the results showed that after taking part in the course students performed significantly better on these problems, with the post-course students making more progress towards the solutions than the pre-course students. The process data revealed two distinct problem-solving approaches for each problem, one adopted primarily by experts (the expert approach) and one by pre-course students (the novice approach). The post-course students were more likely to adopt the expert approaches, which were strongly associated with better performance on each problem. The study found that participation in the course made the high-school students better able to “see” the various components and their ways of interacting, making them more like expert engineers.
EDM

Towards Modeling Learner Performance with Large Language Models

Seyed Parsa Neshaei, Richard Lee Davis, Adam Hazimeh, and 3 more authors

In Proceedings of the 17th International Conference on Educational Data Mining, Feb 2024

DOI PDF
Stepping Outside the Ivory Tower: How Can We Implement Multimodal Learning Analytics in Ecological Settings, and Turn Complex Temporal Data Sources into Actionable Insights?

Bertrand Schneider, Richard Davis, Roberto Martinez-Maldonado, and 3 more authors

Feb 2024

PDF

2023

The Future of Maker Education

Paulo Blikstein, Richard Davis, Leah F. Rosenbaum, and 11 more authors

Feb 2023

PDF
Protected Attributes Tell Us Who, Behavior Tells Us How: A Comparison of Demographic and Behavioral Oversampling for Fair Student Success Modeling

Jade Mai Cock, Muhammad Bilal, Richard Davis, and 2 more authors

In LAK23: 13th International Learning Analytics and Knowledge Conference, Mar 2023

Abs DOI PDF

Algorithms deployed in education can shape the learning experience and success of a student. It is therefore important to understand whether and how such algorithms might create inequalities or amplify existing biases. In this paper, we analyze the fairness of models which use behavioral data to identify at-risk students and suggest two novel pre-processing approaches for bias mitigation. Based on the concept of intersectionality, the first approach involves intelligent oversampling on combinations of demographic attributes. The second approach does not require any knowledge of demographic attributes and is based on the assumption that such attributes are a (noisy) proxy for student behavior. We hence propose to directly oversample different types of behaviors identified in a cluster analysis. We evaluate our approaches on data from (i) an open-ended learning environment and (ii) a flipped classroom course. Our results show that both approaches can mitigate model bias. Directly oversampling on behavior is a valuable alternative, when demographic metadata is not available. Source code and extended results are provided in https://github.com/epfl-ml4ed/behavioral-oversampling.
Fashioning the Future: Unlocking the Creative Potential of Deep Generative Models for Design Space Exploration

Richard Lee Davis, Thiemo Wambsganss, Wei Jiang, and 3 more authors

In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, Apr 2023

Abs DOI PDF

This paper investigates the potential impact of deep generative models on the work of creative professionals, specifically focusing on fashion design. We argue that current generative modeling tools lack critical features that would make them useful creativity support tools, and introduce our own tool, generative.fashion1, which was designed with theoretical principles of design space exploration in mind. Through qualitative studies with fashion design apprentices, we demonstrate how generative.fashion supported both divergent and convergent thinking, and compare it with a state-of-the-art diffusion model, Stable Diffusion. In general, the apprentices preferred generative.fashion over Stable Diffusion, citing the features explicitly designed to support ideation. We conclude that the exploration and development of novel interfaces and interaction modalities that are theoretically aligned with principles of design space exploration is crucial for unlocking the creative potential of generative AI and advancing a new era of creativity.

2022

VET Analytics

Richard Lee Davis, Son Do-Lenh, Mina Shirvani Boroujeni, and 1 more author

In Educational Technologies for Vocational Training: Experiences as Digital Clay, Apr 2022

PDF
Educational Technologies for Vocational Training: Experiences as Digital Clay

Pierre Dillenbourg

Oct 2022

Abs PDF

This book summarises 15 years of research (2007-2021) on the exploitation of digitaltechnologies for VET. In other words, our activities started the same year as the firstiPhone was launched! Four research groups formed the leading house on the topic,named DUAL-T, where T stands for technologies and DUAL refers to the Swiss VETsystem, relying to a great extent upon the alternating of days at school and days inthe workplace. This uniquely long-term research scheme has gathered more than 50research scientists, produced 13 doctoral theses, led to the development of multipledigital learning environments and enabled dozens of empirical studies on thousandsof apprentices and hundreds of teachers and company trainers. DUAL-T developeda network of stakeholders, including vocational schools and companies, but alsomany professional organizations and cantonal or national public entities. This bookdoes not provide a detailed account of all these activities, which have been publishedelsewhere.But it proposes several answers to the question:“Which digital technologies contribute to the enhancementof vocational education?”In VET systems, as well as in other education sectors, the choice of educational platformsis not made by teachers but is often a decision taken at the school, the districtor a higher hierarchical level. Therefore, along our 15 years of negotiationswith schools, many teachers declined to use one of our tools because they were constrainedby these top-down decisions. In this book, we aimed to make design choicesas independent as they could possibly be from school platforms. Some constraintscould not be avoided, but some ideas that used our tools could be implemented withWhatsApp, Instagram, MS Teams and Google Drive.
GANs for All: Supporting Fun and Intuitive Exploration of GAN Latent Spaces

Wei Jiang, Richard Lee Davis, Kevin Gonyop Kim, and 1 more author

In NeurIPS 2021 Competitions and Demonstrations Track, Oct 2022

PDF
GANs for All: Supporting Fun and Intuitive Exploration of GAN Latent Spaces

Wei Jiang, Richard Lee Davis, Kevin Gonyop Kim, and 1 more author

In Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, Jul 2022

Abs PDF

We have developed a new tool that makes it possible for people with zero programming experience to intentionally and meaningfully explore the latent space of a GAN. We combine a number of methods from the literature into a single system that includes multiple functionalities: uploading and locating images in the latent space, image generation with text, visual style mixing, and intentional and intuitive latent space exploration. This tool was developed to provide a means for designers to explore the "design space" of their domains. Our goal was to create a system to support novices in gaining a more complete, expert understanding of their domain{’}s design space by lowering the barrier of entry to using deep generative models in creative practice.
Mixplorer: Scaffolding Design Space Exploration through Genetic Recombination of Multiple Peoples’ Designs to Support Novices’ Creativity

Kevin Gonyop Kim, Richard Lee Davis, Alessia Eletta Coppi, and 2 more authors

In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, Apr 2022

Abs DOI PDF

The ability to consider a wide range of solutions to a design problem is a crucial skill for designers, and is a major differentiator between experts and novices. One reason for this is that novices are unaware of the full extent of the design space in which solutions are situated. To support novice designers with design space exploration, we introduce Mixplorer, a system that allows designers to take an initial design and mix it with other designs. Mixplorer differs from existing tools by supporting the exploration of ill-defined design spaces through social design space exploration. To evaluate Mixplorer, we conducted (1) an interview study with design instructors who reported that Mixplorer would “help to open the minds” of novice designers and (2) a controlled experiment with novices, finding that the design-mixing functionality of Mixplorer provided significantly better support for creativity, and that participants who mixed designs produced more novel designs.
On the Radar: Predicting near-Future Surges in Skills’ Hiring Demand to Provide Early Warning to Educators

Ramtin Yazdanian, Richard Lee Davis, Xiangcen Guo, and 3 more authors

Computers and Education: Artificial Intelligence, Jan 2022

Abs DOI PDF

The AI-driven Fourth Industrial Revolution and the COVID-19 pandemic have one important thing in common: they both have caused significant and rapid changes to the skill set landscape of various industries. These disruptive forces mean that the early identification of the newly rising skills in a labour market — which we call its “emerging skills” — is crucial to its workforce. It is also crucial to the educators who, in order to provide lifelong training to the workforce, need to quickly adapt their curricula to the new skills. We propose a classification methodology that uses the past job ad trends of skills to predict the emerging skills of a future period, defined as the skills that have experienced a surge in hiring demand in said period. This general definition allows for freedom in specifying the criteria for a skill being emerging (through thresholds on hiring demand and its growth), which could be important to educators. Applying our methodology to the Information and Communication Technologies (ICT) labour market in Singapore, we show that we are able to predict future emerging skills with good precision and recall and beat two baseline classifiers for multiple threshold sets. Our methodology also allows us to see where job ads fail to provide sufficient predictive signals, pointing to auxiliary data sources (such as Stack Overflow for ICT) and skill ontologies as potential remedies. The success of our method shows how AI can be used to empower learners and educators in the ICT domain (and potentially other domains) with useful and well-curated insights at a moment’s notice, thus helping speed up the process of curricular change.

2021

Neural Design Space Exploration

Wei Jiang, Richard Lee Davis, Kevin Gonyop Kim, and 1 more author

In 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Jan 2021

PDF
Teaching With Hapkit: Enabling Online Haptics Courses With Hands-On Laboratories

Tania Morimoto, Melisa Orta Martinez, Richard Davis, and 2 more authors

IEEE Robot. Automat. Mag., Sep 2021

DOI PDF

2020

Electronic Textiles and Ambient Belonging

Richard Lee Davis and Paulo Blikstein

In , Jun 2020

Abs DOI PDF

There is a growing body of evidence showing that electronic textiles provide a promising way of closing gender gaps in computing and electrical engineering. The prevailing explanation for this is framed in terms of attractive forces—women are drawn to e-textiles because of their alignment with “historically feminine practices” (Buchholz et al., 2014). However, these gender differences may also be due to repulsive forces. Traditional tools and materials may broadcast ambient identity cues that indicate women do not belong. This hypothesis leads to two predictions: (1) Women and girls are less likely to be interested in workshops that involve tools like wires and soldering irons because they associate those tools with feelings of exclusion and alienation, and (2) e-textiles may be more appealing to women and girls because these tools do not broadcast messages of exclusion. We tested these predictions in an online study with 42 university students (22 male, 20 female). We found that “standard” items like soldering irons and wires broadcast ambient identity cues that lowered women’s sense of belonging while having no effect on men. Additionally, we found that e-textiles did not have a similar discriminatory effect. Thus, “standard” items are not neutral, but charged with meaning, and these items broadcast messages to women that they do not belong.
Opening the Black Box: Investigating the Impact of Engineering Design on Mechanistic Problem Solving and Mechanistic Understanding

Richard Lee Davis

Stanford University, Jun 2020

Abs

The goal of this dissertation was to better understand how working on projects in makerspaces might affect the development of students’ skills and knowledge. However, because makerspaces support an enormous variety of activities, it was necessary to restrict the focus to a specific type of activity and a specific set of outcomes. The activity was building mechanistic systems through the engineering design process, and the outcomes of interest were the development of mechanistic problem-solving skills and the construction of mechanistic understanding. In the first study, high-school students took part in a year-long digital fabrication course in a makerspace. Over the course of the study, they worked on two long-term projects using the engineering design process. Both of the projects involved designing a single object that consisted of many interacting parts, and completing the projects required progressive refinement over the course of multiple iterations of the engineering design cycle. In the second project in the course, the students worked together to design and build a mechanistic system, the Rube Goldberg machine. After taking part in the course, the students received significantly higher scores on a set a hands-on problems involving mechanisms, and a fine-grained analysis of their problem-solving approaches indicated that they had become significantly more like expert mechanical engineers. This study suggested that designing and developing complex systems over multiple iterations of the engineering design process supported the development of mechanistic problem-solving skills in the high-school students. The second study was designed to learn more about how building a mechanistic system through the engineering design process might support the construction of mechanistic knowledge. This study used a 2x2 factorial design to understand the impact of different aspects of the engineering design process—problem-centered making and reflection—on specific dimensions of mechanistic understanding. Students in this study were tasked with building a LEGO pendulum clock, and they all had access to a webpage containing text, diagrams, and videos about how pendulum clocks worked. The two factors in this study were problem-centered making [building the clock with step-by-step instructions vs. having to figure out how to build the clock using information on the webpage] and reflection [self-explaining the text on the webpage vs. not self-explaining]. Crossing these factors resulted in four groups: Read, Build, Explain, and Build+Explain. By comparing these groups on a set of posttest questions designed to measure different dimensions of mechanistic understanding, it was possible to learn more about how each of these activities supported learning about the pendulum-clock mechanism. The first finding was that problem-centered making and self-explaining each supported learning about distinct dimensions of mechanistic knowledge. The students who self-explained learned more about what the purposes of the parts in the clock, and scored higher on a set of questions designed to assess declarative knowledge about the clock. The students who worked on the building problem (as opposed to the students who simply rebuilt the clock using instructions) learned more about the structural organization of the parts in the clock. However, neither of these activities was superior at supporting the construction of causal knowledge about the parts in the clock. The second finding was that the students in the Build+Explain group who engaged in both activities learned significantly more about the causal relations between components in the clock than any of the other groups. Additionally, this group of students were significantly more likely to transfer their knowledge to infer how an analogous, novel mechanism worked. These findings suggested that the combination of problem-centered making and reflection was most effective at supporting the construction of complete, transferable knowledge about a mechanism. Together, the studies in this dissertation suggest that building mechanisms through a problem-centered, reflective process may support the development of mechanistic problem-solving skills as well as the construction of mechanistic understanding. This work has implications for researchers who are interested in how people learn about mechanisms, and for practitioners who are interested in supporting students in learning about how things work. Future work is needed to better understand the mechanisms through which skill and knowledge develop during the process of problem-centered, reflective making. These findings point to future directions for research, including the investigation of educational engineering design processes and the use of virtual, problem-centered, reflective making activities to support learning about a larger number of mechanistic systems.
Measuring Ability-to-Learn Using Parametric Learning-Gain Functions

Chris Piech, Engin Bumbacher, and Richard Davis

In , Jul 2020

Abs PDF

One crucial function of a classroom, and a school more generally, is to prepare students for future learning. Students should have the capacity to learn new information and to acquire new skills. This ability to "learn" is a core competency in our rapidly changing world. But how do we measure ability to learn? And how can we measure how well a school has prepared their students to learn? In this paper we formally pose the problem, and introduce a grounded theory of how to measure ability to learn. Using simulations of students learning we provide initial evidence that this theory provides an elegant solution to this problem. We further validate our ideas using real world data from 70k middle-school students and show that our theory is more accurate and interpretable than current state-of-the-art models of learning gains. We consider our results a modest yet interesting first step for a novel type of test. [For the full proceedings, see ED607784.]
Variational Item Response Theory: Fast, Accurate, and Expressive

Mike Wu, Richard L Davis, Benjamin W Domingue, and 2 more authors

In Educational Data Mining, Jul 2020

Abs PDF

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing public policy. Yet larger datasets pose a difficult speed / accuracy challenge to contemporary algorithms for fitting IRT models. We introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scaleable without sacrificing accuracy. Using this inference approach we then extend classic IRT with expressive Bayesian models of responses. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and improvements in imputing missing data. The algorithm implementation is open-source, and easily usable.

2018

Making Connected Messages: Designing Community-Relevant Murals With Youth in Public Libraries

Yasmin Kafai, Orkan Telhan, Richard Lee Davis, and 2 more authors

In Reconceptualizing Libraries: Perspectives from the Information and Learning Sciences, Jul 2018

Abs

Maker activities are already popular in fabrication spaces, children’s museums, science centers, community organizations, and maker fairs. Connected Messages is a community-oriented maker activity based out of public library branches in Philadelphia, where murals are an existing part of the local art culture and the joint construction of interactive murals can create opportunities for community-relevant Making. This chapter describes the motivation and design rationale for the Connected Messages project, technical background for how this project was executed, and how mural-making took different forms at different library branches. As discussed, this is an example of a library-based experience that can build on the local resources and individual connections to sources of local pride and personal concern.

2017

Fake News, Real Consequences: Recruiting Neural Networks for the Fight against Fake News

Richard Davis and Chris Proctor

Jul 2017
The Haptic Bridge: Towards a Theory for Haptic-Supported Learning

Richard Lee Davis, Melisa Orta Martinez, Oliver Schneider, and 3 more authors

In Proceedings of the 2017 Conference on Interaction Design and Children, Jul 2017

Abs DOI PDF

Haptic force feedback systems are unique in their ability to dynamically render physical representations. Although haptic devices have shown promise for supporting learning, prior work mainly describes results of haptic-supported learning without identifying underlying learning mechanisms. To this end, we designed a haptic-supported learning environment and analyzed four students who used it to make connections between two different mathematical representations of sine and cosine: the unit circle, and their graph on the Cartesian plane. We highlight moments where students made connections between the representations, and identify how the haptic feedback supported these moments of insight. We use this evidence in support of a proposed theoretical and design framework for educational haptics. This framework captures four types of haptic representations, and focuses on one – the haptic bridge – that effectively scaffolds sense-making with multiple representations.
Making the Invisible Visible: A New Method for Capturing Student Development in Makerspaces

Richard Lee Davis, Bertrand Schneider, and Paulo Blikstein

In Making a Difference: Prioritizing Equity and Access in CSCL, Jul 2017

Abs DOI PDF

The contribution of this paper is twofold: we introduce a new kind of assessment developed to capture students’ learning in makerspaces, and we present a new perspective on how participating in a maker workshop impacts students. As opposed to traditional pen and paper tests, we designed a series of hands-on task that participants complete before and after a maker workshop. In this paper, we contrast high-school students’ performance with experts (graduate students in mechanical engineering) and found evidence that the students’ behavior became more similar to experts’ after participating in a maker workshop. For the scope of this paper, we focus on a single task and describe in detail our coding scheme and analyses. Additionally, we show how a combination of qualitative and computational analysis helped us develop metrics to compare novices’ and experts’ performances. We conclude by discussing the potential of this type of assessment for supporting students’ learning in makerspaces.
Predicting Unit-Test Scores for Graphic Representations of Simple Computer Programs

Richard Lee Davis

Jul 2017

PDF
Solder and Wire or Needle and Thread: Can the Tools We Use Change the Way We Think?

Richard Lee Davis, Christopher Proctor, Michelle Friend, and 1 more author

In Knowledge to Action: Achieving the Promise of Equal Educational Opportunity, Apr 2017

Abs DOI PDF

The gender gap in computing has persisted—and grown—over the past 40 years. One class of solutions offered as a way to close the gap are software and hardware design tools created for girls. This study compares one such construction kit—the Adafruit Flora—to a comparable kit that was not designed with girls in mind—the Arduino Leonardo. Using an experimental design, we found that at the start of the study the girls held largely stereotypical attitudes, and that these attitudes became less stereotyped for the girls in the experimental groups. Furthermore, we found little difference between the two experimental groups, casting doubt on prior claims as to the effectiveness of gendered construction kits on girls’ interest in and attitudes towards computing.
Libraries as Emerging Spaces for Computer-Supported Collaborative Learning in Schools and Communities

Victor R. Lee, Carrie Tzou, Megan Bang, and 15 more authors

Jul 2017

Abs PDF

Libraries are undergoing a reconceptualization in their roles as lifelong learning centers for local communities, with STEM content areas and Maker activities receiving special emphasis. This represents a critical and unique research, design, and partnership opportunity for learning scientists and computer-supported collaborative learning scholars. This symposium brings together project teams from four different locales in the United States that have partnered with libraries to bring about new resources and activities that emphasize computer-supported collaborative learning. These projects represent major urban libraries, special collections, community branch libraries, and school libraries. By bringing together these different teams, this symposium aims to promote dialogue about the affordances and constraints associated with CSCL-oriented activity design in libraries, identify commonalities and differences across region and library types, and ground-truth what specific challenges and solutions have been identified in researcher-library partnerships.

2016

3-D Printed Haptic Devices for Educational Applications

Melisa Orta Martinez, Tania K. Morimoto, Annalisa T. Taylor, and 7 more authors

In 24th IEEE Haptics Symposium, APR 08-11, 2016, Philadelphia, PA, Jul 2016

Abs PDF

DiVA portal is a finding tool for research publications and student theses written at the following 50 universities and research institutions.
HandsOn: Enabling Embodied, Creative STEM e-Learning with Programming-Free Force Feedback

Gordon Minaker, Oliver Schneider, Richard Lee Davis, and 1 more author

In Haptics: Perception, Devices, Control, and Applications, Jul 2016

Abs DOI PDF

Embodied, physical interaction can improve learning by making abstractions concrete, while online courses and interactive lesson plans have increased education access and versatility. Haptic technology could integrate these benefits, but requires both low-cost hardware (recently enabled by low-cost DIY devices) and accessible software that enables students to creatively explore haptic environments without writing code. To investigate haptic e-learning without user programming, we developed HandsOn, a conceptual model for exploratory, embodied STEM education software; and implemented it with the SpringSim interface and a task battery for high school students. In two studies, we confirm that low-cost devices can render haptics adequately for this purpose, find qualitative impact of SpringSim on student strategies and curiosity, and identify directions for tool improvement and extension.
Modelling Student Knowledge as a Latent Variable in Intelligent Tutoring Systems: A Comparison of Multiple Approaches

Qandeel Tariq, Alex Kolchinski, and Richard Davis

Jul 2016

PDF

2015

Artifacts as Pollen: What Makes a Makerspace Successful

Richard Lee Davis and Paulo Blikstein

In Toward Justice: Culture, Language, and Heritage in Education Research and Praxis, Apr 2015

Abs PDF

We set up a three-month workshop where students from a predominantly low-SES, Latino school came into a fablab and worked on two large-scale projects. Using a mixed-methods approach, we found that students became more comfortable with technology, developed autonomy and confidence, learned to accept failure, and valued working with their peers. We also discovered that the influence of the fablab reached into the community as well. The artifacts produced by students in the fablab spread beyond the fablab like pollen, helping to combat negative perceptions of science and math in the community.
Sketching Intentions: Comparing Different Metaphors for Programming Robots

Richard Lee Davis, Engin Bumbacher, Oceane Bel, and 2 more authors

In , Apr 2015

DOI PDF
Connected Messages

Orkan Telhan, Yasmin B. Kafai, Richard Lee Davis, and 2 more authors

In Civic Media Project, Apr 2015

2014

Learning Electricity Through Exploration and Instruction – Does the Order Matter?

Richard Lee Davis, Shima Salehi, and Engin Bumbacher

In FabLearn, Apr 2014

Abs PDF

Electricity is a difficult subject to learn because it is filled with counter-intuitive concepts like voltage, current, and resistance. Instructional materials provide learners with formal explanations and abstract knowledge, while hands-on experiments give learners opportunities to explore phenomena on their own and form practical knowledge. In this study we explored how different combinations of instructional materials and hands-on experiences affect learning outcomes. We designed two inquiry-based activities in the PhET Circuit Construction Kit and a reading passage with the goal of guiding participants towards a better understanding of basic electrical phenomena. The experiment consisted of manipulating the order in which participants went through the materials: One group received the reading passage after engaging in both activities; another group had the passage between the two activities, with the idea that the hands-on activities would prepare participants to learn from reading passages. We found no difference on posttest scores between these groups. However we did find that the number and type of circuits recorded in the second activity significantly predicted posttest scores. The study is a first step towards understanding how exploratory activities translate to learning.
Playing in the Arcade: Designing Tangible Interfaces with MaKey MaKey for Scratch Games

Eunkyoung Lee, Yasmin B. Kafai, Veena Vasudevan, and 1 more author

In Playful User Interfaces, Jan 2014

Abs PDF

Most tools for making games have focused on-screen-based design and ignored the potentially rich space of tangible interface design. In this chapter, we discuss how middle school youth (ages 10–12 years) designed and built their own tangible game interfaces to set up a game arcade. We conducted two workshops in which students used the MaKey MaKey, a low-cost tangible interface construction kit, to build touch-sensitive game controllers using everyday conductive materials for games they remixed in Scratch. We address the following research questions: (1) What types of tangible interfaces do youth create for their games? (2) How do youth designers deal with the complexities of coordinating the design of tangible interfaces with online Scratch games? (3) What do young users have to say about their tangible interface designs? We found that youth designers mostly replicated common controller designs but varied in their attention to either functionality or esthetics. An unexpected finding was how these different approaches followed traditional gender lines, with girls more focused on esthetics and boys more focused on functionality. These findings might point toward different expectations and informal experiences that need to be taken into consideration when bringing tangible design activities into educational settings. During the arcade, the youths’ perspectives on their games and controllers changed as they observed other people playing their games. They expressed pride in their creations and saw ways to refine their designs in order to improve usability. In our discussion, we address how the inclusion of tangible interface design can extend game making activities for learning. Ultimately, we want youth to move beyond and experiment more with conventions, not just to increase their technological understanding and flexibility but also as a way to more critically approach the design of everyday things.
Community-Embedded Design: Connected Messages at the Free Library of Philadelphia

K-Fai Steele, Richard Lee Davis, Yasmin B. Kafai, and 2 more authors

Mar 2014
Connected Messages: Library as Youth Maker Space

Orkan Telhan, Yasmin B. Kafai, Richard Lee Davis, and 2 more authors

Mar 2014
Connected Messages: A Maker Approach to Interactive Community Murals with Youth

Orkan Telhan, Yasmin B. Kafai, Richard Lee Davis, and 2 more authors

In Proceedings of the 2014 Conference on Interaction Design and Children, Mar 2014

PDF

2013

The Education Arcade: Crafting, Remixing, and Playing with Controllers for Scratch Games

Richard Lee Davis, Yasmin B. Kafai, Veena Vasudevan, and 1 more author

In Proceedings of the 12th International Conference on Interaction Design and Children, Mar 2013

PDF
Joystick Designs: Middle School Youth Crafting Controllers with MaKey MaKey for Scratch Games

Veena Vasudevan, Yasmin B. Kafai, Eunkyoung Lee, and 1 more author

In Games, Learning, and Society, Mar 2013

Abs PDF

While there are many tools for making games, most of these have focused on designing screen interfaces leaving aside the potentially rich space of designing tangible game interfaces. This paper reports on a workshop with middle school youth who created game controllers with MaKey MaKey, a tangible construction kit, to interface with their remixed Scratch games. The analyses focus on the design of game interfaces and programs and indicate that youth mostly replicated common controller designs but varied in their attention to either functionality or aesthetics. The interface designs followed traditional gender lines with girls more focused on aesthetics and boys more focused on functionality and were, to some extent, replicated in the remixes of the Scratch games. In the discussion we address the pedagogical and technological opportunities and challenges of including the design of tangible interfaces in game making for learning.