F&M Stories
Students Decode Zoom Gestures
A team of Franklin & Marshall College students combed through six hours of Zoom class footage to identify gestures exhibited by participating professors and students.
The results were overwhelming. Over the span of four recordings, the team identified 1,497 gestures.
"That's 1,497 cases of communication where the person performing it may not even be aware they were doing so," said sophomore Ayako Belka, a biology and psychology double major.
Under the tutelage of Elena Cuffari, assistant professor of psychology & scientific philosophical studies of mind, Belka was joined by senior Daisy Zang and sophomore Wendy Huang.
While gestures have been widely studied, the limitations of digital screen borders provide an unexpected twist.
"That's rare in gesture scholarship because we just haven't had a lot of conditions or studies where the gesture is partly blocked," Cuffari said.
"I also wanted to build up a corpus. This is a cool moment to have a bunch of data of people gesturing on Zoom. Yes, people meet on Zoom all the time, but it's not always so recorded and available and abundant. We had ninety-five hours of classes and we got to see professors' and students' behavior in a lot of detail in different class meetings," Cuffari said.
Research was conducted by taking recorded classroom meetings from four different F&M professors during the 2020-2021 school year, and coding certain aspects of all observed gestures through a linguistic annotation system, EUDICO Linguistic Annotator (ELAN).
To decode the clutter, the team created their own coding template "so that we could focus on how much information the gestures are bringing to an interaction, as well as their spatial orientation within the Zoom frame," Belka said.
Overall, the study showed that the most common type of gestures were ones that were only partially visible in the Zoom window, contradicting the team's initial hypothesis that more meaningful gestures would have better visibility.
To further determine the impact of these partial gestures, the research team is conducting a follow-up study that examines the effects of gesture visibility and attention span.
Ayako Belka '25
- Majors: Biology and Psychology
- Hometown: New Orleans
Did anything surprise you about this research?
What surprised me most about this research was the sheer amount of subconscious gesturing that we do every day. Gesturing is a form of communication that is so routine to us that we hardly notice it, yet it serves a huge role in daily interactions by communicating emotions and ideas that speech can't. Even over a virtual interface such as Zoom, we still saw significant evidence that people were continuing to make meaningful gestures. Over four videos (all around 1.5 hours), we saw 1,497 gestures. That's 1,497 cases of communication where the person performing it may not even be aware they were doing so. The impact of gestures on the person they are directed to is something we are planning on investigating, but I would also like to find out how gestures impact the person performing them. What does gesturing bring to the person performing the gesture?
Daisy Zang '23
- Major: Psychology
- Hometown: Kunshan, China
What inspired you to take on this research?
During the pandemic, I was struggling with online classes and the time difference, but I was also unsure about why the efficiency of learning was lower in online classes. Therefore, I was very excited to have the opportunity to explore what may impact the class efficiency. In addition, I am also interested in utilizing eye-tracking equipment in visualizing how the attention was changed when partial gestures are presented in comparison with the presentation of full gestures. Our follow-up study plans to start to use eye-tracking equipment to track attention changes of participants when watching partial gestures in comparison with the presentation of full gestures.
Wendy Huang '25
- Majors: Sociology and Mathematics
- Hometown: Nanjing, China
What inspired you to take on this research?
I am very interested in Zoom fatigue and how this affects the efficiency of online teaching. Gestures are unique components of teaching which are usually ignored by the audience, but facilitate our understanding unconsciously. Hence, I think it is significant to investigate how the Zoom frame's restriction will affect people's perception of gestures.
Related Articles
May 21, 2025
Faculty Earn Recognition Across Range of Academic Fields
This spring, F&M faculty members received numerous grant and fellowship opportunities, with several award announcements still pending. Faculty and staff have been recognized both nationally and globally for their contributions to a range of academic fields.
May 13, 2025
F&M Professor Awarded $500,000 Mellon Grant to Amplify Local Grower Voices Amid Climate Change
Franklin & Marshall College Associate Professor of Environmental Studies Eric Hirsch is the recipient of a $500,000 grant from the Mellon Foundation to lead a transformative three-year research initiative titled, “The Agricultural Futures Archive: Rural and Urban Growers in the Shadow of the Solar Panel.”
May 6, 2025
‘You’re in a Place Where Everyone Is Rooting for You’
Shreya Punj ’25 found her time at F&M to be a period of personal growth, fueled by a vibrant campus life that encouraged learning beyond the classroom. “F&M is home to people with diverse experiences and interests,” she said. “Learning from my peers and becoming friends with people who have lived drastically different lives than me has been an enriching experience and contributed significantly to my growth in the last four years.”