Supporting Collaborative Learning through Surface Haptic Displays
This project explores the opportunities and challenges of using variable friction touch-screen displays to support collaborative learning among children and their parents. Our work involves prototyping and evaluating various early reading, math, and science learning applications for tablet- and phone-based surface haptic technology.
Project team members: Elham Beheshti, Katya Borgos-Rodriguez, Sarah Pila, Anne Marie Piper, Ed Colgate, Ellen Wartella
Image:project
New Interfaces for Creativity, Self-Expression, and Sharing among Older Adults
We are conducting field work to understand the practice of art therapy for older adults and through this have begun to prototype new hybrid interfaces that enable older adults and art therapists to capture, record, and share artwork. We are exploring the opportunities and challenges around content creation and sharing among various groups of older adults online. As part of this project, we are also understanding the role of caregivers in online interaction for individuals with cognitive impairments and/or speech-language impairments.
Image:project
Voice-Based Interfaces for Late-Life Social Communication and Content Generation
We are designing new hybrid systems, which blend the affordances of offline and online technologies, to support older adults in staying socially connected in the later stages of life. Our work involves long-term field observations and interviews to inform the design of novel hybrid technologies and evaluation of these systems in local retirement communities. Specifically, we are designing and studying new voice-based interactions that exploit the familiarity and pervasiveness of traditional telephones.

Image:project
Understanding Older Adults' Experiences Online: Blogging, Crowd Work, Social Media
We are investigating older adults' experiences learning to use social media and their motivations for interacting online, particularly as one's socio-emotional goal change over the lifespan. We are also researching ageism and age-bias in the context of online interaction.
Image:project
Audio-Enhanced Photos for Older Adult Communication and Reminiscence
This project examines the use of audio-enhanced photos and physical objects to support older adult communication and reminiscence. We enable creation of and interaction with audio-enhanced paper photographs through custom software deployed on a digital pen.
Image:project
Haptic E-Books for Children
This project examines the design and use of haptic feedback within the context of electronic books (e-books) for children. We are using the TPaD tablet to prototype and evaluate the ways in which haptic feedback may enhance digital story reading experiences for young children and their parents.
Image:project
Exploring Affective Communication with Surface Haptics
This project uses variable friction surface haptics enabled by the TPaD Tablet to explore affective communication between pairs of users. We developed three haptic applications for the TPad Tablet (text messaging, image sharing, and virtual touch) and evaluate the applications with intimate couples and strangers. This work illustrates that users readily associate haptics with emotional expression and that the intimacy of touch in the contexts we study is best suited for communications with close social partners.
Image:project
Improving Accessibility through Surface Haptics
With our collaborators in Mechanical Engineering, we are exploring the ways in which haptic feedback can improve accessibility for people with vision impairments. In one study, we designed and developed wearable information displays for blind individuals using a haptic smart watch. We found that a physical clock face, compass, and numerical keypad are productive representations for presenting information haptically, and these models build on existing tactile and spatial understandings of our target user group. In another part of this work, we explored the use of haptic feedback at the operating system level on Android devices to improve touch-screen accessibility.
Image:project
Authoring Multimodal Language Activities
Hybrid paper-digital interfaces are a promising approach for supporting language activities. The familiarity of pen and paper makes it a particularly attractive media for many user groups, including young children. Digital pens enhance interaction with traditional paper content by playing and recording audio and recognizing handwriting and gestures. Currently, generating custom interactive paper documents involves some programming, limiting its use by many user groups (e.g., educators and families) who might especially benefit from application of hybrid paper-digital interfaces in their practices. To address this need, we developed the TAP & PLAY end-user authoring system, which allows creation of custom language activities using off-the-shelf digital pen technology. Project Video.
Image:project
Documentation in Early Childhood Education
We are studying how teachers in one early childhood education center document development for children from three months to five years old. Our research examines the use of paper records and printed photos in order to understand the challenges of transitioning to a digital record keeping system.
Image:project
Communication and Speech-Language Therapy for Aphasia
This project involves a year-long field study of the process of speech-language therapy with an emphasis on the practices of therapists working with adults with aphasia and apraxia of speech. We observed that paper is a pervasive and important medium for this therapy context. Pen-based computing combines the familiar and flexible nature of low-fidelity communication aids (e.g., paper picture boards) with the benefits of high-fidelity computer-based technologies. We present the design of a hybrid paper-digital interface using multimodal digital pens and report on two early case studies of use by older adults undergoing speech-language therapy. Following this, we examine use of this technology by a single therapist and older adult with aphasia over a 12-week field deployment. Project Video.
Image:project
Older Adult Health Care Support and Communication
This project examines accessibility issues of surface computing with older adults and explores the appeal of surface computing for health care support. We present results from a laboratory study involving 20 older adults (age 60 to 88) performing gesture-based interactions on a multitouch surface. Older adults were able to successfully perform all actions on the surface computer, but some gestures that required two fingers (resize) and fine motor movement (rotate) were problematic. Ratings for ease of use and ease of performing each action as well as time required to figure out an action were similar to that of younger adults. Older adults reported that the surface computer was less intimidating, less frustrating, and less overwhelming than a traditional computer. Finally, the idea of using a surface computer for health care support was well-received by participants. Project Video.
Image:project
Improving Medical Communication for Deaf Patients
Shared Speech Interface (SSI) is an application for a tabletop display that facilitates communication between deaf and hearing individuals. Our approach uses a mulitmodal tabletop system to support medical conversations that may be very personal and require communication through a means other than an ASL interpreter. With SSI, the deaf patient can ask the doctor questions by typing on a keyboard, and the doctor speaks into a headset microphone. As both people communicate, their speech spills out onto the table in the form of speech bubbles. Speech bubbles can be moved and rearranged around medical diagrams, transforming the ephemeral nature of conversation into a shared, lasting representation. Project Video.
Image:project
Supporting Small Group Learning with Surface Computers
This work compares the affordances of presenting educational material on a tabletop display with presenting the same material using traditional paper handouts. Ten pairs of undergraduate students used digital or paper materials to prepare for exams during four one-hour study sessions over the course of a term. Students studying with the tabletop display solved problems on their own before resorting to answer keys and repeated activities more often than students studying with paper documents. We summarize study activities and discuss the benefits and drawbacks of each medium.
Image:project
Social Development for Children with Autism
SIDES is a multi-player tabletop computer game designed to help adolescents with Asperger's Syndrome practice effective group work skills. Individuals with Asperger's Syndrome traditionally have difficultly understanding accepted social protocols and navigating social situations. These individuals often describe the computer as a comfortable and motivating medium. On the other hand, tabletop displays are an effective medium for encouraging face-to-face social interaction around one computer. Through this project we explored how the design of multi-player tabletop computer games could encourage positive group work behaviors among this audience. Project Video.
Image:project