Expressions of Mental Health and Illness Online
In this work, we explore how individuals with mental illness express themselves online and off. Through digital ethnography, including interviews with Instagram users and manual collection of public content on Instagram, we have holistically examined the experience of mental illness as expressed through social media. This user-centric approach reveals and addresses the limitations of computational techniques, many of which rigidly classify content, overemphasize the similarities of online communities or populations, and assume that traces of behavior are indicative of a static, off-screen reality. In contrast, our work demonstrates components of mental illness online that are not rigidly categorical or explicitly linked to offline behaviors. As part of this project, we are rethinking what it means to design for mental health and mental illness by incorporating literature from critical fields, such as Mad Studies.
Project team members: Jes Feuston, Anne Marie Piper
Image:project
Supporting Collaborative Learning through Surface Haptic Displays
This project explores the opportunities and challenges of using variable friction touch-screen displays to support collaborative learning among children and their parents. Our work involves prototyping and evaluating various early reading, math, and science learning applications for tablet- and phone-based surface haptic technology.
Project team members: Elham Beheshti, Katya Borgos-Rodriguez, Sarah Pila, Anne Marie Piper, Ed Colgate, Ellen Wartella
Image:project
Accessibility in Content Creation and Collaboration
Using both qualitative methods and design-based research approaches, we are collaborating with several communities to understand how disability impacts their work practices and creative process. We are especially interested in how professionals or hobbyists with disabilities collaborate in different domains such as programming, academic writing, visual arts, weaving, and audio editing. Some communities we have worked with include Deaf and hearing professionals, artists and academics with vision impairments, and adults with learning disabilities.
Project team members: Emily Wang, Maitraye Das, Abir Saha, Anne Marie Piper, Darren Gergle
Image:project
Designing for Accessible Language Expression
This line of work involves a series of projects on how technologies can be designed to support people who have disabilities that affect their experiences with written or spoken language, such as people with aphasia and adults with dyslexia. We are using qualitative methods to study how these individuals communicate and generate language in various settings, such as professional workplaces as well as in more informal settings such as restaurants. We are also studying how current technologies are adapted by these individuals and designing tools to support novel and evolving communication in these settings.
Project team members: Emily Wang, Mmachi Obiorah, Anne Marie Piper, Mike Horn
Image:project
New Interfaces for Creativity, Self-Expression, and Sharing among Older Adults
We are conducting field work to understand the practice of art therapy for older adults and through this have begun to prototype new hybrid interfaces that enable older adults and art therapists to capture, record, and share artwork. We are exploring the opportunities and challenges around content creation and sharing among various groups of older adults online. As part of this project, we are also understanding the role of caregivers in online interaction for individuals with cognitive impairments and/or speech-language impairments.
Image:project
Voice-Based Interfaces for Late-Life Social Communication and Content Generation
We are designing new hybrid systems, which blend the affordances of offline and online technologies, to support older adults in staying socially connected in the later stages of life. Our work involves long-term field observations and interviews to inform the design of novel hybrid technologies and evaluation of these systems in local retirement communities. Specifically, we are designing and studying new voice-based interactions that exploit the familiarity and pervasiveness of traditional telephones.

Image:project
Understanding Older Adults' Experiences Online: Blogging, Crowd Work, Social Media
We are investigating older adults' experiences learning to use social media and their motivations for interacting online, particularly as one's socio-emotional goal change over the lifespan. We are also researching ageism and age-bias in the context of online interaction.
Image:project
Audio-Enhanced Photos for Older Adult Communication and Reminiscence
This project examines the use of audio-enhanced photos and physical objects to support older adult communication and reminiscence. We enable creation of and interaction with audio-enhanced paper photographs through custom software deployed on a digital pen.
Image:project
Haptic E-Books for Children
This project examines the design and use of haptic feedback within the context of electronic books (e-books) for children. We are using the TPaD tablet to prototype and evaluate the ways in which haptic feedback may enhance digital story reading experiences for young children and their parents.
Image:project
Exploring Affective Communication with Surface Haptics
This project uses variable friction surface haptics enabled by the TPaD Tablet to explore affective communication between pairs of users. We developed three haptic applications for the TPad Tablet (text messaging, image sharing, and virtual touch) and evaluate the applications with intimate couples and strangers. This work illustrates that users readily associate haptics with emotional expression and that the intimacy of touch in the contexts we study is best suited for communications with close social partners.
Image:project
Improving Accessibility through Surface Haptics
With our collaborators in Mechanical Engineering, we are exploring the ways in which haptic feedback can improve accessibility for people with vision impairments. In one study, we designed and developed wearable information displays for blind individuals using a haptic smart watch. We found that a physical clock face, compass, and numerical keypad are productive representations for presenting information haptically, and these models build on existing tactile and spatial understandings of our target user group. In another part of this work, we explored the use of haptic feedback at the operating system level on Android devices to improve touch-screen accessibility.
Image:project
Authoring Multimodal Language Activities
Hybrid paper-digital interfaces are a promising approach for supporting language activities. The familiarity of pen and paper makes it a particularly attractive media for many user groups, including young children. Digital pens enhance interaction with traditional paper content by playing and recording audio and recognizing handwriting and gestures. Currently, generating custom interactive paper documents involves some programming, limiting its use by many user groups (e.g., educators and families) who might especially benefit from application of hybrid paper-digital interfaces in their practices. To address this need, we developed the TAP & PLAY end-user authoring system, which allows creation of custom language activities using off-the-shelf digital pen technology. Project Video.
Image:project
Documentation in Early Childhood Education
We are studying how teachers in one early childhood education center document development for children from three months to five years old. Our research examines the use of paper records and printed photos in order to understand the challenges of transitioning to a digital record keeping system.
Image:project
Communication and Speech-Language Therapy for Aphasia
This project involves a year-long field study of the process of speech-language therapy with an emphasis on the practices of therapists working with adults with aphasia and apraxia of speech. We observed that paper is a pervasive and important medium for this therapy context. Pen-based computing combines the familiar and flexible nature of low-fidelity communication aids (e.g., paper picture boards) with the benefits of high-fidelity computer-based technologies. We present the design of a hybrid paper-digital interface using multimodal digital pens and report on two early case studies of use by older adults undergoing speech-language therapy. Following this, we examine use of this technology by a single therapist and older adult with aphasia over a 12-week field deployment. Project Video.
Image:project
Older Adult Health Care Support and Communication
This project examines accessibility issues of surface computing with older adults and explores the appeal of surface computing for health care support. We present results from a laboratory study involving 20 older adults (age 60 to 88) performing gesture-based interactions on a multitouch surface. Older adults were able to successfully perform all actions on the surface computer, but some gestures that required two fingers (resize) and fine motor movement (rotate) were problematic. Ratings for ease of use and ease of performing each action as well as time required to figure out an action were similar to that of younger adults. Older adults reported that the surface computer was less intimidating, less frustrating, and less overwhelming than a traditional computer. Finally, the idea of using a surface computer for health care support was well-received by participants. Project Video.
Image:project
Improving Medical Communication for Deaf Patients
Shared Speech Interface (SSI) is an application for a tabletop display that facilitates communication between deaf and hearing individuals. Our approach uses a mulitmodal tabletop system to support medical conversations that may be very personal and require communication through a means other than an ASL interpreter. With SSI, the deaf patient can ask the doctor questions by typing on a keyboard, and the doctor speaks into a headset microphone. As both people communicate, their speech spills out onto the table in the form of speech bubbles. Speech bubbles can be moved and rearranged around medical diagrams, transforming the ephemeral nature of conversation into a shared, lasting representation. Project Video.
Image:project
Supporting Small Group Learning with Surface Computers
This work compares the affordances of presenting educational material on a tabletop display with presenting the same material using traditional paper handouts. Ten pairs of undergraduate students used digital or paper materials to prepare for exams during four one-hour study sessions over the course of a term. Students studying with the tabletop display solved problems on their own before resorting to answer keys and repeated activities more often than students studying with paper documents. We summarize study activities and discuss the benefits and drawbacks of each medium.
Image:project
Social Development for Children with Autism
SIDES is a multi-player tabletop computer game designed to help adolescents with Asperger's Syndrome practice effective group work skills. Individuals with Asperger's Syndrome traditionally have difficultly understanding accepted social protocols and navigating social situations. These individuals often describe the computer as a comfortable and motivating medium. On the other hand, tabletop displays are an effective medium for encouraging face-to-face social interaction around one computer. Through this project we explored how the design of multi-player tabletop computer games could encourage positive group work behaviors among this audience. Project Video.
Image:project