Tangible User Interfaces workshop:
Design Challenge Exercise A

This page describes results from one of three design challenges considered in the afternoon session of the CHI 2007 workshop on Tangible User Interfaces in Context and Theory.

Design Question:

"Augmenting Everyday Objects for Healthcare Applications"
(introduced by Beverly Harrison, Intel Research)

Background to the Question:

(to come)

Tangible User Interfaces

Discussion Notes:

When data is collected about patterns of use over time and anomolies in these patterns how does the user indicate exceptions/annotations as explanations? E.g., my grandkids came over today so my normal routines were disrupted, altered

How does system provide feedback to individuals? Is this a closed loop feedback system where the feedback and perhaps user corrections or input are then passed back into the system and reflected? Response - to some extent yes since system could show status indicator that person did not take medication X and if they subsequent took X the status would change. Also we are investigating various forms of feedback not only to person whose objects are tagged but also to social network, caregivers, family members so there are questions about what to show broader group about state.

How does system learn typical use - through sequences of actions? Would you do something like create/apply HMMs (Hidden Markov Models) to personalize? Issue - How much do you personalize versus create general models for broad groups of users? Also want to allow system to work out of the box rather than need to have users feed it training examples of what each behavior so it can learn before it works.

Has the approach delegated behaviors to artifacts rather than social structures that would normally occur? (note that a key issue to investigate is how communication patterns increase, decrease or change in topical nature when such a technology is used within a social network or between family/caregiver and elder).

If info provided to caregivers and feedback appears on visual display how do we respect privacy? Does this change the way we provide feedback?

Can we assume individuals are not trying to thwart system? E.g., tip medication bottle over but throw meds in garbage can rather than take and system reports meds as taken? Or what about example where elder know family will call to check up if status says meds not taken so they don?t take meds in order to create more social interactions with family?

Is such a system about enforcement? Could it be about motivation? Positive enforcement? Rather than telling people what they didn?t do and need to do? (Response - it is typically a blend of time critical reminders for things like meds where you want absolute compliance because consequence of non-compliance is high, versus the skipped a meal situation where it is more passive status indicator, and highest level general wellness indicators (like orb lamps).

What models of feedback can be used to support compliance (versus different model to support status)? Sound? Actuation? Social networks or rely on people? (Note we are trying to provide alerts of varying urgency and escalation to primary caregiver and individual depending upon how health critical the task is).

System as a way to indicate openness to visitors? And social interaction? For instance, a reminder to social network to call or stay involved by detecting presence of elder? Does tech make caregivers less involved because they now know about activity indirectly and so they do not have to call every day to find out this information?

Can this detect other types of activities like falls? (Note - fall detection is very hard to do accurately due to the variation in types of falls? believe we can make some headway but accuracy and reliability unknown at this point).

Can life patterns be augmented with other data like notes, emails, photos, etc.?

Are there other killer apps and what are these?

Is this approach similar to SenseCam approach from MSR Cambridge? Where problem space was discovered after tech was developed (in their case pendant camera - purpose uncertain, value uncertain - then found episodic memory application). Maybe this is a model for that style of approach?

Note for thwarting the system - for this type of instrumentation of objects - faking the system like pretending to cook is just as hard as actually doing the task so it is much more difficult to pretend to do something. You still need to touch all the objects and in a time sequence/time lapse that is believable and fits the models.

What about ordinary but sedentary adults as users? Are there other groups beyond elders that could use this in different application contexts? What is target population?

In Scotland - had project looking at people?s eating habits and kids would get points towards prizes like new bike for healthy eating habits. Could such positive incentives and games be used here? What about the social community posting its data and the effect this would have? Abstract rewards versus tangible rewards? Negative vs. Positive feedback. What happens if the feedback (as positive incentive) is increasing social contacts with other people?

Are there larger social implications? Are they ways to increase social capital?

Could you add in physiological sensors as indicators too? Pulse, O2, body temp, heart rate, etc.

Can users provide error feedback when system is wrong and how? Maybe they could record voice complaints?

Can you augment value of bracelet by providing other capabilities of value - e.g., digital voice memos to leave yourself reminders and messages?

Can system detect anything about emotional aspects - laughter? Socialization? Depression? (at present this is only done by the loose association between observed behaviors that are correlated with depression - e.g., not eating, sleeping too much, not taking care of personal care like combing hair, shaving, etc.) What about detecting speech affect? (Note this is a separate research effort at Intel Research Seattle - in current plan we are not doing any audio or video) .

What about pushing more on tangible instantiated objects as feedback - one tangible UI objects for each activity that is being tracked?