Investigating the applicability of user models for motion-impaired users
Simeon Keates, John Clarkson
Department of Engineering
University of Cambridge
Trumpington Street, Cambridge CB2 1PZ, UK
+44 1223 332673
lsk12@eng.cam.ac.uk, pjc10@eng.cam.ac.uk
Peter Robinson
Computer Laboratory
University of Cambridge
Pembroke Street, Cambridge, CB2 3QG, UK
+44 1223 334637
Recent discussions about the concept of Universal Access have shown that traditional HCI approaches are not the best way to achieve universal accessibility and that more specific studies are required to reach this goal [9]. Two core themes from this work are proactivity - addressing the issue of accessibility at design time, and adaptation - the ability for the interface to be tailored to the user [8].
Designing for motion-impaired users requires the adoption of strongly user-centred design practices, complemented by rigorous usability testing. Usability engineering techniques enable a designer to produce an end-product that is ultimately more usable [6]. This covers a wide range of attributes including user ability to understand, ease of learning, and clarity of input. However, central to the theme of user-centred design is having an understanding, or model, of the user. Looking at the interaction process involves investigating a wide range of parameters with complex interdependencies. A framework is needed to achieve successful descriptions of interaction.
User modelling techniques provide such a structure. The origins of such techniques are in neuropsychology and the attempts to understand the brain and its functions through empirical models. User modelling is widely used in the field of human computer interaction because of the clear benefits it brings to the design process [4]. It requires an understanding of the user, the interface and the system. This paper concentrates on evaluating how well a model can be used to obtain a detailed understanding of button activation for motion-impaired users.
It is important to note at this stage that practical limitations, principally involving the variable availability of individual users, restrict the usefulness of detailed statistical analysis. Consequently, the evidence is presented as primarily qualitative.
Computer-based assessment tools can be developed from accepted neuropsychological theory to provide information on user performance parameters [3]. If designed well, they can provide the basis for a detailed user model.
Total time = xt p + yt c + zt m
where x, y and z are integers and t p, tc and tm correspond to the times for single occurrences of the perceptual, cognitive and motor functions. On neurological grounds, the model assumes that it is only possible for each constituent of the cycle to occur in integer multiples of the base time. In other words, the theory states that it is impossible to have half a cognitive cycle or a third of a motor response [3].Although this is a very simple model, it was selected because it is very easy to understand and to observe deviations from the predicted behaviour. It is also the basis on which many more sophisticated models are based.
Given the variable nature of motion impairments across the different disability types, it may be expected that motor functions would behave differently for individual users. It is therefore necessary to validate any user model before assuming that the theoretical basis is correct for a particular user group. To this end, a first set of user trials was established at the Papworth Trust, a local residential disabled community, with the users detailed in Table 1. It is worth noting the heterogeneity of the users. They were specifically chosen to represent a broad range of impairments and severities, to avoid condition-specific, or impairment-specific results.
User |
Condition |
PJ3 |
Tetraplegia (from head injury) |
PJ4 |
Muscular Dystrophy |
PJ5 |
Spastic Quadriplegia Cerebral Palsy |
PJ6 |
Athetoid Cerebral Palsy |
PJ7 |
Friedrich’s Ataxia |
PJ8 |
Athetoid Cerebral Palsy |
Table 1. The users from the first set of Papworth trials.
User trials offer a valuable source of observational data. Such observations can provide qualitative reinforcement of quantitative data and also indications of other behaviour should the theory prove to be in need of modification.
The first task, Delay, involved the user observing the motion of the circle whilst time delays were inserted into the motion at random points. The purpose of the experiment was to determine whether the user could successfully detect the locations of the delays for given delay durations. An initial delay of 500ms was used and then incrementally decreased until the user could not determine the location of the delay. The smallest delay successfully observed was recorded as the perceptual response time, t p.
A second task, Smooth, was also used, in which the circle was moved in the same circular pattern in discrete segments of motion. The duration of each movement segment was varied between 10ms and 150ms. The user was asked to state whether the motion was smooth (continuous), jerky (discrete) or borderline. An analogy to an aeroplane propeller was used to describe differences between the motion types. As a propeller begins moving slowly, it is possible to discern each blade. However, as it gains in speed, the blades begin to merge until all that can be seen is the impression of a circle. The borderline between these cases, which corresponds to tp [3], was the time recorded.
Both the Delay and Smooth tasks were performed with able-bodied and motion-impaired users. Tables 2(a) and 2(b) summarise the results observed. The mean times recorded are rounded to the nearest 10ms. The perceptual response time from Card et al is approximately 100ms.
User |
t p Delay (ms) |
t p Smooth (ms) |
A |
75 |
70 - 80 |
B |
95 |
70 - 90 |
C |
75 |
70 - 80 |
Mean |
80 |
80 |
Table 2(a). Perceptual response times: able-bodied users.
User |
t p Delay (ms) |
t p Smooth (ms) |
PJ3 |
115 |
110 - 120 |
PJ5 |
105 |
90 - 110 |
PJ6 |
95 |
90 - 100 |
PJ7 |
90 |
90 - 100 |
PJ8 |
75 |
70 - 80 |
Mean |
100 |
100 |
Table 2(b). Perceptual response times: motion-impaired users.
The accepted theory states that the time recorded at the end of the key-up corresponds to one perceptual cycle to perceive that the stimulus has been seen, one cognitive cycle to decide that the stimulus has been perceived and then two motor function times to press and release the key. From the theory, the key-up action should be automatic for this task and not involve any perceptual or cognitive steps. Consequently, the process can theoretically be described by the simple reaction time t p + t c + 2t m.
Instead of the brightly coloured background, the second task had the single OK button accompanied by either a large green triangle or blue square. The users had to recognise the colour before activating the button. This required an additional cognitive step for the extra decision process required. Consequently, the time to complete this task was theoretically t p + 2t c + 2t m..
Finally, the users were presented with a letter above the OK button and were asked to recognise the letter before activating the button. This involves a second additional cognitive step, classifying the shape. The time for recognising the letter then pressing the button corresponded to t p + 3t c + 2t m.
User |
t c (ms) |
A |
100 |
B |
95 |
C |
90 |
D |
89 |
Mean |
93 |
Table 3(a). Cognitive cycle times: able-bodied users.
User |
t c (ms) |
PJ3 |
105 |
PJ4 |
116 |
PJ5 |
101 |
PJ6 |
121 |
PJ7 |
107 |
PJ8 |
128 |
Mean |
110 |
Table 3(b). Cognitive cycle times: motion-impaired users.
Tables 3(a) and 3(b) show the cognitive cycle times obtained from the computer-based able-bodied and motion-impaired user trials by finding the time differences between each of the different cognitive tasks. Again the mean times shown are rounded to the nearest 10ms. The calculated cognitive response time from Card et al is 70ms.
User |
t m (ms) |
A |
78 |
B |
81 |
C |
70 |
D |
67 |
E |
61 |
Mean |
70 |
Table 4(a). Motor function times: able-bodied users.
User |
t m (ms) |
PJ4 |
120 |
PJ6 |
96 |
Mean |
110 |
PJ3 |
223 |
PJ7 |
198 |
Mean |
210 |
PJ5 |
306 |
PJ8 |
297 |
Mean |
300 |
Table 4(b). Motor function times: motion-impaired users.
The above tables of results show that the able-bodied users are consistent and compare well with the 70ms from Card. However, the motion-impaired results appear to show three bands of times. The lowest value band is sufficiently close to the able-bodied values to be explained away as the effect of the motion-impairment producing slower motion. However, the other bands require more careful analysis.
Taking the values for t
p and t
c derived above, this process gives a total time of
t
p+t
c+2t
m »
(100+110+110*2) = 430ms. When divided by a factor of two, as for Table 4.b, a time of 215ms
results (cf. 210ms observed). However, with the approximate similarity in value between
t
c and t
p, it could equally be that two cognitive cycles are present somewhere
in the loop.
Similarly, the third variation involves extra cognitive and perceptual steps in recognising that the key has been pressed.
This gives a total interaction time of 2*(100+110+110) = 640 giving a time of 320ms (cf. 300ms
observed). Again, the perceptual cycle could be replaced by a cognitive cycle to achieve similar
times.
The assertion of extra cognitive and perceptual cycles is supported by two different sources. The first is empirical observations. Watching the users, particularly their direction of gaze, whilst they interacted with the computer, showed when cognitive processing was occurring rather than physical motion. It was this observation that initially suggested the presence of the extra cycles.
Secondly, the times taken to respond to a simple stimulus for two of the users showed an apparent discrepancy between the times recorded for t p, t c and t m for the motion impaired users and the reaction time to a simple stimulus. The values obtained are shown in Table 5.
Reaction to simple stimulus (ms) |
Able-bodied: theory |
Able-bodied: recorded |
Motion-impaired: recorded |
Predicted |
310 |
320 |
420 |
Observed |
- |
320 |
620 |
Table 5. Response times to a simple stimulus.
The 200ms discrepancy for the motion-impaired users is not explained by the standard MHP paradigm. However, it is approximately equal to an extra cognitive and perceptual step under the above regime (210ms) or two additional cognitive steps (220ms).
User |
Condition |
PI3 |
Athetoid Cerebral Palsy |
PI5 |
Athetoid Cerebral Palsy, deaf, non-speaking, ambulant |
PI6 |
Athetoid Cerebral Palsy, ambulant |
PI7 |
Friedrich’s Ataxia |
Table 6. The users from the second set of Papworth trials.
The users for the second set of trials were selected to form a small group that was representative of a wide range of capabilities. Two of the users, PI5 and PI6 were mildly impaired in their hands movements. The other two users, PI3 and PI7, had more severely impaired hand motion. PI3 displayed symptoms of spasms and weak movement. PI7 had continuous tremor and clenched hands. The full range of users is shown in Table 6. Note that only user PI/PJ7 participated in both this and the previous experiments. The other three users were new to the trials.The trials were not repeated with able-bodied users because the results from the first set of trials for this group were in accordance with the MHP theory. It was therefore decided that those results would still be applicable for the discussion of the second set of trials.
User |
t p (ms) |
t c (ms) |
PI3 |
95 |
120 |
PI5 |
100 |
100 |
PI6 |
90 |
110 |
PI7 |
90 |
110 |
Table 7. The perceptual and cognitive cycle times.
The results from the motor function trials are shown in Table 8. The mean times are rounded the nearest 10ms. Results are only available for the mouse for user PI3 because he had to withdraw from the trials for an operation.
With the single exception of the mouse for user PI5, the results for users PI5 and PI6 reflect the interaction predicted by theory of one motor cycle for each button-down and button-up action. The overall average motor function time is 100ms for both of these users (cf. 70ms for able-bodied users).
The longer times from user PI5 with the mouse are because the user was left-handed and experienced difficulty using a ‘right-handed’ mouse.
The results from user PI3 and PI7 exhibit the longer motor function times seen from the more severely impaired user in the first set of trials. The motor function times observed for both users are significantly longer than those for PI5 and PI6. Only the key-down time for PI7 with the space bar varies from this pattern.
User |
Input Device |
Button down |
Button up |
||
Mean time (ms) |
Std dev. |
Mean time (ms) |
Std dev. |
||
PI3 |
Mouse |
230 |
35 |
210 |
34 |
PI5 |
Mouse |
170 |
42 |
150 |
26 |
Space bar |
80 |
5 |
115 |
22 |
|
Trackpad |
90 |
14 |
100 |
19 |
|
PI6 |
Mouse |
100 |
28 |
90 |
23 |
Space bar |
80 |
14 |
110 |
13 |
|
Trackpad |
80 |
17 |
110 |
20 |
|
PI7 |
Mouse |
210 |
71 |
180 |
43 |
Space bar |
120 |
39 |
240 |
86 |
|
Trackpad |
200 |
43 |
230 |
58 |
|
EasyBall |
220 |
67 |
310 |
80 |
Table 8. Motor function button-down and button-up times.
Looking at the results more closely, it can be seen that for PI3 the button-down and button-up times are very similar, implying the same interaction stages for both actions. The same applies to PI7 with the mouse and the trackpad. However, the results from PI7 using the space bar and EasyBall show approximately 100ms difference between the button-down and button-up times.
The results from this task are shown in Table 9. Again, the mean times recorded are rounded to the nearest 10ms.
From the MHP theory, the button-down time should correspond to t p + t c + t m and the button-up time should be t m. For user PI6, the expected button-down and button-up times were 300ms (90+110+100) and 100ms respectively. The button-down and button-up times in Table 9 generally correspond well with the predicted values for PI6. Only the trackpad button-down time appears to differ significantly and the high standard deviation of that value implies that the discrepancy could be because of experimental noise.
User PI5 also had the same predicted times for button-down (100+100+100 = 300ms) and button-up (100ms). The button-down times recorded in Table 9 agree well with the predicted time, however the key-up times vary quite considerably.
User |
Input Device |
Button down |
Button up |
||
Mean time (ms) |
Std dev. |
Mean time (ms) |
Std dev. |
||
PI5 |
Mouse |
320 |
52 |
260 |
26 |
Space bar |
320 |
46 |
140 |
18 |
|
Trackpad |
320 |
49 |
210 |
30 |
|
PI6 |
Mouse |
320 |
49 |
110 |
22 |
Space bar |
330 |
42 |
120 |
28 |
|
Trackpad |
380 |
70 |
100 |
17 |
|
PI7 |
Mouse |
360 |
57 |
180 |
20 |
Space bar |
450 |
87 |
190 |
38 |
|
Trackpad |
450 |
122 |
190 |
39 |
|
EasyBall |
490 |
82 |
170 |
50 |
Table 9. Reaction button-down and button-up times.
User PI7 appears to have consistent key-up times, but a spread of approximately 130ms across the mean key-down times.
The motor function times for PI3 are approximately twice those for users PI5 and PI6. With the exceptions of the space bar key-down and the EasyBall button-up, the same is true for PI7. Given that the number of motor function cycles is fixed at one for each time recorded, then extra cycles of another kind were being inserted into both the button-down and button-up actions, or the motor function times are much longer for those users.
The space bar key-down times show that PI7 is capable of producing faster motor function times than the 210ms button-down average for the other devices. This implies that the apparently slow times are because of the insertion of extra cycles into the interaction and reinforces the assertions made at the conclusion of the first set of user trials.
If the 120ms space bar time is taken as the true value of t m, then the time difference between the expected and observed button-down actions is approximately 90ms. This broadly corresponds to the values of both t p and t c for this user. Looking at the average button-up time of 220ms (excluding the EasyBall time) this again gives a similar difference of 100ms. Consequently, the most likely description of the interaction is one extra cycle being inserted into both the button-down and button-up actions and that it is the same type of cycle being inserted into both.
Further evidence of this comes from examining the times recorded in more detail. Figure 1 shows
the motor function task times recorded for PI7 using the EasyBall.
Figure 1. The motor function task times for PI7.
The major peaks in the button-down times occur at 110, 210 and 310ms. The 110ms peak most likely corresponds to the true value of t m and compares well to the observed 120ms space bar key-down time. The central peak, at 210ms, represents the most frequent time observed and coincides with the t p + t c time from the interaction model above. The third peak, 310ms, most likely corresponds to the insertion of a second cognitive cycle, but could also represent the insertion of a perceptual cycle. The data is inconclusive about this.The times for the button-up action align well with those for the button-down action, but displaced by 100ms. This implies that for this input device, extra cognitive cycles are being inserted compared to the button-down activity. This is reflected in the 310ms button-up time for the EasyBall in Table 8. For the other input devices, the 100ms offset in button-up times is not observed, and the peaks in button-down and button-up activities are coincident.
Looking at the reaction task, the button-up times for PI7 are similar to those obtained for the motor function task. This implies the presence of a cognitive cycle in that phase of the task. The predicted button-down time for this user should be 90+110+120 = 320ms. This is broadly similar to that recorded for the mouse, but approximately 130-170ms faster than for the other input devices. Figure 2 shows the full range of times obtained for the button-down activity for user PI7 with the trackpad button.
Figure 2. Trackpad button-down times for PI7.
The first peak in Figure 2 occurs at approximately 320ms, in accordance with the time predicted by the MHP theory. This is followed by a broader peak between 400 and 600 ms and then a narrower peak at 650ms. A possible explanation of the central broad peak is that there are actually two peaks present, one at approximately 450ms and the other at 550ms and that the high frequency of the 450-500ms time band is a result of the two peaks overlapping.
Figure 3.Trackpad button-up times for PI7.
The button-up times for reaction task for PI7 using the trackpad button are shown in Figure 3. There are two principal peaks that can be seen. The first occurs at around 110ms and provides further evidence of t m being approximately that value for PI7. The second peak is broader, but centred on 210ms. This implies the presence of an extra cognitive cycle as for the button-up activity in the motor function task.Consequently, the most likely resultant interaction process for user PI7 reacting to a stimulus is:
CONCLUSIONS The results show that even using a very simple user model, such as the Model Human Processor, can offer valuable insights into how motion-impaired users interact with computers.
The results from both sets of user trials show that the individual components of the Model Human Processor are comparable for able-bodied and motion-impaired users. As expected, the largest observed difference was the motor function time. For this the motion-impaired users were approximately 50% slower than their able-bodied counterparts. However, a variation of approximately 20ms was also noted between the times for the cognitive cycles, with the motion- impaired group being slower. The precise cause of this cannot be identified from this experiment alone. However, the assertion is that it there is an additional delay due to the extra effort required to plan and particularly to control physical movements by the motion-impaired users.
When combining the observed times into a known interaction process, such as pressing a key in response to a simple stimulus, all of the able-bodied users produced response times in accordance with the predicted results. However, the motion-impaired users did not. This at first appeared to show that the model was not working. However, a careful study of the users during the interaction process and of the times obtained showed that extra cognitive cycles were being inserted.
The most likely explanation for the different interaction patterns between able-bodied and motion-impaired users is the extra cognitive effort required to control physical motions in the presence of an impairment. The extra effort not only manifests itself as slightly slower cognitive cycle times, but also additional cognitive cycles in actions that theory predicts should be automatic. The purpose of those extra cycles is unclear at the moment. It is probable that they arise either from the users’ desire to be certain about each movement or from the user being perpetually in a learning mode when interacting with a computer.
Consequently, the implications for those relying on models of interaction for designing interfaces or usability tests, is not to rely on the accepted able-bodied models and ‘add a bit’, but to actually measure the differences in the interaction styles between users with different capabilities. This is the only way of ensuring that false assumptions about behaviour are not made. It is also clear that it is necessary to support the users wherever possible to minimise the need to insert the extra cognitive cycles into the interaction. This can often be achieved by offering additional supportive feedback to the users, through positive reinforcement of actions.
Further research is being performed to quantify the nature of the differences between able-bodied and motion-impaired users and to quantify the cognitive overhead associated with different input movement types.