Interaksi Manusia dan Komputer : Teknik-Teknik Evaluasi (1)
Overview
Evaluation tests the usability, functionality and acceptability of an interactive system. Evaluation may take place: in the laboratory or in the field. Some approaches are based on expert evaluation:
- analytic methods
- review methods
- model-based methods.
Some approaches involve users:
- experimental methods
- observational methods
- query methods.
What is Evaluation?
We have discussed a design process to support the design of usable interactive systems. However, even if such a process is used, we still need to assess our designs and test our systems to ensure that they actually behave as we expect and meet user requirements. This is the role of evaluation. Evaluation should not be thought of as a single phase in the design process (still less as an activity tacked on the end of the process if time permits). Ideally, evaluation should occur throughout the design life cycle, with the results of the evaluation feeding back into modifications to the design.
The Goals of Evaluation
Evaluation has three main goals:
- to assess the extent and accessibility of the system’s functionality
- to assess users’ experience of the interaction,
- to identify any specific problems with the system.
The system’s functionality is important in that it must accord with the user’s requirements. In other words, the design of the system should enable users to perform their intended tasks more easily. This includes not only making the appropriate functionality available within the system, but making it clearly reachable by the user in terms of the actions that the user needs to take to perform the task. It also involves matching the use of the system to the user’s expectations of the task. For example, if a filing clerk is used to retrieving a customer’s file by the postal address, the same capability (at least) should be provided in the computerized file system. Evaluation at this level may also include measuring the user’s performance with the system, to assess the effectiveness of the system in supporting the task.
It is important to assess the user’s experience of the interaction and its impact upon him. This includes considering aspects such as how easy the system is to learn, its usability and the user’s satisfaction with it. It may also include his enjoyment and emotional response, particularly in the case of systems that are aimed at leisure or entertainment. It is important to identify areas of the design that overload the user in some way, perhaps by requiring an excessive amount of information to be remembered, for example.
The final goal of evaluation is to identify specific problems with the design. These may be aspects of the design which, when used in their intended context, cause unexpected results, or confusion amongst users. This is, of course, related to both the functionality and usability of the design (depending on the cause of the problem). However, it is specifically concerned with identifying trouble-spots which can then be rectified.
Evaluation through Expert analysis
Carry out user testing during the design proces:
- Expensive
- Difficult to get accurate assessment (incomplete design)
A number of methods have been proposed to evaluate interactive systems through expert analysis. The basic intention is to identify any areas that are likely to cause difficulties because they violate known cognitive principles, or ignore accepted empirical results. However, they do not assess actual use of the system, only whether or not a system upholds accepted usability principles. Four approaches to expert analysis:
- cognitive walkthrough,
- heuristic evaluation,
- the use of models
- use of previous work.
Cognitive Walkthrough
Cognitive walkthrough was originally proposed and later revised by Polson and colleagues as an attempt to introduce psychological theory into the informal and subjective walkthrough technique. The origin of the cognitive walkthrough approach to evaluation is the code walkthrough familiar in software engineering. Usually, the main focus of the cognitive walkthrough is to establish how easy a system is to learn.
Experience shows that many users prefer to learn how to use a system by exploring its functionality hands on, and not after sufficient training or examination of a user’s manual. So the checks that are made during the walkthrough ask questions that address this exploratory learning.
To do a walkthrough, you need four things:
- A specification or prototype of the system. It doesn’t have to be complete, but it should be fairly detailed. Details such as the location and wording for a menu can make a big difference.
- A description of the task the user is to perform on the system. This should be a representative task that most users will want to do.
- A complete, written list of the actions needed to complete the task with the proposed system.
- An indication of who the users are and what kind of experience and knowledge the evaluators can assume about them.
Given this information, the evaluators step through the action sequence to critique the system and tell a believable story about its usability. To do this, for each action, the evaluators try to answer the following four questions for each step in the action sequence.
- Is the effect of the action the same as the user’s goal at that point?
- Will users see that the action is available?
- Once users have found the correct action, will they know it is the one they need?
- After the action is taken, will users understand the feedback they get?
Cognitive Walkthrough Example
UA 1: Press the ‘timed record’ button
Question 1: Is the effect of the action the same as the user’s goal at that point?
The timed record button initiates timer programming. It is reasonable to assume that a user familiar with VCRs would be trying to do this as his first goal.
Question 2: Will users see that the action is available?
The ‘timed record’ button is visible on the remote control.
Question 3: Once users have found the correct action, will they know it is the one they need?
It is not clear which button is the ‘timed record’ button. The icon of a clock (fourth button down on the right) is a possible candidate but this could be interpreted as a button to change the time. Other possible candidates might be the fourth button down on the left or the filled circle (associated with record). In fact, the icon of the clock is the correct choice but it is quite possible that the user would fail at this point. This identifies a potential usability problem.
Question 4: After the action is taken, will users understand the feedback they get?
Once the action is taken the display changes to the timed record mode and shows familiar headings (start, end, channel, date). It is reasonable to assume that the user would recognize these as indicating successful completion of the first action
Heuristic Evaluation
Nielsen’s ten heuristics are:
- Visibility of system status Always keep users informed about what is going on, through appropriate feedback within reasonable time. For example, if a system operation will take some time, give an indication of how long and how much is complete.
- Match between system and the real world The system should speak the user’s language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in natural and logical order.
- User control and freedom Users often choose system functions by mistake and need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialog. Support undo and redo.
- Consistency and standards Users should not have to wonder whether words, situations or actions mean the same thing in different contexts. Follow platform conventions and accepted standards.
- Error prevention Make it difficult to make errors. Even better than good error messages is a careful design that prevents a problem from occurring in the first place.
- Recognition rather than recall Make objects, actions and options visible. The user should not have to remember information from one part of the dialog to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
- Flexibility and efficiency of use Allow users to tailor frequent actions. Accelerators – unseen by the novice user – may often speed up the interaction for the expert user to such an extent that the system can cater to both inexperienced and experienced users.
- Aesthetic and minimalist design Dialogs should not contain information that is irrelevant or rarely needed. Every extra unit of information in a dialog competes with the relevant units of information and diminishes their relative visibility.
- Help users recognize, diagnose and recover from errors Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
- Help and documentation Few systems can be used with no instructions so it may be necessary to provide help and documentation. Any such information should be easy to search, focussed on the user’s task, list concrete steps to be carried out, and not be too large.
Each evaluator assesses the system and notes violations of any of these heuristics that would indicate a potential usability problem. The evaluator also assesses the severity of each usability problem, based on four factors: how common is the problem, how easy is it for the user to overcome, will it be a one-off problem or a persistent one, and how seriously will the problem be perceived? These can be combined into an overall severity rating on a scale of 0–4:
- 0 = I don’t agree that this is a usability problem at all
- 1 = Cosmetic problem only: need not be fixed unless extra time is available on project
- 2 = Minor usability problem: fixing this should be given low priority
- 3 = Major usability problem: important to fix, so should be given high priority
- 4 = Usability catastrophe: imperative to fix this before product can be released (Nielsen)
Sumber
HCI Slides: Evaluation Technique 1
Posting Komentar