User Experience and Cognitive Engineering

Last year, the government of Quebec asked Cognitive Group to do a heuristic evaluation (expert evaluating usability with a list of usability heuristics) and usability testing of their web portal. First, heuristic evaluation is not reliable because:

1. Experts cannot assess terminology choice and evaluate task compatibility. Terminology and task compatibility account for the majority of usability problems. To my knowledge, the only effective approach to evaluate terminology and task compatibility is through usability testing and field study.
2. The importance of heuristic (guidelines) depends of the context of use. For example, is the user going their once in a lifetime? Or is he a power user going there all the time on a mandatory basis.

As I always do, I complied telling the client I will do heuristic evaluation with the help of informal usability testing instead of pure heuristic evaluation. That way, I don’t loose time explaining clients why they are wrong. I asked a few colleagues or friends to act as subjects for the informal usability testing who could be representative of the audience.

Following the evaluation, I found the design of the portal was poor but I was not surprised, I have often seen bad design throughout my consulting practice. As usual, most usability problems were associated with confusing wording, task compatibility issues and funky use of interaction techniques. Up to now nothing unusual. I went on presenting my findings. But, while presenting, the client constantly interrupted me saying this or that; we cannot change it because it is a decision from the upper management. In some cases the minister himself got involved in the design choices.

Only later, after the formal usability testing where all users failed on all tasks, I got the client to admit the design was poor. At that point they had no choice but accept my recommendations. This situation is painful because it places the management in a bad position. They were involved in poor design decisions.

More recently, a client in the airline industry told me he does not want to hear about usability because everybody has a different opinion. They often end up asking the CEO to decide. Again, I will have a very challenging consulting project because it might show the CEO made the wrong decision. Of course, I will try to find ways around it but nevertheless, it is not easy.

For many organizations, design (user interface design, user requirement, and functional specifications) is a painstaking process involving fierce battle between stakeholders. When they do not agree, they escalate to upper management for decision.

Escalating design decision that otherwise should be taken by the design team is a sure recipe for politics. If decisions are wrong, it will be uncovered sooner or later. Then, you will be in serious trouble.

How to prevent this?
If you’re an executive, never get yourself involved in detailed design aspect. Keep yourself at the goal level and let your team design solutions. If your team tries to get you involved in design deadlock, it is because they have not done their homework or they lack expertise. Your project manager might say there is a deadlock; he needs your help to move forward. He will try to reassure you by saying he will fix the usability issues later. Don’t believe that, it will never happen and you will get stuck with a poor design for years. Stress it to your team you don’t care if the screens are blue, white or yellow as long as users were able to achieve their goals rapidly, without hesitation and errors. Get an external usability company to evaluate the design of your internal team so you will get an objective assessment of the design. Don’t worry about the susceptibility of your people, there is an easy way to manage that. If you don’t show leadership, you risk to be stuck with poor design for years if not forever.

Never forget, usability equals productivity. The first reason why we use technology is to improve productivity, so good design is extremely important.

If you are member of a project team, base your design decision on facts instead of opinion. How? Learn usability best practices, build a prototype and test your design with users with usability testing. If you do not have this expertise, don’t hesitate, get and external consultant in usability.