1. Experts cannot assess terminology choice and evaluate task compatibility. Terminology and task compatibility account for the majority of usability problems. To my knowledge, the only effective approach to evaluate terminology and task compatibility is through usability testing and field study.
2. The importance of heuristic (guidelines) depends of the context of use. For example, is the user going their once in a lifetime? Or is he a power user going there all the time on a mandatory basis.

As I always do, I complied telling the client I will do heuristic evaluation with the help of informal usability testing instead of pure heuristic evaluation. That way, I don’t loose time explaining clients why they are wrong. I asked a few colleagues or friends to act as subjects for the informal usability testing who could be representative of the audience.

Following the evaluation, I found the design of the portal was poor but I was not surprised, I have often seen bad design throughout my consulting practice. As usual, most usability problems were associated with confusing wording, task compatibility issues and funky use of interaction techniques. Up to now nothing unusual. I went on presenting my findings. But, while presenting, the client constantly interrupted me saying this or that; we cannot change it because it is a decision from the upper management. In some cases the minister himself got involved in the design choices.

Only later, after the formal usability testing where all users failed on all tasks, I got the client to admit the design was poor. At that point they had no choice but accept my recommendations. This situation is painful because it places the management in a bad position. They were involved in poor design decisions.

More recently, a client in the airline industry told me he does not want to hear about usability because everybody has a different opinion. They often end up asking the CEO to decide. Again, I will have a very challenging consulting project because it might show the CEO made the wrong decision. Of course, I will try to find ways around it but nevertheless, it is not easy.

For many organizations, design (user interface design, user requirement, and functional specifications) is a painstaking process involving fierce battle between stakeholders. When they do not agree, they escalate to upper management for decision.

Escalating design decision that otherwise should be taken by the design team is a sure recipe for politics. If decisions are wrong, it will be uncovered sooner or later. Then, you will be in serious trouble.

How to prevent this?
If you’re an executive, never get yourself involved in detailed design aspect. Keep yourself at the goal level and let your team design solutions. If your team tries to get you involved in design deadlock, it is because they have not done their homework or they lack expertise. Your project manager might say there is a deadlock; he needs your help to move forward. He will try to reassure you by saying he will fix the usability issues later. Don’t believe that, it will never happen and you will get stuck with a poor design for years. Stress it to your team you don’t care if the screens are blue, white or yellow as long as users were able to achieve their goals rapidly, without hesitation and errors. Get an external usability company to evaluate the design of your internal team so you will get an objective assessment of the design. Don’t worry about the susceptibility of your people, there is an easy way to manage that. If you don’t show leadership, you risk to be stuck with poor design for years if not forever.

Never forget, usability equals productivity. The first reason why we use technology is to improve productivity, so good design is extremely important.

If you are member of a project team, base your design decision on facts instead of opinion. How? Learn usability best practices, build a prototype and test your design with users with usability testing. If you do not have this expertise, don’t hesitate, get and external consultant in usability.

This graph shows the evolution of George W. Bush’s approval rating over time. His approval rating rose to 68% in March 2003, at the beginning of the Iraq war, and declined steadily after that. It is now at 26%. Not only did the war not proceed the way he had told people but there is a growing number of Americans questioning the idea of going to Iraq in the first place.

How to lead?
First: be right
No matter how good you are at executing and communicating, and how much charisma and resolution you have, if you are wrong, you will fail. Churchill was resolved but he was right. In the hiking story above, the leader was sure of himself but wrong. He overestimated how lost we were.
To be right, you need facts.
In real life, we do not know with certainty the actual situation. In the context of uncertainty (most real life situations), the more certain you are the more likely you will be wrong. Facts are even more important in uncertain situations. Why? What you know and what don’t know are facts. ”I am lost in the forest” is a fact. If the leader in the hiking story would have admitted the true level of uncertainty about how lost we were, he could have proposed a series of tests. For example, he could have propose a short walk in a different direction to gather more facts about the current location. A negative result would then have been seen as new information on where not to go instead.
Second: Communicate facts (reason for change)
It is very difficult to convince people to change something if they are not convinced about what the problems are and their urgency, see John Kotter. Two economists might disagree even if they use the same economic model. By separating fact from opinion, we might understand the input or assumption and understand why those economists disagree. Whenever you make a decision where people must follow your lead, facts are the essence of getting people agreeing on the current situation and the course of action.
In business transformation (change Management), facts are more than data “Not everything that counts can be counted, and not everything that can be counted counts.” (Sign hanging in Albert Einstein’s office at Princeton), facts are like an observable situation that can be described and shared.

Once facts are gathered and separated from opinion, they can be shared during working sessions. Those sessions permit the group to agree on problems. Consequently, it will be much easier to get the group to agree on a course of action once they have agreed on the problems. During those working sessions, the focus is first on the current situation, not the course of action. Ensure everybody that they are respected and never discard people expressing bad news (don’t shoot the messenger). Often they might be your best source of facts. To ensure the free flow of information, remove red tape, formal presentation and favor a more informal organization with a high degree of respect, see ‘‘The Death to Bureaucracy” from Jack Welch.
Third: gather more facts
Often, you need to collect more facts. Creativity and imagination are required to gather facts. For example, in the above story, a series of short walks in different directions is an approach to gather more facts on the current location. Gathering facts is boiled down to: observation, experiment, calculation and simulation. Fact gathering is not a consensus driven process. You are collecting evidence. Never rely on opinion. Humans are biased and opinions rarely correlate with facts.
In business transformation initiative, the best sources of fact are field studies. The field is rich and wild; people develop very sophisticated tactics in real work environment. Knowledge lies in the field, not in meeting rooms. The worst source of facts is the traditional hierarchy : line manager, director and executive. Facts gathered through this path will surely be altered, trimmed down or hidden.
Fourth: prevent bias
Facts may be distorted. When relying on human sources, multiple independent sources are required. Be careful with surveys or focus groups, they do not gather facts but opinions. They provide the tip of an iceberg. Avoid group meetings. Groups are only good to solve problems, to brainstorm or share information.
Objectivity does not exist; you have to cope with that.
No matter how good the information is, if you are biased, or already convinced of the answer, you are in danger. A bias could, for example, lead one to accept or deny the truth of a claim, discard evidence or look for evidence that confirms a preconceived idea (confirmation bias). In the hiking story, the guy seemed so sure of himself that we trusted him even if the direction he gave did not ring a bell to us.
To reduce bias, observe yourself to see if you feel emotional. You need to detach yourself. It is well known that a lawyer should never defend himself or a medical doctor should never diagnose himself. Ask different persons in a one on one session to interpret facts. Don’t give them your thoughts. Observe the difference in their interpretation. Try to think as fresh as you can. Gather all facts prior to a decision. In order to avoid bias, good investigators will collect evidence and then conclude. After all this work, let the decision emerge naturally. If you still do not feel not comfortable, your instinct will tells you there is something wrong, listen to it. It means there are still uncertainties that you have to work with, or simply wait.
The four steps described here relate to making the right decision and communicating that decision (vision) to a group of people. Once you have the right direction and a group of people agreeing with you on the course of action, executing change is a piece of cake.
Leading people in the right direction requires hard work, time, experience and wisdom. You can get there faster by practicing the right thing. For more on the subject, I suggest reading John Kotter on this subject and “Engineering psychology” by Christopher Wickens.

The levels at which impacts of change are major are not understood while the levels at which impacts are minor are easily understood.

Why?

Most managers believe people resist change because humans have a natural desire to keep the status quo. They believe that resistance to change is a defense mechanism caused by frustration and anxiety. With this diagnostic, executives belief that good change leadership comes with a strong message from the top, effective communication plan, mandatory training and sometimes, removing undesirable parties that are labeled as strong change resistant.
In fact, Individuals aren’t really resisting the change, but rather resist the loss of status, loss of pay, or fear they will not be able to achieve the same level of performance. While causes of losses of pay or statuses are visible, causes impacting performance involve factors at the unconscious level that are difficult to communicate.
We observe that in many business transformation initiatives those fears were justified. New processes would not permit anymore efficient tactics evolved over years of trial and errors.
How can we distinguish between fear and facts?
First, let’s look at the nature of change in human work, the impact of business transformation initiatives and to finish, what to do about it.

Why change is difficult in work setting?

Change happens on an individual and organizational basis. Change is everywhere: mergers, new policies, new technology, new management . . .
Employees resist change because they have to learn something new. They fear the unknown future and their ability to perform.
The unknown comes from subtle and nonverbal aspects of human work that are difficult to communicate.
The nature of work
Jens Rasmussen observed that human work is either at: the skill level, rule-based level or knowledge level. Don Norman also came to the same conclusion.
When driving a car, turning the wheels is at the skill level while letting the other car passes at the intersection is at the rule-based level. Deciding to choose a route over another one is the knowledge level.
In addition, we observe that there is a fourth level, usually called “meta-cognition” but I prefer to use “wisdom”, which involves self-awareness about our own limitation. For example, if tired or drunk, hopefully, we might decide not to drive.
Any human activity: tasks, sport, arts or leisure lies within those levels.

The impacts of change

Changing something at the knowledge level is relatively easy. If someone shows you a better route to go somewhere, you only need to understand the advantage and good indications and voila, you can execute the change.
At the rule-based level, it is a bit more difficult but manageable. For example, if the state decides to change the law and permit right turn on red light, an action previously forbidden, at first you need to pay a little bit more attention to apply the new rule. You may even forget to apply it but with little time and patience, turning on the red light will become natural.
The real challenge is at the skill level. Let’s say the car manufacturer decides to invert the position of the break pedal and gas pedal. It will take you months of practice and you might make many errors that might event kill you. Change will be very difficult to accept and you might express a high degree of frustration. You will certainly not use this car if you have the choice.
Those examples show that: efforts to execute a change at the skill level are much higher than at the rule-based or knowledge based level. The level of energy required is dependent of the level of work affected by change: being at the skill, rule-based or knowledge based level.
Convincing a golfer to change his swing is a much more difficult challenge than convincing him to change club number to reach the green. It requires much more energy to acquire the new swing.
The proposed law of change:

“The effort to execute change will be equal to the amount of energy required to execute the same level of performance.”
“The amount of energy required to change is the highest at the lowest level of wok affected”. Skill based being the lowest level while knowledge based being the highest level.

Business Transformation

In business transformation initiatives, people only understand impact of change at the knowledge and rule-based level but not the ones at the skill level. Skill level is mostly non-verbal and unconscious.
For example, during a word processing task, users might select backspace to delete few letters or they might select the mouse to delete a paragraph. The choice is usually unconscious. This is a good example of a natural way humans optimize energy. According to the keystroke model, it takes 1.2 seconds to delete 6 letters with backspace and it takes 2.6 seconds with the mouse.
Now, let’s say a vendor convinces management to adopt a new word processor. While nicer and with more features, it doesn’t permit using backspace anymore. It is likely that this subtle change would be unnoticed even during a demonstration. Only after implementation employees will complain and express their frustration. As good performers, employees will be frustrated because they cannot achieve the same level of performance. They might even express anger.
This is only the tip of the iceberg. In fact, in a real work setting there is a myriad of such optimizations tactics: being a particular noise, a visual clue, a physical arrangement, a specific sequence…

What to do about it

There is a science, Cognitive ergonomics (called Human Factors in USA), which permits the understanding of human work at all levels including the skill level. Cognitive task analysis permits gathering both verbal and subtle non-verbal operations such as eye movement, cognitive processes and unconscious manual movements. Understating work at all levels permits the understanding of the impact of change at all level of work.

Managers can stop managing change with brute force.

First: stop doing endless meetings and politics and go in the field. Wake-up and forget the meeting rooms, the real world is in four dimensions.
Second: do a field study and cognitive task analysis to fully understand the current situation. Field is rich and wild; people develop very sophisticated tactics.
Third: select a solution or design a new system that will permit skill transfer, keep efficient tactics and strategies and remove problems.
Fourth: before any implementation and at the earliest stage possible, design mock-ups, simulate and test them in the most realistic manner as possible. This is even truer with physical devices because there are so many subtle interactions with the environment.

Easier said than done but at least try to change the change management practice if you want to bring change in your organizations.