User Experience, how to measure the un-measurable…
Tuesday, July 27th, 2010User Experience (UX) is one of those tricky terms we see banded around the digital universe, but what’s it all about I hear you bellow with underwhelming enthusiasm? The clear cut answer is simply that its not clear cut! When the Web was in its infancy the term HCI was used extensively around software development circles (boy threes a group of people to join on a night out!) HCI originally translated as ‘Human Computer Interaction’ but this was soon replaced with ‘Human Computer Interfaces’ largely driven by the need to study web interfaces in order to maximise conversions and ensure return on investment. However UX has now become the industry standard expression to encompass a number of former disciplines including:
HCI
Interface Design
Information Architecture
Interaction Design
Usability
Human Factors Engineering
The key similarity of each of the above is that they are fundamentally qualitative in their measurement. Yes it could be argued that methods such as web analytics provide hard statistical quantitative evidence for decision making, but all this really tells you is the ‘how’ not ‘why’ e.g. ‘how many web visitors originate from London’ to work out the why you need to further investigate your UX, which normally results in some form of subjective analysis.
So that we can highlight the problem, lets have a quick Wikipedia provided description of each of the principals.
Interaction design is a sub-discipline of design which examines the role of embedded behaviors and intelligence in physical and virtual space.
Information Architecture (IA) is the art and science of structuring knowledge (technically data), and defining user interactions.
Usability is the measure of the ease with which particular people can employ a particular tool or other human-made object in order to achieve a particular goal.
Human-computer interaction (HCI) is the study of interaction between people (users) and computers.
Human factors engineering, also referred to as Ergonomics is the study of optimizing the interface between human beings, and the designed objects and environments they interact with.
User interface design is the overall process of designing the interaction between a human (user) and a machine (computer). It includes graphic design, information design and a wide variety of usability methods.
Notice anything? Well I pick out these three words to critique the above: crossover, ambiguity and well, ‘soft’ (meaning study by opinion with limited hard measurable techniques) Don’t get me wrong I love UX as a concept and I love designing to maximise user journeys, but can we truly gain valuable insight via focus groups, workshops and surveys?
The answer techno-fans is yes… I have been involved in various analytical studies where ‘soft’ techniques were the only option for measurement and I definitely believe that subjective user opinion can refine and transform digital experiences. However it can go wrong, very wrong. I have seen cases where due to poor planning, workshop data and survey data has been ‘tweaked’ by analysts to bias a certain decisions or replace non-understanding from test subjects.
Now, not saying that I am lord of UX and designing measurement techniques (I am actually lord of dragons and other fire breathing sky dwellers) but here are some guidelines to ensure that your subjective measurement is fair, accurate and meaningful:
1. Ensure that your test groups are an accurate demographic of real users
2. Ensure that tasks / questions are clear and concise, all testers must understand everything
3. Organise everything to account for unexpected questions / strain on your system
4. Rehearse workshops with colleagues to allow for problems
5. Reward attendees with the lure of bourbon biscuits
6. Ensure that the test environment is accurate to reality
7. Plan an efficient data collection strategy
8. Plan an efficient analysis strategy (don’t just bung everything into excel and see what happens)
So there we have it, common sense points? Absolutely, but I have seen numerous occasions where tiny problems can have a large impact on results, if you are accurate and organised real insight can be measured which can dramatically increase website goals and ultimately that magic word that we all love: conversions..
Ta for reading..









