I keep hearing "testing with 5 users unveils 85% of usability problems". Who defines 100% ?! Is there an endgame? Seriously curious. #ux
And I know the original source is Nielsen, but there is very little info on what a usability test is; magnitude, time or number of features.
In this article: nngroup.com/articles/why-y…
The premise at least seems to be that all things remain equal between studies... which in my experience is rarely the case in real life.
There seems to be a lot of assumption about when a usability test is carried out (w/ regards to website lifespan) and how it is carried out.
I understand how the math makes sense in a lab environment ceteris paribus, but UX is affected by many factors not even addressed in tests.
My worry here is that this "rule" tends to undercut what UX brings to the table.
Companies will go: "let's test with 5 users and fix 85% of the problems". And be done with it and not test again in a year.
"Fixing" will of course create new problems as will changes in environment, other touchpoints and competing activities.
Making UX a numbers game diminishes a need to always be chasing design faults. They are never 100% fixed because the world is never static.
Yes, I get what Nielsen means but the approach needs to be much more clarified and wrapped in disclaimers.
Our focus has to be goal fulfillment, not measuring how many usability problems are fixed.
There, I think I'm done now... 😂
If you want to read the original article and study: dl.acm.org/citation.cfm?i…
Working on a revised wording...
