Usability testing seems to be the new black in Libre Software Land these days. And while I won’t discount its importance one bit, I am a bit frustrated reading this recent post titled “When users first encounter Ubuntu: six showstoppers” over on the Canonical Design blog.
Why am I frustrated? There is no real information about the testing itself (perhaps it’s published somewhere else?). Who are the ‘users’? How many were part of the test? Are they part of the Ubuntu’s target audience? (Who is that anyway?). What age are the people? What are their backgrounds like? Where are they from? Were they female or male? There are a million questions (and perhaps even some answers) that I think should go along with any report about usability testing. The post gives some typical remarks and a summary of what are apparently major problems, but without context these seem just like the myriad of assumptions we make about “users” all the time anyway. These results would be much more meaningful with context and focus. And like I said, all that info may be recorded and published somewhere else. Why not provide it?
Now a few comments (not necessarily answers) about some of the “showstoppers”:
1. File compatibility
I would think this is largely an OpenOffice concern, at least in the context of Office documents. I’m not sure if this will ever be solved without moving to an open format anyway. At work I have dealt with several people who can’t open docx files in their version of Office. File compatibility is a concern even for MS-Office users on Windows.
2. Lack of feedback on system behaviour
This is one of those areas that we could do a lot better than our non-free OS competitors. There is nothing holding us back from providing something that solves the problem creatively and uniquely.
3. Use of jargon
This problem is rampant all over FOSS-land. Thoughtful consideration of every menu and dialog is required, at application and OS level. Again, this would be so much easier with a defined audience.
4. Getting flash
Rather than addressing the problem, I have to take issue with the summary of this point. “Most didn’t know what to do at that point.” Most of who? How many tried? Are we talking about 3 people or 30? And if most didn’t, does that mean 2 or 16, or 29? This is where context would be so helpful. And maybe the more pressing question is: If ‘most’ people didn’t know what to do, then at least ‘some’ actually did. And if they did, why is it that none of them were able to download it? I don’t get it.
5. Software centre
I think a good chunk of this point relates directly back to 2. and 3.
6. Adding a printer
I don’t remember encountering any of this. I opened the printing dialog from the Administration menu, clicked ‘New Printer’ and chose my model and driver. While not perfect, I don’t remember having to list anything about device URI’s etc. Were these people installing an unsupported printer model? Maybe a network printer? There is not enough in the description to tell. It sounds like the printing dialog is horribly broken for everyone and I just don’t think that is the case (again – publish the data and let’s see).
Well, I guess it’s good that there is some usability testing. But how good is the question. And without more information about the testing itself there is no answer to that.