Since tree testing allows you to easily collect data from a large group of users, aim for at least 50 users
Tasks per participant. Ensure that each participant performs only 10 tasks (or fewer).
Once someone has clicked through the same menu 15 times, they are in quite a different state of mind than an average user who has just landed at a website and may have never seen the menu before at all.
Tree-Testing Metrics
Success rate: The percentage of users who found the right category for that task
Directness: The percentage of users who went to the right category immediately, without backtracking or trying any other categories
Time spent: The average amount of time elapsed from the beginning to the end of the task
Path measures:
- Selection frequencies for each category
- First click: the category most people selected first
- Destination: the category most people designated as their final answer
Success Rate
In order to calculate the success rate, you must assign at least one ‘correct’ answer for each task. The success rate for that task indicates the percentage of users who found the correct location in the tree and identified it as the right place to complete that task.
Because tree tests are so basic, success rates are often much lower than in regular quantitative usability studies.
Instead of expecting to achieve a 100% success rate, use a more realistic frame of reference to evaluate what success rate is acceptable for each task, taking into account:
- The importance of that task to the overall user experience
- How each success rate compares to other similar tasks (e.g., tasks which target content at the same level in the hierarchy)
Rather than comparing these two success rates, it would be more realistic to compare either:
The success rate for the food-stamps task to that of another task which also targets content that is 6 levels down; or
The success rate of the food stamps task performed on two different trees with different labels — one which uses the term Food Assistance and one with the term Food Stamps.
+ optimal workshop
Each task also receives an overall score out of 10 (with an 8 or above score considered well-performing).
Directness and Time Spent
In addition to measuring how many users got to the right place, it’s important to also consider how much they struggled on the way. Two common tree-testing metrics signal this: time spent, which indicates how long it took users to find the right answer, and directness, which captures how many users went immediately to the right answer, without backtracking or changing categories. Direct navigation is also sometimes called the ‘happy path’ because it suggests smooth interaction, with minimal confusion or detours.
Pathways: First Clicks to Final Destinations
Success rate and directness tell you whether a category is findable; detailed pathway analysis helps you figure out how improve categories that don’t work well.
The first click is critical because it often predicts whether a user will eventually be successful in finding the right item.
Examine the first click data carefully when:
1.A task has low success rate and/or directness. The first clicks indicate where users initially expected to find that information, and suggest locations where the item should be moved (or at least crosslisted).
overlapping
If you have many tasks where first clicks are distributed across multiple categories, you may have too many overlapping categories. Do a card sort, or review the tree-test results again and look for other possible organization schemes.
Review the final destinations selected by users when the first clicks are correct, but the success rates are low. This pattern suggests that lower-level categories overlap too much.
Turning Data into Action
Although tree testing yields quantitative data, the conclusions are by no means black and white. Task success rates are just the first step, and must be interpreted within the context of how much users struggled to get to the right answer (directness), and where they expected the right answer to be (first clicks).
Once this analysis is complete you can identify appropriate solutions. For example:
- When first clicks are evenly distributed in multiple areas, list topics in multiple categories. If this issue occurs for many tasks, consider changing the overall organization scheme.
- When success rate is low but first clicks are correct, change the labels of subcategories to be more distinct.
ChatGPT
The wording of the task: The language or phrasing of the task may have been confusing or unclear, making it difficult for users to understand what was required of them.
The location of the task in the tree: If the task is located in a less intuitive or less expected place within the tree structure, users may have difficulty finding it.
The complexity of the task: If the task involves more steps or requires a higher level of understanding than other tasks, users may struggle to complete it successfully.
User demographics: The demographic characteristics of the users may affect their ability to complete certain tasks. For example, if a task requires familiarity with technical language or concepts, it may be more difficult for users who are less technically literate. - because it is clinical area?
- more appealing or familiar to the users.
- Another reason could be that the description or content under those labels is more compelling and informative, making users believe that it is the correct path.
- It is also possible that the correct path, 'Building Healthcare Software', is not as well-defined or easily understandable to the users.
레퍼런스
https://www.nngroup.com/articles/interpreting-tree-test-results/
Tree Testing Part 2: Interpreting the Results
Analyze tree-testing results including success, first click, and directness to improve information architecture and navigation labels.
www.nngroup.com
댓글