Alden DeSoto
2y ago
Bringing products and ideas to life with compelling content
Metrics like Pageviews, Time on Page, etc aren't useful for measuring content effectiveness. Try this instead to get better signal.
Pageviews, Time on Page, Bounce Rate, Satisfaction Rate (% of users who answered Yes to "Was this article useful?") and similar metrics aren't helpful.
They measure engagement with the documentation, not the product.
Let's say you have a high Time on Page for an article. Is that good? Maybe. Maybe not. And what if Time on Page is low? Do you know what to fix? Probably not.
If an article has a low Satisfaction Rate, does that tell you what to fix? It does not.
Try this instead:
1. Collect all the comments left by the users who were not satisfied. Load all the comments into a spreadsheet.
2. Categorize the comments into themes. (Anywhere from 3 - 10 themes, depending upon how many comments you have. Don't get too granular.)
Example: If a lot of comments complain that the information is too complicated, create a "too complicated" theme. Group all those comments together.
Example: If a lot of comments complain that they couldn't complete Procedure XYZ, create a "Couldn't complete Procedure XYZ" theme.
Tip: Use AI to scan your comments and identify the top themes.
3. Count the number of comments in each theme. Calculate the percentage of comments that fall into each theme. Often, you'll have a very large percentage of comments in one or two themes.
You now have data that communicates the nuances of user challenges. For example, "X% of negative comments complain that they can't complete XYZ and Y% of negative comments don't understand the terminology."
The product team will love you for sharing this data. More importantly, it gives you actionable information.
4. Now update your docs to address your top problem themes.
5. In two weeks, come back and repeat steps 1, 2, and 3. If you've done a good job addressing the problem areas, you'll find that your top themes HAVE CHANGED. In other words, users are still complaining, but now they're complaining about something else!
For example, "In September, X% of comments complained that they couldn't complete procedure XYZ, but in October, negative comments shifted to A, B, and C ... etc"
You can now see and communicate progress to your Product and UX teams, which they will appreciate.
Don't be satisfied with vanity metrics. Dig deeper and find the data that helps you iterate.