Why You Shouldn't Trust App Store Reviews (and What to Trust Instead)
App reviews at Google Play and the iTunes App Store are pretty broken. They can be helpful, but you have to sift through the tech support rants, device-specific complaints, and half-legible reviews first to find something useful. There are better ways to tell if an app is any good.
Why App Store Scores Suck
Too often people toss review scores out as the only measure of an app. In a perfect world, an app's score would be indicative of the objective quality of an app judged by people who have actually used the app in question. Unfortunately, it doesn't always work that way. "Well it's got terrible reviews at Google Play," isn't a fair critique of an app, especially compared to a real, hands-on review.
The problem with app reviews isn't just trolls one-starring apps, though. It goes a bit deeper than that:
- App store ratings suffer from self-selection bias. This isn't a problem unique to app stores, but everywhere online: The people who leave reviews are usually the most vocal (often the ones complaining) or people who have a positive incentive to do so (who traditionally give high marks.) In either case, their experiences don't offer an accurate picture of what most people-the people who don't like or hate an app enough to leave a review-experience when they use it.
- Most reviews are poorly written, use criteria that's not useful to anyone, or are highly personal or device-specific. It doesn't take much to find reviews that are barely legible, and if you've ever seen a review that said "one star until it works on Xperia Z," or "four stars until it gets pull-to-refresh," you know how a useless review can impact an overall score. This also underscores the fact that a lot of reviews are so author-specific, either praising or complaining about specific features that you may or may not care about, that they're not useful.
- PR companies and developers flood app stores with positive reviews before even launching their app. This is a huge issues with reviews, especially for new apps. Sometimes those upbeat, five-star reviews come later, but other times they're posted in the time between the app goes up in the app store and its official "launch." In that in-between time, which can be hours or days, a few five-star reviews go a long way towards making the app look good on launch day, assuming they don't get caught doing it.
- Competing developers urge users and friends to one-star competing apps to keep their reviews down. This doesn't happen too often, but it has happened-asking "friends and family" of one app to sabotage the ratings of another competing app isn't unheard of. At the very least, developers will have their own teams try to influence a competing app's ratings. At worst, they'll just try to bump up their own. The inverse is true too-often app developers nag users to leave good reviews if they continue using the app. So much so that some people think it's annoying.
- Changes to the app store itself can kill review scores. Not too long ago, Google introduced a new recommendation feature that suggests apps based on apps you have installed. It also lets you star that suggestion-making you think you're rating the suggestion and saying whether the app interests you. In fact, you're rating the app itself, which-as many developers have seen, led to crashing ratings. All because of a poorly-understood and poorly-implemented feature at Google Play. There's a great discussion at Hacker News about this, and how it damages the credibility of app store review scores.
So we've established that app scores and reviews can be pretty useless if you don't take the time to separate the wheat from the chaff. If it's not an issue with the app store itself, the reviews themselves are at best highly specific and useful only to certain people or circumstances, and at worst spam or promotional campaigns designed to influence your opinion. Of course, you can fight the tide by sifting through individual reviews to find the really good ones.
Alternatively, you can push back against the trend by learning to write genuinely useful reviews yourself
What To Read Instead
If app store reviews aren't terribly useful, what can you trust? You do have other options if you want to learn more about an app before downloading it. Here are a few useful options:
- Read trustworthy, curated third-party reviews. Regardless of the type of phone you have, there are plenty of blogs that specialize in app reviews and ratings. Whether it's here on your favorite productivity blog, over at TUAW for your iOS needs or maybe over at Android Police for Android reviews, the key is to find reviews for the app you're looking for on a site that does them frequently and does enough that they're likely to have a worthwhile opinion. Read the review and see if it helps you glean the information you want to know-if the article is recent, you may be able to join a discussion with the author on the topic.
- Check YouTube reviews and video demos. If you're looking at an app and wondering how it works, head over to YouTube and search for the app's name. You'll almost certainly find a number of reviews of the app from YouTubers and bloggers who want to post their video review to their site. While those reviews can be useful, they're really good at showing you how the app works, what the interface looks like, how snappy or quickly they run, and what options they have. An in-depth video review can be just as good as installing the app yourself, and watching one can help you decide.
- Check a developer's other apps and stick with quality developers. Both the iTunes App Store and Google Play both let you see other apps by the same developer if you're looking at the app page for one. Check the developer's other apps and see how well they're regarded. You can check their reviews for general sentiment, but you'll do better if you look around for third party reviews and opinions about the developer and their work. If it's a developer behind other apps that you trust, that's a point in the app's favor. Similarly, if you know a quality developer, stick to them and take note when they launch a new app. Even if you're unfamiliar with the dev, see if they have a website or an easy way to get in touch with them. A serious dev will make their address available, and in the case of Android apps, they may even have a Google+ group you can join. See if they-or the app-has a presence on Twitter or Facebook.
You could always look into app direcovery apps, like our favorite discovery app for iPhone
In the case of some apps-especially free ones, or ones that fill in an important part of your life, like calendar apps, email clients, Twitter clients, or note-taking apps-you'll never really know if they're what you need until you actually try it out and use it for a while. In those cases, don't hesitate to give them a try and form your own opinion. Paid apps (and free apps that hide their useful features behind in-app purchases), on the other hand, make it difficult to get a good feel for it before you can possibly get a refund (if at all), so tread carefully and use the tips above to decide whether it's worth your hard earned cash.