In my last post I presented some evidence showing how easily customers can end up being faced with self-published works with questionable reviews. That post answered arguments that deny the existence of such reviews or their visibility. This post deals with another "It isn't an issue" argument that comes from a different angle.
Customers don't use reviews as guides. They're just not important.
This is easy enough to counter in the absolute sense because I use reviews and I'm a customer. I do exist -- philosophical beard-stroking arguments aside. The broader implied argument that not enough readers use reviews to make these issues important is more difficult to deal with because "enough" is a slippery concept. I suspect that most people making this argument are just generalising from their own opinion, rather than basing their argument on anything solid, but I'll give them the benefit of the doubt and lay out some reasons for believing that a significant number of readers do use reviews.
As you might imagine, a lot of the information concerning customer habits is of commercial interest and that means that those in the best position to have hard data (Amazon, B&N, etc.) probably aren't inclined to share it. However, you can certainly find some research that has relevance. Here's a poll about Australians using reviews to decide whether to deal with a company, for example. Here's a paper on hotel reviews that finds a link between sales and reviews. Here's a page listing headline quotes from marketing research about customer reviews, pretty much all asserting that they affect decisions and sales.
Perhaps you are unimpressed with the above. You don't think they're relevant because they're not about books or you just don't trust marketing companies to be honest about marketing (I don't blame you). How about this then? Here's a paper from Yale's School of Management that specifically looks at book reviews on Amazon and B&N and finds that "an improvement in a book’s reviews leads to an increase in relative sales at that site". In terms of research data from an academically objective source and specifically about the topic I don't think you're going to get any better. I'm certainly not going to look any further.
But maybe you still aren't happy. You don't trust any of the above for whatever reason. Okay, how about a plain argument based on the evidence that's in front of your eyes every time you go to Amazon or B&N or pretty much any online retailer? Here goes.
Space on a web retail site is very much like window and shelf space in a bricks-and-mortar business -- you don't just throw it away on stuff that doesn't work for you. The very fact that user reviews are still present on the most mature and competitive commerce sites is evidence that those businesses see value in them. How much value? On Amazon, every product has the star rating right next to it and the ability to sort both by Top-Rated and Average Customer Review is provided for most listings. At almost every turn Amazon wants to show you a rating or a review or get you to give a rating or review. If you review a lot, Amazon rewards you with badges and free stuff. In short, Amazon puts cash-money behind maintaining, improving and promoting the review system. That would seem to suggest that it matters to some people.
Even if you don't believe Amazon's motives are anything to do with customers using reviews (odd, but I'll run with it), the reviews are there. People have written reviews (unless you believe every review is fake) and people respond to those reviews with comments and ratings on the reviews. Hell, there are reviews that mention how they were duped by reviews they now believe are fraudulent.
And there I rest my case. If none of this is enough for you to think that maybe a decent number of customers actually look at reviews to at least steer them towards or away from further investigation of a book then I have to congratulate you on the depth of your convictions. If you're now shifting your objection to "Yeah, but all this proves is that customers shouldn't trust reviews. They should read the samples." then don't worry, I'll get around to that shortly.
Wednesday, February 9, 2011
Monday, February 7, 2011
Self-Published Book Reviews -- Problem? What problem?
A large part of the proposed purpose of this blog is based on the assertion that there are existing problems with the way that customers and indie fiction come together. Much of this comes down to the user review system and it's time to present some evidence of what I'm talking about. This is a long post, but hopefully it will be enlightening.
A bit of context: before I started this blog, I posted about the problem of misleading reviews when looking for new books on a forum frequented by indie authors. Some authors shared my concern but the overall tone was more than a little defensive. One argument that came up was that there simply isn't an issue to tackle and a poster taking this position pointed to the Amazon Top Rated Fantasy listings, basically saying "Nothing wrong there". Everybody's entitled to an opinion, of course, but I think that poster missed a few things and I'd like to use the same genre to demonstrate that real problems exist for reader looking for honest review guidance.
Here are the first twelve Kindle books listed under Fantasy when sorting by avg. customer review (US region on 3rd February 2011).
A bit of context: before I started this blog, I posted about the problem of misleading reviews when looking for new books on a forum frequented by indie authors. Some authors shared my concern but the overall tone was more than a little defensive. One argument that came up was that there simply isn't an issue to tackle and a poster taking this position pointed to the Amazon Top Rated Fantasy listings, basically saying "Nothing wrong there". Everybody's entitled to an opinion, of course, but I think that poster missed a few things and I'd like to use the same genre to demonstrate that real problems exist for reader looking for honest review guidance.
Here are the first twelve Kindle books listed under Fantasy when sorting by avg. customer review (US region on 3rd February 2011).
- Lover Awakened (Black Dagger Brotherhood, Book 3) by J.R. Ward
- Magic Bleeds by Ilona Andrews
- The Kingdoms and the Elves of the Reaches II (Signature Illustrated Edition, Keeper Martin's Tales Book 2) by Robert Stanek
- White Night (The Dresden Files, Book 9) by Jim Butcher
- Magic Strikes by Ilona Andrews
- Magic's Price by Mercedes Lackey
- Fablehaven, vol. 4: Secrets of the Dragon Sanctuary by Brandon Mull
- The Kingdoms and the Elves of the Reaches III (Signature Illustrated Edition, Keeper Martin's Tales Book 3) by Robert Stanek
- Fablehaven, vol. 3: Grip of the Shadow Plague by Brandon Mull
- Blackbringer by Laini Taylor
- Kingdom Alliance (Ruin Mist Chronicles, Book 2) by Robert Stanek
- Time Will Tell by Eddie Upnick
Now this isn't the same list that poster I mentioned was looking at but the same names cropped up in the listings that they did use (which was the top 100 books in the Top-Rated tab). Unfortunately, the poster in question failed to recognise a number of books as being self-published, including those in the list above. Considering their nature it's quite an oversight. Of the books above, the following are self-published:
3. The Kingdoms and the Elves of the Reaches II - Robert Stanek
8. The Kingdoms and the Elves of the Reaches III - Robert Stanek
3. The Kingdoms and the Elves of the Reaches II - Robert Stanek
8. The Kingdoms and the Elves of the Reaches III - Robert Stanek
11. Kingdom Alliance - Robert Stanek
12. Time Will Tell - Eddie Upnick
On the face of it that's remarkable. We've got self-published authors taking a third of the top twelve spots with solid five-star overall scores. Awesome, right? Joe Fantasy-Fan gets to read some fresh fiction and the review system is working just fine to raise self-published authors up to where they'll get noticed.
On the face of it that's remarkable. We've got self-published authors taking a third of the top twelve spots with solid five-star overall scores. Awesome, right? Joe Fantasy-Fan gets to read some fresh fiction and the review system is working just fine to raise self-published authors up to where they'll get noticed.
Not quite.
See Robert Stanek there, holding the top three spots? Well, Mr Stanek has something of a reputation. There's no point in me going into great detail analysing the reviews of his books as other people have covered the ground already. Anyone who looks into the background will take it as understood that the reviews for those books are somewhat questionable. I'm no psycho-analyst but if you're a self-publishing/indie proponent and don't see a problem with Robert Stanek hitting the top end of the ratings then you might be in denial.
I would like to look a little more closely at the other book in the list: Time Will Tell by Eddie Upnick. If the cover isn't enough to tip you off that this is self-published then you can look at the publisher. It is published by Eloquent Books, a well known vanity press. Eloquent Books is also known by various other names including the Strategic Book Group.
Time Will Tell has 53 reviews, 47 of which give it five stars and the remaining six give it four stars. Those numbers aren't damning in themselves but there are two things that immediately strike me as problematic about them:
It doesn't get any better if you look more closely at the text of the reviews themselves. The review listed as the most helpful is by William White, who manages to hold his enthusiasm down to a mere four stars, but writes:
Offering an alternative view on the 16th January, 2011, fellow reviewer M. Stanhope gives the book five stars and writes:
Okay, I'll save the space. They're identical reviews posted on the same day by supposedly different people.
Bad, huh? Actually, it's even worse than it looks and I'll get into that in a minute, but first an aside. The reviews from J. Cormier and M. Stanhope aren't just identical to each other, they're also clearly ripped off from an earlier review by one Sondlo Leonard Mhlaba, PhD, who on June 2, 2010 gives five stars:
To give credit where it's due, Dr Mhlaba does appear to be writing his own material, but then you would expect him to as he's the author of With or Without God: Life's Mysteries Continue Ruminations on God, Life, Death, Spirits, Reincarnation and the Future of Humankind, published by -- guess who? That's right! Eloquent Books! Small world, isn't it?
Anyway, let's get back to Cormier and Stanhope. So far you might be thinking that I'm making a bit of big deal about nothing here. Sure, Stanek is known for dodgy marketing practices and maybe Mr Upnick and his fellow authors at Eloquent Books are being a bit over-zealous with their efforts, but perhaps these are unusual examples. Well, let's take a closer look at the four reviewers, including Cormier and Stanhope, that posted reviews on the 16th January. Here are the links to their Amazon pages:
You can look at any single individual and see a couple of things that are odd. What immediately struck me was their tagging; most reviewers don't tag at all but these guys are tag-crazy. Then I started seeing something very odd about their reading habits. I can't claim to have checked every book but I've checked a fair few and every one has been obviously or most likely self-published. They're published by one of XLibris, iUniverse, CreateSpace, AuthorSource, BookSurge and a smattering of publishers that only have one or two authors in their stable which can be taken to be closely linked to the authors if not directly owned and operated by them.
That's not all though, if you look at the specific books these "individuals" have reviewed you'll see a commonality in taste that's beyond remarkable. Not only are they all big fans of Eddie Upnick, but two or more of them also share an enjoyment of Becky Due's romance fiction, Paul Torcivia's contemporary humour and Mimi Mathis' US Civil War fantasy. It's not just fiction they share an interest in though. They're also fans of Fitness Leisure's martial arts manuals and even more amazingly have a common need to revise for the ITIL V3 examinations. Oh, and they all really love Carl Tuchy Palmieri.
It doesn't stop there. These four aren't alone. You can just keep following book to reviewer to book to reviewer and finding these amazingly like-minded people. If you don't believe me, look at the selection below. It's as if they're part of some kind of mind-melding experiment.
I could go on, but you get the idea -- there are a fair number of "reviewers" apparently working in a team and a fair number of self-published books apparently being serviced by this team.
I want to make it clear that I'm not claiming that all, most or even a large proportion of self-publishing authors are engaged in this sort of thing. The point of this post is to show how a customer looking for good new authors can immediately find themselves presented with books and reviews that will set off alarms. If customers start to connect this with self-publishing as a whole then the honest players will be judged guilty by association. I think the evidence I've put forward here should be enough to convince anyone that there is some seriously dubious practice in these reviews and that there's a real danger of readers getting a poor impression of self-publishing.
Time Will Tell has 53 reviews, 47 of which give it five stars and the remaining six give it four stars. Those numbers aren't damning in themselves but there are two things that immediately strike me as problematic about them:
- The sheer number of reviews is rather unusual considering the Amazon sales rank. Time Will Tell has a sales rank of over 22,000 for Kindle books as I'm writing this. Based on the figures that various Kindle authors have posted, that ranking suggests sales of a few tens of copies a month, if that. Despite these figures, the book has gathered a number of reviews that you normally see for books that have sold thousands of copies.
- I've read the sample chapters and Time Will Tell is an abysmal piece of work. The writing manages to be both simplistic and yet difficult to follow and the story's ridiculous events are narrated with all the storytelling flair of a seven year-old describing the movie they just saw. Can I believe that someone could think it's a fabulous book? Sure, it takes all sorts, but I can't believe that pretty much everyone who has ever bought it thinks so and feels the need to write a review. Something smells funny. If you don't trust my opinion then go ahead and read the sample on Amazon. I'm fairly sure I'm not mistaking a rose for a sewer here.
It doesn't get any better if you look more closely at the text of the reviews themselves. The review listed as the most helpful is by William White, who manages to hold his enthusiasm down to a mere four stars, but writes:
"What a story! ... Keep up the good work Eddie. There is a place for you in English literature."Now maybe I'm a bit of a stick in the mud but that's a rather familiar way to refer to an author you're reviewing, isn't it? Perhaps there's a clue as to why this is the case in William White's full username, which announces that he's the author of "The Treasure of San Miguel Island". The information for William's novel reveals that his book is published by Eloquent Books/Strategic Book Group -- the same vanity publishing company that Eddie uses. Hmm...
There are some interesting things to be seen in Mr White's reviews, but let's not get sidetracked. Back to Eddie Upnick's novel. In his five star review of "Time Will Tell" on the 16th January, 2011, J. Cormier writes:
'If you are into science fiction, you may want to read a book titled "TIME WILL TELL" by Eddie Upnick. It starts out with "Prepare yourself, young man, for a story that will shutter your known truths." A psychologist at a nursing home is listening as Jeff begins his story. Jeff is a Time Traveler from the 22nd Century. The only way to make things right is to go back. In this story, his world has been ruled by the Nazis since the 20th Century. His goal is to change the course of history, Jeff a rebel along with a few others were sent back to 1938. $90K in cash without anything else will buy him a roof for some time in this nursing home.
I liked the fact that book keep me so interested I could not put it down. I wanted to know what was going to happen to kudos to the author. Even though it is 2133 you are quickly brought back in time. In the end, looking back I loved the ties to history. The author ties it all together and brings you a great book to read. One that you will not be sorry you picked up.'
Offering an alternative view on the 16th January, 2011, fellow reviewer M. Stanhope gives the book five stars and writes:
'If you are into science fiction, you may want to read a book titled "TIME WILL TELL" by Eddie Upnick. It starts out with...'
Okay, I'll save the space. They're identical reviews posted on the same day by supposedly different people.
Bad, huh? Actually, it's even worse than it looks and I'll get into that in a minute, but first an aside. The reviews from J. Cormier and M. Stanhope aren't just identical to each other, they're also clearly ripped off from an earlier review by one Sondlo Leonard Mhlaba, PhD, who on June 2, 2010 gives five stars:
'"Prepare yourself, young man, for a story that will shutter your known truths." This is how Jeff begins his story, as told to Eddie Upnick, the author of TIME WILL TELL.
The first encounter between Jeff and Eddie takes place at a nursing home where Eddie Upnick works as a psychologist. Jeff appears as if from nowhere. He claims to be homeless and has no family...'...and so on.
To give credit where it's due, Dr Mhlaba does appear to be writing his own material, but then you would expect him to as he's the author of With or Without God: Life's Mysteries Continue Ruminations on God, Life, Death, Spirits, Reincarnation and the Future of Humankind, published by -- guess who? That's right! Eloquent Books! Small world, isn't it?
Anyway, let's get back to Cormier and Stanhope. So far you might be thinking that I'm making a bit of big deal about nothing here. Sure, Stanek is known for dodgy marketing practices and maybe Mr Upnick and his fellow authors at Eloquent Books are being a bit over-zealous with their efforts, but perhaps these are unusual examples. Well, let's take a closer look at the four reviewers, including Cormier and Stanhope, that posted reviews on the 16th January. Here are the links to their Amazon pages:
You can look at any single individual and see a couple of things that are odd. What immediately struck me was their tagging; most reviewers don't tag at all but these guys are tag-crazy. Then I started seeing something very odd about their reading habits. I can't claim to have checked every book but I've checked a fair few and every one has been obviously or most likely self-published. They're published by one of XLibris, iUniverse, CreateSpace, AuthorSource, BookSurge and a smattering of publishers that only have one or two authors in their stable which can be taken to be closely linked to the authors if not directly owned and operated by them.
That's not all though, if you look at the specific books these "individuals" have reviewed you'll see a commonality in taste that's beyond remarkable. Not only are they all big fans of Eddie Upnick, but two or more of them also share an enjoyment of Becky Due's romance fiction, Paul Torcivia's contemporary humour and Mimi Mathis' US Civil War fantasy. It's not just fiction they share an interest in though. They're also fans of Fitness Leisure's martial arts manuals and even more amazingly have a common need to revise for the ITIL V3 examinations. Oh, and they all really love Carl Tuchy Palmieri.
It doesn't stop there. These four aren't alone. You can just keep following book to reviewer to book to reviewer and finding these amazingly like-minded people. If you don't believe me, look at the selection below. It's as if they're part of some kind of mind-melding experiment.
I could go on, but you get the idea -- there are a fair number of "reviewers" apparently working in a team and a fair number of self-published books apparently being serviced by this team.
I want to make it clear that I'm not claiming that all, most or even a large proportion of self-publishing authors are engaged in this sort of thing. The point of this post is to show how a customer looking for good new authors can immediately find themselves presented with books and reviews that will set off alarms. If customers start to connect this with self-publishing as a whole then the honest players will be judged guilty by association. I think the evidence I've put forward here should be enough to convince anyone that there is some seriously dubious practice in these reviews and that there's a real danger of readers getting a poor impression of self-publishing.
In future posts I'll discuss other examples of behaviours that result in misleading ratings but also how things might be improved and the ways in which some of those involved in self-publishing are trying to change things.
First Post and PAQ
Hello. How are you? I hope you're well. Welcome to another blog. A blog about indie fiction. Likely more often than not, a blog about what's wrong with indie fiction.
I'm sure various hackles are rising at that, but hang on a second while I explain in the form of a PAQ or Predictably Asked Questions.
What's indie fiction?
Good question. You're sharp. I like you. Indie fiction is the prose fiction equivalent of indie games, films and music. It's a kind of casual way of talking about independently published fiction -- you might know it better as self-published fiction.
So why not call it self-published?
Between you and me, I do and I will. To me the terms are interchangeable but some people have decided that self-published is a tainted term and so would prefer to use "indie". If it avoids a silly argument, I'm perfectly happy to oblige at least some of the time.
So why are you looking to tell people what's wrong with indie fiction? What's your agenda?
I like fiction and I like the idea of a functioning independent scene for authors. There has been a lot of talk about how technology is democratising the media. To truly democratise a market it is important that there are as few barriers to entry for producers as possible but also that consumer decisions are enabled by the provision of good information. The market needs to be as transparent as we can make it. Great strides are being made in opening the market for producers with the Web and virtual stores for devices like Kindle, Nook and iOS (iPhone/iPad). The second part seems to be where there's a struggle. How do customers find great products by talented producers in the crowded market that results from removing barriers to entry?
You didn't answer my question. Why are you looking to tell people what's wrong with indie fiction?
You caught me. Okay, from what I just said about the problem of consumers finding products, here's what I think is wrong:
There's nothing I can say that will convince anyone otherwise if that's what they want to believe but no, I'm not. I don't believe that self-publishing means bad writing. What I do believe is that it's very easy to come to that conclusion or at least that it's so painful looking for the good stuff that you may as well just act as if it's all terrible. I do, if you really want to find something to be offended by, think that the average self-published writer is worse that the average traditionally published writer and that will probably always be true.
But you believe that all the reviews are fake?
Nope. I'm saying that there are a lot of examples of highly questionable reviews but questionable doesn't mean fake. There's even a question about what constitutes a "fake" review. If a writer gives a dozen copies to their friends and asks them to post reviews without asking for any favours, and they all put five star reviews because they're nice people, those clearly aren't as "fake" as reviews bought and paid for or reviews posted by the writer. In most cases there's no clear evidence what exactly is going on.
But you do think some of them are fake, don't you? Like really, dishonestly fake?
Well... yeah. It seems naive not to consider that as likely.
What's your angle? Are you some sort of traditional publisher goon?
Primarily I'm a technologist with a general interest in the technology-driven movement towards open publishing platforms in all media. In the mid-90s you could have found me arguing with people from the world of traditional publishing about the coming impact of technology. In the argument about self-publishing, I was generally to be found in the middle, disliked by the traditional publishing crowd for supporting the idea that self-publishing would grow and could be legitimate, but also attacked by self-publishing cheerleaders because I warned about the problems with discarding the publishing filters. Little has changed except that I don't get scoffed at for suggesting that people will see advantages in electronic reading over paper.
Other than that I'm a consumer and I want to be able to find new stuff. I don't want to be frustrated in my search. I would like a market that allows people to find new creators that they appreciate and which encourages new entrants because it functions so well in letting consumers find things to consume.
I'm sure various hackles are rising at that, but hang on a second while I explain in the form of a PAQ or Predictably Asked Questions.
What's indie fiction?
Good question. You're sharp. I like you. Indie fiction is the prose fiction equivalent of indie games, films and music. It's a kind of casual way of talking about independently published fiction -- you might know it better as self-published fiction.
So why not call it self-published?
Between you and me, I do and I will. To me the terms are interchangeable but some people have decided that self-published is a tainted term and so would prefer to use "indie". If it avoids a silly argument, I'm perfectly happy to oblige at least some of the time.
So why are you looking to tell people what's wrong with indie fiction? What's your agenda?
I like fiction and I like the idea of a functioning independent scene for authors. There has been a lot of talk about how technology is democratising the media. To truly democratise a market it is important that there are as few barriers to entry for producers as possible but also that consumer decisions are enabled by the provision of good information. The market needs to be as transparent as we can make it. Great strides are being made in opening the market for producers with the Web and virtual stores for devices like Kindle, Nook and iOS (iPhone/iPad). The second part seems to be where there's a struggle. How do customers find great products by talented producers in the crowded market that results from removing barriers to entry?
You didn't answer my question. Why are you looking to tell people what's wrong with indie fiction?
You caught me. Okay, from what I just said about the problem of consumers finding products, here's what I think is wrong:
- There's a whole load of bad self-published fiction out there.
- The existing mechanisms for consumer to consumer guidance appear to be failing or being subverted. If you want the blunt language, there are a whole load of books where the reviews and ratings are so at odds with the quality of the book that the reviews have to be questioned. I think there's enough of this to cause problems for the acceptance of self-published works even as we enter a period where it could flourish.
There's nothing I can say that will convince anyone otherwise if that's what they want to believe but no, I'm not. I don't believe that self-publishing means bad writing. What I do believe is that it's very easy to come to that conclusion or at least that it's so painful looking for the good stuff that you may as well just act as if it's all terrible. I do, if you really want to find something to be offended by, think that the average self-published writer is worse that the average traditionally published writer and that will probably always be true.
But you believe that all the reviews are fake?
Nope. I'm saying that there are a lot of examples of highly questionable reviews but questionable doesn't mean fake. There's even a question about what constitutes a "fake" review. If a writer gives a dozen copies to their friends and asks them to post reviews without asking for any favours, and they all put five star reviews because they're nice people, those clearly aren't as "fake" as reviews bought and paid for or reviews posted by the writer. In most cases there's no clear evidence what exactly is going on.
But you do think some of them are fake, don't you? Like really, dishonestly fake?
Well... yeah. It seems naive not to consider that as likely.
What's your angle? Are you some sort of traditional publisher goon?
Primarily I'm a technologist with a general interest in the technology-driven movement towards open publishing platforms in all media. In the mid-90s you could have found me arguing with people from the world of traditional publishing about the coming impact of technology. In the argument about self-publishing, I was generally to be found in the middle, disliked by the traditional publishing crowd for supporting the idea that self-publishing would grow and could be legitimate, but also attacked by self-publishing cheerleaders because I warned about the problems with discarding the publishing filters. Little has changed except that I don't get scoffed at for suggesting that people will see advantages in electronic reading over paper.
Other than that I'm a consumer and I want to be able to find new stuff. I don't want to be frustrated in my search. I would like a market that allows people to find new creators that they appreciate and which encourages new entrants because it functions so well in letting consumers find things to consume.
Subscribe to:
Posts (Atom)