Book recommendation system comments

[John Smith]

I don't know if someone else has already posted on something like this - as I have not read all of the forums - but here it is:

Currently, I am a user of librarything, a website I use to keep track of all my books that I read and to find new books that the site automatically recommend to me. However, I find that the recommendation system is very shallow, in that it mainly uses the authors that you have read and the genre of the book to recommend new books, and that is all - at least so far as I can tell - despite the rating that you give it. This is a real downturn for this site becuase if there is a book that is horribly written, and I presume that the rest of this author's writing is of the same calibre, the recommendation system won't really filter out the results. (For example, I read a book that I hated, and I rated it at its lowest, and lo and behold, it was in my top 15 books for recommendations.) For this reason, I think that it is vital that the INDIVIDUAL user is able to rate any book, and that your recommendation system should alter the results listed to appeal to that individual, not just based on the content of the book. That way, the same crummy authors don't keep appearing in the recommendations, mucking up the potential books that a reader would likely enjoy.

The potential of this site seems to lie in the objective nature of its recommendation system, and I think it would be entirely worthwhile to catalogue as many books as you can so that any user could build up a library in order to get recommendations that have to do with their entire library, not just a single book, or even a small number of books.

Personally, I would definitely start with the classics of literature. (I know, this site will try to break from traditionalist methods what books are good, but at least this would give you an appeal for a good number of readers.)

I think that the recommendation system that you have highlighted near the end of your video on the main page would be excellent. However, I would add at least one more factor other than perspective, action, dialogue, density, pacing, and description. This would be user recommendation, a seemingly hot-button topic in these forums. This would be  a general "safety net" for your recommendation system so that people who really care about what is popular - having something later to talk about with their friends - will still feel comfortable using this system; counterwise, people who don't care at all what other people think can at least find books that are similar enough in style to what they like despite others' thoughts/opinions muddying the waters. (I have to say that I would be in the latter camp!)

Yet another feature that you could integrate would be a "partial library search," or some such feature, where a user could find a recommendation on a specific group of books listed in his/her library that s/he is particularly fond of, even if the span across a few different genres. Maybe this is hoping for too much, but perhaps your site could analyze the algorithms and such of all of those books, and average them out to some degree to find other books that fall within the same range, ordering the books based on accuracy.

Perhaps hoping for too much yet again would be to say that your recommendation system could incorporate vocabulary in its results. That is to say, if you could find algorithms that decipher the range of vocabulary used in a book, a user could more intelligently decide on his/her next read. For example, if a reader sees that a potential read has a very narrow range of vocabulary, then perhaps it will be a very light read, easy to get through, and ultimately not challenging. Or a reader could find a book that has a very wide range of vocabulary and the appeal would lie in its challenging, heterogenous - yet specific and intentional - nature of word choice.

If this could even be taken a step further - that is to say, if an algorithm could be found not only on the range of vocabulary, but the frequency of vocabulary usage, then I think that would be a miniature gold mine. Suppose there was a reader who had not been exposed to books with made-up vocabulary - such as A Clockwork Orange - who wanted to find one such book, then all s/he would have to do would be to prioritize vocabulary on the preference scale and specify the degree of common usage in books.

One obvious and easily overlooked feature of a recommendation system would be the length of the book. Perhaps a reader is interested in short stories above full books; perhaps then this same reader could come to your site and find a desirable short story that fits within the spectrum of his/her fancies. Furthermore, the same reader could also find lengthy novels that are appealing, and could likewise find a good, long book to read throughout the near future.

Yet another obvious - yet seemingly covert - addition to the recommendation function would be time. That is to say, the time range wherein a book a) was published or b) takes place would also have a direct appeal to the reader. Futuristic, modern-day, historical - all of these would be held within the reader's preference when looking for a new book to find as well as old books or ultra-new ones.

As to whether or not this site should be open source, a capital partner, a function service, etc. - I am not sure; I don't know about these things. MY preference - and I'm positive that I'm not alone - is that you should do whatever you can do do get a VERY large amount of books in your system as humanly possible; combine this with a recommendation system that is as comprehensive as possible, considering the above points (and others, as people may add to this topic), and you will have a website that is far and away second to none. I think it is also important to get as many of these things down as possible in the starting stages of your website, that way you attract a specific - yet not too specific - group of people to use your website, and you don' t have to worry about changing things later on. I think other popular social book networking web sites may have this problem; they have pigeon-holed themselves into something that they started with, and they cannot really get out of it now that they are so far in. Any big changes will negatively affect many of the users that currently use their sites.

Overall, I think the more multi-faceted a recommendation system is, the better user-response you will get. I have spent the last two days searching for a better site (which is how I found this one) to recommend books to me, and from what I can tell, no one has really come up with anything like this.

I hope that this has been helpful to any degree, and again, I must say that I feel that there are many who are like me who want a deeper recommendation system than one based on a single book. The same goes with recommendations systems that incorporate many books, but that are shallow in how they go about recommending other books. So I say kudos, to you, and I'm excited to see where you go. To echo some who have came before me in these forums,

I HEAR YOU. Best of luck.

[Daniel Bowen]

John,

   It is always great to hear from individuals who understand the value that BookLamp has the potential of creating.  We will see how the public reacts in the coming months.  As for your sentiment toward librarything, your anecdotal comments are very similar to a market study that we did here at BookLamp.  To be brief, we found that 99% of the 150+ titles recommended by librarything were either bestsellers or books written by an author already listed in my librarything user library.  To clarify, we determined that a bestseller is a title that sells >100 copies - 97% of published books sell less than 100 copies.

   It is our full intention to include every possible title into the system.  This technology could also incorporate other book categories such as technical titles and text books.  We have no limits with regard to what titles to eventually include in the system.  For now, we are inclined to focus on general trade until we have more evidence of the validity of the algorithms.  When i say validity, i mean with respect to any working relationships we have.  Though, we know in our heart that this is real and forward-looking (we've seen too much evidence to think otherwise), we have to take every opportunity convince others (mostly to acquire titles to add the the system).

   I believe that user recommendations will be a part of the service (I see no reason why not).  When a full-fledged working BookLamp is in play, there are plans to build a social community supported by the recommendations we make.  To be honest, I am currently working on something related.  Right now, obviously, there is not much of a service available (everything's been happening behind the scenes), so we don't have that option available yet.

   Your 'partial library search' is very similar (if not identical) to some of the tools that have been considered for development.  Again, we need a working service with users (SOON to come), but I am glad to see that the public finds this as thought provoking as we do.

   As for your incorporating vocabulary into the results, that is a definite possibility.  Off the top of my head we do have a couple of measures that can be considered a bit of a proxy to the more intensive vocabulary idea that you may have in mind.  As I understand what you are suggesting (and my experience at BL so far), those thoughts are very possible and only a matter of time and money.

   Your thoughts on incorporating the length of the book is one of the first things I remember hearing in one of Aaron's videos describing the possibilities.  Something like "If you liked Stephen King's IT, but want something half the length, BL can recommend a title under those circumstances" - he said something like that, I don't remember.

   I understand the use of time that you are suggesting, and all I can say there is that it won't be overlooked.  Now, taking the setting into account is something that we hope to eventually work out algorithmically - so that we don't have to individually review the 100,000 titles published every year.

   We have every intention of adding as many titles as possible.  If the opportunity presents itself to include every book in written history (currently impossible) we will not hesitate to do so.

Thank you for your commentary John.  Feel free to voice any other thoughts/concerns/ideas here at the BookLamp forums.

-Dan

PS  stay in touch in the upcoming months and the recommendation service you've been looking for will show itself.