The Possible Ramifications of BookLamp.org

[Al Green]

When BookLamp.org was unveiled I was quite curious and excited by the prospect of a system that categorises books to allow for fast and effective searching to find new texts similar to those you've enjoyed in the past. However, being a lover of literature and young writer myself, I tend to enjoy extrapolating things and to see where they take me - it's a great technique for story ideas (1984 comes to mind as a great example of taking the status quo and "running with it.") I was reminded of a short story by Roald Dahl, The Great Automatic Grammatizator [link], I read a number of years ago, that raised a couple of questions and actually caused to me to begin to worry a little.

I’m not sure I agree with the quantifying of literature; I don't think it can ever accurately be done. A book is not simply the sum of its words. BookLamp.org may well be robbing people of some important experiences, clumsily trying to assign numerical values to thoughts, ideas, style and expression. If a reader solely, or even heavily, relies on BookLamp.org for guidance, they begin to read very selectively. This denies them the experience of reading a bad book and thereby lessening their appreciation of a good book when they finally happen to come by one - diluting the entire reading experience. Part of the fun of reading is, at the conclusion of a series of books you’ve enjoyed, experiencing a range of styles, types and ideas on your journey to finding another gem.

It also removes the need for the period of reflection at the conclusion of a book; during which time one usually determines for himself what he enjoyed about the story, plot and characters – and what he didn’t like so much. Why bother when you have an online calculator to analyse it for you?

Assuming that BookLamp.org does go on to become quite large - and I think it has the potential - it may become quite influential not only in the selection of literature, but the writing of it, too. For example, take a young, budding writer who wants to strike it big in the world. All he has to do is log on to Booklamp.org, look up the most popular reader profile for his chosen genre - readily available for your curiosity and mine - and suddenly he has what he thinks is a perfect formula to write the next best seller. Suddenly all authors will be writing to a formula. If the authors don’t take it upon themselves to adopt them, then you can count on the publishers enforcing them. After all, a proven ideal ratio of pace and description is too good to refuse for a company out to sell books. Whether you chose to publish such information publicly, or make a little (or a lot) of money selling it privately, I think it has the potential to be quite influential.

Anyway, these are just thoughts – probably exaggerations, but I thought I’d throw them out there and see what sort of response I got back from you guys.

[Aaron Stanton]

Or, possibly not exaggerations, who knows?    Of the different possible complaints about the system, this is actually the one that I think has the most philosophical implications to it:  Will BookLamp lead to a narrowing of literature?  Will everyone start writing to a formula?

To be honest, I don't know, though we've put a fair amount of thought into trying to find ways around that.  One of the more interesting features of the system is what we call the Press My Limits button, which looks at what you've liked and what you haven't liked, and tries to return a book result that is just at the edges of your limits, different enough that you probably wouldn't choose it on your own, but not so far out that you'll reject it out-of-hand.  Knowing what you like does not mean that you have to read only what you like; you can use that same data to expand your horizons, too.

I think it would be pretty cool to have a feature in your profile somewhere, maybe call it Book To Try, or something like that, that is the MOST radically different from your preferences.  Maybe a different books will show up in your profile every week that's specifically chosen because it's well rated socially, but the opposite of your writing style preferences - might be an interesting grab bag.

Basically, what I'm saying is that the system can be implemented to serve both functions; it can be used as a tool to find whatever you're searching for, whether that's a good book or a bad book.  The concerns about it being too easy to find good books... well, that one I can't really address, because I'm not sure if there's an answer.

BookLamp has a motto that I think is worth mentioning here, and we have it partly for this exact reason:  BookLamp will never tell you if a book is good or bad.  Period.  We'll never even try.  While that seems counterintuitive to what the project is about, it's not.

If you bring us a book, we will find you matches based on what you're looking for, but we will never, ever say, "You don't want to read this, it's a bad book."  You have to decide and tell us what is good and bad to you - the worst we'll ever do is suggest that a book might be different from your normal tastes, but not bad.

BookLamp is not a system for passing judgment.  That's still the reader's job. 

And as for the market pressures forcing books into a similar pattern, we've debated that a bit, so let me put forth the two arguments, pro and con, in case you haven't thought about the pros in this regard (not to lessen the importance of the cons, but just for well-rounded consideration).

Con: As patten X becomes more and more popular, publishers seek pattern X, and it reinforces itself.  - (I have to note, by the way, that considering the depth of the analysis and the depth of some of the formulas we run, I'm not sure it'd even be possible to write matching a specific formula, but it's a reasonable concern.)

The other side of the coin:  Part of the reasons that current, certain formulas sell is because readers don't have easy access to finding alternatives.  Once you know you like author A, you stick with him until you have to move on.  This is partly why name brand is so important, and why Stephen King can sell millions of books, while Richard Bachman (S.K.'s pen name) never did as well.  People knew and stuck with King, and never explored books of equal quality by an author they didn't know.

But now brand name isn't the biggest factor, which means you can flow from author to author far easier.  It also means that people with more unique tastes now have a tool to find books that match their styles off the beaten path, books that may never appear on a best seller list or be well received in social networks driven by popular demand.  There will always be the mainstream books that sell the most, but I think there's a fairly good possibility that BookLamp will have the opposite effect of what you think - the books on the fringes will get a boost as well.

Basically, it lowers the difficulty of exploring - ideally - to the point that reading from a new author is as painless as reading someone you're comfortable with.

I think that's far more likely to be a boast to the unknowns and unusual in the world of literature.

But I could be wrong - that's a debate we've had amongst ourselves here as well, and I'm not sure I have an answer for you.  But maybe you'll at least find it encouraging that we've thought about it, and have it in mind, and worry about it, too. 

Aaron

[mirandafte]

Al: I totally had the same thoughts running through my mind as I watched the introduction video. I actually listened to an NPR broadcast today about being melancholy and how it's a necessary part of life if we want to truly experience happiness. This mirrors one of your first points exactly. We must experience bad to see the good.

I am happy to see Aaron's response to these ideas. It seems like you guys are a group that I would enjoy working with: a group that is sharing and bouncing around ideas with concern for all those involved. I work for a company that believes in the opposite, so it's so refreshing to see a collaborative team at the brink of success.

The issue of all these suggestive and search engine sites is a broader topic that I hope to study more in my library science classes in the coming years. The Internet and sites like Book Lamp are amazing tools that need to be used in conjunction with the, lets say 'old school methods' of walking into a library and pursing the shelves and speaking with your librarian.

[Al Green]

Although the Press My Limits and Book To Try sound like interesting features, for a project that advertises itself as a service for finding books that are similar to the reader's favorites, do you think these features will actually get used all that often? It seems counterintuitive to sign up for a service to use a feature that does the very opposite of the projects primary function. I could be wrong, though. And by no means am I suggesting that it will be a wasted feature - just it won't be used by the majority of your users and won't go very far in reversing any "damage" that may occur as a result of BookLamp.org.

And, unfortunately, you will again be choosing how far outside their reading "comfort zone" you're willing to suggest a book - using numbers and ratios. So, although you've managed to include a larger group of books that satisfy your ideal formula, by making it less sensitive and by selecting some outliers, you're still narrowing things down. You're still taking away people's decision of how far outside their 'typical' reading material they're willing to venture. Perhaps you'll have a slider, or a quotient to turn down the sensitivity, but there's those numbers again, trying to represent something that's just not quantitative, if you ask me.

I understand and expect that your formulas and algorithms are indepth and complex, but if sold or guessed by publishing companies, although it would be near impossible - or, at least, very difficult - to write to a formula, it would be exceedingly easy to, when faced with two prospective manuscripts, run them both through the algorithm and take the one that comes out best, before a human eye even gets a chance to read it.

I agree with you on Stephen King. I recently read his Cell and notice that the author's name was at least twice the size of the novel's title - readers were expected to buy the author, not the book, it seems. I support anything that helps the 'little guy' get a fighting chance.

I would caution against including books that rate well socially - even if your reason for including them is to illustrate good books outside the user's normal reading comfort zone. Users can hear about these books elsewhere and I don't see how you could maintain the claim of being "impervious to advertising" if you started pulling books of best sellers lists. After all, part of the reason they're there is because someone's been spending the advertising budget wisely.

[Paul Kreiner]

Hi Al,

Toward the beginning of the project, we in the team discussed and debated exactly the points you raise.  I wish we had recorded some of those discussions for posterity, but it's nice to see that other people such as yourself have recognized the exact same potential misapplication (or overapplication) of BookLamp technology that we recognized.

As far as readers are concerned, based on my own experiences, I think the positives vastly outweigh the potential negatives.  I'll speak for myself -- I don't try new authors that often because my reading time is limited, and I've had several experiences where I selected a random book from a friend, the library, etc, spent a weekend trying to get into it, and found that I just couldn't do it because of the writing style.  Maybe it was too dense, or slow-paced, or I didn't like the way the author portrayed the accents of his characters (Huck Finn?)  Point being, I came away from the experience unhappy and not feeling that I had gained any entertainment or educational value from the experience, other than I'm not good at picking books at random.

I read a lot of technical, non-fiction stuff for my work (don't we all?) -- I don't typically read for enjoyment, unless I'm on vacation.  When I do choose to read for fun, I want to be sure I'm actually going to enjoy the experience, or at least not feel it was a waste of my time.  For me, the BookLamp technology often will suggest a book I may never have heard of, and instead of walking right by it (like I would have before), if it's recommended as a writing style that I like, or that's close to my preferences, then I'll pull it off the shelf, and at least scan the summary and sample a few pages.  So for the casual reader (like myself), it opens up more horizons and more authors, and it does so in an objective fashion.  Since starting this project, I've read about 4-5 novels, and at least two were directly because of analysis recommendations -- authors I'd never heard of and books I didn't know existed.  Before that, I'd read maybe 1 or 2 novels per year.

This technology has broadened my horizons by reducing the risk that I might randomly select a book that I'm not compatible with and waste a bunch of time trying to read it (and getting little/negative enjoyment).  At this time in my life, I'm apt to put the book down and go do something else if I'm finding it consistently too un-enjoyable.  (Perhaps I'm just a bit too hedonistic!)  I know based on conversations (one I had today, in fact) that there are a lot of folks similar to me.  If we can get these folks reading and enjoying it again, IMHO that's a net positive, compared to the status quo of not reading at all.

My $0.02.   Thanks for the concern!  I agree about not overpopulating from best-seller lists (yes, we had that discussion too), although we can't avoid them.  Our approach would be to keep the representative ratios of the total population from being swayed toward best-sellers.  As the database grows, this naturally becomes less of an issue.  Any other ideas (short of canceling the project) about how to minimize some of these misapplications of the technology?
Paul Kreiner
Project BookLamp

PS - a tangentially related side note: Does anybody know of a copy of "The Killing Star" I could borrow?  I've been wanting to read it for months (as Aaron can attest) and have been unable to find anything for a reasonable price, nor anything in our local or university libraries.

[Al Green]

G'day, Paul.

I'm encouraged to hear that such social implications were discussed early on in this project - if nothing else, it certainly illustrates this is a well thought-out idea.

You don't have to convince me of BookLamp.org's usefulness. I can definately see it being a great tool for a lot of people worldwide - or, rather, the English speaking population (at least to begin with.) And for all my reluctance to quantify literature or even simplify it into a few discrete facets, I acknowledge that it can be done for the purpose of making certain generalisations - and you guys seem to have done it quite well. I only seek to draw people's attentions to "the possible ramifications of BookLamp.org" (or another company like it) just because I'm curious as to what you guys, and the community, think. I'm not shouting a warning or trying to discourage BookLamp.org; I think it's a good idea and, ultimately, inevitable. We've categorised just about everything else and I suspect novels have been left relatively untouched because of the difficulty of designing a system of analysing and quantifying the literature. I don't think we'll ever get it perfect - which I have to admit, I'm quietly smug about - you guys seem to have made some impressive progress with it.

Another thought that re-occurred to me while I was writing this: once you guys hit it big, there's assuredly going to be imitation websites who try to get their leg in the door with a new or exciting feature - much like hundreds of social networking websites - and they may be willing to venture where you've decided not to, and may not be so considerate, or even aware, of the possible side effects. Obviously that's something beyond your control, but I think it's something to think about, none-the-less.

Now, to some practical concerns I have: It's great that each book will be given an equal footing, but I'm wondering if availability will take any part in your calculations. There will no faster way to irritate a reader than have your algorithms return a book that is out of print or a small release and all but impossible to get their hands on. After all, a shopping list is only as good as the stores you're able to visit. Amazon.com and websites like it will help people locate a copy of the book they're after, but not everyone (myself included) finds the idea of virtually shopping for a book appealing and even Amazon doesn't have everything.

As for ideas to suppress any possibly damaging influences BookLamp.org may have: well, ultimately, I think the idea represents progress and it's very difficult (and usually wrong) to try and prevent progress. If BookLamp.org gets as big as you seek to make it, it's going to be influential and related industries are going to flock to take advantage of that. I don't see a way to change that. As I've stated already, though, I do take comfort in the fact that you guys have already considered this thing from just about every angle and I'll personally be glad if it's you that make this work. Because somebody else will if you guys don't. 

[Aaron Stanton]

Al,

You know, your comment about book avaliability is a really good point, and one I'll have to think about.

I'm not going to respond with a really long one, this time - partly because I don't want to flood people with responses; but it's also a topic that interests me, and as I said, the objection to the system I expect is the most legitimate.  I think we can convince people that it works, because we have data.  And I think we can convince people it's interesting.

But I'm not sure how to respond to this objection.  I acknowledge the concern, but I'm not sure how to solve it, aside from building tools that allow people to search for anything they want (good or bad) and then making them as attractive and easy to use as we can.

So, let's see if we can shift the discussion a little.  We all see the potential issue; what can we do to help it?  I know that you, Al, already mentioned that nothing has lept out at you yet... but think about it a bit and post if you have something.  I'm interested in making this project great in all aspects, and if that means doing my best to be useful and protect the creativity of an industry I love... well, sounds good to me. 

Aaron

P.S.  And Paul really has been looking for a copy of The Killing Star for a while now.  It's become like a quest for him.    He hits all the used book stores; gives him something to do, I guess. 

[Soren Morton]

I would caution against including books that rate well socially - even if your reason for including them is to illustrate good books outside the user's normal reading comfort zone...

After reading this two thoughts came to me...

My first thought was that putting in best-sellers is one of the most useful things you can do starting out. I say this because it will allow more users to find matches using your system. If you have very few best-sellers and an average reader comes to the site they won't be able to find a book that they have read in the past to compare against other books. This would inhibit their ability to find a similar (Or different) book by an unknown author.

The second was that if you were to deliberately limit the number of best-sellers in the database then this site would do exactly what you don't want it to do. You would create a bias against a book based on its popularity... I think it is also important to note that just because a book is a best-seller doesn't mean it isn't worth reading (Or that it IS worth reading ).

Maybe you could enter at least 1 unknown book for every best-seller to make it fair?

[Al Green]

Soren,

You have a good point and I now realise I was not very clear. Naturally best sellers will have to be included in the database to avoid 'reverse bias' (as you pointed out) and to provide a good starting point for things. As I briefly touched on above, I believe more readily available books will need to form the basis of the search system so readers are able to come away with the name of a book or author they're likely to be able to get their hands on. And, as you pointed out, readers first need a book they've read to able to request texts like it - and these are usually the more popular works, either past or present. So, if the guys decided to build up their own database, I think they would have to begin with the best sellers and the classics to get this up and running any time soon.

Now, what I meant was I would discourage a feature that suggests a book primarily based on its current popularity, which is what I believe Aaron was suggesting. I'm referring to this sentence, specifically:

Maybe a different books will show up in your profile every week that's specifically chosen because it's well rated socially, but the opposite of your writing style preferences - might be an interesting grab bag.

This brings me to the related dilemma of whether or not to publish data concerning popular user searches. I'm undecided as to whether this is likely to end up mirroring the current best sellers list and assist in perpetuating the hype surrounding them; or whether it will help combat their current monopoly of the "hot literature scene" by bringing to people's attentions less well known works.

Has a Recommended Read section been discussed? Where members of the team would suggest books to the public. It's against your "we won't tell you what's good or bad" motto, I know, but there seems to be a Moderator's Recommendations on a lot of websites with large databases. Youtube for example. Just curious as to what you're thoughts are on the subject, Aaron and Paul.

[Matt Pascoe]

Just wanted to add another comment to this fun and interesting thread.

We need to also remember, no matter what the goals and desires for how this system could influence things, that it is still only as good as the data available to it.

Currently it suffers from lack of data at ~200 books.  At this point that is a totally understandable position to be in.  As the system grows there will always be an ebb and flow to how the results come back.  Until the system reaches a large and broad array of books there will probably a somewhat perceptible slant to things.

I'm thinking of the situation in that data is loaded from publishers who decide to share their books.  In a similar way, it could be easier to get "best sellers" into the system vs the obscure books (or the other way around who knows).  The point is that as the system is loaded, there will be certain influences due to the amount and types of data fed into it. 

This is not something that is hugely avoidable unless things work out that the data loads can be controlled in some way because there are multiple sources (Google, direct publishers etc) to choose from.  But if the willing participants trickle in it could come to a decision of loading the data at hand even though its not as broad as desired.  I think however there are acceptable "risks" (if you want to call it that) in this regard.

With all of that said, it's a matter of decisions that have yet to be made because sources of data have not been provided yet.  Having met with Aaron and worked with Paul and Brandon in a previous life, as well as the comments you see on this forum, I am more than confident that they will proceed appropriately.  The fact that they have opened this forum for just this type of discussion is a good testament to that fact.  As has already been alluded to, this is the world we live in and certain things are inevitable.  I'd rather have someone with an "Author" background instead of a "Publisher" background working on this! 

[Stephen Rollins]

I’m not sure I agree with the quantifying of literature; I don't think it can ever accurately be done. A book is not simply the sum of its words.

I had this thought when I first heard of BookLamp, though I didn't know how to phrase it and therefore post about it on here.  XD  To me, a book isn't something that you can measure based on the words within it; the important thing is how the story speaks to you.  I don't care so much about the difficulty or simplicity of reading a book if it contains certain elements I enjoy reading about.  But then, I also understand that there are people who would find something like BookLamp incredibly useful for finding new and enjoyable books to read.

Anyways, I'm still a bit skeptical about using a computer to measure out a book, however accurate it may turn out to be.  But then, it's impossible to satisfy everyone, right? 

I don't really have anything to say about the popularity issue... none of the books I read are ever best-sellers anyways.  I'm not interested in what 99% of the world is interested in. XD

[Al Green]

Matt,

I think your idea of publishers providing copies of their releases is an interesting one. It would certainly be beneficial for both parties: the publishing companies would get greater exposure for their books, but without comprising the objectivity of the system; and BookLamp.org gets a bigger database.

You would have to be careful what was included in any such deal struck with publishing companies, though. Naturally they are going to want to know the data surrounding how often their books show up in searches to determine whether their money is being well spent - assuming there is a financial component to the deal and it's in Booklamp's favour; and if not, publishers get curious too, right? That information in itself isn't dangerous; let them make of it what they will. I don't see any advantages they can gain from such information that couldn't be equally easily extracted from some basic market research. But, as stated before, there are other desirable snippets of information that may aid in monopolising all sorts of areas of the publishing industry.

Just a quick question: Are books that are yet to be released going to be included in the database? I think users will be interested in hearing about books that match their search criteria that are soon going to be available as much as those that already are, but it seems a little like promoting to me. 

[Aaron Stanton]

You have an interesting question about books that have not yet been released.  I think Paul (and you) may have touched on this earlier - that the books in the system have to be available to the reader in one form or another.  I think listing a book before it's available to be picked up at the library does smack a little of advertising; however, I suppose there could be a compromise - that as long as we have the data, we could introduce the book to the search the week it is released.

We've actually been contacted over the last few days by a number of different people that work in the publishing industry, and at least one publisher that seemed to be looking at the system from this perspective.  As I think they should - since we don't present any copyright issues and never expose the copyrighted data on a public server, there is little in the way of drawback to be listed in the database.  My only concern is that - because of the copyright battles that are taking place with companies like Google over full-text scanning programs - that the publishing industry may be a bit weary of such a program.

I personally look at BookLamp as a new bridge - a middle ground - between making a full text database useful to users in the way that Google does, and a way that the publishers can be happy with.  It's certainly worth noting that BookLamp has some basic copyright advantages.

Stephen, I wanted to address your comment directly, but it'll stretch this post too long, if I do.  So, I'll write that response tomorrow.    It's an interesting conversation.

Aaron

[Stephen Rollins]

I think books that aren't released should still be listed if you can do that with all books, not just certain ones.  It wouldn't be so much of an issue of advertising that way, since you're advertising for them all.

Stephen, I wanted to address your comment directly, but it'll stretch this post too long, if I do.  So, I'll write that response tomorrow.    It's an interesting conversation.

Yay!  That makes me feel special. 

[LEHorwitz]

A few thoughts about all of this.  I think that, as with many things, if you get any kind of volume at all some "chaos theory" is going to come into play and have organic and unpredictable outcomes that you cannot predict nor manage - so you need to let it go.

If the system is "pure" then, Pollyanna-ish or not, I think a lot of this will take care of itself.

From the writer's or editor's standpoint, writing is generally hard enough without ALSO writing to fit a complex algorithm - I suppose perhaps a computer could do a nice job of it, but that's another conversation.

From the publisher's standpoint - as interesting as the activity on this site may ultimately be, what they (in my opinion) are going to continue to look at is their sales numbers - while there may somewhere be some correlation, it's a gigantic leap to assume that a system designed as yours is will elicit purchasing activity that is significantly different from what exists independent of it - at worst/best, I would imagine, it would increase the sales of lesser known writers...but perhaps I underestimate the possible influence.

Lastly - I love the two features you suggested - the "wild card" books - recommend keeping them as an option for the open-minded.