Using the human mind for analysis?

[Daniel Bowen]

I can't sleep and thought of something that sounds interesting to me.  Assuming BookLamp builds up some momentum (I think that's a fair assumption) and has 'regular' users...  has BookLamp entertained the idea of having academics (or other credible persons) thoroughly analyze each book on literary structure, and whatever may be important to the subjective mind of a reader.  This is (at least) part of Pandora's analysis of music, tracking music on different variables such as 'use of 12/8 time signature,' and 'African Influences' - I think there's something like 600 musical attributes that are tracked (I found this on wikipedia, so that number could be completely wrong, but the idea is clear).  I'm not sure, exactly how the the mechanics of it all work, but they do have credible persons ("experts") who sit down with a song and analyze it.  It takes around 20 - 30 min. 

It does take much longer to read a book, so having in-house experts would be kind of unreasonable, at least for the time being.  There's two ways this could happen: the fast and slow way.  The fast way is that credible users hear about this and want to help; then it's easy, just find credible users and ask for their help.  Send them a book that matches their writing style preferences and wait for its return.  The slow way requires compensation, which is completely feasible also, but much, much slower.  Standards, with interpretive flexibility, will eventually need to be put in place.  To decrease the bias that one critic could have on the analysis, you can have the same book sent to two 'analysts' and have them review each others analysis - where they will be relating it to their own analysis as well as considering the others thoughts with respect to the text.  Maybe more than two will be necessary.

In Pandora's case, they don't have the luxury of page after page of hard information as well as the artistic aspect of the piece, they are relying solely on expert opinion, so I don't see any reason the same method couldn't be used here.  It's already being done, the medium (though more complex) is the only difference.

I'm curious what yall have to say...

[Jo Red]

I can't sleep and thought of something that sounds interesting to me.  Assuming BookLamp builds up some momentum (I think that's a fair assumption) and has 'regular' users...  has BookLamp entertained the idea of having academics (or other credible persons) thoroughly analyze each book on literary structure, and whatever may be important to the subjective mind of a reader.  This is (at least) part of Pandora's analysis of music, tracking music on different variables such as 'use of 12/8 time signature,' and 'African Influences' - I think there's something like 600 musical attributes that are tracked (I found this on wikipedia, so that number could be completely wrong, but the idea is clear).  I'm not sure, exactly how the the mechanics of it all work, but they do have credible persons ("experts") who sit down with a song and analyze it.  It takes around 20 - 30 min. 

It does take much longer to read a book, so having in-house experts would be kind of unreasonable, at least for the time being.  There's two ways this could happen: the fast and slow way.  The fast way is that credible users hear about this and want to help; then it's easy, just find credible users and ask for their help.  Send them a book that matches their writing style preferences and wait for its return.  The slow way requires compensation, which is completely feasible also, but much, much slower.  Standards, with interpretive flexibility, will eventually need to be put in place.  To decrease the bias that one critic could have on the analysis, you can have the same book sent to two 'analysts' and have them review each others analysis - where they will be relating it to their own analysis as well as considering the others thoughts with respect to the text.  Maybe more than two will be necessary.

In Pandora's case, they don't have the luxury of page after page of hard information as well as the artistic aspect of the piece, they are relying solely on expert opinion, so I don't see any reason the same method couldn't be used here.  It's already being done, the medium (though more complex) is the only difference.

I'm curious what yall have to say...

I think the centerpiece of booklamp is the algorithm, and so the idea of human analysts is probably not an alternative.  IMHO.

[Daniel Bowen]

I would think of it as supplemental to what they are already doing, not an alternative.

i agree, but i don't mean to have a written analysis made about the literary aspects of the book.  What I mean is to track things like "use of irony" or "use of "sarcasm," etc.  For example, it is entirely possible that you can read a book that uses absolutely no sarcasm, where you can also read a book that uses a lot - the two extremes.  So what's the difference?  It's in what's written on the page... but irony and sarcasm are quasi-subjective, so who's opinion will the public trust to say what is 'ironic' and what is 'sarcastic'?  Someone with credibility.  Then, with a credibly minimized error, BookLamp could quantify these literary concepts in the same way that it does Pacing, Action, etc.  Whether or not this seems possible probably requires some imagination, because it would be a very long, arduous project and as you said, "the idea of human analysts is probably not an alternative" right now - again i think a better word is to use is 'supplemental.'

I think this falls along the same lines as the thematic elements that are mentioned in the video - "all of your books occur on the water" or "they all have spaceships" - but i can imagine ways that those can be tracked using strictly their algorithmic systems.

I'm a graduate student studying economics, and spent a lot of time thinking about how to use Pandora's data, but I found the more I think about what data could be available from BookLamp the more I find that its much different, and I think this could solve the problem.

[Daniel Bowen]

Another thing... a graph of "use of irony," for example, would make no sense.  But it's the aggregate value across all scenes that's useful.

[Daniel Bowen]

... jo i just read your signature, and from reading other posts of yours I think it fits your view on this whole deal pretty well.  Good Job.  Sounds like something a wordy accountant would say about life.

[Aaron Stanton]

There are a number of different things we are and have considered as supplemental elements to BookLamp's recommendations - human input is certainly ultimately going to be a contributing factor.  Jo's mostly right in this instance, though - BookLamp's primary focus will most likely always be our algorithms, which are then strengthened by other information.  In that regard, we have actually sat down with a number of professors in figuring out our formulas, and will probably continue to do that as we progress.  We also use a number of other techniques to evolve our formulas backwards.

Before I get to the rest of it, there's one overreaching reason that algorithms will most likely always hold sway on BookLamp, and it's simply that we have access to much greater data than any single human reviewer can hope for.  The algorithms used by BookLamp can be dynamic, meaning that they're specific to each user derived from the data on books they've read in the past.  Those custom formulas can be applied to the entire database at once, and modified as we gain more information.  A human review would have to be very carefully done in order to achieve the same level of flexibility and feedback.

My concern with using human reviewer is time, completeness, and bias.  My background is in what's called I/O Psyc., and I've worked a fair amount with conducting studies.  Training your raters, as you would have to train your professionals, is easily where you spend the majority of your time.  If two professionals read the same book and rate it differently, then anything they rate can't be directly compared with any level of sensitivity - there has to be an objective element or the matching doesn't work.  Algorithms have the advantage of being equally applicable to every book at once.

And then things like time and such does play a huge factor.  There are advantages to using human reviewers, because human reviewers do many things better than computers, and we're building systems to both improve our algorithms and gracefully combine human feedback without throwing off the objectivity of the system.  It's an interesting idea, and one worth exploring.  We'll have to chat about it more in detail when we get a chance. 

You can rest assured that integrating the knowledge of people that are experts in the field of literature is certainly of high priority for us - we're already doing that.  Whether it's using them to improve the automatic system by developing new formulaic approaches, or through direct analysis, or some combination of those, well... the exact mix is still developing. 

Aaron