New Black List Thread - Franklin Leonard answers your questions

[Bitter Script Reader]

Re: New Black List Thread - Franklin Leonard answers your questions

So you're telling me you know exactly how a reader is gonna rate a script as soon as it's assigned to said reader?

If not - then then the reaction is random or to use a synonym unexpected or arbitrary. By your own description of the process, there is no methodology for the readers to follow. Absent methodology, each reaction is by definition random. Every evaluation simply boils down to that individual reader's subjective taste, no? And that can easily differ between two readers. Which is exactly what happens at any prod co, contest, or anywhere you send a script.

Because that's what I'm talking about -- not any specific situation -- but how in general, no one can know how the reader is going to react to a script. Thus, random. Just pointing it out as a reality of the business.

A pitcher tosses one across the plate. The instant it leaves his hand, can you predict the outcome?

If you can't predict the outcome, does that mean the outcome is so random that it's just as likely he'll plant it in the strike zone as he'll bean the batboy? And even if we start with the assumption that he will always plant it in the strike zone, can we always predict how the batter will swing. If he gets a strike on one pitch and then puts it in the bleachers on the next pitch, would we call that result "random?"

Extrapolating further, does that mean that every play in baseball is random? If so, does that not mean that the outcome of the entire game is random? And if so, does that not mean everything stemming from that - from playoff eligibility to the result of the World Series - is random?

ran*dom, adj - made, done, happening, or chosen without method or conscious decision.

[Alfred Parker]

Re: New Black List Thread - Franklin Leonard answers your questions

We're not talking about low score from one or a high score from another. We're talking about near identical scores from two consecutive readers. Not exactly a random walk.

So you're telling me you know exactly how a reader is gonna rate a script as soon as it's assigned to said reader?

If not - then then the reaction is random or to use a synonym unexpected or arbitrary. By your own description of the process, there is no methodology for the readers to follow. Absent methodology, each reaction is by definition random. Every evaluation simply boils down to that individual reader's subjective taste, no? And that can easily differ between two readers. Which is exactly what happens at any prod co, contest, or anywhere you send a script.

Because that's what I'm talking about -- not any specific situation -- but how in general, no one can know how the reader is going to react to a script. Thus, random. Just pointing it out as a reality of the business.

[Jai Brandon]

Re: New Black List Thread - Franklin Leonard answers your questions

Apologies for the delay. This one was escalated to me personally and between travel issues getting to the East Coast from Sundance and then a brief bout with the Sundance flu, I was a bit behind. Check your inbox.

Another thumbs up for the Blacklist customer service! Thanks, Franklin.

[JeffLowell]

Re: New Black List Thread - Franklin Leonard answers your questions

Franklin, I've got a quick question. Can your pro members rate scripts they haven't downloaded?

[FranklinLeonard]

Re: New Black List Thread - Franklin Leonard answers your questions

Hmmm... Maybe the Sundance FF is responsible for the delay. Thanks, guys!

Apologies for the delay. This one was escalated to me personally and between travel issues getting to the East Coast from Sundance and then a brief bout with the Sundance flu, I was a bit behind. Check your inbox.

[FranklinLeonard]

Re: New Black List Thread - Franklin Leonard answers your questions

Franklin, what's the distribution of pro ratings?

I'm guessing pros probably don't give out as many very low ratings (1-3) as readers, since they would probably stop reading scripts they dislike that much (and then presumably not rate them).

On the other hand, if they really like something enough to give it a 9 or a 10, that means they might actually want to sign the writer, in which case why alert other pros that the writer has talent?

Anyway, it would be interesting to see how the pro ratings distribution compares to the reader ratings distribution.

If you could then also show the reader ratings distribution for only those scripts that have been rated by pros (to make it a more direct comparison, since pros will only tend to read better scripts), that would also be interesting.

Should be able to get you numbers on this in the next few days. Anecdotally, the mean rating of hosted scripts by industry pros is higher than that of our readers, largely because they're less likely to download, read, and rate the scripts that receive low ratings from our readers.

As far as rating scripts go, there's ample incentive for industry pros to rate scripts either good or bad: the more data we have on their personal taste, the better recommendations they'll receive from our recommendations algorithm, which is quite powerful. (Created with the advice of two Netflix prize winners and recently updated with the help of Sean Owen, founder of Myrrix and current VP of data science at Cloudera.)

[FranklinLeonard]

Re: New Black List Thread - Franklin Leonard answers your questions

Hey Franklin,

Thanks for the quick response. Do you know a typical reason why that might occur? They accidentally deleted the first copy? Just curious.

If people read the script online, that counts as a download. It's possible that they started reading it, didn't finish, and returned to it later (which then gets counted as a non-unique download.)

[AP Oz]

Re: New Black List Thread - Franklin Leonard answers your questions

Franklin, what's the distribution of pro ratings?

I'm guessing pros probably don't give out as many very low ratings (1-3) as readers, since they would probably stop reading scripts they dislike that much (and then presumably not rate them).

On the other hand, if they really like something enough to give it a 9 or a 10, that means they might actually want to sign the writer, in which case why alert other pros that the writer has talent?

Anyway, it would be interesting to see how the pro ratings distribution compares to the reader ratings distribution.

If you could then also show the reader ratings distribution for only those scripts that have been rated by pros (to make it a more direct comparison, since pros will only tend to read better scripts), that would also be interesting.

[CthulhuRises]

Re: New Black List Thread - Franklin Leonard answers your questions

Correct.

Hey Franklin,

Thanks for the quick response. Do you know a typical reason why that might occur? They accidentally deleted the first copy? Just curious.

[FranklinLeonard]

Re: New Black List Thread - Franklin Leonard answers your questions

My script has 2 pro downloads, 1 reader download, but 4 non-unique downloads. Does that mean the reader (or one of the two pros) downloaded it twice?

Correct.

[CthulhuRises]

Re: New Black List Thread - Franklin Leonard answers your questions

My script has 2 pro downloads, 1 reader download, but 4 non-unique downloads. Does that mean the reader (or one of the two pros) downloaded it twice?

[FranklinLeonard]

Re: New Black List Thread - Franklin Leonard answers your questions

What it does do his highlight the Russian Roulette nature of Black List evaluations. And any coverage or evaluation service for that matter. You're not being evaluated by the "Black List" -- you're being evaluated by an individual reader who is randomly assigned your script by chance. Luck of the draw equals maybe a low score from one or a high score for another. But that's all it is -- one person's reaction.

We're not talking about low score from one or a high score from another. We're talking about near identical scores from two consecutive readers. Not exactly a random walk.

[Exponent5]

Re: New Black List Thread - Franklin Leonard answers your questions

For those who have submitted complaints to support, how many days does it typically take for the ticket to be addressed?

Same for me - very fast response and turnaround.

[Jai Brandon]

Re: New Black List Thread - Franklin Leonard answers your questions

I had a response within a few hours. The second review was done about 4-5 days later I believe.

I received a response within a day and the review in the same 4-5 day turn around-- very professional.
FA4

Hmmm... Maybe the Sundance FF is responsible for the delay. Thanks, guys!

[finalact4]

Re: New Black List Thread - Franklin Leonard answers your questions

I had a response within a few hours. The second review was done about 4-5 days later I believe.

I received a response within a day (could've been hours) and the review in the same 4-5 day turn around-- very professional.
FA4