Blacklist Secret Shopping Experience

Collapse

Announcement

Collapse
No announcement yet.
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Re: Blacklist Secret Shopping Experience

    Originally posted by Bitter Script Reader View Post
    They're explicitly told NOT to judge on that criteria. They don't second guess what other people MIGHT think of the script. They are to evaluate it based on their own professional judgement.
    They may not be told to second guess what other people MIGHT think of the script but I'm sure they do. I'm sure they second guess how they'll be judge themselves for recommending this or that script. Or whether they'll get fired for championing a script that pisses most people off. It's human. Everyone is looking to save face on some level. Our egos will have it no other way.
    Advice from writer, Kelly Sue DeConnick. "Try this: if you can replace your female character with a sexy lamp and the story still basically works, maybe you need another draft.-

    Comment


    • Re: Blacklist Secret Shopping Experience

      Originally posted by MJ Scribe View Post
      But how does one reader fathom what might be viably commercial for a myriad of some 2300 industry folks while reading/analyzing scripts.

      For that reason, I echo Geoff's & Margie's thoughts as I mentioned earlier...
      Which is exactly what Franklin has spent the last 400 pages of this thread saying. A lot of criticism has been that though a reader might not rate a piece, an industry pro would, so a reader should think of whether someone might want to produce it. As Franklin has repeated, a person cannot legislate for the opinion of others so can only go off their own judgement.
      M.A.G.A.

      Comment


      • Re: Blacklist Secret Shopping Experience

        Originally posted by UpandComing View Post
        Actually, they put Sanjaya through to the Top 12, so this is not the best analogy
        Yeah, but Sanjaya was also utterly unlike anyone out there and he was getting attention in a way that suggested an audience might find him compelling. He's the crazy polarizing outlier.

        If you must think of him in screenwriting terms, he's the BALLS OUT of American Idol performers, loved and hated in equally intense measures for being such a contrast to the norm.

        Comment


        • Re: Blacklist Secret Shopping Experience

          Originally posted by Bitter Script Reader View Post
          Yeah, but Sanjaya was also utterly unlike anyone out there and he was getting attention in a way that suggested an audience might find him compelling. He's the crazy polarizing outlier.

          If you must think of him in screenwriting terms, he's the BALLS OUT of American Idol performers, loved and hated in equally intense measures for being such a contrast to the norm.
          Okay. Then based on my polarizing score history, I am the Sanjaya of the Black List, and should be getting a deal any day now
          "I love being a writer. What I can't stand is the paperwork.-- Peter De Vries

          Comment


          • Re: Blacklist Secret Shopping Experience

            I don't know if there's a proper end to this debate.

            The site has its rating system and it's got a big element of the subjective, because that's inherent in any creative assessment.

            They went with 1-10, which personally I don't mind. It's got to be more informative than 'pass, consider, recommend,' which is restrictive and doesn't give as much flexibility.

            That said, a document explaining to the readers (and writers) what each number means can't hurt, and for all I know, already exists.

            Comment


            • Re: Blacklist Secret Shopping Experience

              For me a quality to look for in a pro-reader would be the ability to evaluate scripts beyond the narrow field of their own opinion. The industry shouldn't be recruiting readers for their opinions, they should recruit for insight and expertise.

              To be fair, I'd guess many experienced readers do exhibit these qualities and that reading numerous scripts would have the effect of teaching these skills. Hopefully the Black List is able to cherry pick these higher quality readers.

              I imagine most of us have encountered readers who allow their opinions to override their evaluation of a script. I'm sure I've done it myself! Often these reviews take the form of "The script you should have written is...-

              Sometimes those will be valid opinions and you'll come to see that a major rewrite makes sense. Other times your response is "No, it's not, because...- and you'll conclude that you the artist made a valid artistic choice. You are then at an impasse with that specific reader.

              In the case of Jeff's feedback, the lower scoring reviewer is stating that the script he should have written is one in which the social attitudes of key players are more acceptable in a modern context. Jeff doesn't agree and has made a good case for what he has chosen to do.

              So it comes down to the unanswerable question of who is right? Jeff has earned the right to say, "I am.- The rest of us have to take our lumps with a $50 hole in our pocket.

              As a reviewer it is a true challenge to first accept the writer's intent. Anyone can second guess artistic choices based on the choice they would have made. The person who has put the brutal work in is entitled to a default assumption that their intent is valid. To base an evaluation on an assessment that you know better than the writer you need to be able to explain the flaw or mistake, rather than provide an alternate opinion.

              Suggestions to make things better are fine. Dinging a project based on your opinion when you're second guessing the writer is not.

              Comment


              • Re: Blacklist Secret Shopping Experience

                Crazy thread.

                I think the last few pages are very interesting, and I like where Geoff is headed with his comments.

                I don't have a problem with the 1-10 scale. I did have a problem with is when there are more than the average point spread on a high-low basis for reviews, but now I've accepted that my script can be polarizing.

                I had one review that clearly (at least to me) the reader was ultra-sensitive to the violent nature of my spec. But even in that review, the reader had a lot of really good things to say about it, too-- which made the rating somewhat confusing.

                What I'm not sure I believe yet, is that a '7' doesn't (generally speaking maybe) get lost statement because I did receive a 7 for my spec on 1/27 and the result:

                4 pro views on 1/28
                1 pro view on 1/29
                1 pro view on 1/30
                Downloads: zero

                That's two full weeks. Not sure how long it should take. My spec is flagged as a high concept, so I don't know 'how' a spec doesn't get lost if there's nothing to notify the membership that there was a spec that received a new review and it was a 7.

                Honestly, I'm not that upset about not receiving downloads, because the reviews have been insightful and I'll be rewriting this spec when the last review comes in-- most likely this week. Then I'll give another wackattit-- to see if it's actually ready to be queried. I don't want anyone taking a look at it until it's really ready, you know?

                Admittedly, and in fairness to Franklin Leonard, I do not even pretend to understand the algorithms they use-- so maybe the bases are all covered.

                It might be nice to consider having a list link that can be clicked on that lists scripts with new ratings/reviews. I can't think of the arguments against that, though I'm sure there are some.

                So, bottom line is that I do think the site is worth my money and my time-- do I think it's perfect? No. Do I think it can be improved-- sure.

                FA4

                Post Edit: I think that Howie428 (above) has some very valid comments.
                Last edited by finalact4; 02-09-2014, 01:43 PM. Reason: post edit
                "Arguing that you don't care about the right to privacy b/c you have nothing to hide is no different than saying you don't care about free speech because you have nothing to say." -- Edward Snowden

                Comment


                • Re: Blacklist Secret Shopping Experience

                  Originally posted by LIMAMA View Post
                  Re Jeff's review (and I haven't read his script, so I can't comment on that one way or the other, Susan C I hate you)...

                  I have done many, many critiques/notes/reviews, both paid and unpaid. Probably the most important lesson I learned is the ability to recognize a script that is great but not one that I would necessarily want to see and give it high marks. Maybe the subject matter was polarizing. Maybe it left me uncomfortable. Upset my comfort zone. Challenged me. So I have to put my personal prejudices and feelings aside and still say with passion and conviction, this is great and it has a place. It might not be for me, but it will be for someone else. It shouldn't receive a low score because the reader wouldn't pay to see it. If it's great, it should stand alone.

                  In my opinion, it's not acceptable for a reader to say, I like comedies, therefore only give me comedies to crit. A good reader should be capable of handling all genres and give constructive, meaningful comments even if the script or subject matter is not their cup of tea.
                  Fundamentally disagree.

                  An exceptional reader with a great deal of experience may have developed a varied and deep enough knowledge base to deliver insightful notes across all genres, but there's a big difference between what makes a horror film work and what makes a romantic comedy work. Sure there's overlap, even significant overlap, but there's definitely specialized knowledge involved in each genre. There's absolutely no need to put a writer at a disadvantage by having a reader read a script they're likely to be predisposed against when it can be avoided.

                  That's why our readers aren't asked to read genres in which they have no interest. At a minimum, we try to give the writer the best opportunity to have their script liked.

                  Comment


                  • Re: Blacklist Secret Shopping Experience

                    Originally posted by Anagram View Post
                    That said, a document explaining to the readers (and writers) what each number means can't hurt, and for all I know, already exists.
                    Exactly. I asked this almost a year ago in another Blacklist thread; is there a standard that the readers use when assigning scores. It seemed to me the answer was "Not really." We writers are okay with a numerical value being assigned to our script, as long as ALL the readers are grading on the same curve. As an example, I read a script that received a hiigh rating on the Blacklist. There was no way I would have rated that script so high. Based on this, I was sure the script I subsequently uploaded would receive a 10. It got a 6. I'm sure it deserved a 6, (it was an early draft) but I promise you that my reader judged on an entirely different scale than the reader of the high-scoring script. But if the readers are not told, 6 means this, 7 means this, etc. but instead are told to quantify a feeling, it's no wonder there is so much confusion and disparity in the scores. That is why we writers are asking for a more clearly defined rating system (consider/pass etc.) It's not so much so we can understand the rating system, but so we can have confidence that the readers do.

                    Also, everybody is saying: See, the system works, Jeff got a 9. But what if he only purchased one evaluation, the one that came back a 6. Wouldn't people be saying the process needs to be fixed?

                    Comment


                    • Re: Blacklist Secret Shopping Experience

                      Originally posted by Howie428 View Post
                      For me a quality to look for in a pro-reader would be the ability to evaluate scripts beyond the narrow field of their own opinion. The industry shouldn't be recruiting readers for their opinions, they should recruit for insight and expertise.

                      To be fair, I'd guess many experienced readers do exhibit these qualities and that reading numerous scripts would have the effect of teaching these skills. Hopefully the Black List is able to cherry pick these higher quality readers.

                      I imagine most of us have encountered readers who allow their opinions to override their evaluation of a script. I'm sure I've done it myself! Often these reviews take the form of "The script you should have written is...-

                      Sometimes those will be valid opinions and you'll come to see that a major rewrite makes sense. Other times your response is "No, it's not, because...- and you'll conclude that you the artist made a valid artistic choice. You are then at an impasse with that specific reader.

                      In the case of Jeff's feedback, the lower scoring reviewer is stating that the script he should have written is one in which the social attitudes of key players are more acceptable in a modern context. Jeff doesn't agree and has made a good case for what he has chosen to do.

                      So it comes down to the unanswerable question of who is right? Jeff has earned the right to say, "I am.- The rest of us have to take our lumps with a $50 hole in our pocket.

                      As a reviewer it is a true challenge to first accept the writer's intent. Anyone can second guess artistic choices based on the choice they would have made. The person who has put the brutal work in is entitled to a default assumption that their intent is valid. To base an evaluation on an assessment that you know better than the writer you need to be able to explain the flaw or mistake, rather than provide an alternate opinion.

                      Suggestions to make things better are fine. Dinging a project based on your opinion when you're second guessing the writer is not.
                      This is brilliant. And the paragraph I BF'd is so good I think everyone, even those who simply give notes to friends, would do well to keep it in mind.

                      It's hard enough in an industry where non-writers have the final say on what gets developed or produced. Readers -- who are writers themselves, more often than not -- need to give the writer the benefit of the doubt.

                      One problem I see is that readers, for the most part, should be paid more to do what they do. So that they're compensated for the time needed to mull over the writer's intent and make a truly fair call.

                      For example, in Jeff's script (which had a number of Whoa! moments for me), there was one scene that made me so uncomfortable my knee-jerk thought was, "Do we really need this scene?" And I read it again. And then a third time because that's what it took for the discomfort pass and I realized, "Ah - I see exactly what he's doing. Yes. the scene works."

                      I'll be honest, though, if I was getting paid $25 per read and had a stack of scripts to get through PLUS was tasked with writing a cogent assessment of the script, I may have gone with my knee jerk reaction and moved on.

                      It sort of boggles my mind, in an industry where so much money is risked, readers, the first filter of material, get paid so little.
                      Advice from writer, Kelly Sue DeConnick. "Try this: if you can replace your female character with a sexy lamp and the story still basically works, maybe you need another draft.-

                      Comment


                      • Re: Blacklist Secret Shopping Experience

                        Originally posted by Bitter Script Reader View Post
                        "Achieved a minimum of quality in total execution?" Wow, where do I sign up?

                        "Here, read this. It's more or less adequate." Try telling that to a boss and seeing how quickly they jump to read it.

                        The site is designed to shine a spotlight on the best of the best and to match industry pros with scripts they might be looking for. The superlative scripts get the extra nudge of an email blast, but all scripts are still visible and searchable.

                        And as Franklin has pointed out, a number of success stories underline the fact that it's possible to be discovered even with ratings below a 6. And the top lists don't have an "8 or above policy."

                        It's pretty clear than a 8 is the threshold where a reader says, "I put immense support behind this. This WILL be worthy of your time."

                        They don't put people through to the Top 36 on American Idol just because they sing all the notes in the right order.
                        You've completely missed my point. I'm saying that *assuming* a certain level of execution, the job of the reader should be to put it in play regardless of personal biases.

                        Comment


                        • Re: Blacklist Secret Shopping Experience

                          Originally posted by sallain View Post
                          Exactly. I asked this almost a year ago in another Blacklist thread; is there a standard that the readers use when assigning scores. It seemed to me the answer was "Not really." We writers are okay with a numerical value being assigned to our script, as long as ALL the readers are grading on the same curve. As an example, I read a script that received a hiigh rating on the Blacklist. There was no way I would have rated that script so high. Based on this, I was sure the script I subsequently uploaded would receive a 10. It got a 6. I'm sure it deserved a 6, (it was an early draft) but I promise you that my reader judged on an entirely different scale than the reader of the high-scoring script. But if the readers are not told, 6 means this, 7 means this, etc. but instead are told to quantify a feeling, it's no wonder there is so much confusion and disparity in the scores. That is why we writers are asking for a more clearly defined rating system (consider/pass etc.) It's not so much so we can understand the rating system, but so we can have confidence that the readers do.

                          Also, everybody is saying: See, the system works, Jeff got a 9. But what if he only purchased one evaluation, the one that came back a 6. Wouldn't people be saying the process needs to be fixed?
                          THIS.
                          "I love being a writer. What I can't stand is the paperwork.-- Peter De Vries

                          Comment


                          • Re: Blacklist Secret Shopping Experience

                            Originally posted by Geoff Alexander View Post
                            However, yours is in fact a matchmaking venture, scripts to people. But, you aren't trying to find the match between the right script and the right person, you are trying to find a script which could be a match and then broadcasting its qualities, so that the right person will have the opportunity to find it, correct?
                            Actually we do both, but the only way to do the latter is by assessing the script in the only way that I believe is legitimate - by having readers evaluate based on their point of view, not the imagined point of views of over 2000 industry professionals (which only introduces considerably more opportunity for error.)

                            Originally posted by Geoff Alexander View Post
                            Claiming that opinions of art can only be subjective is an interesting philosophical position, but the way you have constructed the BL isn't entirely consistent with the position, i.e., you tout the fact that your readers are qualified. If training or qualification can create an outcome that reflects "quality", then you are not actually looking for 100% subjectivity.
                            I'm not sure this tracks. Are you saying because we opted to hire readers with industry experience that that somehow implies that we don't actually believe that subjectivity exists in art? We aren't looking for subjectivity. It's a fact of existence.

                            We hired the readers we did because we believe that with at least a year's experience reading scripts in a professional capacity, our readers would have read considerably more screenplays than the average person on the street and at least enough to know what works for them and what doesn't.

                            Furthermore, they're likely to have read the same scripts that many of our industry professionals have, which means that there will be considerable overlap amongst the scripts they've rated (the fact of rating them, not the numbers themselves necessarily) which will allow our recommendation algorithm to more accurately predict which uploaded scripts industry pros are likely to like.

                            Finally, I don't believe that I've ever used the term "qualified" to describe our readers because that would imply that I'm somehow the adjudicator of who is qualified or not to read scripts in the industry.

                            What I have said is that they're experienced, all having at least one year at a major reputable Hollywood company wherein reading scripts was a significant part of the job, which they are and have.

                            I've said that they have strong critical and analytical reading and writing skills, which they do.

                            Originally posted by Geoff Alexander View Post
                            So, if the above is true, that the BL is set up in order to identify "quality" material, then you need to do everything possible to further the goal. This means requiring your readers to the greatest extent possible to put aside personal taste and to work to objectively identify scripts which could find a fan in the production community that could get the material made. Just that simple. That doesn't require the reader to "know the minds" of the professional community at all, as the BL simply puts the project in play and allows the right match to find the material.

                            If Jeff's script is "good enough" to have a shot, that is, if it has achieved a minimum of quality in total execution, then the reader, IMO, should have the duty to put it in play. It appears that this was the case with Jeff's script, as per the reader's description of the material and the disconnect with the scores over a single issue, so the fact that it was scored to low to put it in play looks to me like a failure on the part of the reader.
                            I think I've already established that the above isn't true so I don't know that I need to rebut this conclusion.

                            Respectfully, the reader would have "put it into play" if they thought that it had reached a level of execution that would merit them recommending it enthusiastically to a peer or superior in the industry. It didn't in their minds, so they didn't. It's that simple.

                            Go back and read the actual weaknesses. There was more than one issue, as I've explained several times in this thread.

                            Comment


                            • Re: Blacklist Secret Shopping Experience

                              Originally posted by madworld View Post
                              I really don't either. I was just illustrating I think a pass/consider/recommend could be a helpful accompaniment, particularly in situations where the numbers are disconnected from the reader's description of the material. Maybe it's just my vestigial instinct for the established system. But hey, it could be time for a new one. There are lots of things that need fixed in the movie business.

                              Truly, it doesn't matter, especially to me. Franklin's site, Franklin's rules. Very happy for the people who have gotten 8s. Hopeful for the people who haven't - yet.
                              If it makes people feel any better, think of the #s like this:

                              9-10 - Recommend
                              8 - Consider
                              1-7 Pass, in varying degrees

                              Comment


                              • Re: Blacklist Secret Shopping Experience

                                But the real question is, how ugly is the person doing the rating?

                                Are they really qualified to be judging and rating others?

                                1 year industry experience = not very qualified.

                                At some point people will start to ask, WHO ware these BL readers to judge my work? And are they qualified?

                                As JL has demonstrated, not everyone who uses BL is total newbie. Some have much more industry experience than FL.

                                Originally posted by Rochjeff View Post
                                If someone told me that I was a 6, I would not think that I was Halle Berry or Brad Pitt in the looks department. But, I wouldn't necessarily know if they were saying that I was hideous, moderately attractive, or average.

                                Clearly 9, 10 and sometimes 8s are easier to gauge on any scale. 5-7 (not so much). Hence why there's a need for a clearer relationship between the scores and a recommendation. Yes, a 6 is higher than the average on the site, but that could still mean that my work is shitty. But, if someone gives me a 6 and a consider w/reservations, I get a better sense of where I stand. Even an 8 is unclear. Is it a recommend or a consider?

                                Most studies that ask for numerical evaluations also have definitions associated with those numbers.

                                Comment

                                Working...
                                X