Another perspective on judge's sheets

[FedoraDave]

When I received my scoresheets from Homebrew Alley VIII, I didn't just read them, I studied them. I compared the observations to my own recollections of what I saw, smelled, and tasted. I wanted to learn and improve.

With the Pearly Pils, both judges noted diacetyl. One noted a sulphur aroma, and the other noted a green apple aroma. Both judges recommended healthy yeast rate and temperature control.

I did not recall any off-flavors or aromas, but the recommendations made sense for any beer, especially a German Pils. So I resolved to make the recipe again, and concentrate on those two items, and to making a minor change in the hops bill, which I had already decided to do on my own.

I also talked to the guy at my LHBS about it, and he confirmed my process regarding D-rest and lagering were okay. He also pointed out that judges are human, everyone's different, and judging beer, while being mindful of style standards and guidelines, is still subjective to some extent. One thing he suggested kind of hit home, though, and I hope I'm not using this as an excuse for a score that was lower than I felt it should have been.

He pointed out that it's possible the fact that it's a lager puts the problem of diacetyl in their minds, and the power of suggestion takes over, so they're hypercritical and looking for anything at all to hang that particular hat on.

Maybe. And I hope I'm not adopting a "sour grapes" perspective, but I honestly didn't perceive off-aromas or off-flavors, nor did anyone else. I even compared it to a Victory Pilsner, and thought mine was really damn close to a commercial example used by the BJCP for style comparison. But I told myself that I'm not BJCP, so what do I know?

At any rate, I can only shrug my shoulders over this, and concentrate on my yeast health and my procedures.

[Funky Skunk Brewing]

I've ran into these same "issues" myself, Dave. How can one, two or even all three judges seem to find something in a beer when no one else can?

I went back through some old score sheets and noticed that judges that detect certain diacetyl notes, they have gone to the temperature control feedback as an option to perhaps resolve those issues. Though I go back to something I have always thought on judging and score sheets that if all the judges do not comment on something, it could be the "human factor" as well as what you mentioned in your original post from your LHBS is that a lager puts the problem of diacetyl in their minds and could be already supplanted in their minds that it is/will be present.

You're not getting a "sour grapes" approach, this is the same response I see and hear from a lot of brewers who send beer off to competitions. Although as you said, we're not BJCP judges so what do we know? I know that my beer doesn't seem to taste all that bad to me and others, so I must be doing something right here. Just providing us with some feedback to help tighten up our process to make better beer in the end.

What would be interesting is to send bottles to another competition (from the same batch as this one) to see if other judges perceive the beer has these same notes as these judges did. If they do or do not, you may get a better understanding of it was just "human" or there really is off flavors in the beer.

[mashani]

I don't know how much of this would apply, and honestly I think a judge should not be such a person to judge fairly (or at least not have their opinion when it comes to those specific flavors/aromas counted towards the score if nobody else tastes it) but:

There are people who are "super tasters" of some flavors and "super smellers" of some aromas. Someone like that might pick up diaceytl or acetaldehyde or smell Simcoe or Cluster or Citra and think "cat piss" where others do not share their impressions. I actually would be one of the acetaldehyde people, so if judging I personally would only want to consider it a "flaw" if other judges taste it. That's probably not really the way it's supposed to be done, but I know what my taste is like, and lots of folks wouldn't taste it where I do. But I am not a "cat piss smeller" and a bit of diaceytl doesn't bug me, as I drink lots of English beers that have some and I don't freak about it as being overwhelming.

[Beerlabelman]

You never know as it's so subjective. I'm sure your brew was fine. Maybe they served it too warm or something like that. Keep sending in entries & soon you'll get a winner.
I'll stop by your house & you can give me some to test. I'll send you the score sheet. No charge.

[FedoraDave]

@Mashani, it's possible some of the judges might be "super tasters". I've long thought my son is one, since the slightest bit of spice sends him over the edge. Or it may be that, with their experience and training, the judges are just more sensitive to certain aspects of beer tasting. We know from our own experience how our ability to taste things like hops and certain malts have changed as we become more mature and experienced brewers, and if you spend any time at all watching Food Network, you know how master chefs can detect all sorts of subtle flavors and textures in food.

There's no way I can know if what they detected was there in any amount; all I can say is I didn't detect it, nor did anyone I served it to around my circle.

@Beerlabelman, thanks for the kind offer! Thanks for the kind words, too, but getting a prize, while gratifying (and I've been there, so I know), and something I'll continue to pursue, isn't the be-all and end-all of entering competitions, for me. I learned a lot more from these "also-ran" entries than I did from the beer that won me a ribbon last year. It's the comments and recommendations that are valuable, because they'll help me evaluate and change my recipes/processes and get me brewing at a higher level. I intend to tweak my Bitter recipe and enter it again next year, just to see if my score improves and what kinds of comments I get on it.

[oly]

Mashani makes a very good point, although I would qualify it somewhat. Everyone perceives certain aromas and flavors differently, and in varying degrees. I personally have a tough time perceiving subtle esters, and I'm really bad at differentiating between different hop varieties, although I get the cat piss aroma from Citra hops big time (but since I'm aware that cat piss is a common characteristic of that hop variety, I would never nick the beer for it, unless it was overpowering, in which case I might suggest a greater variety of C hops in the recipe) . On the other hand, I've judged sours with Master judges who could not perceive enteric aromas or flavors when they hit me in the face like a baseball bat. Diacetyl is one of those characteristics that some people are hypersensitive to, and some not, although there are relatively few beer styles where it is not considered a flaw. Some judges mistake caramelization characteristics for diacetyl. Some can't differentiate diacetyl from butyric aroma/flavor (they're similar, but butyric has a rancid quality). It can be a real bitch when you have a beer that commonly has diacetyl in it, like a Bo Pils, and run into a judge that is either overly sensitive to it, or has not learned the style well enough to realize that (to a point) it is not a flaw in that beer. I'll even qualify that though. If you've had any of the Pilsner Urqell in the old style cans, you may have sampled a really bad batch of beer, with so much diacetyl that it was practically undrinkable. Even the well established breweries f..., er, mess up a batch now and then, although most will chuck it and cut their losses. But if a novice judge had only sampled that batch as his/her benchmark, he might think your beer was great, while the judge he is partnered with finds it hideous.

When you are reviewing your score sheets, look at the rank of the judge. You will usually get better input from a higher ranked judge, but not always. I recently judged with a lower ranked judge who was far better qualified, and had much greater perceptive abilities than I, for the category we were judging (witbier). It can be a bit of a crap shoot. I've scored a 42 and a 22 with the same beer, on the same weekend. No one wants to send a twelve pack of beer to various competitions, but sometimes, that is about what it takes to get a fairly accurate assessment of your beer. You try to be objective, but it's really not possible (Beerlabelman was right about that, as well as being right about winning a medal or ribbon sooner or later, unless you consistently brew undrinkable beer). That's why we have at least two judges for every flight, and try to arrive at a rough consensus. Judges are strongly encouraged to be within 5, sometimes 7 points of each other; so when the score sheets are filled out, there is generally some discussion of the score each judge has arrived at, and individual judges will often alter their score, both to bring the scores closer together, and often because one of them has pointed out features of the beer that the other did not notice as much originally. That's also the reason that you will sometimes have a judge who really likes (or is hypercritical of) your beer, but does not give a score that seems to reflect the feedback. Score sheets are filled out individually, with little to no conversation between judges (this allows each judge to note his impressions without being biased by the other), while the scores for each feature, and overall impression of the beer are sometimes worked up after the judges have conferred.

Bottom line is that some of you are making better beer than the people judging your beer do. Take the feedback for what it's worth, which is sometimes valuable, and sometimes not worth much. If you think you're making good beer, and people you trust to be honest agree, then you probably are. You'll win some and lose some, and sometimes with beers you would not expect.

Sorry for rambling. I saw this ancient post, and being under the influence of a kickin' habanero ale, had a hard time shutting up.

BTW, new style guidelines coming soon: http://www.bjcp.org/docs/2014%20BJCP%20 ... AFT%29.pdf

This will probably be used for most competitions by summer of 2015. I expect that the 2008 guidelines will still be used through spring, to give time for judges and competitors both to familiarize themselves with them.

[Inkleg]

Glad you decided to ramble. I'm not a judge myself, but have helped steward for a few and that is a spot on accurate account of beer judging.

[John Sand]

Thanks for the input. I have two brews in a competition in two weeks. We'll see.

[FedoraDave]

I'm glad I revisited this thread. I had missed Oly's contribution, and it was very enlightening, especially as I just delivered my entries to Homebrew Alley IX yesterday.

I enter this competition to get feedback, rather than win ribbons (although I'll take the ribbons, make no mistake). And as long as those comments give me something to think about, that's a positive thing. I still don't understand the scores I got, but I also have to take the other entries into consideration. They may have been really great examples of those styles, and by comparison, mine was lower.

Overall, I still prefer the BJCP competitions over the barbecue competitions I've been in. Those things have no standards or guidelines other than "it tastes good" for that particular judge. And I call BS on that.

[poodie]

Good comments by Oly.

Pull order also has something to do with it. I know I am best on beers 2-6/7. I am not warmed up on beer 1 and by the time I get to 6/7 my palate is shot, especially with hoppy styles. I usually note which number beer it is on the top of the score sheet (6 of 9) to give the brewer an idea. I have written "this is my 8th beer and my palate is shot" to let the brewer know. At the last NHC I judged over 40 beers on Saturday + 4 mini-BoS. Even though I tried I know my afternoon sheets were not as good as I wanted them to be but I had consumed a lot of beer. Again, I noted that on my sheets. I also put my email address on my sheets. I have yet to have someone contact me. It would be nice to receive some feedback on my sheets.

We do go through off flavor training by doctoring beer (usually budweiser) with off flavors. So, we have smelled/tasted most flavors. With that said we are only human. Most judges I know want to do the best they can for the brewer. We are brewers too. We know you have taken time and spent money to send us your beer. In general we want to do a good job, at least I do. Their are jerks in every crowd.

I have seen judges "search" for faults. That is not how I judge. If a fault hits me in the face then I will note it but I am not looking for anything to be wrong. I am trying to find what is right.

[gwcr]

We don't have a "Like" button so...^^^Like^^^

Great comments and good to hear from a judge. Hope all the ones who taste mine have the same mind set (although I know they don't).

[Inkleg]

Thanks for your input poodie and your contrabutioin of being a judge. I'm one that completely understands that judges are human, taste is subjective, there are good days and bad days and you can just get plum worn out on the same beer time after time.

Sorry that you've gotten no feed back from any of your sheets.
I entered a Brett IPA fermented 100% with White Labs WLP644 Brett Trois. It scored 39 in the speciality category, but got dinged pretty hard as an IPA, a 22. Both judges noted an "infection" in the beer and we're pretty descriptive about it, when they thought it might have happened and offered tips to help prevent it. I was rather impressed with their comments and suggestions. Like you, their email was on the sheets, so I sent them a note explaining how much I appereated them taking the time to judge and how well tuned their taste buds were, as the beer they had dinged had been brewed infected. They both responded and ask if I would send them a copy of the score sheets as having tasted way to many IPAs that day, they couldn't remember my beer exactly, but would love to see their comments on a blind judged Brett beer.

This brewer thanks you and all the BJCPs for your time.

[BlackDuck]

Nice comments poodie. Like Jeff said, the judges are human. Even though there is an attempt by the BJCP to be consistent across the board in judging, the beers are still being judged by people, all who have different perspective of things. As a good example, I entered a pale ale into the Ohio State Fair this year, it scored a 25.5. I entered the SAME batch into the Ohio Brew Week competition a few weeks later and it scored a 36.5 and won a silver medal in the category. Same batch with an 11 point diffence...just goes to show you that every judge tastes them a little different.

[MadBrewer]

Poodie da man. Just ask Mr.Bill's Mom.

I was going say similar to Olly, these guys are trained to taste and pick up on what we can't or do not know about.

But since we got an old thread going, what is your lager and D-rest technique Dave? Dry Yeast or Liquid, what kind of starter, what size pitch...etc. Pitching warm for a lager and then crashing it down to lager temps is a good ticket to diacytel or however it's spelled.

[Foothiller]

Many of us judges do try for consistency in what the numbers mean, but still there are differences. There are differences among what judges assign, but there are also differences in what you're competing against. In a recent competition(114 entries total), I got the interesting result that one of my entries took first in its category with a score of 32.5. Another took second in its category with a 34. Another was third place with a score of 39, with comments that some attributes were world class. My fourth entry was honorable mention with 38.5, in the same category as my third place. So, the scores were in opposite order of the placings -- just how the cookies crumbled that day. In all cases I did get good feedback from respectable judges.