If you were testing judges before 2020, this will serve as a refresher for you, because the process is almost exactly like the old process was. If you weren’t testing judges back then, this guide will tell you how to do so.
Broadly, all Judge Foundry exams for promotion are closed-book, in-person, untimed tests. We believe that this type of test is superior to the alternative, because it lets us test knowledge, not ability to google knowledge, and can be calibrated for “These are the things we expect you to know without having to look them up,” understanding that judges should look up the answers to questions that they don’t know.
For judges for whom in-person testing would be oppressively difficult due to geography, we can make accommodations, just as with the interview process. Contact joe.klopchic@judgefoundry.org if you believe you would not be able to proctor an in-person exam for your judge candidate.
Step 1: Interview the Candidate
For Levels One, Two and Three, as well as Event Admin certification, the interview should precede the test. Knowledge is just one part of these certifications, and we don’t want to create pressure on interviewers of “C’mon, I passed the test, so I’m basically a judge already, and it wouldn’t be fair to not pass me on the interview.” Note that the interviewer doesn’t have to be the same person who proctors the test, but most often will be.
If you’re satisfied that the candidate meets the requirements…
Step 2: Verify that the Candidate is a Judge Foundry Member.
Judge Foundry exams can only be given to Judge Foundry members. Candidates will need to become members, including paying the pro-rated first month dues. It’s likely that anyone taking the L2+ exams is already a member, unless something weird happened, so you can go right to…
Step 3: Create the Exam
Once the candidate is a Judge Foundry member, you can create an exam for them. Go to Exams > My Exams on the right sidebar. Then click “Create Exam” for the appropriate test.
Note that the exams are labeled with the level to which the member is trying to advance – the L1 test is the test to become an L1, not the test L1s take to level up to L2!
Enter the name of the candidate in the picker. If the name isn’t appearing on the list, verify with the candidate that they joined Judge Foundry and paid their dues. If the name still isn’t showing up, ask the candidate to go to the Account Assistance page. Once you have the candidate name loaded up, click “Create Exam.”
Take care when creating exams right around the time of a set release, as creating it locks the questions in their current state, and the candidate will miss any rules and policy changes. It might be best to only create the exam right before you’re ready to…
Step 4: Print the Exam
Once you’ve created the exam, there’s a confidentiality reminder. This is very important, so please don’t skim it! It’s an incredible amount of work to create an exam, and if questions get leaked, that invalidates the efforts of our volunteers and reduces the integrity of the process, and ultimately, the value of the judge brand.
After reading that statement, print the exam and answer sheet for the candidate. You can also print an answer key for yourself. Don’t click that last link yet – we’ll get there soon!
Now, find a time to meet your candidate so that…
Step 5: The candidate takes the exam
Find a quiet place for the candidate to take the exam. You don’t need to hover over the candidate during this process (indeed, it’s probably better if you don’t), but just be available for clarifying questions. Many judges took their exams at an LGS while their mentors were playing in FNM, and that’s a great way to do it – none of these exams should take too much longer than your average event, and that gives you a way to be accessible without being bored. Stores often also have printers, which comes in handy if you don’t.
You can answer questions the candidates might have where the question is ambiguous, but don’t give away the answers. If you hit a question where the candidate needed clarification, note that to give feedback later, since it will probably also be ambiguous for others.
Have the candidate mark all answers on the answer sheet. Tell them that that is the canonical source you’ll be evaluating, so while they can write all over the exam itself, only the answers they ultimately mark on the answer sheet will be considered final when you…
Step 6: Score the exam
Compare the answer sheet and answer key. A quick tip: if you printed out the answer key, you can just overlay it with the answer sheet and hold them up to the light to get a ballpark idea of how your candidate did.
Walk through each incorrect answer with the candidate. Some candidates will want to start with “did I pass” and some will want to go question-by-question. If you get to an incorrect answer and you believe the question and answer are wrong, do not overrule it yourself. Instead, contact us via the feedback and we may overrule the test result and change the question. Or we’ll point out something you may have missed that makes the answer correct.
Tell the candidate how they did, and congratulate or commiserate with them. The cooldown for failed exams is listed on the exam info page – for L1s, it’s 45 days, for L2s, it’s 60 days for each exam, and for L3s, it’s 90 days.
Lastly, gather all exam materials, including the test and answer sheet. Dispose of the test and answer key, but hold on to the answer sheet, since you’ll need it for…
Step 7: Enter the candidate’s results
Here’s where we come back to that last link on the exam info page. Enter the candidate’s answers to each question. It’s important to get these answers so that we can tune these questions for future test-takers.
Once you’ve entered in the results, submit the exam. At this point, the candidate will be able to see the exam in their exam history, including reading the explanation attached to each question. This is useful so you and they can…
Step 8: Submit Feedback
In addition to the quantitative feedback we receive from seeing which questions candidates are getting right and wrong, qualitative feedback is also helpful. If a question has a typo, cites the wrong rules or policy reference, seems unfair, or in any way can be improved, please click the feedback link for each question to submit your feedback about it.
To give feedback, go back to the exam info page and select “view completed exam.” Then click on the feedback link for each question to provide feedback about it.
While we may not reply to each piece of feedback, we do read them all and consider them in adjusting existing questions and developing new ones.
Now, as important as entering feedback is, it won’t actually level up your candidate. To do that, you need to…
Step 9: Write an Advancement Review
The advancement review is your chance to impart on your candidate some words of wisdom as they advance to a new level. You might remind them of some things on which they should study more, based on test results. Or review some of what you went over in the interview. Or just remind them that you’ll be there for them after they level up – the job of a mentor doesn’t end after a candidate advances!
An advancement review can be created like any other review in JudgeApps. For the type, just select “Advancement Interview (Pass or fail)”. Then, in the dropdown that appears, select for which level you interviewed the candidate. Lastly, select whether it was a successful or unsuccessful interview. This is the final step in the advancement process, and it will promote the candidate to their new level.
In the event that you’re merely proctoring the exam, and the candidate already did an advancement interview with another judge, you can have that just enter the advancement review after you’ve uploaded the exam results.
Conclusion
That’s all you need to know to give a Judge Foundry exam! If you have questions about this process, please let us know by emailing Joe at joe.klopchic@judgefoundry.org and/or johnbrian@judgefoundry.org.