Judge Foundry started sharing our plans for our new level system 146 days ago, in mid-October. Since then, we’ve received quite a bit of feedback from judges about our philosophy and requirements – thank you to everyone who’s shared your experiences and opinions with us!
As a result of feedback, we’re making a few small changes to our levels. We’re calling this a tune-up, and members should expect these about once a quarter or so (at least for Judge Foundry’s first year) as we see how our theory plays out when put into practice. None of these changes are individually significant enough for its own article so we’re presenting them all together. In the spirit of transparency, we wanted to share the changes with you and explain our thinking, rather than just quietly updating our official definitions.
As with our earlier articles defining the levels, the bold text is considered Judge Foundry policy, while the italics are our commentary on it.
Maintenance and Update Quizzes
Level Two+ Judges may complete their rules and policy maintenance by passing three update quizzes since the previous maintenance.
We’re patching a couple holes with this one and making a very small functional change. The old requirement was to pass three out of four update quizzes throughout the maintenance year (April to March). This generated two problems:
- Some years might have more than four update quizzes. This year, we’re going to see Modern Horizons III over the summer, and we might be writing a quiz for that (it’s too soon to say – this isn’t a spoiler!). Sometimes, Wizards prints more sets into Standard. Sometimes the calendar gets shifted.
- The Murders at Karlov Manor update quiz closes on April 9th (L2+ judges, take yours today if you haven’t already!). If we were doing maintenance in 2024 (we’re not), toward which year would this count?
The solution to both these problems, and probably some others that we hadn’t considered, is to just require judges to take three quizzes, without specifying a denominator. This is the requirement for 2025 maintenance, due March 31, 2025. Depending on the number of sets and update quizzes each year, this might change in the future. But we’ll likely keep it a simple number rather than a fraction.
The other change here is that we’re specifying “since the last maintenance,” so judges don’t put off their MKM quiz until April 1. There hasn’t been a maintenance pass yet, so quizzes you take now (like MKM) or in the past (like LCI) count toward your requirement.
F2F Tour and “Multi-Day Events”
For requirements that specify “Multi-Day Events”, Face-to-Face Tour events count for these requirements.
We recognize that the requirements as originally written were pretty difficult for Canadians who can’t or don’t want to leave their home and native land. While we want the requirements for Levels Three, Four, and Five to encourage judges to work at events with many other judges, the multi-day nature of the event was largely shorthand for “Big destination event.” Since F2F Tour stops are mostly one day, they didn’t count, and mono-Canada judges would have a difficult time meeting these requirements.
We’re still only counting one Headline Event per show for the purposes of advancement requirements – some F2F Tour events have multiple Super Qualifiers, and only the one projected to be the largest will count as a Multi-Day event, because we still want this requirement to involve working with a team of judges on a large tournament for high stakes.
As the tournament landscape changes and evolves, we’ll continue to monitor our wording around requirements like these. If you know of an event that you think should count whose current status is unclear, let us know via JudgeApps or by email.
L1s and Event Formats
Level One Judges must judge at a Constructed, Sealed, or Draft event, and be able to explain how to launch all three.
We formerly required candidates for advancement to Level One to run a draft, plus a Sealed or Constructed event. This changes that “and” to an “or,” but it also adds an expectation that the Level Two Judge issuing an endorsement will make sure the candidate knows how to launch a draft.
We learned that drafting isn’t as popular everywhere as we thought it might be. New judges should be aware of what a draft is and be able to help get one started, but if that’s not something that happens often in their community, we don’t need them to wait for it to happen.
L1 Endorsements
The endorsement from a Level Two+ Judge can be given verbally after an interview before proctoring an exam. The proctor must still submit an advancement review to certify the candidate.
This change is more of a clarification, and cleanup of the wording to what we intended. We want judge candidates to receive written feedback from their mentors as they start their judge journeys, but this doesn’t need to happen until after they take the exam.
We expect most mentors will sit down with the candidate, talk for thirty minutes or so (including making sure the candidate knows how to run constructed, sealed, and draft!), and then say “Okay, I’m confident that you embody the qualities of a judge. Let’s print you an exam.” After the candidate passes the exam, the mentor jots down some thoughts based on how the candidate did into an advancement review.
We want it to be easy to become a judge and easy for a mentor to make a candidate a judge – this clarification will hopefully make that process a little easier, without doing anything to reduce judge quality.
Exam Cooldowns
Exam cooldowns are changing to 45 days for Level One (was 30), 60 days for Level Two (was 45), 90 days for Level Three (was 60), and 150 days for Advanced Rules and Policy (was 90). The window for Level Two candidates to take both exams is similarly expanded so candidates need two passing scores within 75 days. Candidates looking to take an exam a third or subsequent time need permission from that exam’s manager before doing so.
When we initially set our advancement exam cooldowns, we set them pretty short, because we wanted to jump-start the promotion pipeline, both to expand out to the new level system and to get more judges out in the field who could promote judges. We also recognized that the exams, which are comprised entirely of new material, could have some questions that are over- or under-tuned on difficulty.
Now that Judge Foundry has been operational for a few months, we’re extending the length of the cooldown after failing an advancement exam. We want candidates to have enough time to study the material that tripped them up, and to take their time improving their rules and policy knowledge before making additional attempts. We also can only write exam material so quickly, and extending these cooldowns reduces the pressure on the volunteers on our exam teams.
Additionally, because we currently only have enough material for two tests for each candidate for each exam, we’re adding a requirement that a candidate speak with the manager of an exam before taking it again after failing twice. This is important for two reasons: first, it ensures that candidates have a reasonable chance of success on their next attempts. We don’t want candidates to keep failing, we want them to pass! Second, it provides a chance for the exam manager and the candidate to have a conversation about when the question pool will be deep enough for a third attempt. We don’t see much evaluative value in serving a test with questions candidates have already seen, and candidates can work with the exam manager to understand how long it might be before a third test is ready.
If this change is going to cause you significant hardship (for example, if you were planning on a re-test at an upcoming event that now falls in your cooldown window), email us and we’ll work with you.
Conclusion
Those are all the changes we’re making to levels at this time. If you have feedback about these changes, or about anything else about levels, you can email us or post about it on JudgeApps. Thanks for reading!