Maintenance is around the corner, so as you might guess, we’re keeping a close eye on level requirements right now. While we were thinking about maintenance, there were some other things that looked like they could use some adjustments as well. We’ll repeat what we said in our Winter ‘24 and Spring ‘24 tune-up articles:
We’re calling this a tune-up, and members should expect these about once a quarter or so (at least for Judge Foundry’s first year) as we see how our theory plays out when put into practice. None of these changes are individually significant enough for its own article so we’re presenting them all together. In the spirit of transparency, we wanted to share the changes with you and explain our thinking, rather than just quietly updating our official definitions.
A number of these changes are a result of changes to organized play since we last adjusted our levels. With StarCityGames expanding the American Regional Championships and the addition of the Spotlight Series, alongside the expansion of Star Wars Unlimited and Lorcana events, there are more judging opportunities at major tournaments in 2025 than there have been in a long time. But the return to Grand Prix-sized events also means we need to tighten up our expectations around Level Three, Level Four, and Level Five in order to ensure a quality staff with the skills to run events of this size.
Each change here comes with a note about when it’s effective. This is a warm-up to an article we’ll be publishing on February 1st, kicking off Judge Foundry’s first maintenance sprint, and a few of these changes affect that, though we don’t think there are any that will cause anyone who was going to pass to suddenly not pass. If you fall into this camp, please email us at testing@judgefoundry.org and we’ll work with you to figure it out.
As in other articles of this nature, bold text is considered Judge Foundry’s policy, while the italics are the commentary.
Definitions
Several new terms are introduced with this update. These are Judge Foundry terms defined to assist with evaluating advancement and maintenance requirements. Some TOs use different terms for the same roles, so we designed these terms to be well-described so that judges can determine whether their judge role meets a requirement.
Sides Lead
A sides lead is a role typically seen at multi-day events in charge of many or all judges working on scheduled side events. Sides leads may lead as few as two or three judges at a small multi-day event, but up to fifteen or twenty or more at a large one.
Primary Sides Lead
The primary sides lead is the main sides lead judge at a multi-day event. Typically there is one primary sides lead at a multi-day event, but very large events may have more than one. Primary sides leads usually have their role for an entire event weekend, are on an early shift, and plan the schedule for all of the judges for the day. Primary sides leads get to count leading each unique judge assigned to them on the weekend, up to a limit.
Secondary Sides Lead
A secondary sides lead backs up, relieves, or assists a primary sides lead. Typically this will mean being assigned a later shift. Some secondary sides leads may help with scheduling and planning for the day, but it isn’t their primary role. Secondary sides leads get to count each unique judge assigned to them for the weekend, up to a limit.
Assigned
A judge is assigned to a lead if they appear on the schedule with their lead listed as such, and are expected to attend that lead’s beginning of day briefing. Judges moved between events/leads during the day to adjust for event sizes only count as assigned to their first lead. Some edge cases may exist, such as a large event starting two hours into a shift, where a judge might be assigned to a lead starting in the middle of a shift. If this is the case, they only count as assigned to one lead, and only the lead who they reported to for the majority of the day.
Appeals Judge
An Appeals Judge is an additional judge assigned to assist the Head Judge of a large competitive event. Typically their title will be Appeals Judge or Support Judge. In general, a large event will have at least one Appeals Judge, but not more than one per 250 expected players. Appeals Judges must be the Appeals Judge hired by the Tournament Organizer, covering a Head Judge’s lunch break or covering appeals when there is an overload is not sufficient.
Removal of a Distinct Policy Exam for Maintenance
Under Maintenance, for Level Two and Level Three: Remove “Pass the annual L2/L3 policy refresh exam.” Change “Pass the annual L2/L3 rules refresh exam” to “Pass the annual L2/L3 rules and policy refresh exam.”
Under Maintenance, for Level Four and Level Five: Change “Pass an Advanced Rules Practice and Advanced Policy Practice test” to “Pass the annual L4/L5 rules and policy refresh exam.”
Effective: Immediately
When we were first concepting maintenance, one of the first things we knew was that we wanted it to include regular certification on changes to policy. At the time, we weren’t sure how frequently policy would be changing – changes were pretty sparse in 2023 and a few years before.
But with most set releases since Judge Foundry’s founding, Toby has made some policy changes, and we’ve been able to include testing on those changes as part of our regular update quiz process. Where we didn’t get policy changes, the Update Quiz team took the opportunity to quiz judges on policy questions that judges were regularly getting wrong at events.
The result is that we don’t see the need for a comprehensive dedicated policy refresh for every Level Two+ Judge. Instead, if you’ve been keeping up with your update quizzes, you’re already keeping up on policy. If you haven’t been doing your update quizzes, the Refresh Exam will include some policy questions so we can make sure you’re staying sharp there.
It’s one less step for each judge to complete each year, and one less thing our exams team has to build each year. That’s a win for everyone.
Adding Community Project Grants to Benefits
Under Benefits, for all levels: Add “Ability to lead and participate in projects funded by Community Project Grants.”
Effective: Immediately
This change is just catching up to the launch of Community Project Grants in the fall. In order for a project to be funded, it must be led by a Judge Foundry member, and a majority of the project members must be members of Judge Foundry.
Applications for Community Project Grants are still open – fill out this form if you’ve got an idea with which we can help!
Preparation Tests will be Required
Under Advancement, for Level One: Add “Take the L1 Preparation Test”
Under Advancement, for Level Two: Add “Take the L2 Rules Preparation Test and L2 Policy Preparation Test”
Under Advancement, for Level Three: Add “Take the L3 Policy Preparation Test”
Effective: April 1, 2025
In the article announcing Preparation Tests, John Brian noted:
Prep Tests aren’t currently required. I say “currently” because it’s probable that we’ll include these in our next levels tune-up, coming very soon. Joe and I have talked a bit about what we’ll require, and it will likely be a requirement to take a Prep Test, but not a requirement to pass one.
Very soon is almost here, as Prep Tests go from “It’s a good idea” to “You have to take them before you can take your exam” on April 1st. We’re already hearing from judges that these tests are working – judges are putting off taking an exam to study more after doing poorly on the Prep Test, or they’re scheduling their exams with confidence after passing it. Note that we’re not requiring candidates to pass this test, just to take it – with only one attempt per judge per test, it wouldn’t serve its purpose if we required candidates to pass.
There’s no requirement for the Advanced Rules and Policy Exam, because we don’t have a Prep Test for this one, and won’t until we have more data on how the exam is performing. Expect to see this late in the year, probably in the fall, at which point it’ll be required as well.
We’re giving a short window before implementing this requirement for a few reasons. One is technical – we’d like to work with JA staff to make it impossible to generate an exam for a judge who hasn’t yet taken the Prep Test, in case interviewers miss the requirement. But we also want to give time for this information to circulate around the community, so we don’t have people scheduling exams they can’t take.
Still, taking one remains a good idea – read the article to learn more!
Test Cooldowns Escalating
Certification exam cooldowns will now increase for each time a candidate has failed an exam.
The L1 Rules and Policy Exam cooldown is now 45 days per previous attempt.
The L2 Rules and Policy Exam cooldowns are now 60 days per previous attempt
The L3 Policy Exam cooldown is now 90 days per previous attempt
For L2, under Advancement, remove: “, within a 75-day period”
Effective: April 1, 2025
Since Judge Foundry launched, we’ve been working hard to deepen our exam pools to allow candidates to make additional attempts on exams. As of today, we have enough questions for two attempts on the L1 Rules and Policy Exam, four attempts on the L2 Policy Exam, two attempts on the L2 Rules Exam, three attempts on the L3 Policy Exam, and one attempt on the Advanced Rules and Policy Exam. Many of the additions to these exams have come through the Update Quizzes, which provide pre-tested questions so we can add new material where difficulty is relatively calibrated for candidates.
However, because most of our questions come from the updates, we can only add new questions so fast. If you read some of John Brian’s update quiz wrap-up articles, you’ll see that only a handful of questions are promoted to the certification exams from these quizzes. It can take a long time to grow a new exam from just quizzes. And adding new questions outside the quiz framework can be challenging – not only are these questions not screened for difficulty, but there are only so many ways you can ask “HCE or LEC?”
On the other hand, we’re seeing exam scores generally go up in the last few months. Part of this might be attributed to Prep Tests, part of it to judges recognizing that they need to study more than they might have expected, and part of it that judges are just getting better at rules and policy. This makes us more comfortable extending the cooldowns.
Test cooldowns will now scale based on the number of previous failed attempts on a certification exam. For example, if you fail your L2 Rules exam on January 1st, the soonest you could retake it is March 2nd. If you fail that attempt, the soonest you can retake it is June 28th. This is designed to encourage candidates to take more time to study, and also to reduce the “cliff” that appears after failing the last available attempt on an exam, where the cooldown becomes “until we generate enough questions for another attempt.”
Adding Appeals Judges as an Option
Under Maintenance, for Level Four:Change “Head-judge an event with at least five judges” to: “Head-judge or Appeals-judge events where the event was assigned at least five other judges.”
Under Maintenance, for Level Five: Change “Head-judge an event with at least twelve judges” to: “Head-judge or Appeals-judge events where the event was assigned at least ten other judges.”
Effective: Immediately
Appeals Judge, Support Judge, Assistant Head Judge… by whatever title, we’re giving some consideration to the judges who wear the same color shirt as the Head Judge but are not responsible for the primary prep work before the event. Each event by definition only has one Head Judge, but can have multiple Appeals Judges – if you’re the “PM Head Judge” or some similar title, you fall in the AJ bucket for maintenance.
This one was tough to quantify, and we still aren’t ready to put it into the advancement requirements, because Appeals Judges can mean a lot or it can mean very little, depending on how active you choose to be. Similarly, there’s not a ton of accountability for Appeals Judges, compared to Head Judges, Team Leads, and Floor Judges. So we’re not ready to make it a requirement on which you’ll be tested.
That said, a number of our Level Five members had a weird situation where, in the last year, they served as Head Judge or Appeals Judge a lot, and consequently didn’t serve as Team Lead as much. This is the first of two changes meant to get our requirements in line with reality, and to avoid having judges feel like they need to not check the “Appeals Judge” box on their application even if they want to do it and the TO wants them in that role.
Additional Head Judge Experience as an Option
Under Maintenance, for Level Four: Add to the list of service electives: “Head-judge or Appeals-judge three events where the event was assigned at least five other judges.”
Under Maintenance, for Level Five: Add to the list of service electives: “Head-judge or Appeals-judge three events where the event was assigned at least ten other judges.”
Effective: Immediately
As foretold above, this is the other half of aligning our requirements to reality. We wanted to make this one a little tricky, because we expect most judges who focus on events to get their maintenance completed by team leading and working plenty of events. But we also recognize that some corner cases come up, and if tournament organizers are entrusting you with the captain’s chair for three major events in a year, it’s a sign that you’re probably operating at the right level.
Note that we are capping it at three, because the next escalation would be to serve as Head Judge for six events, and at that point, you’ve worked six events, so you won’t need the option.
Separating out Sides Leads
Under Maintenance, for Level Four: Replace “Lead a total of 20 judges as a Team Lead at events” with:
- Lead a total of 20 judges as a Team Lead at events
- Primary Sides Leads may count unique judges assigned to them over an event weekend, up to 5 per day they were a Primary Sides Lead
- Secondary Sides Leads may count unique judges assigned to them over an event weekend, up to 2 per day they were a Secondary Sides Lead
- Other Team Leads may count each judge assigned to them each day
Under Maintenance, for Level Five: Replace “Lead a total of 30 judges as a Team Lead at events” with:
- Lead a total of 30 judges as a Team Lead at events
- Primary Sides Leads may count unique judges assigned to them over an event weekend, up to 5 per day they were a Primary Sides Lead
- Secondary Sides Leads may count unique judges assigned to them over an event weekend, up to 2 per day they were a Secondary Sides Lead
- Other Team Leads may count each judge assigned to them each day
Effective: April 1, 2025
This requirement was designed to be “Be a team lead a few times.” However, with the size of the sides team, especially on Fridays, counting sides leads for this requirement meant a lead at a very large event would complete this requirement in one weekend. That wasn’t the intent, but also ignoring sides leads doesn’t feel right either – judges are using their leadership skills as a sides lead.
By putting some caps on these, we’re recognizing that sides leads are in a leadership role, but we’re scaling the number of judges who count for each day so the result is similar to leading a team on a competitive event.
You’ll note that this requirement isn’t coming into effect for this maintenance cycle. While we felt the rules needed some tightening up to fit closer to our initial intent, we didn’t feel it was fair to judges who completed the requirement and thought they were done.
On Our Radar: Update Quizzes
In the past, we’ve done a set update quiz for every set that introduced new mechanics (so, everything but Foundations). But in 2025, we’re getting six standard-legal sets. That’s a lot! And our Update Quiz team isn’t sure if they want to commit to six quizzes a year. So we’re going to wait and see what the Final Fantasy set looks like.
If it has a similar complexity to Lord of the Rings, we’ll probably eschew the full update quiz (unless there’s also a major policy change). We can add a few questions on a single major mechanic to the next set’s update.
If it has similar complexity to most standard-legal sets, with multiple new mechanics, it’ll likely be a sign that the Universes Beyond sets are going to be as complex as previous Standard sets, so we’ll need to run update quizzes. If we do look like we’re going to have to do update quizzes for every set, we’ll probably also bump up the number of quizzes required each year from three to five.
Either way, Level Two+ Judges should still expect a healthy cadence of update quizzes in the next year.
Tune in Next Time
This tune-up seemed pretty dense, but it was mostly about aligning things so we have a smooth maintenance. Many of these changes apply just to the highest-level judges, but we’re still making them public so that folks can see that continued iteration is being done. If you’ve got feedback, you can post on the Feedback Forum, ask a question for our next Board of Directors meeting, or talk to us at an upcoming event.