How to Build a Candidate Evaluation Framework: Hard Gates, Soft Scoring, and Rubrics

Most AI sourcing tools will hand you a candidate marked a 100% match who you already said does not qualify. A recruiter we spoke with watched a tool rank someone "a perfect match" who was working on electronics design at the wrong kind of company, without the years of experience she had asked for. The score itself was fine. The real failure was that nothing dropped the wrong people before the scoring ran, so a candidate with one fatal gap still ranked at the top on the strength of everything else.

A candidate evaluation framework fixes that by running in a fixed order. First a set of hard gates removes anyone who fails a non-negotiable, before a single point is assigned. Then a weighted rubric scores only the survivors. Then you calibrate that rubric against your own judgment so the bands mean something. This guide walks through all three layers, with working filters you can run from your terminal. For which signals actually predict fit once you reach the scoring layer, we link out to a companion piece so this one can stay on the structure.

Why does candidate evaluation break before any scoring?

The recruiting team we spoke with sources outbound for hard technical roles, often as the founder rather than the recruiter. They ran their searches through the usual AI tools and kept getting the same result. One recruiter typed her real must-haves into a tool, "five years of experience, must have PCB or design experience, must have robotics experience," and got back 701 candidates where only about 200 were worth her time. The criteria were stated. They just were not enforced.

Keyword matching has no concept of disqualification

When a tool turns your prompt into keyword filters, anyone who contains the keywords clears the search, regardless of context. As the recruiter put it, "Anybody with PCB on their profile and five years cumulative experience is going to show up in the search, and that's not correct." A candidate who did a three-month internship touching the right skill, then spent the bulk of their career somewhere unrelated, looks identical to the keyword layer.

Her colleague described the sharpest version of this. A tool counted "eight months at an internship, and I've been at AWS for seven years" as significant contribution to the thing the role needed. The internship matched. The seven years that actually define the person did not. Nothing in the tool knew the difference between a credential you hold and a word that appears on your profile.

Why cumulative experience overstates current relevance

The same gap shows up with time. The team told us these tools "don't take into account recency," so a robotics project from fifteen years ago pulls a candidate into a search for someone doing the work today. A flat keyword match treats every line of a career as equally present. A real evaluation has to know which facts are live and which are history, because a current role and a long-finished internship carry very different weight.

What is a hard gate in candidate evaluation?

A hard gate is a non-negotiable that drops a candidate out of the pool entirely, before any scoring. It is binary. The candidate either clears it or they are gone, and no amount of strength elsewhere buys it back. The whole point is to run gates first, so your rubric never wastes a point on someone who was never eligible.

The team we spoke with learned this on a defense search where work authorization was the constraint. In their words, "One of the big hard gates is that the people have to be a U.S. person because it's a defense tech company." On the first pass they had not flagged it as a gate, and "some non-U.S. people leaked in." The fix was to make the gate fire first. "Please, before anything else, make sure that the people that you're surfacing meet that. And then after that, it was good." That ordering, gate before everything else, is the core of the framework.

The four gate categories that come up most

Across the searches the team ran, the same kinds of hard gates kept appearing. Each one is binary and each one belongs before the rubric.

• Credential or qualification: a license, a degree, a clearance, or a specific must-have skill that the role cannot proceed without. Treat it as a pass-or-fail check that sits outside the scoring math entirely.

• Location and onsite policy: a metro you can support, or an in-office requirement. One recruiter described collapsing a list "from 11,000 people that fit your profile in your mind to 68, and only 12 of them are in the Bay Area" once the real constraints went on.

• Work authorization: the U.S.-person gate above is the clearest case, where a candidate who fails it can never move forward no matter how strong they are.

• Current versus past role: whether the qualifying experience is where the candidate is now, or something they did briefly years ago. This is the gate the keyword tools miss most often.

Gating on the current role rather than the whole history

The current-versus-past distinction is worth its own filter, because it is where the framework earns its keep. Crustdata's structured career data separates the two. The path experience.employment_details.current.title filters on what someone is doing now, while experience.employment_details.past.title filters on what they did before. The bare experience.employment_details.title matches across every role a person has ever held, which is exactly the behavior that lets an old internship clear a gate it should not.

So to gate on current relevance, you filter the current path and let the rest of a career feed the score while staying out of the gate. The hard-gate pass below keeps only people whose current title fits, who sit in a supported country, and who clear a tenure floor, run against Crustdata's Person Search API:

The (.) operator is a case-insensitive regex match, so Electrical Engineer|Hardware Engineer catches the same role under either label, which is the recruiter's "electrical engineers in disguise" problem. The => operator means greater-than-or-equal, and Person Search expects => and =< for the inclusive comparisons rather than the more familiar >= and <=. To gate on location by radius rather than company country, swap in a geo_distance or geo_exclude filter on professional_network.location.raw. Everyone this query returns has already cleared the gates, so the scoring layer starts from a clean pool.

What is soft scoring over the survivors?

Once the gates have run, soft scoring ranks who is left. This is where weighting lives, and where judgment belongs. A soft score sums weighted criteria into a number you can sort on, with the heaviest weight on what matters most for this role and lighter weight on the nice-to-haves. Nothing here removes a candidate. It only orders them.

The recruiter described exactly how she wants to read that order. "My 100% matches, I probably shouldn't even have to look at those. I should have enough confidence in the tool to be like, I trust you. I'm not even going to touch those. I'm going to spend my time on the 80 to 60% where I'm going to make that final call. That's where the judgment is." Her bottom band is just as decisive. "If there's someone who's below a 50% match, not worth my time."

Three bands instead of a single cutoff

That gives you a three-band rubric rather than a single pass-or-fail line. The top band is high enough that you trust it and push straight to outreach. The middle band is where a human reviews and makes the call. The bottom band you discard. The teaching point is that the score is a triage tool, and the middle is the only place your time pays off, because the bookends decide themselves.

The deployed version of this showed up as green, yellow, and red flags on each profile. As one recruiter described his greens, "Very rarely is somebody that's green like bad. The greens are almost always somebody that should be spoken to." The bands are only useful when the top one is trustworthy enough to act on without a second look, which is the bar the whole framework is built to hit.

Weighting decides which gaps you forgive

Weighting is what keeps a strong match on a minor attribute from outranking a gap on something that matters. For which variables to score and how to source the signal behind each one, we cover that in detail in our companion guide on building a candidate scoring model with custom variables. This piece stays on the structure itself, on how gates remove people, how scores rank the ones left, and how weights decide which gaps you are willing to forgive.

How do you keep weak signals out of the gates?

A common mistake is promoting a weak signal into a gate, so a candidate gets dropped for missing something that was never essential. The team we spoke with were explicit about this with public code and papers. In one recruiter's words, those signals "need to be like a cherry on top of a sundae, because some of the best engineers we hire, if you go to their GitHub, there's nothing. They've made no commits." He was blunt about it. Do not over-index on that stuff.

A simple test for gate versus score

The rule that falls out of this is clean. If a missing signal should kill a candidate outright, it is a gate. If a present signal should only lift them up the ranking, it is a scored, weighted criterion. Public code, conference talks, and a co-authored paper almost always belong in the second group, because their absence says little. A strong engineer can have an empty profile, so absence of a public footprint is weak evidence and a terrible gate.

This is also why title alone is a poor gate. The recruiter pointed out that "not everybody's called an electrical engineer, sometimes you might be called a hardware engineer," so a strict title gate quietly drops qualified people. The fix is to gate on the underlying must-have where you can state it precisely, and score the fuzzier signals so a near-miss on wording costs points instead of elimination.

How do you calibrate the rubric?

A rubric is only worth its bands if those bands match your judgment, and they will not on the first run. Calibration is the loop that tunes the weights until the scores agree with a recruiter who knows the role. The team we spoke with did this by feedback, moving profiles between bands and explaining why each one moved.

Calibrate by contrast, on both kinds of error

The fastest way to calibrate is to look at both error types together. You correct the false positives, the candidates the rubric ranked too high, and the false negatives, the ones it ranked too low, and the contrast between the two pulls the weights toward the right setting. One recruiter caught a false negative that came from over-weighting location. In her words, "I know you're saying this person because of location, you don't want to talk, but no, that would be somebody I'd talk to." That single correction told the rubric to treat location as a flag for review rather than grounds for elimination on that search.

A practical way to seed calibration is to score people whose outcomes you already know. As one recruiter described his method, he would "go through the people I've reached out to for roles before, and based on their profile, score them on the one to four," then check the rubric against reality. If the rubric agrees with your past calls, the weights are close. Where it disagrees, you have found a weight to fix.

Set a high bar without over-indexing

There is a real trap here, and the team named it. One recruiter worried she might "over-index on something and then send the tool in the wrong direction." Setting a high bar and over-indexing are different moves. A high bar means demanding strong evidence across the rubric. Over-indexing means letting one criterion dominate after you correct on it once. Calibrate by adjusting weights across the whole rubric rather than hammering a single criterion every time a candidate slips through, and the bands stay honest.

Run it in Claude, or call the API directly

There are two ways to wire this framework into a real workflow, and both hit the same data. A Claude Code agent with Crustdata's MCP server configured can run the whole sequence from plain language. You describe the gates and the rubric, the agent runs the gated search, enriches the survivors, scores them against your criteria, and hands back a banded list. It is the fastest way for a non-technical recruiter to operate the framework without writing code.

A natural-language brief to that agent reads roughly like this:

If you would rather own the orchestration, call the same Person Search endpoint over REST from Python, Node, or whatever your stack runs on, and build the gate-then-score logic in your own code. The gate filters go in the and block, exactly as the curl example above, and the scoring runs on the returned survivors. Most teams prototype on the MCP path and move to the direct API once the framework is stable. For enriching the survivors with contact data before outreach, our roundup of candidate enrichment APIs covers the options.

Conclusion

A candidate evaluation framework works because it runs in order. Hard gates drop the non-qualifiers first, so a fatal gap can never be outranked by a strong attribute. Soft scoring then ranks the survivors into bands you can act on, with the heaviest weight on what the role actually needs. Calibration keeps those bands honest by correcting both the false positives and the false negatives against your own judgment.

Three steps to put this to work this week. Write down the three or four non-negotiables for your open role and turn them into gates that fire before any scoring. Move every weak signal, public code, papers, a fuzzy title, into the scored layer where its absence costs points instead of an elimination. Then score ten candidates whose outcomes you already know and adjust the weights where the rubric disagrees with you. To build the data layer behind it, sign up for Crustdata's free tier (100 credits included) and wire the gated Person Search into your evaluation step.

Crustdata is the data layer for recruiting teams building their own candidate evaluation, where you set the gates, own the rubric, and see the evidence behind every band. Come and see what gated, scored sourcing looks like across your own recruiting workflow.

Frequently asked questions

What is a candidate evaluation framework?

It is a fixed-order process for deciding who advances. Hard gates first remove anyone who fails a non-negotiable like work authorization or location. A weighted rubric then scores only the survivors and sorts them into bands. Calibration tunes the weights so the bands match a recruiter's judgment.

What is the difference between a hard gate and a scored criterion?

A hard gate is binary and removes a candidate before scoring, so a missing must-have ends the evaluation. A scored criterion only adds or subtracts points and orders the survivors. The test is simple. If a missing signal should kill a candidate, it is a gate, and if a present signal should only lift them, it is scored.

Why should hard gates run before scoring?

Because a candidate with one fatal gap can still rank high on the strength of everything else if you score first. Recruiters told us non-qualifiers "leaked in" until they made the gate fire before anything else. Gating first also saves time, since you only spend scoring effort on eligible people.

How do you evaluate current role versus past experience?

Filter on the current role for gates and let past roles inform the score. Crustdata's structured data separates experience.employment_details.current.title from .past.title, so a brief internship years ago does not clear a gate meant for current work. This prevents the cumulative-experience match that keyword tools produce.

How do you calibrate a candidate scoring rubric?

Score candidates whose outcomes you already know, then correct both the false positives and false negatives. The contrast between the two pulls the weights toward the right setting. Adjust weights across the whole rubric rather than over-indexing on one criterion, so a single correction does not send the scoring off course.

Can you evaluate candidates before you have their contact details?

Yes. Gating and scoring both run on public profile and structured career data, which need no email or phone number. Score and band the pool first, then spend on contact enrichment only for the candidates in the trust band you are about to reach out to.