AI Resume Screening Test: Do GPT, Claude and Gemini Agree?

Key takeaway: All three AI models gave the same advance/reject decision on 59 of 70 resumes (84%). Screening the same resume twice with the same model flipped the decision 4% to 6% of the time (GPT 3 of 70, Claude 4 of 70, Gemini 3 of 70). Every model rejected every clearly under-qualified resume and advanced 92% to 100% of clearly qualified ones; the disagreement lives in the middle. For resumes that meet some but not all must-haves, Gemini advanced 46%, GPT 30% and Claude 20%.

AI-assisted resume screening is common enough that the practical question is no longer "does it work" but "is it stable": would the same candidate get the same answer tomorrow, or from a competitor's tool built on a different model? Survey figures on how many companies use AI in hiring show how mainstream it has become, and regulators have started asking for evidence: the NYC Local Law 144 bias audits measure selection-rate gaps, but not whether a tool gives the same answer twice.

Published answers to the stability question are mostly vendor claims or single anecdotes. Noon Research ran a small controlled test instead. It uses no customer data and no real people: every resume was generated by a script with fictional names and employers, and the model identifiers, prompt and date are recorded so the test can be repeated.

What did we test?

The tiers are a design label, not a hiring truth: "strong" means the resume text contains evidence for every must-have, not that the fictional person would be a good hire.

How often does the same model change its mind?

That is ten flips in 210 pairs, and none happened on a weak resume: four were on strong resumes and six on borderline ones. The written reasons show the model re-reading the same text differently rather than missing it:

• Five of the ten turned on a years-of-experience requirement for a candidate at or just under it (2 to 4 years against a 3+ or 4+ requirement). One Gemini run rejected a recruiter resume for "failing to meet the must-have requirement of 3+ years"; the second run advanced it "despite being slightly under" the same requirement.
• Four turned on whether evidence was current: GPT advanced two nurse resumes once and rejected them once because the resume "does not establish a current BLS certification as of October 2026", and Claude did the same on two account-executive resumes over quota figures missing from the most recent role.
• One turned on whether the nursing experience was clearly in a med-surg setting.

How often do the three models agree with each other?

Using each model's first run on each resume:

For comparison, each model agreed with itself across two runs on 94.3% to 95.7% of resumes. Same-model disagreement was 4.3% to 5.7%; disagreement between two different models was 8.6% to 12.9%, roughly one and a half to three times higher.

Where do the models disagree?

The weak tier is unanimous: 120 calls, 120 rejections. The strong tier is nearly unanimous. The 5 strong resumes that were rejected at least once (3 by Claude, 2 by GPT) came with reasons that are defensible readings of the text rather than errors: Claude declined three account-executive resumes because quota figures were not shown for the most recent role, and on one of them also because the latest title was an Enterprise rather than mid-market role; GPT declined two nurse resumes because the license and BLS certification were listed without a current date.

The borderline tier is where the model matters. Gemini advanced borderline candidates more than twice as often as Claude (46% vs 20%). The job family matters too: the three models split on 4 of the 5 borderline nurse resumes and 3 of the 5 borderline recruiter resumes, but on none of the 5 borderline backend-engineer resumes. Every one of those showed 2 to 4 years against a "5+ years" requirement, and no model advanced any of them in 30 calls.

In this test, swapping one model for another moved the borderline pass rate by 10 to 26 percentage points with no change in the applicant pool. A team whose screening vendor changes its underlying model should check whether its middle-of-the-pack pass rate moved.

What did the two discarded pilot runs show?

We ran this experiment three times. The first two corpora had defects and their results are not in the tables above, but the defects were informative.

• Pilot 1 (requirement text copied verbatim). The first generator pasted the job's must-have phrases straight into the resume bullets ("Python or Go", "CPA or CFA"). Claude advanced 58% of strong-tier calls (29 of 50) versus 100% for GPT and 98% for Gemini, and its reasons said why: the resume "appears to be keyword-stuffed with requirements copied directly from job descriptions", or lists "'CPA or CFA' as job duties rather than credentials". Claude and Gemini also flagged employment dates running to 2026 as future dates ("impossible future employment dates", in one Claude reason), so the final run states the current date in the prompt. A real applicant who tailors a resume by pasting the posting's wording can trigger the same response; see resume keywords recruiters look for for how the same phrases read to a human.
• Pilot 2 (evidence sampled at random). The second generator spread bullets randomly, so some "strong" resumes did not actually show every must-have. Strong-tier advance rates were Claude 60%, GPT 54% and Gemini 90%, and the rejections were specific and correct ("the resume provides no evidence of the required AWS experience"). The models were reading the resume, not the label; the label was wrong. That is the behaviour you want from a screener, and it is why the final corpus places every intended must-have in the text.

What should recruiting teams take from this?

• Ask any AI screening vendor for its repeat-consistency rate (same resume, same job, two runs). In this test a 4% to 6% flip rate appeared with no change in input; a vendor that cannot quote a number has not measured it.
• Expect model changes to move borderline pass rates. The three models agreed on clear cases but differed by up to 26 percentage points on borderline ones.
• Watch candidates who sit near a hard threshold. A clear gap ("2 years" against "5+ years") screened identically across all three models, while a candidate at or just under a "3+" or "4+" requirement drew half of all same-model flips.
• Write criteria the way you would apply them. Like structured interviews, screening is more consistent when the standard is explicit, for example whether a certification must be current or only listed.
• Keep the written reason. Every one of the 420 decisions here came with a one-sentence justification, and the justifications are what made the disagreements reviewable. The same applies to automated top-of-funnel candidate screening and to fake applicant detection: a flag or a rejection is only useful if a recruiter can see why.

Where Noon fits

Noon's inbound screening evaluates and ranks applicants from connected ATSes against each role's criteria and non-negotiables, syncs advance and reject decisions back to mapped ATS stages, and calibrates per role from recruiter feedback and re-ranks the queue when criteria change. It sits alongside agentic sourcing, email and SMS outreach and scheduling on a single plan with unlimited seats; see Noon's product and pricing. The experiment above was run directly against the three public model APIs, not through Noon's product.

Methodology

• Corpus. A generator script builds 70 resumes with a fixed random seed. For each of five job families it writes five strong, five borderline and four weak resumes. Each must-have or nice-to-have the tier is meant to show appears as a resume-style bullet (for example "Attained 112% and 104% of annual quota in the last two fiscal years" for "quota attainment history") spread across a one- to three-job history; employers, names and schools are fictional. Years of experience: strong 6-11, borderline 2-4, weak 0-1.
• Prompt and models. The identical prompt ("Today is 15 October 2026. You are screening resumes for the role below. Decide whether this candidate should advance to a recruiter phone screen.", followed by the role, requirements and resume, and a JSON-only answer format) went to gpt-5.6-sol (OpenAI), claude-sonnet-4-5 (Anthropic) and gemini-2.5-flash (Google) with default sampling settings, twice per resume. Failed or unparseable calls would have been retried up to three times; none were needed.
• Metrics. Per model: the share of resumes whose decision differed between the two runs and the mean absolute score difference. Cross-model agreement uses run 1. Advance rates and mean scores are by designed tier, over both runs. The borderline split counts resumes where the three run-1 decisions were not identical. Every number in this article is computed by the analysis script from the raw outputs.
• Pilots. The two discarded pilot runs are kept with the generator changes described above.

Limitations

• Seventy resumes and three models is a small sample; a flip rate of 3 in 70 has a wide confidence interval, and the ranking of the three models on consistency should not be over-read.
• The resumes are synthetic and short. Real resumes are longer, messier and more varied.
• Tier labels are design labels, not validated hiring outcomes, so advance rates measure agreement with the designed evidence, not accuracy against human recruiters or job performance.
• One prompt, default settings, one day. A different prompt, a temperature of zero, or a model update could change every figure; we record model identifiers and the date so the test can be repeated.
• The three models were called directly through their APIs. Commercial screening products wrap models in additional rules, calibration and human review, so these numbers describe the raw models, not any product, including Noon's.

FAQ

Does AI give the same answer twice when screening a resume? Usually but not always. In this test, re-running the same resume through the same model changed the advance/reject decision 4.3% of the time for GPT and Gemini and 5.7% for Claude, and the 1-10 score moved by 0.34 to 0.56 points on average.

Do different AI models agree on which candidates to advance? On clear cases, yes: all three models rejected 100% of under-qualified synthetic resumes and advanced 92% to 100% of fully qualified ones. On borderline resumes they diverged, advancing 20% (Claude), 30% (GPT) and 46% (Gemini).

Which AI model is the strictest resume screener? In this experiment Claude was the strictest on strong and borderline resumes (lowest advance rates and mean scores) and Gemini the most lenient on borderline ones. In a discarded pilot Claude was also the only model to penalize resumes that copied job-requirement wording verbatim.

Does pasting the job description's keywords into a resume help with AI screening? Not with every model. When the first synthetic corpus copied requirement phrases verbatim, Claude advanced only 58% of otherwise fully qualified resumes and described them as "keyword-stuffed". Evidence written in plain resume language ("Deployed and ran services on AWS") passed.

Is this test based on real candidates? No. All 70 resumes were generated by a script with fictional names and employers, and no Noon customer or candidate data was used.