Zenodo - technical report (not peer-reviewed)
AI Hiring in a Live Company: A 36-Day Case Study of an AI Hiring Agent and an AI Interview
Evgenii Arsentev · ARSENTEV.AI · ORCID 0000-0002-9120-7298 · 2026-11-03
Abstract
This technical report (part 2 of the series "AI managers in a live company") shows how the figures in the article "AI Hiring in a Live Company: The Agent Filled the Funnel. The AI Interview's Labels Did Not Line Up With Who Is Still Working" were calculated. AI hiring means software agents take over parts of a recruiter's job: reading applications, deciding who gets an interview, booking it and sometimes running the first interview. The subject is a Claude hiring agent and an AI interview (pre-recorded avatar clips, a voice AI and a separate scoring model) used by a US-facing telemedicine company to hire for a phone role.
In 36 days the agent read 1,973 applications in detail; 142 candidates entered the CRM, 138 human interviews were booked and 81 held (show-up rate 68.6%). Of 41 candidates invited to the AI interview, 22 completed it (HIRE 7, MAYBE 12, NO 3). On day 36, 2 of 7 HIRE and 6 of 12 MAYBE candidates were still working; with these group sizes the difference is not statistically meaningful (Fisher exact test, p ≈ 0.63). All 3 NO candidates were rejected on that verdict, two of whom had passed the human interview, with no record that a person read the transcript first.
The report compares time to first call in an equal window and measures what candidates were told against NYC Local Law 144 and the Illinois AI Video Interview Act as a yardstick. The company operates under a non-disclosure agreement: only the published figures, formulas, assumptions and limitations are given; the underlying data is not published. Part 1 of the series: AI Call Center Supervisor in a Live Company (doi:10.5281/zenodo.23926850).
Keywords: AI hiring, AI interview, AI recruiting, AI hiring agent, AI manager, agentic AI in operations, AI agents, human in the loop, automated employment decision tools, NYC Local Law 144, candidate screening, future of work
Cite
@techreport{arsentev2026ai,
author = {Arsentev, Evgenii},
title = {AI Hiring in a Live Company: A 36-Day Case Study of an AI Hiring Agent and an AI Interview},
institution = {ARSENTEV.AI},
year = {2026},
doi = {10.5281/zenodo.23927438},
url = {https://doi.org/10.5281/zenodo.23927438}
}