Talent Lens — The Question That Made Me Rebuild a Hiring Tool

I built a tool to help hiring teams score candidates against a job description. I called it Talent Lens. It ingested resumes and JDs, built an evidence scorecard, and gave recruiters a ranked shortlist with the reasoning attached — not just a number, but why.

I was proud of it. Then someone asked me one question that made me take the ranking logic apart and start over.

"Why does the score include how confident the model is that the resume is authentic?"

I'd added it without thinking too hard about it — an "authenticity confidence" signal, meant to flag resumes that looked machine-generated or templated, folded into the overall match rank alongside actual skills and experience. It seemed reasonable at the time. Catch the noise, help the recruiter focus on real signal.

Here's the problem, and it took me an uncomfortably long pause to see it once it was pointed out: that signal doesn't measure whether someone is qualified. It measures whether their resume happens to read a certain way to a language model — and there's no reason to believe that correlates with anything except how someone was taught to write, which correlates with education access, native language, and frankly the parts of someone's background that have nothing to do with whether they can do the job.

Folding it into a rank doesn't filter noise. It quietly punishes people for reasons that would never survive being said out loud in an interview room.

That's not a minor bug. That's exactly the shape of adverse impact that governance frameworks exist to catch before it ships — and I'd built it myself, without catching it.

I want to be honest about what I did next, because the flattering version of this story is "and then I immediately fixed it," and the real version has more sitting-with-it than that.

My first instinct, if I'm being completely honest, was to defend the design — explain the intent, argue it was a minor weighting, move on. I've sat in enough governance reviews to know that instinct isn't unique to me; it's the default reaction to being told your own work has a problem. I didn't act on it, but I felt it, and I think that's worth admitting rather than skipping past.

What I actually did was pull the authenticity signal completely out of the ranking. Not down-weight it — remove it. If it's useful at all, it's useful as a separate, clearly labeled flag a human can look at and decide what to do with, never as an input that silently moves someone up or down a list.

Then I went back through the rest of the scoring logic looking for anything else doing the same thing quietly. That rebuild took the product from version 7 to version 13 in about ten days — the fastest I'd iterate on anything all year, which tells you something about how motivating "I got this wrong" is compared to a normal feature backlog.

The reason I'm writing this up isn't to make Talent Lens sound impressive. It's because I spend a fair amount of my actual day job writing the AI governance rules that are supposed to catch exactly this kind of thing before it reaches production — bias testing, human review points, audit trails.

It's a completely different exercise to write "test for adverse impact" in a framework document versus sitting alone at 10pm with your own ranking algorithm, realizing the thing you built is the exact failure mode you lecture other teams about. I don't think that makes the framework wrong. I think it makes it a lot more real to me than it was in February.

The review that catches the problem in your own work matters more than the one you write for everyone else's.

I got lucky that someone asked the question out loud. I try to ask it of myself now before anyone else has to.