AI Sourcing: Promise, Bias, and the Human-in-the-Loop

Claire Monroe

Welcome to the show—Edwin, if a recruiter types “account executive, SaaS, Chicago” into a search bar, they might get 200 profiles. But if an AI sourcing tool looks for adjacent signals instead—pipeline ownership, vertical experience, even patterns in career moves—it suddenly surfaces people no one explicitly searched for. The quiet candidates.

Edwin Carrington

Yes, and that idea—quiet candidates—is really the center of this. The strongest prospects are often not actively looking. They’re doing good work somewhere else. And historically, they were invisible unless their profile matched the exact language a recruiter happened to search for.

Claire Monroe

Right, and language is messy. One person writes “customer expansion,” another writes “account growth,” and a traditional search treats those as completely different—even though they’re basically the same capability.

Edwin Carrington

Exactly. So now you’re not just automating search—you’re defining competence. And that’s where this gets more serious. A well-tuned system expands your understanding of talent. A poorly tuned one just automates a narrow definition of it.

Claire Monroe

So when vendors talk about “faster sourcing,” you’re basically saying… speed isn’t the interesting part?

Edwin Carrington

Speed matters, but it’s not the differentiator. Consistency is. A human recruiter on Monday morning and that same recruiter late Friday are not evaluating candidates the same way. An AI system applies the same logic every time. That can be useful—but also dangerous.

Claire Monroe

Because “same logic every time” sounds fair… until you realize the logic itself might be flawed.

Edwin Carrington

Precisely. Consistency is not the same as accuracy. If the model overvalues certain employers, certain career paths, or uninterrupted trajectories, it will apply that preference perfectly—at scale. It will be wrong very efficiently.

Claire Monroe

And it looks clean while doing it. That’s the unsettling part. You get a dashboard, neat scores—82, 61—and suddenly people treat those numbers like they were discovered, not designed.

Edwin Carrington

That distinction matters. These systems don’t discover truth. They reflect decisions. What data was included, what success was defined as, what patterns were rewarded. If past hiring favored linear careers, the model may quietly penalize anything that deviates from that.

Claire Monroe

So AI sourcing is helpful when it finds patterns humans miss—but risky when it just repackages old habits as data?

Edwin Carrington

That’s close, but I’d sharpen it. It’s not just old preferences—it’s accumulated organizational habits. Things that were never formally decided, just repeated. The model treats those as signals, even if they were never intentional.

Claire Monroe

And a lot of those habits come down to… comfort. Same background, same industry, same type of profile. It feels safe.

Edwin Carrington

Yes, and the system can reinforce that without anyone noticing. If every shortlist keeps surfacing the same kind of candidate, it can look like quality—when it’s actually repetition.

Claire Monroe

There’s also the outreach side. “Personalized messaging” sounds great—until it turns into very polished, very irrelevant emails.

Edwin Carrington

Exactly. Good personalization reflects actual relevance. Something like, “I noticed you’ve led post-merger integrations”—that’s meaningful. Referencing a university or a generic title is just noise. Bad sourcing doesn’t improve connection. It just scales interruption.

Claire Monroe

So if someone wants the upside without turning hiring into a black box… where do they start?

Edwin Carrington

Start small. One use case. For example, passive candidate discovery for a difficult role. And define success clearly—better response rates, less manual sourcing time, broader candidate pools without lowering quality.

Claire Monroe

That constraint matters. Because otherwise teams roll this into everything at once, and when something goes wrong, nobody knows where it started.

Edwin Carrington

Exactly. Narrow scope allows for real evaluation. And anything that influences decisions—especially shortlists—needs human review. Not symbolic review. Actual scrutiny.

Claire Monroe

“Not symbolic review” is important. Because sometimes the human is technically there—but they’re just approving whatever the system suggests.

Edwin Carrington

And that’s not oversight. That’s delegation of judgment. A shortlist determines who gets seen, and in hiring, being unseen is often the deciding factor.

Claire Monroe

Let’s make this practical. If I’m evaluating a tool through something like the OAD lens—what do I need to understand upfront?

Edwin Carrington

Three things. First, transparency—what data is being used. Second, exclusions—what’s missing. And third, auditability—can you track outcomes over time and see if certain groups are consistently ranked lower.

Claire Monroe

Because bias doesn’t always show up as rejection. Sometimes it’s quieter. A candidate just keeps landing lower in the list, so no one ever reaches them.

Edwin Carrington

Exactly. Bias often shows up as ranking patterns, not outright exclusion. That’s why you need to compare input and output. Who entered the pool, who rose to the top, who received outreach, who moved forward.

Claire Monroe

And then there’s data quality. If profiles are uneven, outdated, or overly polished, the system might just reward whoever sounds the best on paper.

Edwin Carrington

Correct. Poor data doesn’t improve through processing. It becomes consistently misleading. So the real question isn’t whether the AI is sophisticated—it’s whether the signal it’s using is reliable.

Claire Monroe

Let’s test a scenario. A team uses AI sourcing for engineering managers. The tool finds strong passive candidates quickly. Recruiters spend less time searching and more time talking to people. That’s the ideal.

Edwin Carrington

That’s the productive version. The system handles breadth and initial ranking, and the recruiter focuses on relationships and fit. In that case, AI removes repetitive work and increases meaningful interaction.

Claire Monroe

But the failure version is… the same speed, just aimed in the wrong direction. The shortlist leans too heavily on certain companies or titles, and everyone moves faster toward the wrong candidates.

Edwin Carrington

Exactly. Faster toward the wrong candidates. That’s the risk. Velocity feels like progress because activity increases, but if no one questions the assumptions behind the ranking, speed becomes acceleration without direction.

Claire Monroe

So the real question isn’t whether AI can move faster. It’s whether a team can stay deliberate when speed is no longer a constraint.

Edwin Carrington

Yes. And the organizations that handle this well will make a conscious choice to keep judgment slower than the software. If you’re looking to operationalize that kind of structured, evidence-based hiring, you can test OAD’s tools—like behavioral assessments—for free at o-a-d-dot-a-i. It’s a practical way to introduce structure without losing oversight.

Claire Monroe

Which is probably the balance everyone’s actually trying to get to—speed where it helps, and thinking where it matters.