Beyond the Frankenstack: How to Choose the Right AI Partner for Hiring

Let’s get something out of the way: I still hate the word “Frankenstack” (or its close relative, the “Frankensuite”). 

It’s one of those hackneyed cliches that’s been floating around the world of SaaS for way longer than it should to come up with an alternative definition for a bunch of half-functioning point solutions patched together through a bunch of integrations and configurations into what’s commonly referred to as a “stack.”

The thing is, a “Frankenstack” isn’t really a stack at all. It’s a cry for help, and one that’s only grown louder since we added AI to what was already an overcomplicated, overly complex and chaotic mix of point solutions, plug ins and middleware. 

With the proliferation of HR Technology, both as a category of spend as well as a relatively mature ecosystem, this bric-a-brac approach to leveraging technology to drive more efficient, effective processes and increasingly optimal outcomes has become untenable. 

It’s always been something of a Quixotic challenge to manage the chimera of TA Technology, but it’s become an even more daunting task with the rise of “artificial intelligence,” since in addition to considerations like implementation, configuration and adoption, recruiting organizations now have to focus on the shiny new “AI” powered tools. 

9 out of 10 times, of course, this means absolutely nothing, but the premium pricing and significant margins entailed in AI adoption put a new impetus on recruiters to capitalize on the promise of these tools to deliver improved bottom line results – from time to fill, to cost per hire, to quality of hire and retention, among myriad other ostensible outcomes of AI adoption.

The Downside of Digital Transformation

As we’ve discussed before, however, the fundamental challenge in adding AI adoption to a broken system is that artificial intelligence doesn’t (despite its name) make that shitty system any smarter, nor work any better.

The relative success or failure of AI adoption has nothing to do with technology, and everything to do with how end users are leveraging those technologies.

And so far, those results look anything but promising. Sure, the recent MIT Study which reported 95% of all workplace AI initiatives fail was enough to spook the tech markets and garner headlines. 

But it’s the reason why those initiatives fail that bears closer scrutiny. The reasons are complex, but ultimately, they can be reduced to unrealistic expectations, lack of end user training, lack of defined strategy and, above all, the accelerated accumulation of technical debt that results from a disconnect between problem and “solution.”

This is the overwhelming majority of “AI enabled” products on the market, and it’s an issue that doesn’t look like it’s going to be resolved any time soon.

Because here’s the truth that no vendor wants to admit (at the risk of repeating myself, but these points are important): AI can make your hiring process better, but only if that hiring process is already working.

If your job descriptions suck, if your interviewers are inconsistent, or your recruiters are undertrained (or some combination thereof), AI can’t close these foundational capability gaps. It’s just going to surface and replicate your existing flaws, but faster and at scale.

AI, unlike so many other technologies, carries the risk of not only making your entire recruiting function less efficient and effective, but to entrench those worst practices into an inextricable, immutable part of your processes.

Worse, AI is uniquely capable of obscuring its flaws and limitations behind really shiny UI/UX, a bunch of colorful performance dashboards, and some “insights” that, like a Fortune Cookie, seem pretty profound, until you realize that they’re not even predictive, much less prescriptive – and often, largely theoretical.

If any of this sounds at all familiar (and if you work in recruiting or HR, it should), if you’re struggling to make your current tools work better together and improve your talent acquisition capabilities, well, piling some “AI” on top of them isn’t going to solve your problems.

It’s going to make them a whole hell of a lot worse. The good news is, there are tons of tactics and strategies HR and people leaders can use to mitigate the risks associated with AI – and even drive the sort of promised outcomes normally reserved for press releases and product marketing.

And, like all else in HR Tech, it all starts with selection.

Why AI Is So Hard to Evaluate

Traditional HR tech is (relatively) straightforward. You look at features. You test workflows. You pilot the product. You can usually see what the thing does, how it does it, and whether it integrates with your other systems. AI? Not so much.

AI models are black boxes by design. They make decisions you can’t easily explain, using training data you can’t see, based on logic you don’t control. Most vendors can’t (or won’t) tell you how their algorithms actually work.

They’ll dazzle you with phrases like “neural embeddings” and “natural language inference.” They’ll show you dashboards with accuracy percentages and correlation graphs. But ask them what specific signals their models prioritize when screening a candidate (or whether those signals vary by location, industry, or seniority) and suddenly it’s “proprietary.” 

Or, if it’s vaporware, “agentic.”

Questions to Ask Any AI Vendor (Before You Buy)

If you’re evaluating an AI product and feel like you don’t fully understand what it does or how it works, that’s not your fault. That’s the point. Most AI is built to impress, not to explain.

And that makes it exponentially harder to buy well, because you’re not just evaluating a tool. 

You’re betting on a black box. It’s not a great bet, but there are a few things you can do to improve your odds. So, if a vendor is pitching you AI, here are the questions I always ask (and recommend you do, too).

And if they can’t answer these pretty simple and straightforward questions clearly, then your answer is equally simple and straightforward: walk away. 

It’s not only the easiest approach, but it’s the only one that’s guaranteed to avoid flushing money down the drain on crappy AI tools or technologies: 

1. What decisions is the AI actually making or influencing?
   If it’s just generating a list of similar resumes, fine. But if it’s making rejection decisions independently, that’s a different conversation.
2. What data was your model trained on—and how frequently is it updated?
   Generic internet data from 2019 isn’t going to help you hire engineers in 2025. Nor is market data from Q1 likely to be applicable in Q4. In a dynamic market, it’s about real time, all the time.
3. Can we see performance across different groups (race, gender, age, etc.)?
   If they can’t show you fairness metrics, they haven’t done the work, or worse, they don’t care. Which is a pretty damning indictment, to be honest.
4. Can your product be configured to match our hiring process, or do we need to change our workflows to match yours?
   AI should adapt to your process, not the other way around.
5. If the model makes a mistake—who’s accountable?
   If they dodge this, they’re not ready for enterprise adoption.
6. Does your AI improve outcomes—or just automate tasks?
   Be wary of tools that claim “efficiency” without measurable impact on quality, speed or experience.

AI shouldn’t be another line item in your budget. It should be the connective tissue that helps reduce tools—not add more. The right AI partner can consolidate workflows you’ve been solving with 3 or 4 different tools:

• Sourcing & outreach can be unified with intelligent matching and automated messaging
• Screening & assessment can be consolidated with AI-first evaluation frameworks
• Scheduling & coordination becomes seamless with integrated calendar logic and conversational bots

But that only happens when you start with consolidation in mind. Instead of asking “What AI tools should we add?” ask “What existing tools can we remove if AI does this better?”

Because the point isn’t to build a shiny new AI stack. It’s to stop building stacks entirely.

AI Can Make Recruiting Better. But Only If You Make Better Choices.

I’m diving deeper into all this in an upcoming conversation, presented by Talk Talent and our friends at Humanly: “Beyond the Frankenstack: How to Choose the Right AI Partner for Hiring.”

We’ll talk about the real questions talent leaders need to be asking: about vendors, about consolidation, about trust. So, join us on Thursday, September 4 at 2 PM ET/10 AM PT. This is one webinar no TA leader or practitioner can afford to miss.

We’re going to skip the buzzwords and talk about what AI is actually good at (and what it isn’t), and how to think about evaluating this stuff without getting steamrolled by sales decks full of hallucinated success stories and borrowed benchmarks.

Hope to see you there.

👉 You can register here.

Remember: The tools aren’t the problem. The process isn’t even the problem. The problem is we keep outsourcing judgment to vendors, hoping that algorithms will save us from making decisions.

They won’t.

AI can make recruiting faster, more equitable, and more effective—but only if you start with clarity. About your process. About your goals. And about who you trust to help you get there.

Hope to see you Thursday!

Disclaimer: The author is an advisor and investor in Humanly, the sponsors of tomorrow’s webinar; however, he is not being compensated for this post nor for the Talk Talent webinar appearance. He just really likes Humanly and thinks that you will, too.