As every HR tech company adds AI to their websites, demos, and messaging, here are some questions to ask when evaluating AI features in a product:
1️⃣ Is the feature impressive, useful, or both?
The latest AI models are incredibly powerful, and that means that we continue to have moments like the first experience with ChatGPT where we're all just amazed that a computer can do something we didn't expect.
BUT it's super important not to confuse a very impressive feature with a very useful one. Just because an AI can answer a question or perform a task that we didn't expect, doesn't mean we benefit much from that impressive feat.
Is this a question you actually ask today? Do you even know what questions you would ask? Is this task that AI can perform something that you do today? Is it actually easier / does it save you time to ask AI to do it vs doing it yourself?
2️⃣ Why wouldn't I just use ChatGPT?
Or Gemini, Claude, etc...
In the push to be seen as adopting AI rather than falling behind, many providers have added very simple integrations of AI capabilities: you have a chat window in the app, and this wrapper over a foundation model has been provided with some basic context (their product documentation, an export of the data in the app, etc.)
This may save you a bit of time vs giving ChatGPT a link to their docs or a report to analyze, but as the major AI providers become stronger, have more context about you, and more integration into your company's policies, that advantage may erode.
You should be especially critical of this type of marginally helpful functionality if you have to pay extra for it.
3️⃣ Is this secure and allowed by my company?
If your company hasn't already created an AI specific security and compliance evaluation, it's coming. We have been seeing these AI questions for multiple years and are starting to see very in-depth AI specific questionnaires from the enterprises we work with.
Startup vendors that are moving fast to integrate AI may not always have the resources to have even thought through all the compliance questions your company will have.
This raises the bar even further for the actual usefulness of the AI capability you're being sold on.
The directive to adopt AI is being pushed from the top at basically every company for good reason: there is a lot of real value out there. But there is a lot of noise too.
We've used these questions to think critically about the AI features we've rolled out at Aeqium over the last year, and I think they can be a really helpful lens to view any AI capability you're considering. What do you think?
