How to evaluate an AI consulting firm
Most firms selling AI services today were selling something else two years ago. That is fine. What matters is whether they can put a working system into your business and leave you able to run it.
1. Ask to see systems running in production
The strongest evidence is a system a real client uses every week. Ask for a live demonstration or a client you can call.
Slides, frameworks and maturity models are easy to produce. A working agent that handles intake, drafts documents or answers customers is hard to fake. Ask what it does, how long it has been running, what broke in the first month and who fixed it.
2. Check that the work survives a change of model
A new frontier model ships every few weeks. A firm that builds everything around one provider leaves you stuck when a better or cheaper model arrives.
Ask where your instructions, company knowledge and tool connections live. They should sit outside any single provider, ideally through an open standard such as MCP, so the model underneath can be swapped in days.
3. Get prices in writing before the first meeting
Opaque pricing usually means the number depends on how much the firm thinks you can pay. Ask for a range by project type before you spend time on discovery calls.
- What does a single automation or agent cost to build?
- What does it cost each month to keep running, including model usage?
- What happens to the price when the scope changes?
Nova3 AI publishes its figures on the pricing page for this reason.
4. Confirm who owns the result
Ask in writing who owns the code, the prompts, the knowledge base and the accounts the system runs on. You should be able to end the engagement and keep everything working.
Watch for systems hosted only in the firm's own accounts, licence fees for your own prompts, or handover documents promised for later.
5. Test how they handle your data
- Which providers will see your data, and are they set to not train on it?
- Where are passwords and API keys stored, and who can read them?
- Can each person in your company see only the records they should?
- What gets logged, and how long is it kept?
A good firm answers these on the first call without needing to check. Vague answers here are the clearest warning sign in this list.
6. Meet the people who will build it
Large firms often sell with senior partners and deliver with junior staff or subcontractors. Ask who will write the instructions, connect the systems and answer the phone when something breaks. Then meet that person.
7. Start with one workflow
A firm confident in its work will offer a small first project with a clear definition of done: one workflow, one agent, a measurable result within weeks. A firm that insists on a six-month transformation programme before anything ships is asking you to carry its risk.
Nova3 AI starts every engagement in the Sandbox with a scoping conversation about one workflow.
Questions to ask on the first call
- Can you show me something you built that a client uses every week?
- What happens to our system when a better model comes out?
- What will this cost to build, and what will it cost each month after?
- If we stop working together, what do we keep?
- Which outside companies will see our data?
- Who on your team will build this, and can I talk to them now?
- What is the smallest useful first project?
For the vocabulary you will hear on those calls, see what AI operations means.
What should an AI consulting firm deliver?+
Working systems your team uses: agents, automations and the company knowledge they run on, plus written documentation and ownership of everything built. A report alone is the smallest part of the job.
How much does AI consulting cost?+
It varies widely by scope. Ask every firm for written ranges for a single agent, a multi-step workflow and the monthly running cost, including model usage, before discovery calls.
Are small AI firms a good alternative to large consultancies?+
Often, for small and mid-sized businesses. A small firm usually puts its builders on the call, prices more plainly and ships a first workflow faster. Check the same seven points either way.
What is the biggest red flag when hiring an AI consultant?+
Vague answers about data handling and ownership. A firm should say plainly which providers see your data and confirm that you keep the code, prompts and accounts.
Mark Jones
Principal, Nova3 AI
Mark Jones is the principal of Nova3 AI and founder of Trifecta Agency in Santa Rosa Beach, Florida. He builds and runs AI operations for companies from small business to enterprise: automation, agents, intelligence layers, and full operating systems on top of whichever model fits the job. He writes and reviews every Nova3 guide and the weekly Current State briefing on what changed at the major AI labs.
Want to put Nova3 AI through these seven checks?
Bring us your project →