Buyer's guide · For operations leaders
The AI Buyer's Guide for Operations Leaders
Choose AI that improves operations, not one more layer of software. Ten checks, the questions to ask every vendor, three worksheets and a scorecard.
A great demo can hide integration work. A low price can leave out what you need. A successful pilot may never reach production.
So the question isn't just which AI tool to buy. It's this:
Which approach will improve how we operate, with measurable results, manageable risk and a cost we understand?
The guide at a glance
Step 1 · Define the problem
Step 2 · Test the claims
Step 3 · Decide and prove
01
Start with the business problem, not the AI
AI pays off when it helps you act on information that's already there, but scattered.
Pick a problem worth solving: a supplier delay that puts production at risk, an engineering change that hits open jobs, a quality exception that takes hours to investigate.
- Supplier email · New date: Oct 26
- ERP · Still shows Oct 12
- Spreadsheet · Expedite list
One problem
A supplier delay puts a production run at risk
- Production plan · Run A-88 needs it Oct 20
- Quality system · Open deviation
- People · "The planner knows"
Ask every vendor
- What outcome are we improving, and by how much in 90 days?
- Where does the work stall today, and why?
- What does the current process cost in time, money, risk or revenue?
- Do we need better information, better decisions, coordinated action, or all three?
What to look for
A vendor should tie its product to a concrete outcome. "Save time with AI" is a hypothesis, not a business case. Take your baseline before you test anything.
Worksheet 1
Define the problem and its cost
Fill this in before you talk to vendors. It becomes your baseline.
Where the information lives
What it costs today
Assumes 50 working weeks: hours × rate × 50, plus incidents × cost per incident × 12. Your numbers, your estimate.
02
Count the full cost, not the subscription
Compare total cost of ownership with verified value.
A low price can hide engineering, data work and add-ons. Usage pricing rises with adoption. Essential features can be extras.
What it takes to run it
- Integration
- Data readiness
- Implementation
- Governance and security
- Ongoing operations
- Organizational change
- Expansion
Ask every vendor
- What do we pay beyond the quoted price?
- Which integrations and features cost extra?
- How does pricing change as usage and complexity grow?
- What internal staff and outside help will we need?
- What will it cost to run over 12 to 24 months?
Watch out for
A business case built on theoretical time savings that leaves out implementation, exception handling and running costs.
Worksheet 2
Total cost over 24 months
One worksheet per option. Ask each vendor to help you fill in every line, including the ones they don't sell.
| Cost | One-time | Per year |
|---|---|---|
| SoftwareSubscription, usage, model and overage fees | ||
| IntegrationConnecting ERP, MES, CRM, email and spreadsheets | ||
| Data readinessCleaning, mapping and reconciling data | ||
| ImplementationConfiguration, workflow design, testing | ||
| Governance and securityAccess controls, reviews, audit | ||
| Ongoing operationsMonitoring, fixes, maintenance | ||
| Organizational changeTraining, ownership, adoption | ||
| ExpansionMore users, workflows or connectors | ||
| Totals | $0 | $0 |
24-month total cost
—
Payback
—
24-month total = one-time + 2 × per year. Payback = one-time ÷ monthly (value − yearly cost). If the yearly cost is more than the value, there's no payback.
03
Test the plug-and-play promise
Easy to demo isn't the same as easy to deploy.
Flagging a late delivery is step one. It also has to know which jobs and customer orders are affected, and who acts next.
In the demo
In your business
Clean sample data
Missing, inconsistent and conflicting records
One system
Email, ERP, spreadsheets and planning
The happy path
Exceptions that need human judgment
The vendor drives
Your team owns it after go-live
Ask every vendor
- What must be configured before it works on our data?
- How does it handle missing, inconsistent or conflicting records?
- What happens when a connector fails or the AI is unsure?
- Who owns deployment and maintenance, and how much of our team's time will it take?
Ask for this
A demo on your real workflow and data, including a conflicting record and an exception that needs human judgment.
04
Make sure it fits how the work happens
Real work crosses systems. The tool has to, too.
Email
Supplier says the parts will be late
ERP
The PO still has the old date
Inventory
Stock on hand won't cover it
Plan
A production run depends on it
People
A planner knows the customer impact
Ask every vendor
- Does it handle work that starts in email or spreadsheets?
- Does it connect systems without asking everyone to adopt another workspace?
- Does it show downstream consequences, or one isolated event?
- What must change in our process, and what can stay as it is?
The principle
Start with the work and the outcome, then fit the software to it. Standardizing a process can be right, but it shouldn't be a hidden prerequisite.
05
Check what happens after the alert
Finding a problem isn't the same as resolving it.
| 1Detect | 2Connect | 3Assess | 4Coordinate | 5Follow through | 6Verify | |
|---|---|---|---|---|---|---|
| Chatbot or assistant | ||||||
| Alerts or dashboard | ||||||
| Closed-loop workflow |
- 1. Detect:
- Spot a meaningful change or risk
- 2. Connect:
- Gather the evidence across systems
- 3. Assess:
- Work out what's affected and how urgent
- 4. Coordinate:
- Name the next action, owner and approval
- 5. Follow through:
- Track progress and surface what's stalled
- 6. Verify:
- Confirm it's resolved and records are updated
Ask every vendor
- Does it stop at an answer or an alert, or move the work forward?
- Can it identify the affected orders, jobs, batches or commitments?
- Can it assign an owner and track the follow-up?
- Can we see the evidence behind each recommendation?
- How do we know the issue is actually resolved?
What to look for
A closed loop from information to decision, action and outcome, with people in control.
06
Make governance a buying criterion
The controls should match what's at stake.
- 1Recommend a follow-upLog it
- 2Draft a messageA person reviews before it's sent
- 3Update a recordApproval and an audit trail
- 4Change a PO or scheduleApproval by role, and a way to reverse it
- 5Touch a quality releaseOnly through your existing QA controls
Higher consequence, stronger control
Ask every vendor
- What data can it access, keep or send to outside models?
- Can permissions be limited by user, system, record and action?
- Is every action logged with evidence and an owner, and can it be reversed?
- What are the retention, deletion, incident and model-training policies?
- What happens if the vendor, model or connector goes down?
For manufacturing
AI should never override a quality hold, release a batch or approve a material substitution without your required controls and approvals.
07
Price vendor dependence like technical debt
Depending on a vendor isn't bad. Not knowing what it would cost to leave is.
Your data and history
Can we export it in a usable format?
Workflow rules and configuration
Do we own them, and are they documented?
Integrations
Built on documented, maintainable interfaces?
The AI model
Can we change model or provider without rebuilding?
Ask every vendor
- What happens to our workflows if the vendor changes its pricing, product or model provider?
- What are the termination, data-export and transition terms?
- What expertise would we need to run or migrate it ourselves?
The goal
Understand the dependence, price it, and keep reasonable options.
08
Know what the price buys
What you saw in the demo isn't always what you signed.
Standard product
In the price, available today
Roadmap
Promised, not yet shipped
Custom work
Extra cost, often a change order
Ask every vendor
- Which capabilities from the demo are in the contract?
- Which integrations, workflows, environments and users are in scope?
- What counts as acceptance, and who pays if a requirement isn't met?
- What triggers a change order or an extra license?
Protect the business case
Put the workflow, assumptions, exclusions, acceptance criteria and success metrics in writing.
09
Buy, build, extend or connect
There's no universal winner. It depends on the workflow.
1Is the problem common, and does a product already cover most of it?
YesBuy
Check workflow fit, integration depth, total cost and the limits of configuration.
2Does your ERP, MES or QMS already have the data and the capability?
YesExtend
Check it can run the cross-system workflow without heavy customization.
3Does the workflow set you apart, and can you maintain it?
YesBuild
Check time to value, lifecycle cost, ownership, reliability and opportunity cost.
4Is the real problem context scattered across tools?
YesAdd an AI layer across your systems
Check cross-system evidence, permissions, exception handling, action controls and resolution outcomes.
A useful rule
Buy commodity capabilities. Build only where the advantage justifies the burden. Extend existing systems when they genuinely solve the problem.
10
Run a pilot that proves operational value
Test the business process, not the model's best answer.
Pick one bounded workflow, such as supplier delivery exceptions, material shortages, engineering change impacts or quality-documentation gaps. Agree the plan before day one.
Measure outcomes, not output
Time from detection to resolution, hours spent gathering context, at-risk orders caught early, avoided expedite costs. Model accuracy and estimated hours saved are diagnostics, not outcomes.
Worksheet 3
Pilot plan
Agree every line with the vendor before day one. Sign it.
How we run ours
Workbench Pilot: $5,000 for 6 weeks, credited in full to setup.
One workflow, your team's real data, read-only. Built on your production setup, so a pilot that meets its criteria becomes phase 1.
Compare vendors consistently
Your AI buying scorecard
Rate up to three options. The weighted score updates as you go.
- Business outcome and measurable ROI · 20%
- Fit with actual operational work · 15%
- Cross-system context and integration · 15%
- Implementation effort and time to value · 10%
- Total cost of ownership and pricing clarity · 10%
- Security, governance and reliability · 10%
- Action, follow-through and outcome verification · 10%
- Vendor flexibility and exit options · 10%
Business outcome and measurable ROI · 20%
Fit with actual operational work · 15%
Cross-system context and integration · 15%
Implementation effort and time to value · 10%
Total cost of ownership and pricing clarity · 10%
Security, governance and reliability · 10%
Action, follow-through and outcome verification · 10%
Vendor flexibility and exit options · 10%
| Dimension | Weight | |||
|---|---|---|---|---|
| Business outcome and measurable ROI | 20% | |||
| Fit with actual operational work | 15% | |||
| Cross-system context and integration | 15% | |||
| Implementation effort and time to value | 10% | |||
| Total cost of ownership and pricing clarity | 10% | |||
| Security, governance and reliability | 10% | |||
| Action, follow-through and outcome verification | 10% | |||
| Vendor flexibility and exit options | 10% | |||
| Critical failure? Security, compliance or a must-have not met | ||||
| Weighted score (out of 5) | 100% | Not rated | Not rated | Not rated |
Rate 1 (poor or unproven) to 5 (strong and demonstrated), using evidence from a real workflow, not vendor claims. Adjust the weights to your risk profile. A high total never makes up for a critical failure. Everything you enter stays in this browser tab.
Red flags
- A great demo, but no clear answer on what it takes to deploy on your data
- ROI built on theoretical time savings, with no baseline
- No integration, maintenance or staffing in the business case
- A new process or workspace, with no operational reason given
- No evidence behind recommendations, and no answer for when it's wrong
- Alerts that leave your team to investigate, assign and verify by hand
- Capabilities shown in sales that aren't in the contract
- No clear way to export your data or leave
The bottom line: buy outcomes, not AI features
The best investment isn't the most features, the most autonomous agent or the lowest price. It's the approach that solves a real problem, fits how you work, earns trust through the right controls, and pays back at a cost you understand. Before you commit:
- 1
Will it help us find and resolve an important problem better than we can today?
- 2
Do we understand the full cost and effort of making it work?
- 3
Can we prove the value, and keep control of our data, workflows and future choices?
Can't answer all three? You're not ready to buy. You're ready to ask better questions.
Get the PDF
Send the guide to your inbox.
The full guide, the three worksheets and the scorecard as a printable PDF, to share with your team or bring to vendor meetings.
A different way to think about operational AI
See across systems. Act on what matters.
Important work often falls between the systems you already use. LevelOneAI connects the information across them, so teams see what matters, what's affected, and move the work to resolution, without replacing your systems.