Evaluating AI Fit for Any Industry
Every industry has specific, high-value problems. Not all of them are well-suited to AI. Applying AI where it does not fit wastes budget and builds scepticism. Identifying where it does fit — and structuring the deployment correctly — is one of the most valuable skills for AI leaders, product managers, and consultants. This page provides a structured framework for making that assessment.
When AI Fits: The Core Conditions
AI excels in specific conditions. Before investing in an AI solution, evaluate whether the problem meets these criteria. The more criteria a problem satisfies, the better a candidate it is for AI.
Clear Pattern in Historical Data
AI learns from data. If there is sufficient historical data where the right answer is known (labelled examples), AI can learn the pattern. Fraud detection works because banks have millions of labelled fraudulent and legitimate transactions.
High Volume, Repetitive Judgments
AI pays off where humans must make the same type of judgment thousands of times per day. Radiology reads, loan applications, customer service queries, invoice processing — high volume tasks where AI operates at marginal cost per unit.
Measurable Ground Truth
AI models need to be evaluated. A task where the correct answer is verifiable (was this transaction fraudulent? did this part pass inspection? did this patient deteriorate?) allows rigorous model evaluation and continuous improvement.
Tolerance for Imperfection
AI models are probabilistic — they make mistakes. A use case where AI acts as a first filter (with human review for edge cases) tolerates imperfection better than one where AI makes final autonomous decisions with no review.
Speed is Competitively Important
When humans operating at human speed are a bottleneck — reviewing contracts, processing claims, flagging anomalies — and faster processing creates business value, AI's throughput advantage is directly monetisable.
Consistent Input Format
AI models perform best on inputs in a consistent format — images, tabular records, standardised documents. Highly variable, context-dependent inputs (open-ended negotiation, novel legal situations) are harder and more error-prone.
When AI Does Not Fit
No Historical Data
A new product, a novel market, a situation without precedent — AI cannot learn patterns that do not yet exist in data. You may need rule-based logic, expert systems, or simulation instead.
Low Volume, High Nuance
A bespoke task done 10 times a year by an expert who applies deep contextual judgment is not a good AI candidate. The cost of collecting training data and validating the model exceeds the value of automating 10 events per year.
Zero Tolerance for Errors
Autonomous AI decisions in safety-critical contexts (flight control, surgical robotics, nuclear plant control) require extraordinarily high reliability standards, extensive validation, and regulatory clearance. Most AI is not ready for zero-tolerance autonomy.
The Problem Is Actually Unclear
"We need AI" is not a problem statement. If the actual business problem is not clearly defined, AI cannot solve it. Many failed AI projects have failed at problem definition, not model training.
Data Requirements Checklist
Data quality and quantity are the primary constraint on AI performance in most industry deployments. Before committing to AI development, assess the data situation honestly.
Questions to answer before starting
- Volume: How many labelled examples exist? Classification tasks typically need thousands to tens of thousands; complex tasks (medical imaging, fraud) need millions.
- Quality: Are labels consistent and accurate? Label noise (inconsistent human labelling) directly limits model ceiling performance.
- Recency: Is historical data still representative of current conditions? Distribution shift (the world changed since the data was collected) breaks AI models.
- Availability: Is the data accessible to the AI team? Siloed data, privacy restrictions, and legacy systems that cannot export structured data are common blockers.
- Balance: Are positive and negative examples proportionate? A fraud dataset where 0.1% of records are fraud requires careful handling; a model predicting "not fraud" always achieves 99.9% accuracy while being useless.
- Coverage: Does the data cover the distribution of real-world inputs the model will encounter? Data collected from one hospital, one region, or one product line may not generalise.
Regulatory & Compliance Constraints by Sector
Regulatory constraints are often the deciding factor in AI fit — not because AI cannot technically work, but because the compliance burden makes deployment prohibitively expensive or slow.
| Sector | Key Regulations | AI Impact |
|---|---|---|
| Healthcare | FDA SaMD, HIPAA, EU MDR, AI Act | Clinical AI requires clearance; PHI requires BAA |
| Finance | SR 11-7, ECOA, GDPR, EU AI Act | Model risk management; fair lending explainability |
| Legal | ABA ethics rules, Bar regulations | Verification obligation; confidentiality of client data |
| Education | FERPA (US), GDPR (EU), COPPA | Student data privacy; parental consent requirements |
| Manufacturing | Safety standards (ISO, IEC), export controls | OT cybersecurity; industrial safety compliance |
Build vs Buy vs Partner
Buy (SaaS/API)
Use when: a vendor solution already solves your problem adequately, speed to value matters, and your differentiation is not in the AI itself. Examples: off-the-shelf contract review, AI customer support, standard fraud detection.
Fine-tune / Adapt
Use when: a foundation model gets you 70-80% of the way and you need to adapt it for your domain, terminology, or data format. Fine-tuning a general LLM on medical notes or legal documents is faster and cheaper than training from scratch.
Build Custom
Use when: your use case is unique, your data is proprietary and a key competitive asset, vendor solutions do not exist, or regulatory constraints require full control over the model. Highest cost, highest differentiation.
Measuring ROI for Industry AI
AI ROI is measurable, but requires establishing a baseline before deployment and committing to a measurement approach upfront. Post-hoc ROI measurement is often gamed or contested.
ROI measurement framework
- Baseline: Measure current performance (throughput, error rate, cost per unit, time per task) before AI deployment.
- Direct cost reduction: Labour hours saved × cost per hour, or defects avoided × cost per defect (recall, rework, warranty).
- Revenue uplift: Faster processing enabling more throughput, improved customer experience metrics, or new products enabled by AI capability.
- Risk reduction: Quantify the expected value of avoiding fraud losses, compliance failures, or downtime events — harder to measure, but often the largest component.
- Total cost of AI: Include model development, infrastructure, data labelling, ongoing maintenance, monitoring, retraining, and human oversight costs — not just the initial build.
Common Failure Patterns When AI Enters a Vertical
The Pilot That Never Scales
A successful pilot in one clinic, factory, or office fails to scale because the conditions that made it work (clean data, engaged users, a champion executive) do not exist elsewhere. Pilots must be designed for transferability, not just success in isolation.
Solving the Wrong Problem
Teams build an AI to optimise a process metric (throughput, speed) that does not map to the actual business outcome (profitability, patient outcome, customer satisfaction). Optimising the wrong metric at scale creates sophisticated failure.
Ignoring the Human System
AI changes workflows. If the humans in the loop are not involved in design, do not trust the system, or work around it rather than with it, AI creates no value regardless of technical performance.
Underestimating Data Work
In most industry AI projects, 60–80% of the effort is in data preparation — cleaning, labelling, integrating, validating. Teams that budget for modelling but not for data work run out of budget before reaching production.
Checklist: Do You Understand This?
- List five conditions that make a problem a good candidate for AI.
- What types of problems are poor candidates for AI, and why?
- What six data quality dimensions should you evaluate before committing to AI development?
- When would you choose to buy a SaaS AI solution vs fine-tune a foundation model vs build custom?
- What is baseline measurement, and why must it happen before deployment rather than after?
- Why do many AI pilots succeed but fail to scale, and how can this be prevented?