Adding AI to Business Software: When Do the Benefits Justify the Cost?
Adding AI to business software pays off when it takes over a specific, frequent task where a mostly-right draft or suggestion saves real time and mistakes are cheap to catch – sorting enquiries, pulling data out of documents, drafting replies for a person to check. It rarely pays off when every answer must be exactly right, when the task is rare, or when an ordinary rule would do the same job. There is no universal return-on-investment figure; the way to know is a small pilot measured against how the work is done today.
Where AI tends to fit
- Turning messy text into structured data – enquiries, emails or supplier documents turned into fields in your CRM or order system, with a person confirming.
- Drafting – replies to common customer questions based on your own documents, or first versions of product descriptions that an editor checks (see what to check before publishing AI-written content).
- Classifying and routing – tagging support requests, detecting language and urgency, sending each to the right person.
- Searching and summarising – finding answers in your own manuals, contracts or long email threads.
Where it usually doesn't
- prices, taxes, invoice totals and anything else ordinary code calculates exactly;
- decisions with legal or financial effect on people, without a human deciding;
- tasks whose rules can simply be written down – a rule is cheaper, faster and predictable.
The costs that are easy to miss
| Cost | What it means in practice |
|---|---|
| Usage fees | Most AI models used through an API are billed by the volume of text processed, measured in tokens. Anthropic's price list, for example, quotes separate prices per million input and output tokens, with output priced higher, and a 50 percent discount for non-urgent batch processing. The bill depends on how many items you process and how long the texts are, so design decisions affect it directly. |
| Development and integration | Connecting the model to your data, building screens for review and correction, handling failures and timeouts. |
| Testing | A set of real examples with known correct answers, used to measure accuracy before launch and after every change. |
| Human review | Time people spend checking and correcting output. It is often the largest cost and the one most easily left out of a business case. |
| Mistakes that get through | A wrong address, a misread amount, an incorrect answer to a customer. |
| Model retirement | Providers retire older models. Anthropic's documentation says requests to retired models fail, and that it gives at least 60 days' notice for publicly released models. When that happens you switch and re-test – a real, provider-driven reason to change, not a routine upgrade. |
| Data protection | If personal data is sent to an AI provider, that provider processes it on your behalf: you need a proper agreement and to know where the data goes. |
Errors are part of the product
The US National Institute of Standards and Technology (NIST) lists "confabulation" among the main risks of generative AI and defines it as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived". The practical answer is to design for mistakes rather than hope they won't happen:
- the AI suggests, a person confirms anything that matters;
- answers are based on your own documents, with the source shown so it can be checked;
- uncertain cases go to a person instead of being guessed;
- corrections are easy, and they are logged so you can see how often they happen.
The more expensive an undetected mistake, the more review you need – and every minute of review comes off the time saved. That trade-off decides many business cases.
What the evidence says – and why to measure your own
Published studies show both real gains and real limits. In a study of 5,179 customer support agents, Erik Brynjolfsson, Danielle Li and Lindsey Raymond found that access to a generative AI assistant increased issues resolved per hour by 14% on average, including a 34% improvement for novice and low-skilled workers, but with minimal impact on experienced and highly skilled workers. The benefit depended on who used the tool.
A 2025 randomised study by METR points to a different lesson. Sixteen experienced open-source developers worked on 246 real tasks, with AI tools allowed or not. They expected AI to speed them up by 24% and afterwards believed it had sped them up by about 20% – yet tasks with AI took 19% longer. METR describes this as a snapshot of early-2025 tools in one specific setting and now marks the results as out of date, so the lesson is not "AI slows people down". It is that impressions are not measurements.
Neither study tells you what AI will do in your business. A pilot will.
How to run a pilot that gives an honest answer
- Pick one task and measure it today. Items per month, minutes per item, number of errors.
- Collect real examples. A few dozen past cases with known correct results; remove or mask personal data where you can.
- Build the smallest working version inside the tool people already use, not as a separate demo.
- Measure the same things. Share of items accepted without changes, time per item including review, errors that slipped through, cost per item.
- Compare with the alternatives. A better form, a simple rule, a template or a cleaner process may deliver most of the benefit – see which process to automate first.
- Decide against thresholds agreed in advance, so the decision doesn't come down to enthusiasm.
The arithmetic is simple: items per month × minutes saved per item × cost of a minute of work, minus usage fees, review time and upkeep. If the result is small or uncertain, wait.
EU rules to keep in mind
The EU AI Act entered into force on 1 August 2024 and became applicable on 2 August 2026, with some exceptions. According to the European Commission, AI literacy obligations have applied since 2 February 2025, and the transparency rules – including that people should be made aware when they are interacting with a machine, as with a chatbot – came into effect in August 2026. Strict obligations for high-risk uses, those that can pose serious risks to health, safety or fundamental rights, apply to most of them from 2 December 2027. The Commission also notes that most AI systems, such as spam filters, are minimal risk and get no new rules. If your use touches decisions about people, such as who is hired or who gets credit, check with a lawyer whether it counts as high-risk. GDPR applies to personal data either way.
When the simpler option wins
- the rules can be written down and rarely change;
- the volume is too low for saved minutes to add up;
- mistakes are expensive and hard to spot;
- the underlying data is scattered or unreliable – fix that first, often through integrations between your systems.
If you are considering AI in your own software, we can help you choose a task, build a small pilot into the system your team already uses and measure the result – and we will tell you if a plain rule would do the job better. Read about our web-based software development or contact us.
Sources
- AI Act – European Commission
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1) – NIST
- Generative AI at Work – National Bureau of Economic Research
- Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity – METR
- Pricing – Claude API Docs (Anthropic)
- Model deprecations – Claude API Docs (Anthropic)