Share to gain more social capita
Written by — Erika Vitikainen, Project Delivery Lead
Written by — Erika Vitikainen, Project Delivery Lead
Share to gain more social capita
Please note this is a translation of Erika Vitikainen's blog "Mikä AI-projektin johtamisessa oikeasti muuttuu?" that was originally written in Finnish.
🇫🇮 Tämä blogi on alun perin kirjoitettu suomeksi. Löydät suomenkielisen version oikeassa yläkulmassa olevasta kielipainikkeesta.
In software development, we're used to defining how a system should work. When the value of a shopping cart exceeds 100 euros, you get a free cat. The rule is implemented and tested.
But what if we want to predict customer churn or build an assistant that understands customer questions? Not every correct answer can be predefined as a rule. The quality of the solution also depends on what the model has learned from data and how it applies that knowledge.
This difference is described by the terms deterministic and probabilistic. The first produces the same result from the same input, while the second is based on probabilities and uncertainty. In other words, we're now operating in a world of uncertainties and probabilities.
An AI-generated answer can be wrong, even when delivered with more confidence than anyone else in the meeting. That doesn't mean all AI systems produce different answers to the same input, or that traditional software is free of errors. The key difference is this: just because the functionality is finished doesn't mean an AI solution performs its task well enough.
Using AI in development also challenges how we measure success. In Atlassian's State of Product 2027 survey, 80% of respondents felt that AI was helping them release products faster without delivering value to customers any sooner. So we're picking up speed, but often still waiting for the results.
.jpeg?width=400&height=533&name=IMG_0845%20(1).jpeg)
Photo: Erika at her first Recordly hackathon
"We should do something with AI." Sure, we probably could. The possibilities are exciting and tempting, and it's easy to want to jump straight into experimenting. But before we fill up our calendars, let's answer a few questions:
After answering these questions, the backlog might look quite different. Perhaps the first step is figuring out whether the data even contains the information the solution needs. Can we identify customers who are about to leave, or can we mainly see those who have already left? The latter is useful information too, but we're a little late if the goal was to keep them.
You can set a timeline and budget for the investigation, but you can't promise a positive outcome in advance. So plan what you need to learn before making the next investment decision. Redefining the scope might turn out to be the best decision of the entire project, even if it means changing a brilliant plan.
When we talk about AI, the focus easily shifts to the model itself. What gets less attention is how much work can go into data availability, quality, preparation, and maintenance. In an AI solution, much of the logic comes from the data, which means data work is part of product development, not just a preliminary step.
Even the most sophisticated model can't decide which of a company's five conflicting guidelines is the correct one. Application development and data work therefore need to go hand in hand. The user interface might be ready even when the information behind it is already outdated.
Who owns the data, and who's responsible for keeping it up to date? These tasks need room in the project plan, even if they don't make for the most impressive demos. AI creates new roles and changes the skills organisations need.
Data maturity is like the foundation of a house on which an AI solution is built. A lightweight experiment might work just fine using a file called jarmon_excel_final, but reliable production use at a larger scale often requires a proper data platform and shared ways of working.
We also need common criteria for evaluating the quality of results, along with evaluation tasks that reflect real-world situations. An awkward sentence in marketing copy and incorrect information in a contract are two very different kinds of mistakes. Defining what's "good enough" therefore requires business expertise. It can't simply be left to the technical team to do some testing.
A limited pilot with real users shows whether the overall way of working actually improves. If an answer takes one minute to generate but ten minutes to verify, it's worth revisiting the business case. Otherwise, we might have automated one task only to create two new ones.
Monitoring also needs an owner and a budget after launch. As data and operating conditions change, a solution that worked once won't necessarily keep working.

An AI project isn't just about introducing a new tool. It's also about agreeing on what the machine is allowed to decide and what remains the responsibility of humans.
One person sees opportunities in the project, another worries about the future of their skills, and a third is concerned about being held responsible for the system's mistakes. These concerns can't be addressed through onboarding training alone, even if there's a whole five minutes reserved for questions at the end. We need conversations about how work will change, who has decision-making authority, and how problems will be handled.
Not all scepticism is resistance to change, either. An expert pointing out an important edge case might provide the most valuable feedback of the entire project. Involve future users in defining success and evaluating results. At the same time, agree together on how work and responsibilities will be divided between people and machines.
Trust grows when people understand both the benefits and limitations of a solution and have a say in how their own work changes.
AI projects still need goals, clear scope, collaboration, and decisions. But they also need evidence that the data and the solution can actually deliver the intended results.
So, in the next project meeting, alongside the timeline, I'd ask one more question: what have we already proven to work, and what are we still just really hoping will work?