In practice I: AI trust in real products

The preceding three articles made three arguments. Trust is the currency for value and infrastructure. Immaturity is measurable and reparable. And people stay for conditions that reflect and respect their nature, rather than for features. Below are some examples of what that looks like in current products, projects and engagements, seen through a few key questions.

Does any of this show up in a live product, or is it a position?

It shows up first as a measurement problem.

A support assistant that changed the metrics

We enhanced the conversational capability of an IT support assistant in a constrained, high-value, customer-facing environment. Before release, the new behavior raised new questions and challenges. They were different from how we built and measured KPIs and SLAs in the past, even in the previous, more controlled version.

We had to answer two questions we did not expect to ask:

• How does a session close, or how do we determine that it has closed?
• How do we know the person experienced the exchange as successful, beyond a satisfaction score?

Organic conversations do not follow a script

Standard support indicators assume a clear path:

1. the ticket opens
2. self-service knowledge is provided
3. the ticket is resolved
4. a CSAT score is collected
5. the record closes on both ends, if there is no further inquiry or need.

Organic conversations, however, do not work the same way. They follow a different build-up that can be divergent or scant and still be a positive experience. That is why two months went into building the behavior and the measurements that fit what was actually happening. Or rather, what the evolution of the engagements was asking of it.

Results after release

After release, negative appraisal and reported defects fell significantly. In addition, escalation beyond first-line support was reduced, and the use of parallel channels and unrecorded routes dropped. New types of tickets also began to be captured and measured alongside it, based on processes and journeys.

What happens the moment someone realizes they are talking to an AI?

The uncanny valley predisposition is activated. As a result, expectations rise, especially without proper instruction or introduction. People then become more unforgiving when something fails or falls short of their expectations.

Why AI loses trust faster

Nobody needs to rebuild their confidence in a water bottle. After all, an ordinary tool that fails is just a tool that fails. By contrast, a system designed to present itself as understanding you, and then fails, has broken something else. Because it simulates human nature, AI produces distrust faster and recovers trust more slowly.

Where AI sits in a journey is a design decision

Our design leadership in Brazil shares the evolving terms we have adopted as part of our workflow. For example, where AI sits inside a journey is a design decision, rather than a user’s choice. The same goes for the designed behavior, which depends on the nature of the experience objective.

Visible and chosen AI is one thing; embedded and invisible AI is another. Both serve a purpose and carry risk. This is why organizations are still exploring the topic while they try to define governance, ownership, accessibility, security limitations and orchestration.

If people are unforgiving, what can you actually do about it?

An AI platform within a client environment offers a clear case with an unexpected finding.

When adoption drops in silence

Adoption slowed down quickly after just a few engagements. The platform did not lack capability or correct knowledge training. Instead, people tried it once or twice, and it did not produce what they expected.

There was no complaint, no ticket and no signal in any channel built to catch dissatisfaction. Perhaps a community channel described bugs, but it was unclear how to express a lack of trust or capability. In short, a poorly framed and poorly explained tool at first impression meant no further engagement and very brief retention.

Instruction and AI literacy

Subsequently, what helped was a social strategy around instruction and AI literacy. This is more than onboarding. Enough context conveys what the system can and cannot do, neither too much nor too little. In this way, it creates a moment of mutual respect that runs in both directions.

The person forms a working model, adjusts how they engage and takes ownership of the exchange. Thus, expectation management becomes flexible and measurable, instead of brittle and lost.

There is, however, a counterexample we run into constantly. Even with an introductory message explaining what a support agent does, a striking number of people ask it for a chocolate cake recipe. That is not misuse. It is a habit imported from using AI in search, which is where most of their AI use comes from.

A conversation between our design leadership in Spain and Brazil

Spain: What I have started to find valuable is when a product stops and tells you to verify something. Or when it says outright that it does not know, shares its limitation plainly and points to where you can look instead. This accountability and shared responsibility do two things. First, they build more trust, not less, based on the actual capability and limitations of the tool. Second, they leave room for the person to bring their own information into the exchange, refining it and creating a sense of collaboration and control: a co-directed mission. When something is delivered with too much confidence, what you get is deference, and deference closes the conversation.

Brazil: It is also a way to spend more tokens.

Spain: It is. Both things can be true. Which makes me wonder: what is the objective we want to achieve here? Can we find a balance, or choose what is co-directed and what is not? Or does this become an opportunity for a new business model altogether?

These questions do not have an answer yet, nor can we say with certainty that this will be our priority. Still, they rise as we continue to create focus and relearn our own processes. Perhaps continuing to find the right questions is exactly where we need to be now. Meanwhile, we keep testing, stay curious and continue to shape a human-centered framework for designing AI experiences.

When does your own data mislead you?

When it measures what it was built to measure, but the product has already evolved beyond it.

A 93 percent satisfaction score that hid problems

A digital employee experience study at a large client engagement reported 93 percent satisfaction with the service desk. However, underneath it sat a perception of the tool that did not match. It was concentrated around device performance and difficulty with work documentation.

A high score can become misleading if we do not try to understand the nuances behind it. Treated as a question rather than an answer, it opened a program of work. We have now seen this trend across several clients. As consultants, it is our responsibility to make sure the practices presented in this series can be audited and support the narrative closest to the employee’s reality.

Metrics that look like failure but are not

The same gap appeared in a platform we redesigned around expectation rather than usability. In the first month, user acquisition rose approximately 20 to 30 percent in some regions and around 40 percent in others, compared with the preceding month. At the same time, the number of pages people visited fell, as did the time they spent. Meanwhile, downloads and completion actions rose, and sessions from returning users grew significantly.

On a conventional dashboard, half of those movements would easily read as failure. Yet all of them point the same way once you ask what the product is for and which behavior we want to promote. In a product whose job is to answer and release the person back to their work, staying becomes the sign of failure.

A working hypothesis, not a conclusion

We hold that as a working hypothesis rather than a conclusion. This is mainly because we are still formalizing how to measure the intended behavior and which data to capture to test it. In addition, three other releases were competing for attention in the same window, and there was no formal campaign. A community channel and an ambassador network were also active. Therefore, these factors may have influenced the outcome, not just the redesigns.

How does the way a system speaks change anything measurable?

It turns out to be a support metric.

Emotional tone as a governed property

At a large enterprise client, we defined emotional tone as a governed property held in a data hub. In other words, it was no longer a copy decision made locally by whoever was closest to the feature. It shaped employee experience directly, as well as whether people regarded the tooling as trustworthy.

Alongside careful work on the chat environment and its affective vocabulary, this improved self-service success. It also reduced avoidable contact with IT support. Those are cost lines. Even so, the connection between how a system speaks and whether a person resolves their own issue is not intuitive. Most delivery organizations treat tone as polish applied once the function works.

What the EU AI Act allows and prohibits

To be precise: designing how a system speaks, with affectiveness and a positive tone, is not an inference of an employee’s emotional state. Nor is it affective mirroring as we understand it in a two-person dialogue. In fact, that kind of inference has been prohibited in workplace settings under the EU AI Act since February 2025 (European Union, 2024, Article 5(1)(f)). The two are easily conflated, but only one of them is lawful here.

Where does this leave the discipline?

Somewhere uncomfortable, and discomfort is the point.

A sales platform built with AI

We consulted on a sales platform for a large organization, built with Claude AI coding and design tooling. The work was good. Its developer captured the sales path with an intricacy we know well because, although building for the department, the creator worked primarily in a sales environment.

What was left out: room to push back

However, the journey design left out space for the user to answer back. The interface is dense with functions and tools the system provides and offers very little room to question it. Its own objective reveals this: the tool was meant to help optimize, not to help validate.

That omission was structurally predictable. After all, a space for a person to push back is not a feature anyone requests. It does not appear in the tooling users are trained on, and it takes up interface real estate that could hold another appealing function. Nothing in the production process surfaces it. It only surfaces when someone asks what happens to the person’s judgment. Or when the person receiving the output analyzes beyond what was produced and sees that “something is off”.

This raises the question that the next article answers: which specific part of this work requires judgment, and what unique value does this practice provide that no other function does?

Reference

• European Union. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). Official Journal of the European Union.