The Importance of Empathy in Automation

To take AI beyond automation into true intelligence, we need to include empathy.

In this post, I want to discuss a real danger we may face with AI agents, drawn from a specific interaction I witnessed on LinkedIn.

Recently, on a post where someone was presenting ideas about “intelligent” AI agents, the author claimed that if a customer fails to pay a SaaS subscription, the system will block access, trigger an automatic charge and apply a late fee, all without any human intervention. This was promoted as something futuristic and advanced.

I saw a disaster waiting to happen, especially considering that the same outcome could be achieved without AI, if that were ever desirable.

What Was Not Considered: The Human Element

There are many situations that can lead to a missed payment. What happens, for example, if the client’s assistant falls ill and cannot process the payment? What if they had a medical emergency and no one took over the task in time? What if the invoice never arrived? What if the client is going through a temporary hardship, but has been a loyal customer for years? Does that count for nothing? Where is the human element in the consumer relationship?

These are not far-fetched hypotheticals. They have happened with my own SaaS customers.

Automation without humanity is oppression at scale.

We could easily implement the AI agent’s suggestion and follow the same path: missed payment, blocked account and a fine applied. But is this the right approach?

My background in neuroscience and communication reshaped how I think about the human side of technology-driven processes. That plus some life experiences lead me to consider the broader context:

• In many jurisdictions, consumer protection laws prohibit unilateral penalties without proper notice. Implementing such a drastic automatic response could have legal implications.
• In society, there is the principle of good faith. Every commercial relationship presumes that both parties act honestly until proven otherwise. This action could have serious consequences for that business relationship and reputation.
• We have to remember to consider the human context. Behind every business and every overdue charge, there are people with stories, setbacks and circumstances that must be considered before any penalty is applied.

What We Should Actually Build

A genuinely intelligent AI agent does not block first and ask questions later. It recognizes patterns. Has this client always paid on time? Then the delay probably has a reason. The system could send an empathetic notification before taking any action. It could offer a courtesy window. It could escalate the issue to a human when the situation is ambiguous. An intelligent agent could prioritize retaining a client, which is often worth far more than punishing a late payment.

For those building AI agents: before you automate punishment, automate empathy. Your system will reflect your values. If you build without considering the person on the other side, you could be constructing a scalable injustice machine, one capable of generating damages that could exceed any original problem the AI was trying to address.

Technology without empathy is not innovation. It is regression with a polished interface.

Conclusion

We are at a moment where the architecture and design decisions we make today will shape how millions of people are treated by systems over which they have no control.

And to help avoid building oppressive and unjust machines, we need to cultivate empathy and uphold our organizations’ principles, with the greater purpose of serving people rather than merely extracting value from them.