AI Phone Assistants in the Enterprise: Transparency, Data Protection and Safe Processes

The essentials in brief

• An AI phone assistant is not just a communication tool but part of corporate IT and data processing.
• Under Art. 50 of the AI Act, people must in principle be able to recognise when they interact directly with an AI system, unless this is obvious.
• Transparency should be established at the start of the conversation — clear, understandable and accessible.
• Data protection questions concern call content, contact data, retention, access and connected systems.
• Organisations should define clear boundaries: what the AI may handle itself, when a human must take over.
• A structured readiness check is often more sensible than a purely technical go-live decision.

Why AI telephony is a governance topic

Telephony in many organisations is still treated as a purely operational channel: someone calls, someone answers, the request is handled. Once an AI system takes over that process, additional questions arise.

These questions sit between IT, data protection, the business unit, information security and process ownership. That is precisely why AI telephony should not be treated as a mere tool rollout.

A concrete example from practice is the AI phone assistant by Telfo. Telfo is developed in Mannheim, Germany, and handles recurring inbound first contacts for companies — from automated call answering and understanding freely formulated requests to preparing callback and appointment processes. This example illustrates well which governance questions should be answered before production use.

• Does the caller know they are speaking to an AI?
• Which personal data is processed?
• Which call contents are stored or shared?
• Which subcontractors or technical services are involved?
• Which internal systems may be addressed?
• Who may access call data?
• What happens on misclassification or uncertainty?
• When is a call handed over to a human?

Art. 50 AI Act: transparency in direct AI contact

Art. 50 of the EU AI Act contains transparency obligations for certain AI systems. For systems intended to interact directly with natural persons, the key point is that affected persons must in principle be informed that they are interacting with an AI system, unless this is obvious to a reasonably informed, observant person. The transparency information must be clear and distinguishable and provided no later than the first interaction or exposure.

For an AI phone assistant this means in practice: the information about the AI should not be hidden somewhere in a privacy policy but meaningfully integrated into the actual interaction. A possible phrasing: “You are speaking with our AI-powered phone assistant.”

Which phrasing is sufficient and appropriate in a concrete deployment depends on the setup and context. Organisations should therefore match regulatory requirements against their concrete role and use. We break down the full systematics of the four duties — including which bind the provider and which the deployer — in our Article 50 transparency article.

Provider, deployer, user: determine roles cleanly

One of the most important governance questions is: which role does which organisation hold in the concrete AI setup? The AI Act distinguishes among others between providers and deployers. Which obligations apply depends on who develops a system, places it on the market under its own name or deploys it in its own organisation.

For a company using an external AI phone assistant, knowing the product name is therefore not enough. What also matters is:

• Who provides the AI system?
• Who configures dialogue logic and content?
• Who decides on purposes and scope of use?
• Which components come from subcontractors?
• Which obligations does the provider take on contractually?
• Which obligations remain with the deploying organisation?

Data protection starts with: which data does the call really need?

A phone call can quickly contain personal information. Name, phone number and appointment request are obvious. Depending on the industry, addresses, customer numbers, contract data or particularly sensitive information may also be mentioned.

A clean process therefore starts with data minimisation: the AI should only ask for the information the respective process actually requires. For a simple callback, name, callback number, the request and, where applicable, a preferred callback time are usually enough.

Additional data should not be collected merely because the technology is capable of it.

Call data, transcripts and retention

Before the go-live, organisations should clarify precisely which call data is technically generated. The answer is product- and configuration-dependent. What matters is that these questions are answered and documented before productive use.

• Are audio recordings stored?
• Are transcripts created?
• How long is call data retained?
• Can retention be reduced or disabled?
• Who has access?
• Which data flows into downstream systems?
• Is information used for model training?

Subcontractors and the technical supply chain

Technically, AI telephony often consists of more than a single system. Depending on the architecture, telephony, speech recognition, speech synthesis, the AI model, hosting, e-mail, CRM or calendar may use different technical components. Organisations should therefore evaluate not only the direct provider but understand the relevant technical supply chain.

Typical questions: which subprocessors are involved? In which countries is data processed? Which contracts and processor arrangements exist? Are there international data transfers? Which components see call content? Which availability dependencies arise?

Limiting access and permissions

Not every employee needs access to all call information. The permission model should follow the actual working process. A sales team needs different information than support or administration. Particularly sensitive data should only be visible where needed for handling it.

• role-based access,
• limited administration rights,
• logging of relevant changes,
• regular permission reviews,
• clear offboarding processes.

Which tasks an AI phone assistant should take on

A safe deployment starts not with “what can the AI do?” but with “which processes do we deliberately free up for it?”. The more clearly the process can be described, the easier it is to define boundaries, escalations and quality controls.

• callback requests,
• appointment requests,
• opening hours,
• location information,
• initial request classification,
• routing according to defined rules,
• structured capture of recurring requests.

Which tasks should remain with humans

Not every phone interaction is suitable for full automation. Organisations should be particularly cautious with complex legal or tax advice, medical or other highly sensitive expert decisions, contract and price negotiations, complaints and escalations, personnel decisions, and situations with high economic or personal risk.

Here the AI may take the initial intake, but should not automatically make the actual decision.

Human handover: escalation must be part of the design

A good AI process needs a defined exit. When a request cannot be understood reliably, falls outside the approved scope, or the caller explicitly asks for a human, there should be a clear handover path — depending on the organisation, for example direct transfer, prioritised callback, a ticket to the responsible team, an e-mail with structured conversation context, or an appointment with an employee.

A fallback is therefore not a sign of poor automation. It is part of controlled automation.

Telfo as a practical example of AI-driven call handling

Telfo develops AI phone assistants for companies. The assistant can take incoming calls, understand freely formulated requests, ask suitable follow-up questions and prepare defined follow-up processes such as callbacks or appointments.

For governance owners, the process view matters most: which call types are automated? Which information may the system ask for? Where is it passed on? What happens on uncertainty? Exactly these questions belong in the system rollout, not after the go-live.

A simple governance frame for AI telephony

The eight areas below work as a minimal control set before a fuller assessment pays off: for simple use cases a compact internal review is often enough. As soon as several systems, business units, data classes or regulatory requirements come together, a more structured assessment is worthwhile — for example the AI Readiness Assessment for the organisational frame or the AI Use Case Assessment for the use-case inventory.

Practical example: introducing an AI phone assistant in a mid-sized company

A mid-sized service provider wants to automate incoming callback and appointment requests. Instead of switching the AI on for all calls at once, the deployment is built up step by step.

This approach does not eliminate every risk. It makes the deployment controllable and auditable.

1. 1Suitable call types are defined.
2. 2A transparency notice for the start of the call is agreed.
3. 3Required data fields are reduced to the minimum.
4. 4Routing and escalation rules are defined.
5. 5Access and retention are specified.
6. 6Technical providers and data flows are documented.
7. 7The process starts with a limited scope.
8. 8Errors and edge cases are analysed.
9. 9Only then is the scope of application extended.

Typical mistakes during introduction

Checklist before the go-live

• Is the purpose of the AI phone assistant clearly defined?
• Is it transparent that callers interact with AI?
• Have the AI Act roles been assessed?
• Is the data-protection role allocation settled?
• Are only necessary call data collected?
• Are retention periods defined?
• Are subcontractors and data flows documented?
• Are access rights limited?
• Is there a clear human handover?
• Are sensitive and high-risk use cases excluded or specially secured?
• Are there quality controls and a complaints process?
• Is the deployment reviewed regularly?

Conclusion: good voice AI starts with clear boundaries

AI phone assistants can noticeably relieve companies in day-to-day business. But production use should not be measured only by how human a voice sounds or how many calls are automated. Transparency, data minimisation, controlled access, a comprehensible technical supply chain and a clear transition to humans matter just as much.

Art. 50 of the AI Act makes transparency in direct AI interaction a central topic. Data protection and information security add the question of how call data is processed and protected.

An AI phone assistant like Telfo can thus be part of a safe digital customer process — provided organisations govern not only the automation but also the governance cleanly.

For a systematic position check rather than judging by voice quality, our free AI Act readiness check offers a structured entry point: roles, gaps and next steps surfaced in minutes. Such assessments do not replace legal advice or a formal compliance review — they make the implementation status manageable.