Explainable AI: Trusted Use of Artificial Intelligence in Customer Service Automation

Get started with the most important terms, explained

AIC4
The AIC4 (Artificial Intelligence Cloud Service Compliance Criteria Catalogue) defines minimum requirements for the secure use of machine learning methods in cloud services.

Bias
Bias describes a systematic error that can result from either insufficient data or judgment errors. A cognitive bias in machine learning models can lead to discrimination of certain people.

BSI
The Bundesamt für Sicherheit in der Informationstechnik (BSI) is the German government's cybersecurity authority and is shaping secure digitization in Germany. The AIC4 criteria catalogue was largely developed by the BSI.

C5
The Cloud Computing Compliance Criteria Catalogue (C5) defines minimum requirements for secure cloud computing and is primarily aimed at professional cloud providers, their auditors, and customers.

Cognigy.AI
With the Conversational AI platform Cognigy.AI, companies can deploy intelligent voice and chatbots across the organization to automate their customer/employee communications at all touchpoints.

Conversational AI
Conversational AI is a branch of artificial intelligence that utilizes software and technologies such as natural language processing, machine learning, and automatic speech recognition to facilitate communication between a human and a machine.

Explainable AI
Explainable AI is a concept that makes artificial intelligence methods, e.g. neural networks or deep learning systems, explainable and comprehensible. Among other things, it is intended to solve the so-called "black box" problem, meaning that it cannot be clearly explained how a machine learning model reaches a decision. The need for explainable - and thus trustworthy - artificial intelligence is an important action field in the AIC4 catalog.

NLP
Natural Language Processing (NLP) enables computers to understand and interpret human language.

NLU
Natural Language Understanding (NLU) is a sub-field of NLP that explicitly deals with the understanding of human language. It is primarily concerned with nuances such as context, mood (so-called "sentiment"), and syntax.

PwC
PricewaterhouseCoopers (PwC) is Germany's leading auditing and consulting firm. The company audits AI services as part of the AIC4 criteria catalog and provides objective audit reports.

A brief summary of Trustworthy AI

This article serves as an implementation-oriented guide on the topic of "Trustworthy AI" for decision-makers in the areas of customer service and compliance. It is built upon the European Union's efforts to formulate ethical frameworks for the development and use of cloud-based AI applications and establishes these in the business world with the help of an audit, the AI Cloud Service Compliance Criteria Catalogue (AIC4).

Especially in the area of customer service automation, the topic of "Explainable AI" - meaning the establishment of trustworthiness and accountability - plays a special role, since a direct interaction with the AI application takes place. Companies should therefore choose Conversational AI applications that meet all action areas of the AIC4 catalog. This is a good way to ensure a future-proof deployment.

In our whitepaper " Trusted Use of Artificial Intelligence in Customer Service Automation", we show in detail which specific requirements AI applications from the Conversational AI domain have to pass an AIC4 audit. Since Cognigy.AI was one of the first AI platforms worldwide to successfully complete the AIC4 audit, we also provide valuable insights into auditing practices as well as the tools Cognigy.AI uses to ensure the trustworthiness of its AI applications.




This article is a brief summary of the whitepaper, focusing on the ethical framework of trustworthy AI ("AI ethics") and the 7 action areas of the AIC4 catalogue.

Even though auditing cloud-based AI services is still a fairly new measure for establishing trust in the field of AI, it can be assumed that AIC4, similar to the Cloud Computing Compliance Criteria Catalogue (C5), will become a globally recognized auditing standard in the future. The AIC4 will thus act as an important source for IT, ethical and legal evaluations of AI services. Companies that rely on appropriate technologies can thus secure a competitive advantage in the strife for global leadership as innovation drivers in their industry by using Trustworthy AI within the framework of the AIC4.

An introduction to Conversational AI and automation

 
The phonebot: Next-generation telephone support

Many consumers already use the convenient option of contacting companies via chatbots. Many customer requests are answered automatically. What is new, however, is that this technique can also be applied to customer service on the phone. Phonebots provided by Cognigy.AI can understand telephone inquiries and conduct multilingual telephone calls. This already works so well that almost natural conversations can be held between humans and bots on the phone. Customers benefit from the elimination of waiting times and round-the-clock availability. This means that the customer's problems and questions can be dealt with at any time.

• The McKinsey Global Institute predicts that AI will add $13 trillion in value by 2030
• The global gross domestic product will grow by 1.2 percentage points as a result of the AI sector. This growth would be equivalent to that of all three previous industrial revolutions combined.
• A look into the future from PwC states that nearly 60% of the expected AI growth will be stimulated primarily by more user-centric products
• The healthcare industry, as well as the automotive industry, will be the biggest winners of AI progress

What is Trustworthy AI / Explainable AI?

The development and perception of many AI applications go through a so-called "hype cycle," as outlined by the market research company Gartner, for example. At the beginning, a new technology triggers a trend that can quickly become a hype. While in the hype phase, this technology is often overestimated and potential future scenarios are drafted that cannot yet be realized, a return to reality can be observed after some time. In the case of AI solutions, it is usually only after a completed hype cycle that it becomes clear whether the technology is really innovative, robust enough, and adapted for further sustainable development and market acceptance. This volatility often leads to social uncertainty.

Companies using AI technologies are therefore faced with the challenge of pushing the boundaries of what is possible with the help of the latest technology, while at the same time creating trust for the (AI) services in use. This trust arises, amongst other factors, from ethical frameworks that define the use of AI in the overall social system.  

Due to tensions in this area, a political process was initiated at the European level in 2018 that measures artificial intelligence against the backdrop of its "trustworthiness" (Trustworthy AI). As a result, ethical guidelines for Trustworthy AI were established in 2019. They further define the term "trustworthiness" in relation to AI applications in seven different action areas and can therefore be used as an (indirect) definition of the term.

Trustworthy AI occurs when

• AI software components are traceable and controllable by humans
• an environment is established that is protected from unwanted access and interference
• principles of data protection and data management are followed
• fundamental rights are respected while promoting diversity, non-discrimination, and fairness
• the focus is on environmental and social well-being
• sufficient transparency is established, meaning results are verifiable and adaptable while providing the necessary legal remedies to legitimize the use of AI

The importance of setting these ethical and technical guidelines become obvious when looking at the numerous IT, philosophical, and legal challenges that arise in connection with AI applications:

The AI component of a software thus influences human-machine interaction more than other parts of the system.

One example of this is the intelligent chatbot of ARAG Insurance. It is connected to numerous backend systems via the Cognigy.AI Enterprise platform and thus, among other features, enables transactional conversations. ARAG customers can, for example, report claims or request insurance quotes and the most important initial information on various insurance topics via the bot 24 hours a day. To establish transparency and trust, it is important to be able to clearly identify which decisions the AI component makes ("Explainable AI").

Prevention of the Black Box Problem

AI components can become so complex that their functioning or decision-making can no longer be clearly understood. Situations like these are then referred to as "black boxes." Making adjustments solely on the basis of externally visible behavior of the AI can be risky because this approach is reactive rather than proactive. This can be helped by so-called "explanatory models" that describe which inputs lead to which results.

Operational Control of Machine Learning Algorithms

Machine Learning algorithms mostly offer the possibility to learn on their own during operation. This can be particularly well illustrated in the area of Conversational AI: A chatbot receives direct feedback from the user during the conversation and can provide supposedly better answers in the future based on the insights gained during past interactions. The emphasis here is on "supposedly" because what the chatbot learns naturally depends on the quality of the feedback. There is a risk that users will intentionally manipulate machine learning algorithms by providing a lot of misleading information so that they give wrong answers. Therefore, one of the most challenging tasks in the field of AI is to define and implement the framework and guidelines for machine learning algorithms.

The Ethics of Artificial Intelligence

Since the field of artificial intelligence is comparatively young, new use cases permanently emerge, and more and more people are involved in its development, implementation, use and there is a growing desire for ethics of artificial intelligence. The goal is to ensure that all those involved in the AI process act morally according to certain standards and do not restrict anyone's rights, autonomy, and freedom. This is where the disciplines of philosophy and sociology are particularly relevant, as these are currently drafting proposals on how artificial intelligence fits into the universal value system of human beings.

Accountability of Artificial Intelligence

In order to enforce ethical and moral guidelines, a legal framework is required. The use of machine learning algorithms in particular raises the question of individual liability and responsibility (“Responsible AI”). Careless use of artificial intelligence can cause individual and societal damage. Although AI applications are generated by humans, an objectification of humans inevitably takes place within the technological system, because this is required for the machine to be able to store and exploit human actions in the form of binary data.

The preservation of rights laid down in the common law implies the traceability of AI applications. Only if there is complete clarification on the operating principles and decision-making paths of AI components, direct accountability to human actors can be established.

In the area of Conversational AI, every human-machine conversation is flanked by a designed communication process. The more structured and differentiated this human setup is, the easier it is to understand how the AI reacted to certain intents in retrospect. For the creation of multi-level communication processes, it is important to develop and visualize the decision paths and the actual dialog with the help of a conversation editor.

Dangers of non-trusted AI / non-explainable AI

An honest discussion of artificial intelligence always includes consideration of its potential dangers. AI applications can only be designed in a secure and trustworthy way if potential dangers are considered and eliminated from the very beginning.

As one of the world's leading providers of Conversational AI solutions, Cognigy with its AI, made in Germany, has been sensitive to the potential risks coming from Artificial Intelligence for years. As early as the development process of the Cognigy.AI platform, potential threats were categorized and evaluated according to their likelihood as well as their impact. These so-called "Threat Categories" can be generalized from our experience and applied to many AI services. We were able to identify the following AI threat categories:

• Intentional, nefarious manipulation of the AI service, e.g.:
  • Adversarial or backdoor attacks ( corruption of classification results by deliberate modification of training data)
  • Deliberate, insufficient validation of ML models or deliberate "model poisoning".
  • DDos attacks (assaults that disable the availability of the AI service, usually triggered by a targeted overload of the data network)
  • So-called "insider threats," i.e., the deliberate manipulation or disclosure of data by employees from within the own company
  • Weak labeling of input data in supervised learning models.
  • Unauthorized access to data sets, code, or data transfer processes by third parties

• Unintentional, harmful manipulation of the AI service, e.g.:
  • Undetected bias introduced into the system by the originators of the data
  • Unintentionally limiting the output quality of machine learning models, e.g., by withholding data for confidentiality reasons or by misjudging the value of certain data
  • Unintentional use of unreleased personal or sensitive data
  • Accuracy reduction of training data, including unintentional intervention in training sets or intermixing of data sets with highly different degrees of quality

• Legal risks, e.g.:
  • Careless handling of / non-compliance with data protection regulations
  • Disclosure of sensitive or personal data, e.g. due to lack of required pseudonymization mechanisms
  • Profiling of customers
  • Breaches of contract by third-party providers with service level agreements (SLA), e.g., to secure the performance of the AI service
  • So-called "lock-in" scenario, i.e., dependence on third-party providers (e.g., cloud providers, AI libraries

• Failures or malfunctions, triggered e.g. by:
  • Lack of expertise in developing AI applications
  • Inaccurate implementation and configuration of machine learning frameworks
  • Inadequate analysis of required resources, vulnerable system infrastructure, or insufficient computing capacity to perform ML model calculations reliably and without errors
  • Lack of data quality checks
  • Missing or incorrect documentation
  • Errors triggered by third-party vendors

• Eavesdropping, interception or hijacking of data, e.g. by:
  • Data theft during data transfer or through data storage media theft.
  • Disclosure of ML model training and configuration information
  • Insufficient encryption regarding data transfers and storage
  
  
  
• Physical attacks, e.g.:
  • Causing disruptions in communication networks.
  • Attacks on data centers holding the relevant resources for the operation of AI services
  • Targeted damage to hardware running ML models 

• Outages and Disasters, e.g.:
  • Damage to the IT infrastructure, for example due to overheating, water, etc.
  • Natural disasters such as earthquakes, floods, fires, etc.

With the help of the AI Cloud Service Compliance Criteria Catalogue (AIC4) - established by independent third parties (German Federal Office for Information Security, BSI) - it is possible to audit AI applications for the first time. Most of the above-mentioned risk areas can be assessed uniformly and objectively. Auditing firms such as PricewaterhouseCoopers (PwC) can now officially confirm the security and trustworthiness of AI applications.

In the process, the AI services are audited with a view to the AIC4 criteria catalogue (7 action fields). In order to evaluate the "readiness" of the AI service before the audit, an assessment is usually performed together with the auditing partner. This is to prevent potential risks from becoming known only during the audit, which could lead to a so-called „exception“, meaning a clear indication in the audit report of an existing risk. The auditing report allows customers to verify the security and trustworthiness of the AI service.

In summary, the AI-specific risks can be reduced through adequate processes and product implementations. Decision-makers should therefore focus on AI service providers that align their management, products, functions and services with reliable AI operation and make appropriate investments in this area.

Assessment of AI applications for trustworthiness with the help of the AI Cloud Service Compliance Criteria Catalogue (AIC4)

The AI Cloud Service Compliance Criteria Catalogue (AIC4) is the world's first concrete set of criteria with operationalizable requirements for testing AI applications published by an official governmental institution. It contains AI-specific criteria that allow for an assessment of the trustworthiness of an AI service throughout its lifecycle. The criteria establishes a baseline level of security that can reliably be assessed by independent auditors.

AIC4 was developed specifically for application in the current state of AI technology and applies to cloud-based AI services that rely on machine learning methods and use training data for iterative improvement. Typical application areas for the above methods are speech recognition and language processing services (NLUs & NLPs), image classification tools, (economic) forecasting tools, and scoring models.

In a Conversational AI system, a set of specific conversational functions are provided, which users can then use to model their own virtual assistants (chatbots or voicebots), including desired conversational flows, and configured speech understanding. Central to an AI system for conversational intelligence are the NLU (Natural Language Understanding) functions. These are used to process speech input and evaluate the intentions of that input in multilingual environments. Through platforms such as Cognigy.AI, users are empowered to configure the system to connect the input and output interfaces and train their individual NLU models. The NLU models used in Conversational AI and the data processing in machine learning models make AI applications like Cognigy.AI particularly amenable to auditing by AIC4.

The AIC4 criteria catalogue developed by the German Federal Office for Information Security (BSI) is a response to demands from market participants, the German government and the EU Commission to establish transparency, traceability, and robustness of AI applications. When auditing according to AIC4, companies and providers of AI solutions assume an international pioneering role in the race for Trustworthy AI applications. This results in a competitive advantage in both end-customer deployments and the B2B sector.

Most AI services offer their products - at least optionally - as Software-as-a-Service (SaaS). Cognigy.AI can also be used as a SaaS service. Therefore, the secure operation of AI services in the cloud is an important component of the AIC4 catalogue. With the BSI's C5 criteria catalogue (Cloud Computing Compliance Criteria Catalogue), minimum requirements for secure cloud computing already exist. Therefore, the AIC4 criteria build upon the C5 catalogue and specify it further in the field of the AI lifecycle.

Companies that use AI applications such as Cognigy.AI as an on-premise solution and consequently not as a cloud service can use the AIC4 catalogue to better assess the trustworthiness of the AI services set up on their own or rented IT infrastructure, and can also have them certified by accredited auditors according to the catalogue specifications. In addition to greater transparency and control, competitive advantages and positive image effects can arise for participating companies.

The 7 action areas for auditing according to AIC4

As of May 2021, the AIC4 criteria catalogue comprises seven different action areas that cover the minimum requirements for auditing AI applications. They are supplemented by the requirements of the Cloud Computing Compliance Criteria Catalogue (C5) already described in the preceding chapter.

In the following, we provide an overview of the significance and goals of the seven action areas. For a detailed look at the implications of each action area for customer service automation, please refer to our free whitepaper. There, we have also included use cases from Cognigy's practical experience to give you an understanding of concrete measures for meeting the respective AIC4 action area.

Action area 1: Security and Robustness

This action area deals with protecting AI applications from manipulation attempts. The ability to react to continuously changing systems and environmental conditions plays a decisive role here. In order to detect malicious attacks, suitable tests must be developed and regularly performed on the part of the AI solution provider. Furthermore, the focus needs to be put on proactively implementing measures against targeted attacks. Through tests and measures, the AI application should gain robustness, the processed data should be protected, and the ongoing secure development of the algorithms utilizing training data should be ensured.

In the area of Conversational AI, meeting the criteria of this action area will lead to increased integrity of virtual agents (phone and chatbots), protection of personal data, reduction of manipulation risks, and secure operation of AI applications in the cloud.

Action area 2: Functionality and Performance

The objective here is to define and measure the performance of an AI service in its particular area of application. This is done by relying on appropriate procedures for training algorithms as well as for validating and testing AI applications. The metrics proposed in the AIC4 catalogue for, inter alia, the accuracy, sensitivity, and the error rate of the AI service, are used, on the one hand, to check the functionality and performance scope agreed upon with contractual partners and, on the other hand, act as a basis for future performance improvements.

In the area of Conversational AI, meeting the criteria of this action area will lead to an increase in reliability of virtual agents in customer use (e.g., during peak load), increased quality and performance of phone and chatbots after each training iteration, and the use of the most advanced and plausible machine learning models.

Action area 3: Reliability

This action area is intended to ensure the reliable operation of the AI service in production environments. In addition, processes must be established to investigate errors and failures of the service. This includes, for example, the provision of adequate resources for maintaining the AI, logging procedures, backups, and processes for error handling. In this context, partial use is made of the C5 catalogue criteria.

In the area of Conversational AI, meeting the criteria of this action item will lead to better accessibility of virtual agents (24/7), continuous maintenance of Conversational AI platforms, and close monitoring of AI services.

Action area 4: Data Quality

The major issue is that data used for ML model development, intent training, and AI service operation must meet certain quality standards. For example, training data should be of appropriate quality and quantity to adequately train the AI for the corresponding area of application. In addition, training data must be easily accessible and consistent in structure. This has a significant impact on the explainability of AI models ("explainable AI").

Meeting the criteria of this action area will lead to better virtual agents in the area of Conversational AI, as the quality of the training data is crucial for the quality of the phone and chatbots. In addition, processes for data acquisition, qualification and evaluation are followed and possible biases (so-called "AI bias") in the ML models are avoided.

Action area 5: Data Management

This control area establishes the framework for the structured collection of data and the use of data from trusted sources. The AI service must also establish a framework for developing and operating its service, using training data. This includes, for example, adequately protecting the training data from access through unauthorized individuals. The training data must also be used for its intended purpose and must be documented.

In the area of Conversational AI, meeting the criteria of this action area will lead to better documentation and more transparency in the training data, legal protection, as only data that also meets legal requirements and is approved may be used, and the establishment of granular access rights in the AI system.

Action area 6: Explainability

The decisions made by algorithms in the context of an AI service must be comprehensible. If necessary, experts must be consulted and suitable techniques must be used for the assessment. Among other things, it must be possible to explain the purpose and operation of the machine learning model in use. This information must be documented and prepared in such a way that an evaluation by external parties, for example, technical and industry experts, but also users, is possible. Only when these actions take place an AI becomes "Explainable AI".

In the area of Conversational AI, meeting the criteria of this action field leads to a better traceability of the decisions of virtual agents. It also creates an opportunity to have ML models externally verified. This accelerates the use of conversational AI technologies because reservations in the company are reduced more quickly.

Action area 7: Bias

In this action area, the possible bias of the AI service is to be identified. This involves, for example, direct and indirect bias as well as systemic and statistical bias. The AIC4 is not explicitly designed to assess biases in the code or training data from an ethical perspective. Rather, the aim is to uncover possible vulnerabilities by applying mathematical methods (for example "adversarial debiasing," "prejudice remover," etc.) and thus make them transparent to the user of the AI service. An assessment must then be made by the user. As part of the AIC4 audit, the AI service operator is also required to reduce any AI bias identified using specific algorithms.

In the area of Conversational AI, meeting the criteria of this action field leads to a reduction or elimination of bias (e.g., discrimination), equal treatment of users and intents - regardless of origin or dialect, and factual-rational communication between humans and virtual agents.

The AIC4 Audit in Practice

For decision-makers, the auditing practice is an important factor for a commitment to the use of trustworthy AI services. In the following, we provide a brief insight into auditing AI services using Cognigy.AI as an example.

You can find further valuable insights - among others described by the auditing partner PwC (PriceWaterhouseCooper) - in the free Cognigy whitepaper.

It can be stated that an AIC4 audit is feasible for a wide range of potential use cases. Even the first steps in the field of artificial intelligence and/or selectively deployed AI services can be accompanied by an AIC4 audit and thus integrated into the company from the outset, taking compliance guidelines into account.

An example of this would be the use of an already trained AI model, as used in image recognition software or in programs for automated text and character recognition (OCR). Equally, however, AIC4 also offers the possibility of auditing entire AI workbenches such as the Cognigy.AI platform. Due to the system, an additional challenge arises, in that not only the AI provider must act in an AIC4-compliant manner, but also the users, since they create value by providing information and data with the help of the platform.

Let’s take Cognigy.AI as an ideal use case: the conversational AI platform provides most if not all AI models, tools and interfaces for developing intelligent, fully automated voice and chatbots and thus acts as the underlying intelligence. However, the added customer value is generated by the user himself by designing natural language dialogs with the help of the Conversational Editor and thus significantly influencing the behavior of the AI (“Explainable AI”). An AIC4 audit takes this use case into account and enables a holistic examination of the resources used in the context of AI automation.

Due to the wide range of applications using AI, there is currently (as of May 2021) a large demand for AIC4 audits from companies of various sizes. Both well-known telecommunication providers such as Deutsche Telekom, as well as ambitious startups from wide-spread industries express a strong interest in the audit or are already in the auditing process. As one of the first AI platforms, Cognigy has successfully had the AI-based applications of the Cognigy.AI platform audited.

The auditing process can be divided into two phases: Phase 1 involves auditing the design and implementation of AI services against the AIC4 action areas. Phase 2 aims to audit the effectiveness of corresponding AIC4-compliant processes and is conducted a few months after the completion of phase 1. The result of an AIC4 audit is a comprehensive report that includes the test object as well as the test activities performed and their result/assessment.

An AIC4 audit is performed according to the standards of ISAE 3000 and therefore requires an auditing company as the auditing institution.

Benefits of using AIC4-audited, trusted AI in Customer Service

In addition to the general harmonization of the German - and in future possibly the international - AI market, the AIC4 audit lays the foundations for legislation at the EU level. Companies that address the AIC4 standards early on and encourage their AI service providers to get audited can already protect themselves from potential legal consequences. Of course, this doesn't just apply to AI services used in Conversational AI.

In customer service automation, an AIC4 audit yields numerous tangible benefits that can have a direct and indirect impact on business success. The following section highlights the key benefits and is intended for business decision-makers who are weighing whether the use of AIC4-audited AI applications is necessary.

We've covered more benefits of AIC4 audition, as well as additional selection criteria for a trusted AI provider in customer service automation, in the free Cognigy whitepaper.