Case Study

As noted in Chapter 1 AI Risk Governance and Framework Integration, Marmot Home Security is a mid-market enterprise with 3,000 employees and a customer base of over 2 million worldwide. Marmot develops and sells innovative smart home products, including smart speakers, security cameras, and automated lighting systems. The company operates globally with direct-to-consumer sales through e-commerce channels and partnerships with large retailers.

Marmot has faced accelerated growth in its products and sales, and as a result, its customer service team has faced a 30% rapid growth in customer inquiries across communication channels. To address these issues, leadership has approved a strategic initiative to leverage AI, specifically the use of AI agents.

After obtaining leadership approval and buy-in, the technology and AI engineering teams evaluated the feasibility of building AI agents internally vs. leveraging third-party pretrained models. After a thorough analysis, Marmot decided to take a hybrid approach:

• Use third-party models for core conversational natural language processing (NLP) capabilities, combined with internally developed domain-specific layers to maintain control over key compliance and ethical requirements.

Marmot is now ready to begin development on the system, and the development team wants to ensure that risk is addressed throughout the process. The team plans on using logs of past service chats, transcripts of recorded phone calls, sentiment analysis, FAQ documents and webpages, and customer profiles to train the agents.

1. What steps should be taken to address possible risk associated with the data pipelines used to train and validate the AI model? (Select all that apply.)
   1. Limit data collection to complex, high-priority cases to maximize data storage and provide the model with the largest amount of information per case.
   2. Categorize cases by region so the model can learn differences in responses based on geographic location.
   3. Normalize terminology, data formats, and data fields across inputs to ensure interpretability.
   4. Obtain new consent forms from all customers and business units to ensure data privacy regulations are followed.
2. How should Marmot address potential challenges in bias and fairness testing to ensure their AI solution remains compliant and equitable throughout its development and deployment phases?
3. When integrating AI agents with existing technologies at Marmot, what challenges should be anticipated? (Select all that apply.)
   1. Ensure seamless communication between AI agents and legacy systems.
   2. Reduce data privacy risk inherent in using third-party models.
   3. Achieve real-time monitoring and reporting of AI model performance.
   4. Streamline the collaboration between AI and customer service teams.
4. Which metric would be MOST suitable for monitoring the performance of Marmot’s AI solution in enhancing customer service once implemented?
   1. Customer satisfaction score
   2. Human resources spend
   3. Social media engagement rate
   4. Repeat visit rate

Chapter 2 Answer Key — Case Study

1. 1. Although complex cases most likely contain a high volume of valuable information, the primary goal of the AI agent is to handle low-priority cases and triage more complex issues to human agents. Complex cases often contain outlier situations, and training solely on these cases can cause the AI model to skew recommendations related to these outliers.

   2. Categorizing data based on geographic location and region can help the AI model to learn differences in common customer service inquiries by region as well as any nuances in culture that can affect how an issue is presented and resolved.

   3. Normalizing data can reduce noise and ensure the model can accurately interpret the training data to produce more accurate predictions.

   4. While ensuring proper consent was obtained when recording calls and chats is important when aggregating data for use in AI solutions, it may not be necessary to obtain entirely new consent in all cases. Often, agents will inform customers that the conversation is being recorded for training purposes or customers will agree to various uses of their data when creating their profiles. The team should first review consent tags and then determine if new consent is required.

2. To address bias and fairness challenges effectively, Marmot should take a proactive approach by investing in a comprehensive framework for evaluating AI models. This framework should emphasize using diverse datasets that reflect the broader customer demographics Marmot serves. Regular audits and testing should be conducted to identify and correct biases within the model outputs, ensuring decisions are fair and equitable. Establishing an AI ethics committee can add an additional layer of oversight, providing insights and guidance on ethical considerations throughout the development life cycle. Furthermore, fostering engagement with stakeholders, including customers and employees, to gather feedback will help identify areas for improvement. Transparency should be prioritized, with clear communications regarding how the AI system processes and leverages data, enhancing trust and compliance.

3. 1. Seemingly seamless communication between AI agents and legacy systems can be challenging due to potential incompatibilities, which may require tailored integration solutions.

   2. While reducing data privacy risk is important, the use of third-party models does not inherently increase this risk. Marmot’s internal controls should mitigate privacy concerns in this hybrid setup.

   3. Real-time monitoring and reporting of AI model performance needs robust systems and processes in place to track deviations and ensure continuous improvement.

   4. Streamlining collaboration between AI and customer service teams is essential to ensure that AI solutions enhance service delivery without disrupting established workflows.

4. 1. The customer satisfaction score provides direct insight into how satisfied customers are with their interactions with the support function. An improvement in this score since implementing the AI agents indicates that customers are pleased with the quality of the response they receive from not only human agents but also the AI triage and response process.

   2. While human resources costs may be impacted by the use of the AI solution, it does not directly reflect how successful the AI implementation is at addressing customer concerns. Other factors, such as continued customer growth or changes in the market, can impact the need for more or fewer service agents.

   3. While social media is a platform for customer service responses, engagement rate does not directly reflect the performance of the AI system in handling queries.

   4. Although repeat customers or visits to the website could indirectly suggest satisfaction with the customer support system, it does not directly measure or indicate how well the AI agents are performing their task.