Case Study

As noted in the previous chapters, Marmot Home Security is a mid-market enterprise with 3,000 employees and a customer base of over 2 million worldwide. Marmot develops and sells innovative smart home products including smart speakers, security cameras, and automated lighting systems. The company operates globally with direct-to-consumer sales through e-commerce channels and partnerships with large retailers.

Marmot has faced accelerated growth in its products and sales, and as a result, its customer service team has faced a 30% rapid growth in customer inquiries across communication channels. To address these issues, leadership has approved a strategic initiative to leverage AI, specifically the use of AI agents.

The AI agents are now deployed across phone, email, chat, and social media to assist customer service agents. Early results have shown improvements in response times and consistency of information provided, leading to increased customer satisfaction.

However, several months into operation, Marmot identifies an emerging threat related to its third-party supplier. Industry reports indicate that the supplier has been involved in a data breach, and there are concerns about the security of customer data processed through their proprietary models. This breach potentially exposes Marmot’s customer data to unauthorized access, raising significant privacy and security concerns.

1. Which of the following is the FIRST step Marmot should take once confirming that the third-party model is affected by the data incident?
   1. Continue all operations while communicating with the supplier to confirm the extent of the incident.
   2. Suspend interactions processed through the affected third-party systems temporarily.
   3. Invoke liability clauses in the contract with the supplier to ensure Marmot is not held responsible for the incident.
   4. Extend a discount code to affected customers to compensate them for the incident.
2. Which of the following would have MOST likely alerted Marmot to the issue with the vendor sooner than reviewing industry reports? (Select all that apply.)
   1. Leveraging vulnerability databases (e.g., OWASP AI Top Ten) to conduct vulnerability assessments for emerging threats
   2. Enabling automated monitoring and alerting systems on the APIs that Marmot uses to connect to the third-party model
   3. Including SLAs in the contract that require timely incident response reporting
   4. Reviewing AI risk scenarios related to model security regularly to ensure any changes are captured
3. What mitigation strategy could Marmot employ after identifying the breach?
   1. Strengthen security protocols and perform encryption of data interfacing with the third-party system.
   2. Prioritize global and internal standards over specific industry standards and regulations.
   3. Redirect resources from customer service to cybersecurity enhancements without increasing overall budget.
   4. Publish all security findings internally to build trust among Marmot employees.
4. After remediating the issue, Marmot asks to review the vendor’s model to assess performance. They discover that the model is a black box, limiting its ability to disclose information on how the model performs. Which of the following actions could help Marmot address this concern?
   1. Assess the risk level of using the black box model to determine if the risk is acceptable.
   2. Include a clause in the contract that requires access to the AI model source code.
   3. Contract with additional AI developers to reverse-engineer the AI model.
   4. Review outputs of model testing sets to review for bias and accuracy.

Chapter 3 Answer Key — Case Study

1. 1. Understanding the impact of the incident is an important step, but Marmot has already identified that the incident has occurred and is affecting their systems. Continuing operations may leave the system vulnerable to additional data leakage.

   2. Temporarily suspending interactions processed through the affected third-party systems is correct, as it isolates Marmot’s data from potentially compromised models, which is logical to minimize risk.

   3. Invoking liability clauses is premature at this phase, as any potential legal consequences or fines are not known. Further investigation and determination of responsibility will occur later in the incident response phase.

   4. Communication with customers and any remediations for the inconveniences caused by the breach will occur after the incident is contained and investigated.

2. 1. Vulnerability databases are useful tools to help organizations stay informed on new and emerging threats to AI solutions. Similar to reviewing industry reports, they are more of a passive means to identifying vulnerabilities.

   2. Automated monitoring and alerting proactively enables Marmot to identify potential incidents or new vulnerabilities in real time.

   3. Including SLAs that require timely reporting of incidents from vendors would help Marmot ensure that they are aware of potential breaches that can affect their customers’ data.

   4. While regular review of risk scenarios helps to ensure that new risk is identified and adjustments can be made, this would not alert to threats in real time.

3. 1. Enhancing security and encryption is a standard mitigation strategy to protect data integrity and provides Marmot with internal control to protect its information.

   2. Compliance is crucial, and prioritizing global and internal standards for industry regulations would risk legal repercussions without proper analysis.

   3. Reallocating financial resources would not necessarily ensure that the risk related to the data breach will be adequately addressed. At the same time, reducing funding from customer service assets could present additional risk.

   4. Publishing security findings internally might aid transparency but does not directly mitigate external security risk.

4. 1. Risk analysis should be performed on the impact of using a third-party AI model that is not transparent. Marmot may determine that any negative effects that result from using a black box model may be outweighed by the value provided from using the AI solution.

   2. The vendor may not have access to the source code of the model, or it may consider it proprietary information that requires protection from external entities. Adding a clause to the contract may not be feasible.

   3. Reverse-engineering the model may infringe copyrights and violate good business practices.

   4. Marmot may run some test cases or demos to see if the quality of the AI model outputs meets its standards and expectations. This may also enable the company to perform initial supervision to ensure the model’s results are fair and unbiased. However, due to the black box nature of the model, they will need to address the concern with the vendor.