Part D: AI Policies, Procedures, and Organizational Training

As with any emerging technology, adoption of AI solutions should align with organizational policies and procedures. In some cases, new guidance may be required to augment existing documentation to ensure that the enterprise’s vision and objectives for the use of AI are adhered to by staff. Clear policies and procedures reduce instances of unintentional misuse of AI or misuse of data used in AI solutions.

In many cases, AI considerations can be inserted into existing policies and procedures. These updates need to be clearly communicated to staff in a level of detail that is commensurate with their role and use of AI. Training and awareness are key to ensuring safe and responsible use of AI.

1.10 AI Acceptable Use Policy

An AI acceptable use policy (AUP) dictates what the enterprise has decided is permitted use and what is not permitted use—for example, the types and classifications of data allowed for training the AI solution. The AI AUP is essentially a tool to inform staff of expectations regarding the use of AI.

In many cases, an AI AUP is an addendum or supplement to an organization’s existing AUP.

What is included in an enterprise’s AI AUP may depend on the level of disruption caused to its industry, but there are some common steps an enterprise should take before creating an AI AUP. These include:

• Understand AI technologies—Ensure that the enterprise understands the AI technologies it wishes to use and its potential applications, risk, and benefits. For example, if it wants to use GenAI, the enterprise should be familiar with common models, such as ChatGPT, DALL-E, Grok, etc., including model types, data sources, biases, and capabilities. Organizational needs, legal obligations, and risk assessment can help determine if the enterprise should leverage a public or private AI solution.
• Assess organizational needs—Understand how the organization intends to use AI. This will inform the AUP’s clauses. Research the legal and regulatory requirements related to AI in the enterprise’s specific industry and jurisdiction.
• Conduct a risk assessment—Identify potential risk associated with the deployment of AI in the organization. This includes both technical risk (e.g., unintended outputs) and ethical concerns (e.g., misleading information).
• Confirm purpose and scope of the acceptable usage of AI—What is the policy’s objective and the scope of systems involved? Specify whether the policy pertains solely to GenAI or includes other AI models. Clearly articulate the policy’s boundaries and its intent, whether ensuring ethical use, meeting legal standards, or offering clarity to users.
• Examine existing IT and information security policies—Before creating something entirely new, check potentially overlapping policies to ensure consistency and clarity. Is there language that could be borrowed? Are there similar responsibilities and expectations articulated within an existing policy? What policies might serve as good templates for an AI AUP?
• Engage stakeholders in policy development—It is crucial to ensure that all relevant stakeholders have a voice in the process, both to make the policy comprehensive and actionable and to ensure adherence to legal and ethical standards. Stakeholders offer technical expertise, ensure ethical alignment, provide legal compliance checks, offer practical operational feedback, collaboratively assess risk, and jointly define and enforce guiding principles for AI use within the organization.
• Adopt a governance and risk framework—A governance framework can help address questions related to ownership, responsibility, and assurance. It can create a structured approach to decision making and ensure management and oversight processes are consistent, transparent, and accountable.
• Prepare for internal and external communication—Policies are meant to be living documents that are communicated to employees and stakeholders alike. Teams within the organization need to be notified of policy updates and changes. Customers and regulators will want to be kept abreast of the enterprise’s intended uses, as well as the ethics and privacy considerations for AI.

1.11 AI Policy Development

In general, corporate policies constitute one of the main mechanisms that enterprises use to establish the guidelines and general rules that govern a specific action. Corporate policies facilitate decision making, guarantee the homogeneity of criteria, align company employees, communicate the strategy of the company, and define its values and brand image for current and potential customers.

The implementation of the corporate strategy and ethical values of the organization is necessary for the use of AI solutions, as is the creation of a policy that clearly establishes the principles, criteria, and rules governing the behavior of employees in relation to AI use.

Some considerations for AI policy development are shown in figure 1.23.

AI policies should clearly define the criteria, basic principles, and guidelines associated with the use of AI systems. These principles should reflect the scope of application. Principles include:

• Authorized uses of AI, including prohibited uses, uses allowed with prior authorization, and uses allowed without prior authorization
• Responsible use of AI (see 1.21 Responsible Use of AI Systems for more information)
• Transparency of decision making
• Fairness and bias considerations
• Third-party considerations
• Governance and supervision of the AI system
• Training and awareness
• Security considerations
• Compliance monitoring
• Policy review/maintenance
• Date of last policy review

1.12 AI Procedures and Manuals

Standard operating procedures (SOPs), manuals, and playbooks are common tools for any robust IT and cybersecurity program. Many existing IT SOPs may apply to the AI solution being developed. However, it is important to understand the unique qualities of AI solutions in the IT space and tailor accordingly.

For example, AI uses vast amounts of data. Therefore, procedures for collecting, handling, and protecting data for use in training and validating AI solutions should be included in AI SOPs and manuals. SOPs can also help standardize how data is processed and sanitized for use in AI solutions, which can address many key concerns with model output.

Additionally, ethical and human rights considerations and procedures for assessing and protecting previously unaccounted for areas need to be addressed within AI SOPs. See Part F: AI Trustworthiness, Ethical, and Societal Implications for more information.

It is crucial that AI SOPs and manuals extend beyond the IT space and include all enterprise uses of AI. Procedures should align with the AI AUP and RAI concepts to ensure consistent secure and trustworthy AI throughout the enterprise.

While best practices for creating AI-specific procedures and manuals are still developing, IT security teams can adapt existing practices to help guide development of AI tools.

1.13 Organizational Culture and AI Risk Governance

Organizational culture encompasses the shared values, beliefs, and behaviors that govern attitudes toward risk taking, compliance, and ethical conduct.

Organizational culture plays a pivotal role in shaping how AI risk is perceived, managed, and mitigated within an enterprise. A thorough evaluation of the organization’s culture regarding AI use and risk management is essential to identify cultural traits that influence risk behavior and to tailor governance frameworks accordingly.

To assess the culture, organizations should consider the following:

• Risk awareness and attitude—An enterprise that is risk-aware will be more inclined to adequately review and assess risk associated with AI. Likewise, understanding if an enterprise is more risk averse or risk accepting can be an indicator of its willingness to implement AI at an accelerated pace or whether it will wait to see how peer organizations act.
• Attitude toward compliance—The jurisdiction or industry in which an organization operates can influence its attitude toward AI risk. In more highly regulated industries with higher financial or reputational penalties for noncompliance, AI use may be severely limited as opposed to less regulated industries.
• Response to negative outcomes—If previous failures have resulted in severe penalties or if a blame culture exists, employees may be less inclined to adopt unproven or emerging technologies or use cases. If employees feel “free to fail,” they may be willing to take more calculated risk and, in some cases, be more innovative with AI technologies.
• Variances in departmental culture—Different departments or teams may have distinct subcultures that affect AI risk behavior. It is important to understand these variances and determine if they are misaligned with the enterprise’s overall risk posture.

Understanding the organizational culture enables the design of AI risk governance frameworks that align with employees’ values and behaviors. For example, in a compliance-driven culture, governance efforts might initially focus on clarifying policies and ensuring regulatory adherence, while in an ownership culture, governance can emphasize empowerment and accountability for AI risk decisions.

AI risk governance structures should incorporate mechanisms to:

• Foster transparency and open communication about AI risk and incidents.
• Encourage critical thinking and a safety-first mindset in AI system design, development, and deployment.
• Align AI risk management roles and responsibilities clearly across the organization.
• Support ongoing monitoring and periodic review of AI risk management processes.

By aligning AI risk governance with the prevailing organizational culture, enterprises can enhance the effectiveness of risk governance activities, promote ethical AI use, and build trust among stakeholders.

1.13.1 Aligning AI Governance With Defined Risk Tolerance

Risk tolerance is the acceptable deviation from an organization’s risk appetite and reflects the level of risk the organization can withstand without having to take corrective action. It is influenced by legal, regulatory, and business considerations and varies across departments or projects. AI risk governance frameworks must incorporate processes to determine the appropriate level of risk management activities based on the enterprise’s risk tolerance.

For instance, an organization with a low tolerance for AI-related privacy risk may implement stringent data handling procedures and restrict certain AI applications, while another with a higher tolerance may allow more experimental AI uses with robust monitoring. Governance policies should clearly articulate these boundaries and include mechanisms for periodic review and adjustment as organizational priorities and external conditions evolve. Organizations should establish periodic review cycles and scenario-based stress tests to recalibrate alignment strategies, considering the security and ethical aspects.

1.13.2 Adapting Policies, Procedures, and Training

To ensure AI risk governance is effective, policies and procedures must be tailored to reflect the organization’s culture and risk tolerance. This includes:

• Policy customization—Developing AI policies that align with organizational values and clearly communicating acceptable AI uses, risk thresholds, and accountability structures. Policies should be flexible enough to accommodate cultural nuances and operational realities while maintaining compliance with legal and regulatory requirements.
• Procedural integration—Embedding AI risk considerations into SOPs that reflect the organization’s risk appetite and cultural expectations. For example, in a proactive culture, procedures may emphasize continuous risk assessment and incident reporting, whereas in a compliance-driven culture, procedures may focus on meeting regulatory mandates.
• Targeted training—Designing AI risk training programs that address the specific cultural context and risk tolerance of the workforce. Training should promote ethical AI use, create awareness of AI risk, and encourage behaviors consistent with the organization’s risk culture.

1.14 Elements of Effective AI Training and Awareness

Effective AI risk training programs are essential to empowering organizational staff to understand, identify, and manage the unique risk associated with AI systems. AI-related training and awareness programs must encompass a comprehensive set of components that collectively build the knowledge, skills, and awareness necessary for RAI use and governance.

Components of AI risk awareness training include:

• Fundamentals of AI technology—In order to address AI risk, employees should understand the underlying technologies that comprise these systems. This understanding helps demystify AI technologies and clarifies their capabilities and limitations. For example, training could cover how AI models are trained on data, the nature of AI outputs, and the differences between narrow AI and more advanced forms such as AGI.
• Security and privacy concerns—Awareness of potential adversarial AI threats and model manipulation techniques is critical to safeguarding AI systems and organizational assets. Given the amount of data used by AI systems, training should also emphasize data security and privacy principles.
• Ethical principles and code of conduct—Responsible and ethical use of AI are key components of AI governance. This includes awareness of the ethical implications of AI use, such as fairness, transparency, and accountability. Employees should be made aware of the importance of adhering to policies that govern acceptable AI use and the consequences of violations.
• Bias detection and fairness—Training programs should provide employees with an understanding of how bias in data and model outputs can lead to unfair or harmful outcomes. Education on how to detect and manage bias, including disparity and fairness metrics, should also be included.
• Model interpretation and explainability—Understanding how to interpret AI model outputs is vital for effective risk management. Training should cover concepts of AI explainability and the risk associated with the black box nature of many AI models.
• Reporting mechanisms and feedback channels—Effective AI risk training must inform staff about established mechanisms for reporting AI-related concerns, anomalies, or incidents. Encouraging timely and transparent feedback helps organizations monitor AI risk continuously and respond appropriately. Training should clarify roles and responsibilities for risk reporting and foster a culture of open communication without fear of retaliation. Feedback channels should also ensure continuous improvement and remediation of data reliability issues that require data cleansing or transforming so that source system owners can evaluate any defects or system enhancements for implementation on source systems.

1.14.1 Impact on the Workforce

How does an organization determine the skills and experience needed to design, develop, deploy, assess, and monitor an AI system? This has become a critical question in many enterprises, as much of the decision making about the use of AI may come from leadership without consideration of the skills needed to support it.

The organization needs to identify the requirements to recruit, develop, and retain a workforce with backgrounds, experience, and perspectives that reflect the users impacted by an AI system. In addition, if the organization is going to make a commitment to incorporating AI into business operations, it must also invest in training current staff to ensure ethical and responsible actions. Additional technical training may be required for the security and development teams.

While AI is not new, most organizations are new to operationalizing AI. Mechanisms (or steps) necessary to close the skill gap should be incorporated into the AI governance program, as well as a determination of the skills required to implement new AI security architectures and the requirements of these architectures.

When adopting new technology, risk managers need to ensure the enterprise has a process for addressing the growing feelings of uncertainty or dissatisfaction that changes to current roles will have on employees. Dissatisfied or frustrated employees can lead to increased intentional or unintentional internal threats. Conversely, other teams may be eager to embrace AI, using shadow AI instances without seeking or following enterprise procurement or AUPs.

Effective awareness programs can help employees understand why the enterprise is adopting AI and what acceptable uses of AI are. If employees are not aware of the business reasons for the change, they will be less likely to embrace the new technologies. In some cases, AI solutions have been implemented, but employees are not using the technologies because they lack understanding as to the benefit the solution can provide. Other employees may feel enterprise policies are too restrictive to their use cases and may not understand the privacy or ethical impacts the policy is addressing. Awareness should also include an open discussion of employee concerns related to the change to help foster adoption and adherence to policy.