Generative AI Should Be Developed and Deployed Responsibly at Every Level for Everyone

The unprecedented growth of generative artificial intelligence (AI), particularly large language models (LLMs), presents both transformative opportunities and significant challenges. Generative AI technology, marked by AI models able to generate synthetic content, including text, photos, audio, and video, has witnessed rapid adoption, reaching hundreds of millions of users in mere months—a steeper adoption curve than any of the previous tech giants’ products. Notable developers, including Big Tech companies Amazon, Google, Meta, and Microsoft, and newer startups (often with large Big Tech investments),¹ such as Anthropic, Inflection AI, and OpenAI, are at the forefront of this revolution, building advanced AI models and providing both their own first-party AI services using those AI models and third-party deployments of those AI models through application programming interfaces (APIs).²

The expansion of generative AI into mainstream applications has been swift, and this rapid growth has outstripped the development of robust safety measures. Generative AI developers have articulated numerous responsible AI principles.³ In execution, however, they have implemented some safety measures at the model level—such as additional trust and safety features in their own first-party deployments of their AI models⁴—while third-party deployments via APIs have limited safety requirements. This raises tremendous concerns about the potential misuse of AI, the protection of users, and the risks to society at large.

There is a significant disparity between the still minimal safety measures taken by developers in their first-party deployments and the almost nonexistent safety requirements for third parties deploying the models via API. In fact, the use of generative AI through third-party APIs can right now be characterized by a deficiency in strict safety standards or requirements, raising questions about accountability and the potential for misuse. This discrepancy is particularly significant as much of the growth and profit opportunities in generative AI are rooted in third-party API utilization.⁵ Even the existing safety tools do not reach the level of sophistication seen in trust and safety systems on other major web platforms,⁶ such as Meta’s Transparency Center⁷ or YouTube’s Community Guidelines.⁸

Potential safety issues from third-party API usage require our immediate attention. There is an urgent need for a standardized framework to ensure responsible use and deployment of generative AI, encompassing both first-party and third-party applications. This framework should prioritize user safety, transparency in policy enforcement, and accountability for both developers and deployers. This report sketches the contours of such a framework, outlining the crucial roles and responsibilities for industry and government.

In order to immediately close the gap and prevent further abuse, developers of generative AI models who allow access to third-party deployers via API must enforce all existing policies and provide additional transparency on their enforcement, especially usage and behaviorally oriented policies; require content moderation features; build adequate tooling to manage API access; and enable reporting from users and third-party deployers.

Policymakers must not ignore the fact that the vast majority of generative AI usage is likely to come from APIs being used by third parties. The executive branch must audit the voluntary AI commitments made by leading companies⁹ to ensure that their APIs and third-party usage can comply with the spirit and the letter of those commitments, a possibility that is difficult to realize when most model developers do not currently require any significant safety features when third parties deploy their AI models via API. Additionally, the National Institute of Standards and Technology (NIST) must ensure that the generative AI companion to the AI Risk Management Framework (RMF)¹⁰ tasked by the recent AI executive order¹¹ outlines the specific risks and mitigations from third-party usage as well as obligations for both the developer and the deployers. The Federal Trade Commission (FTC) should survey the industry, undertaking a 6(b) study to understand third-party safety requirements from leading generative AI providers and provide guidance on what AI developers should require in order to comply with existing law.¹² There is a history of the FTC examining platform safety, as Facebook/Meta’s nearly $5 billion fine was due in part to abuses of its platform by third parties and Facebook’s failure to adequately secure its data and users.¹³

Finally, Congress cannot ignore the responsibility of both developers of AI models and deployers of AI models as it considers new laws to address responsibility and liability for AI usage. A system that allows for accountability and liability for just one party, if at all, is not an outcome that can be tolerated in the new AI age. The status quo leaves us with a flavor of the challenge we face in other aspects of technology policy, wherein it falls to individuals—and occasionally to class actions—for post hoc regulation in lieu of upstream regulation to shape the system.

One of the most effective ways to ensure accountability and responsibility is through legal liability when the law or contracts are broken or when harm comes to an individual. Determining which party is liable—the developer, the deployer, or both—is a topic that courts will soon be ruling on, and Congress will likely need to address this issue as well. Future work from the Center for American Progress will explore the topic of liability for developers and deployers of AI models and recommend legislative solutions.

Given the potential of generative AI to affect nearly every aspect of society,¹⁴ it is crucial to balance technological innovation with adequate safeguards and regulations. This requires a concerted effort from developers, policymakers, and other stakeholders to establish and enforce comprehensive governance structures for generative AI, ensuring its safe, secure, and responsible use.

This report offers definitions to build a shared vocabulary for discussing these issues, highlights the lackluster status quo of first- and third-party safeguards for generative AI usage, and makes recommendations for how developers, deployers, and government can safely and successfully build and distribute generative AI responsibly.

The development and deployment of generative AI may outpace nearly any other technological advancement to date.¹⁵ The White House’s recent Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence defined generative AI to mean “the class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content … includ[ing] images, videos, audio, text, and other digital content.”¹⁶ OpenAI’s generative AI application, ChatGPT, crossed more than 100 million monthly active users two months after launching to become the fastest-growing consumer application in history.¹⁷ Alongside OpenAI, legacy tech giants are developing generative AI technology that will quickly reach millions of users via existing products. In November 2023, Google announced that the rapidly growing Search Generative Experience will soon be available to millions of users in short order.¹⁸ Similarly, Microsoft enumerated plans to integrate the generative AI technology that it licenses from OpenAI¹⁹ into its already widely adopted Microsoft Office suite²⁰ to “make meetings less painful.”²¹ Meta, meanwhile, has begun to integrate generative AI into its messaging products, including Messenger, Instagram, and WhatsApp, with billions of users.²² In addition to the pace of its development and expansion—far faster than traditional social media penetrated wide swaths of users²³—generative AI also has unique distinctions in how it is used and deployed.

Some of the leading U.S. companies developing generative AI technology today include Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI.²⁴ These companies are the developers who build the underlying large language model (LLM) technology that powers generative AI. Additionally, a subset of these developers may deploy those generative AI technologies to host and operate first-party AI services on their own websites or apps, whereby users can gain access to generative AI systems, commonly through a chatbot.²⁵ An entity can be both a developer and a deployer.²⁶ Examples of developers operating first-party generative AI websites or apps include OpenAI’s ChatGPT, Anthropic’s Claude.ai, Google’s Bard, Microsoft’s Copilot, and Meta’s integration of its open-source generative AI model, Llama 2, in Facebook, Instagram, and WhatsApp.²⁷

In addition to managing their own first party websites or apps that provide users access to their generative AI tools, almost every major U.S. generative AI developer also offers third parties the ability to deploy LLM technology to their own use cases via application programming interfaces (APIs).²⁸ These third parties, called “deployers” in the June 2023 draft EU AI Act,²⁹ can use the APIs provided by the developers to access, integrate, and manipulate the generative AI tools they developed into existing or new ways and applications.³⁰ The developer can make these generative AI APIs available for a fee or for free—or in the case of Meta’s Llama 2 open-source AI model, those who can access the model download and run it themselves for free.³¹ The developer can package these APIs together with other services to create a platform that allows third parties to create their own apps on the platform, which has traditionally generated the greatest value in software, such as the Windows Operating System (OS) or Apple’s App Store.³² Every leading generative AI company is working on APIs and platforms.³³ The lion’s share of a developer’s growth, scale, and profit potential is from this third-party API access component.³⁴

For example, in March 2023, OpenAI—the developer of the ChatGPT LLM AI technology—announced that ChatGPT was now available as a service via its API.³⁵ This meant the ChatGPT technology, which had previously only been available as a first-party AI service on OpenAI’s own website, was now available to third parties, who were not OpenAI, who could now deploy ChatGPT generative AI technology in their own apps via the API. A third party, Snap Inc., announced the deployment of ChatGPT into its application Snapchat via the API to create a service called My AI.³⁶ While the majority of generative AI developers utilize closed-source technology,³⁷ which requires deployers to pay for access and use of the models with limited insight into their operation, Meta’s Llama 2 stands apart as an open-source option,³⁸ allowing anyone to access and download the LLM for free.

While generative AI is not the only AI technology to raise safety and policy concerns, its rapid development, significant market penetration potential, and unique API access component pose a set of unique challenges vis-a-vis legacy technology. With these challenges arise bespoke risks, which the primary generative AI developers are well-positioned to mitigate to help ensure society can harness the opportunities of generative AI to the fullest.

Widespread use of generative AI carries risks that must be appropriately mitigated by its developers. However, the current state of developer efforts to safeguard their systems, be transparent with users and stakeholders, and uphold responsibility for their tools paints a picture that is lackluster at best and dangerous at worst.

Developers of large language models have applied a variety of tools to improve safety at the model level. These include Reinforcement Learning from Human Feedback (RLHF) and Reinforcement Learning from AI Feedback (RLAIF), which use humans, AI, or both to train the model responses.⁴⁷ Prior to being deployed for any first- or third-party use, most LLMs are trained against the usage policies of the model developer to encourage and discourage certain behaviors, such as not responding to prompts that would violate the usage policy. The aim is to prevent the model from returning harmful outputs—for example, instructions on how to make a bomb or commit a violent crime.⁴⁸ On top of this baseline training in all use cases, first-party usages have additional content-level abuse filtering turned on that cannot be modified by users. OpenAI’s GPT-4 System Card from March 2023 highlights the suite of derisking efforts taken by OpenAI prior to the model’s release, largely through aggressive testing and “red teaming” to identify and plug vulnerabilities.⁴⁹

While these are commendable steps by the developers to safeguard systems at the most basic layer, they are imperfect. This approach can lead to damaging false positives or false negatives and only covers content-based abuses.⁵⁰ Most notably, it covers model outputs and does not extend to uses and behaviors that violate usage policies. For example, a deployer integrating an LLM into a harmful application or not disclosing to a user that they are interacting with AI may violate the developer’s usage policies, but there are not appropriate protections to prevent and enforce against such offenses.

Beyond the model level, other safety tools can be utilized at the input and output level—including relatively unsophisticated blunt-force tools such as input prompt filters, output block lists, and output classifiers.⁵¹ In addition, simple tools, such as a report button to flag potentially violative content to the developer, appear in some first-party AI systems,⁵² but this is not universal. Current safety mitigations for LLMs are nowhere near the sophisticated trust and safety tools that have been developed for other large platforms on the web, including social media.⁵³ Additionally, the legacy mitigations used to safeguard against risks and protect users across other prominent platforms will not apply in the same manner due to the nature of LLMs and chatbots.⁵⁴

Currently, first-party usage of generative AI has primarily been a single, narrow, context-based application: a chatbot.⁵⁵ Yet the use of generative AI is expected to grow into many other forms.⁵⁶ As developers work to safeguard their systems from abuse, they are well-equipped to apply learnings from existing abuses and iterate their models and systems accordingly in the chatbot context before growing and scaling.⁵⁷

The nature by which generative AI has scaled to hundreds of millions of users—with billions expected to be able to access it in the near future—means that developers will not be afforded the same multidecade time frame they had with legacy products, such as social media, and therefore must prioritize building accountability, liability, and transparency into their systems right away.

Developers have built some reporting, controls, and general protections for first-party usage, but these safeguards still lack the robustness and detail to effectively mitigate risks and protect users. For example, OpenAI, Microsoft, Meta, Anthropic, and Google have acceptable use policies for their generative AI tools.⁵⁸ These usage policies include important prohibitions on the “generation of malware”⁵⁹ and the “planning or development of activities that present a risk of death or bodily harm to individuals.”⁶⁰ But neither the usage policies nor additional documentation⁶¹ enumerates in any detail what exactly constitutes a violation of these usage policies, how potential violations are investigated, or how users who repeatedly abuse the service will be banned.

For example, the recent January 10, 2024, updates to the OpenAI usage policies state: “We use a combination of automated systems, human review, and user reports to find and assess GPTs that potentially violate our policies. Violations can lead to actions against the content or your account, such as warnings, sharing restrictions, or ineligibility for inclusion in GPT Store or monetization.”⁶² Yet no further details are provided. Despite significant criticism over the opacity of social networks, they have provided much more information on the potential enforcement of their usage policies than what is currently being provided by leading generative AI companies.⁶³

This highlights a bare-bones approach to generative AI safety measures, even for a well-resourced and highly technical private company that stresses its commitments to safety as one of its selling points. Even lighter are some of the newer companies’ documentation, such as Anthropic’s Acceptable Use Policy,⁶⁴ a barely one-page policy document that does little to demonstrate the company’s commitment to safeguarding its platforms and protecting users, despite being a company founded on responsible AI use and deployment and with a very robust external commitment to safety.⁶⁵

Even if one wants to report a violation of an acceptable use policy to a first-party developer, it is exceedingly difficult to do so. Within ChatGPT on the OpenAI website or iOS app, for example, one can only give feedback in the interface via the binary “good response” or “bad response” flag.⁶⁶ This does not empower users to say why content may be violative, nor does it confirm that OpenAI will review the report against usage policies. And externally, it does not demonstrate OpenAI’s ability to take input on its own first-party platform.

As of late-January 2024, Anthropic offers two ways for users to report potential violations in Claude: by email and via an in-app feedback button that allows users to select a reason for the report and add additional clarity on the nature of the violation.⁶⁷ This update is aligned with the recommendations in this report, demonstrates a stronger commitment to responsible AI, and sets a standard that other developers should imminently follow.

In addition to building and maintaining first-party AI systems, developers are aggressively expanding the scope of the API access programs, including by significantly reducing barriers to access them. In January 2024, OpenAI announced the launch of the GPT Store, where paying users can access millions of customized GPTs for various uses.⁶⁸ The GPT Store gives users access to custom GPTs without requiring any coding or technical expertise, such as with an API service. Analysts project that by 2026, more than 80 percent of enterprises will have used generative AI APIs or models and/or have deployed generative AI-enabled applications in production environments.⁶⁹ This extensive deployment of generative AI by third parties necessitates clear outlines of safety mechanisms for deployers as well as developers.

While the developers have somewhat prioritized first-party mitigations via efforts such as in-app feedback on responses, abuse filtering,⁷⁰ and externally facing usage policies,⁷¹ they have abstained from accepting responsibility for upholding values of safety, responsibility, and transparency for third-party usage.

Historically, safety tools have been difficult to create, operationalize, and prioritize for APIs and third-party platforms. This is sometimes due to the nature of not wanting to monitor third-party usage for privacy or competition reasons and other times due to it being easier to ignore it. For instance, Facebook’s Cambridge Analytica scandal was due to an abusive third-party and Facebook’s limited monitoring and enforcement of its platform terms.⁷² On the other end of the spectrum, Apple’s App Store requires the pre-approval of apps that fully comply with their terms before deployment, which has also caused tremendous competition concerns.⁷³

The fact that the third-party component of generative AI is such a significant future driver of profitability and growth for the deployers should theoretically correlate with a deepening of their investment in these areas, perhaps to exceed or, at minimum, match efforts for first-party use.⁷⁴ In reality, developers are cutting corners in favor of continued scale, profitability, and wider adoption. Considering their significant growth within a short period of time, with seemingly uninhibited potential for more, developers are not implementing enough safeguards for third-party usage of their systems across numerous axes. From lightweight terms and minimal user controls to unclear enforcement for published policies, developers can be doing more to ensure their systems are protected from abuse.

Terms of service for third-party deployers using APIs

Terms of service, or developer terms, are missing stringent requirements that third parties utilize any safety tools to integrate the technology; they instead simply make reference to being inclusive of usage policies.⁷⁵ When signing terms of service to access developer tools, users agree to abide by usage policies for third-party applications.⁷⁶ However, externally, it is impossible to discern if there is active enforcement against these terms. Indeed, there is no clear way to ascertain follow-up between developers and deployers and therefore confirm that developers are holding deployers to the standards as stipulated.

Usage policies for third-party deployers using APIs

It is unclear how usage policies are enforced on third-party uses of generative AI technology today. The policies themselves are minimal and do not contain examples, commitments to enforcement, or explanations of consequences if a use case is found to violate them. As noted earlier, the most recently revised January 2024 OpenAI usage policy only notes: “We use a combination of automated systems, human review, and user reports to find and assess GPTs that potentially violate our policies. Violations can lead to actions against the content or your account, such as warnings, sharing restrictions, or ineligibility for inclusion in GPT Store or monetization.”⁷⁷ There is also a lack of external explanation of how these violations can be investigated, by whom, or within what time frame. For example, within the first two days of the launch of OpenAI’s GPT Store, it was proliferated by hundreds of AI girlfriend bots,⁷⁸ despite that going against the newly updated usage policy prohibiting GPTs dedicated to fostering romantic companionship.⁷⁹ This highlights a lack of adequate and timely enforcement capacity against all policies and reinforces the need to grow and scale responsibly.

Additionally, all major developers require third parties to disclose the use of AI in human-like chatbots “if your business is using or deploying our products as part of an automated service where your external customers or users interact directly with our products … you must disclose to your users that they are interacting with an AI system rather than a human.”⁸⁰ But there are not any clear oversight or enforcement mechanisms to do so.

Reporting for users using the AI model through a third-party deployer

While there are lightweight ways an end user can inform a developer of a potential abuse in a first-party use case, there is virtually no easy-to-access reporting mechanism for a user to report a potential issue about the deployer to the developer, which means the developer would likely have no idea of any potential violations occurring with the deployer’s use of the model. While reporting mechanisms are not a perfect solution and can be abused or provide unhelpful information, they are an important tool to utilize to ensure user trust and safety.

It is important to have the ability to report a potential abuse directly to a developer, and not just to the deployer, because if the deployer is violating the usage policies or terms, knowingly or unknowingly, they have no incentive to report this violation to the developer. This is not currently a requirement for integration into third-party applications but should be the most basic and strictly enforced safeguard for third parties to specifically include reporting to developers and deployers. Today, Microsoft offers a contact form to report responsible AI considerations,⁸¹ but OpenAI’s model feedback form is not accessible via its help center, usage policies, or terms.⁸² Meanwhile, some OpenAI GPTs in the GPT Store have their own report forms, but it is not required.⁸³ It is incumbent on the developers to reduce user and deployer barriers to report potential abuses and on deployers to offer in-app reporting too.

This is best illustrated by a recent blog post from OpenAI on election protections,⁸⁴ which stated:

We regularly refine our Usage Policies for ChatGPT and the API as we learn more about how people use or attempt to abuse our technology. A few to highlight for elections:

• We’re still working to understand how effective our tools might be for personalized persuasion. Until we know more, we don’t allow people to build applications for political campaigning and lobbying.
• People want to know and trust that they are interacting with a real person, business, or government. For that reason, we don’t allow builders to create chatbots that pretend to be real people (e.g., candidates) or institutions (e.g., local government).
• We don’t allow applications that deter people from participation in democratic processes—for example, misrepresenting voting processes and qualifications (e.g., when, where, or who is eligible to vote) or that discourage voting (e.g., claiming a vote is meaningless).
• With our new GPTs, users can report potential violations to us.

Note that OpenAI listed three potential violations of its usage policy, all of which would be difficult to impossible for an individual to report to the company today. But the company notes the importance of reporting because it does include reporting for its new GPTs,⁸⁵ which are available mainly through its first-party GPT Store.

Safety tooling for deployers

Third-party usage via API integration is severely lacking any safety mitigations, let alone similar ones to what developers do for first usage today. Developers make vague recommendations to deployers interested in using their technology, rather than enforce stringent safety requirements that make developer-responsible AI a precursor to integration.⁸⁶ Most developers simply state deployers must follow an acceptable use policy, which includes all uses of the AI model, including deployers, and which, as noted above, has few details. Only Microsoft’s “Code of conduct for Azure OpenAI Service” has any requirements for responsible AI mitigation, though how to fulfill those requirements is not made clear.⁸⁷ Microsoft’s Azure OpenAI Service also includes a required content filtering system that works alongside core models and is aimed at detecting and preventing the output of harmful content across various harm types and languages.⁸⁸

Other developer documentation recommends responsible AI measures but only in general terms, with few details and no requirements.⁸⁹ It is incumbent on the developers to set external principles around third-party safeguards and to build out-of-the-box safety solutions to match, such as input/output review for deployers and tooling to easily modify the model for safety.

This lack of oversight and enforcement poses significant risks to the safety of generative AI systems and enables the developer companies to essentially pass responsibility to safeguard systems and protect users onto the deployers. However, even if interested in upholding higher safety standards, deployers are left without the appropriate mechanisms, tools, and processes to do so. Safety documentation is scattered across dozens of documents, and safety features are a black box that offer no insight into how deployers can utilize generative AI responsibly. Third-party deployers have no support from first parties, lack ways of reporting potential violations, and must rely on rudimentary tooling, coupled with vague data-sharing policies to protect end users.

Data access for investigations

On data access for investigations, policies and enforcement are inconsistent and opaque. Some developers claim to keep personal data, including inputs and outputs, for as long as reasonably necessary,⁹⁰ while others say they do so only for 30 days for the purpose of abuse monitoring and conducting investigations.⁹¹ Microsoft sets a higher standard by stipulating exactly who can access the data, under what circumstances, and for how long.⁹²

Data need to be both retained and accessible to conduct investigations of abuses, but blanket deletions to uphold privacy dilute the responsibility of developers to investigate in an adequate time frame. It is incumbent on the developers to establish a balance between protecting deployer and user data and ensuring access to safeguard their systems from abuse.

Needed transparency

There are also two severely lacking components of transparency regarding these systems.

First, whether intentional or unintentional, the systems have been designed in a way that obscures their visibility and absolves both the first and third parties of responsibility in investigating reports and prioritizing fixing known gaps. It is incumbent on developers to be transparent about usage policy enforcement, mechanisms for reporting, and data-sharing practices and to shed light on the black-box safety features that exist today. OpenAI’s own Help Center contains examples of confused users unable to decipher why their access was cut, highlighting the weakness of first-party enforcement as well.⁹³ Additionally, there is no requirement for deployers to disclose to end users which large language model they are utilizing.⁹⁴ This is a significant problem in any use case, but especially in open source, where bad actors can easily download the original versions of these AI systems, disable their “safety features,” and make their own custom versions to conduct abuse.⁹⁵

Second, there is no standardized transparency reporting externally for regulators, stakeholders, deployers, and users to help them understand the technologies, potential risks, and limitations. Vis-a-vis traditional social media, where most platforms report on a quarterly basis,⁹⁶ generative AI platforms are not transparent and do little to build trust with key external stakeholders. This is low-hanging fruit for everyone, but especially for well-established developers that are experiencing hypergrowth and expansion.

Examples and questions

Below are example scenarios and associated questions that underscore the importance of strong governance of these systems:

1. A user comes across a third-party application using a developer’s API technology to generate malware that is prohibited by the first-party developer’s terms of service or its developer terms.

How can the user make the first-party developer aware of this if the developer does not require a report function or provide a way to report this on its website?

2. The developer is seeking to review usage patterns and potentially abusive behavior by third-party deployers of their technology.

How can the developer access the necessary information to determine if access should be revoked?

3. In a third-party application, a user interfaces with an undisclosed AI chatbot appearing, by all accounts, to be a human, which is against API usage policies.

How can the user alert both the developer and deployer of this violation? (Just alerting the deployer may not be helpful, since they are likely the one violating the acceptable use policy.)

4. A third party incorporates an LLM into an application meant to deter people from participation in democratic processes, which is currently against OpenAI’s usage policy.¹⁰⁵

How can someone report this misuse to OpenAI or the developer in question? Is there proactive monitoring of all third-party use cases?

Today, the developers are operating in ideal conditions that enable them to pass responsibility, get away with opacity in policy and enforcement,¹¹⁵ and grow user bases despite a lack of rules and unclear data deletion.¹¹⁶ The consequences of minimal governance of these systems range from frustrating user experiences to outright dangerous scenarios.¹¹⁷ In the wrong hands, LLMs can destabilize democratic processes, including elections, and could lead to real-world harm and offline violence.

The AI industry as a whole lacks a general accountability framework for indicating who is responsible for these highly powerful and pliable models,¹¹⁸ which is exponentially riskier with the introduction of the third-party deployment model.¹¹⁹ The bottom line is that the developer companies are building a perfect system of passing responsibility, obscuring systems with opacity, and touting a lack of policies and enforcement of said policies. Ultimately, right now, no one is responsible.

At its current rate of growth, generative AI will continue to reach hundreds of millions of people quickly.¹²⁰ The time to act is now, before it is too late. This report proposes some ideas for how developers can begin to equalize their first- and third-party safety commitments, followed by industry, government, and civil society recommendations to safeguard generative AI systems broadly.

Shoring up safeguards for third-party usage in the short term

Further investing in safeguarding third-party usage is just the start and will help demonstrate private sector commitments to growing responsibly while the industry at large and regulators consider longer-term policies and solutions.

Enforcement of existing policies

• Enforce all existing published policies, such as by requiring disclosure of no “human in the loop” chatbot use cases and suggesting an easily available method for reporting improper usage to the deployer.
• Build and retain adequate internal staff for enforcement and maintenance of all policies, processes, and protocols to keep users safe.
• The developer should have a clear enforcement mechanism for managing API access and should build an enforcement regime to revoke access to third parties who violate usage policies, including with reporting, investigation, privacy-protecting documentation practices, appropriate data retention, and tooling to carry this out adequately.

Abuse prevention

• Default content moderation features—such as OpenAI’s moderations endpoint¹²¹ and Azure’s abuse monitoring¹²²—to be on for deployers using and manipulating developer LLMs and require a submission of justification to the developer to turn them off.
• If a deployer is approved to turn off content moderation, developers should retain access to inputs and outputs to ensure responsible system use and store for an industry-agreed amount of time before permanently deleting.

Data and tooling

• Ensure appropriate data-sharing mechanisms are in place between developers and deployers, with published retention policies for before, during, and after a report is made.
• Build and enhance tooling to manage and revoke API access if a deployer violates developer terms or any other usage policies.

Reporting

• Anyone using an LLM at any time—in a first- or third-party capacity and in any format—should be able to report potential violations of an AI system to the developer and to the deployer of the LLM, if there is one, through a clear and transparent process with appropriate data retention.
• Reporting should be as frictionless as possible—for example, it should be built into the UI, such as in Anthropic’s recently updated Claude reporting flow; display the option on the interface directly; not require cold-emailing an address buried in a help center article; and so on.
• There should be user-facing appeal flows and a commitment to human review appeals in a timely manner.
• Deployers should similarly be required to have a report function directly from the user to the developer, and developers should staff queues appropriately to ensure timely review against usage and developer policies.

The public sector’s role in safeguarding generative AI systems

In addition to the private sector’s responsibility to safeguard these systems, the public sector also has a crucial role to play. The European Union’s AI Act can act as an inspiration for how the U.S. government might set forth policy to manage developers and deployers. For example, it requires deployers to comply with monitoring, record-keeping, human oversight, and transparency obligations once they put a high-risk AI system to use.¹²³

Below are some recommendations that offer ways the government can advance these issues.

• National Institute of Standards and Technology: NIST must include third-party risk management recommendations for generative AI companies in the forthcoming response to its generative AI requirements in the recent executive order,¹²⁴ in the companion to the AI Risk Management Framework,¹²⁵ and as part of the U.S. Artificial Intelligence Safety Institute.¹²⁶
• Executive branch: The executive branch should task an interagency taskforce to audit the seven White House AI commitment signatories¹²⁷ on third-party mitigations and press signatories to agree to voluntary safety frameworks the companies will enforce on third-party users of AI models.
• Federal Trade Commission: The FTC should undertake a 6(b) study to determine what kind of safety requirements leading generative AI developers should require for third-party deployers using their APIs. In addition, it should begin to outline clear steps that developers and deployers should take in order to lawfully protect users and ensure that responsibility is not being passed solely to the other party. The FTC has some precedent here, as the Facebook 2019 settlement was in part due to lax enforcement of its platform terms, including during the Cambridge Analytica scandal.¹²⁸
• Congress: Congress should shed light on the gaps between first- and third-party usage and incorporate ways to mitigate them in legislative proposals. Additionally, it should act to determine the appropriate division of liability between developers of AI models and deployers of AI models. Both parties must have some liability to ensure responsible behavior, and Congress must prevent both parties from passing the responsibility to the other endlessly. The Lawyers’ Committee for Civil Rights Under Law recently proposed Online Civil Rights Act,¹²⁹ which includes a duty of care for both developers and deployers and requires annual reporting from deployers on harms to be reviewed by developers.¹³⁰

Future CAP work will include proposed solutions to the question of AI liability as well as broader AI legislation recommendations.

Further ways to safeguard generative AI systems in the medium-to-long term

Below are additional ways generative AI developers and deployers can mitigate the risks of these systems, beyond equalizing mitigations for first- and third-party usage.

Abuse prevention

• Developers should develop tooling, such as content moderation endpoints and abuse monitoring, and other tools, making them easy for deployers to use and integrate into their apps.
• If an LLM is utilized for moderation of content generated by an LLM, that service should be provided for free or at a discount by the developer of the LLM, such as OpenAI’s GPT-4 for content policy development and content moderation decisions.¹³¹

Transparency

• Deployers should be required to disclose which LLMs they are utilizing in their applications.
• Developers should publish transparency reports for usage of LLMs to highlight prevalence of violations across abuse types and detail reports from deployers of their technology.¹³²
• Developers should publish transparency reports for how AI is being integrated into their first-party apps and offerings as well as what usage or API policy violations are being surfaced.
• Developers should be transparent with users about when they violate usage policies, including what actions or content led to the violations, how to appeal, and what remediations may be required of the user.
• In first-party usage, developers should cite the sources used to generate content in answers, such as Perplexity.¹³³

Data and tooling

• Build appropriate tooling for holistic internal review of reports when a third party or user informs the developer of potential usage policy violations.
• Alter data retention, minimization, and anonymization policies to ensure reports can be reviewed in a timely manner and with all necessary information to make a usage policy violation decision.
• Enhance existing tooling—currently, just inputs and outputs—to reveal necessary information to select, with vetted reviewers to appropriately review reports against usage policies.

Generative AI is poised to affect nearly every facet of society and offers a myriad of opportunities as well as significant risks if not appropriately mitigated. Today, AI developers are not adequately safeguarding their technology from the unique risks posed by third-party usage of their models; nor are deployers offering end users the tools and transparency to be in control of their experience with these powerful systems. If companies execute the recommendations outlined in this report and governmental bodies act by incorporating these considerations into ongoing workstreams, we will all collectively be able to continue to benefit from this groundbreaking technology for decades to come.

The authors would like to thank Ben Olinksy, Dr. Alondra Nelson, Dave Wilner, Audrey Juarez, Steve Bonitatibus, and Will Beaudouin for their contributions to this report.