A 21st Century Privacy Act: Ending the Exploitation of Unlawfully Obtained Government Data

The federal government’s use of Americans’ personal data has expanded well beyond the protections envisioned when the Privacy Act of 1974 was enacted more than 50 years ago.¹ The law was designed to prevent and penalize the government’s misuse of personal data,² but it did not establish clear mechanisms to stop the continued use of data once they were found to have been unlawfully collected or shared,³ as government records of the time were largely analog and accessed in limited ways. This meant that a privacy violation often consisted of a physical file leaving a government office, and fixing that violation simply required returning the file to the correct drawer.

That framework does not account for the fact that government data has evolved to be digital, rapidly disseminated, and vast in scale. Today, data can be copied and transmitted across agencies and into the hands of third parties in an instant. Increasingly, they also fuel artificial intelligence (AI) systems that shape critical government decisions about public services and individual rights. Therefore, the enforcement tools intended to prevent data misuse must also evolve. Their shortcomings are being evinced by the Trump administration’s exploitation⁴ of millions of Americans’ sensitive personal data⁵ that are currently housed within federal systems.⁶

Important efforts are underway to update the Privacy Act of 1974, such as a proposal recently released by Sens. Ron Wyden (D-OR) and Ed Markey (D-MA). While helpful,⁷ these proposed reforms offer incremental updates rather than a full reimagining of how government-held personal data should be governed in the 21st century. In light of emerging threats from the Trump administration and Elon Musk’s Department of Government Efficiency (DOGE),⁸ Rep. Lori Trahan (D-MA) issued a request for information⁹ on March 18, 2025, seeking input on how to reform the Privacy Act and shape a comprehensive framework for safeguarding personal information held by the federal government.

The Center for American Progress recommends that Congress explicitly require both data deletion and algorithmic disgorgement when personal records held by the government are collected, shared, or used in violation of the law. Data deletion refers to the removal of personal information obtained in violation of the law, while algorithmic disgorgement requires the deletion of models or systems that were built or deployed using unlawfully obtained data. The Federal Trade Commission (FTC) already uses this approach in the commercial sector,¹⁰ requiring companies to delete both unlawfully collected data and any algorithms derived from them. Together, these remedies prevent agencies and third parties from profiting from illegal practices.

Congress now has the opportunity to create a modern privacy framework that reflects current government data practices and anticipates emerging threats. To do so, it must reconsider past federal privacy efforts that focused on commercial actors¹¹ while leaving government-held personal data largely unaddressed. As CAP has previously noted,¹² privacy protections must apply not only to private companies and entities but also to government agencies and service providers entrusted with handling sensitive personal data. Recognizing the need for privacy protections across both sectors, this report outlines recommendations to strengthen government accountability through a modernized and expanded Privacy Act. While the recommendations and underlying principles of legally mandated data deletion and algorithmic disgorgement are tailored to the public sector, they should also inform a broader privacy framework that applies across both the public and private sectors.

The Privacy Act of 1974 does not explicitly impose any legal obligation to delete data that have been collected, shared, or used in violation of the statute, nor does it require the deletion of any tools, models, or systems developed using such data.¹³ Instead, the law focuses primarily on the prevention of improper data collection and unauthorized disclosure at the outset.¹⁴ It requires federal agencies to practice data minimization by maintaining only data that are “relevant and necessary” to carry out authorized purposes​. Although this strongly implies an obligation to delete records once they no longer serve an authorized purpose, it does not expressly mandate such deletion. The law prohibits the disclosure of personal records to any persons, including another agency, without prior written request or consent of the individual to whom the records pertain.¹⁵ It also requires agencies to establish administrative, technical, and physical safeguards to protect records, including having policies on storage, retrieval, access, retention, and disposal.¹⁶ However, the statute does not require federal agencies to conduct regular audits or systematic reviews to ensure that retained data continue to meet the statute’s “relevant and necessary” standard. As a result, there is no enforceable duty under the Privacy Act for agencies to periodically assess and delete records that have outlived their authorized purpose.

When violations occur, the Privacy Act provides limited remedies focused on individual harms rather than systemic abuses. Individuals may bring a civil suit if an agency fails to amend inaccurate records, refuses to provide access to requested records, or otherwise violates the statute in a way that causes tangible harm.¹⁷ In these cases, courts may order the agency to amend the record or to disclose improperly withheld information, and they may award monetary damages in instances of willful or intentional misconduct. The statute also provides limited criminal penalties, primarily for willful disclosure of agency records and for knowingly and willfully requesting or obtaining records from an agency under false pretenses.¹⁸

The Privacy Act does not provide any clear mechanism to prevent the ongoing use of unlawfully obtained data. Even after a legal violation is identified, agencies and third parties are not required to delete the data; dismantle any models or systems built or deployed using them; or revoke the decisions that were made based on them. This allows both agencies and third parties to continue operating and profiting from data obtained and tools developed through unlawful practices. In an era of automated, interconnected federal systems, this enforcement gap severely weakens the law’s ability to safeguard privacy. Any modern federal privacy law must close this loophole.

Other federal laws that govern information security, such as the Federal Information Security Modernization Act (FISMA),¹⁹ may need to be updated in tandem with reforms to the Privacy Act. While FISMA requires agencies to maintain information security programs and report breaches, it does not impose penalties when breaches occur.²⁰ This accountability gap reduces the incentive for agencies to adopt meaningful safeguards and turns compliance into a procedural exercise rather than a substantive one.

A modern privacy framework tailored to the public sector must recognize the scale of government data systems, the sensitive nature of the information they contain, and the risks they pose when not properly governed. Strong enforcement must go beyond identifying harms and issuing fines; it must also correct the harm and prevent further abuse. One critical issue is the continued use of unlawfully obtained data, including the operation of systems built from them. This core principle is simple. If data were collected, shared, obtained, or used in violation of the law, they should be deleted. If a system or model was developed or deployed using that data, it should not be allowed to remain in operation. This approach is grounded in well-established legal doctrine. Just as evidence obtained through unlawful searches cannot be used in court under the “fruit of the poisonous tree” doctrine,²¹ data collected in violation of federal privacy protections should not serve as the foundation for ongoing government decisions or be used to benefit third parties, including private entities. Unlike user-initiated deletion rights, which depend on individuals knowing their data were misused, legally mandated obligations ensure that remedies are triggered by official findings of wrongdoing and enforced directly by judicial oversight. Individuals should not bear the responsibility for detecting and addressing government data misuse. Federal law must instead require agencies to take immediate and verifiable compliance actions once a violation is confirmed.

One model for this approach already exists. The FTC has adopted data deletion and algorithmic disgorgement²² as tools to address unlawful commercial data practices. These remedies require organizations²³ to delete both the unlawfully obtained data and any algorithms or models developed or deployed using that data. The FTC first used this approach in 2019 in a case against Cambridge Analytica,²⁴ when it required the company to delete both the data it deceptively collected through the Facebook app as well as “any information or work product, including any algorithms or equations, that originated, in whole or in part, from this Covered Information.”²⁵ Cambridge Analytica ultimately shuttered operations in the wake of the investigation and the enforcement action.²⁶

More recently, in a 2021 enforcement action against the photo app Everalbum, the FTC alleged that the company misled users about its use of facial recognition technology.²⁷ As part of the settlement, Everalbum was required to delete not only the improperly retained data but also the facial recognition models and algorithms built using users’ photos and videos.²⁸ The company was required to submit a written statement to the commission certifying under penalty of perjury that all deletion or destruction obligations were fulfilled.²⁹ This reflects a shift from earlier enforcement approaches, wherein companies were sometimes allowed to keep and continue using systems built on unlawfully obtained data. For example, when Google used data scraped from competitors such as Yelp, enforcement actions did not always require Google to dismantle the tools built from that data.³⁰ By contrast, the FTC’s current approach recognizes the outputs of data misuse as inseparable from the violation itself. Once the data are compromised, so too is everything built on top of them.

This approach is even more critical when it comes to unlawfully obtained or used government data. Federal privacy law governing government data use should follow the same logic applied in the commercial sector. Without a clear legal mandate requiring deletion of unlawfully collected data and disgorgement of systems developed or deployed using such data, agencies and third parties will continue to benefit from privacy violations long after they are identified. The existing penalties are not enough to stop the harm. If the government and third parties are permitted to retain unlawfully obtained data and continue operating tools built or deployed on them, the harm is compounded, and the incentive to violate the law remains. A modern framework must instead require immediate data deletion, algorithmic disgorgement, and publicly verifiable compliance once a violation is confirmed. These enforcement mechanisms are essential for preventing future misuse and for ensuring that the government does not normalize the exploitation of unlawfully obtained data as a routine feature of public administration.

Federal agencies and third parties must be explicitly required to delete personal data that are:

• Collected in violation of statutory limits
• Shared unlawfully across agencies, contractors, vendors, and other third parties
• Used beyond its legally authorized purpose
• Obtained, accessed, or extracted without lawful authority, including through hacking, scraping, or deceptive means, as well as informally shared or transferred without proper legal basis—even if the recipient was unaware the transfer was unlawful

The data deletion requirement should apply not only when data are unlawfully collected at the outset but also when legally collected data are later improperly shared or misused. The fact that another government agency may be using the data within its own authorized scope should not exempt it from the deletion obligation in cases where the original transfer or access was unlawful. Examples of this include public benefits data and IRS data improperly shared with other agencies and misused for immigration enforcement.³¹

Agencies should be required to promptly report the violation to appropriate external oversight bodies such as inspectors general, Congress, and dedicated privacy oversight entities. In this report, the term “external oversight bodies” refers to entities that exist outside the executive branch, rather than internal agency mechanisms responsible for overseeing themselves. In addition, agencies should publicly disclose the violation whenever feasible to maintain public transparency and awareness. The compromised data should be immediately isolated and deleted from the system or operational function where the misuse occurred, and all unauthorized uses should cease without delay. Agencies should then implement comprehensive technical and administrative safeguards to prevent any recurrence of the misuse or unauthorized access.

In extremely limited circumstances where immediate deletion would significantly impair the continuity of essential public services, agencies must be subject to stringent procedural requirements to prevent abuse. Recognizing that claims of “service impairment” could be exploited to justify continued use of unlawfully collected data, federal law must narrowly define and tightly constrain this exception. Any delay in deletion must be temporary, narrowly tailored, and explicitly contingent on achieving full deletion within a defined timeframe, not to exceed 90 days. Agencies may seek one renewal of this delay, also not to exceed 90 days, but only with approval from a federal judge. Renewal requests must include a clear justification, a concrete timeline for deletion, and evidence that continued retention remains necessary to prevent immediate public harm. The approving judge must also be granted authority to enforce the deletion deadline and impose consequences for noncompliance. Under no circumstances should indefinite retention be permitted. The burden to demonstrate that the data could not have been lawfully collected at the time of acquisition and that their continued use is indispensable to prevent immediate public harm must rest entirely on the agency. Mere administrative inconvenience, cost savings, or general service degradation should not be sufficient to justify retention.

Data deletion obligations must be applied across the federal data life cycle and accountability chain. Agencies that unlawfully collect data must promptly delete them upon a determination of wrongdoing. Equally, second-party federal agencies that receive, access, or use data beyond their authorized limits must also face the same mandatory deletion requirements. State and local governments that acquire or utilize federal data through direct sharing or via participation in federal programs must similarly delete any data determined to have been unlawfully collected or shared by federal entities. Furthermore, vendors and contractors who collect, process, or utilize data on behalf of federal agencies must be held to equally rigorous deletion standards. Contracts with these third parties should explicitly mandate the immediate deletion of any unlawfully obtained data and prohibit retention or reuse in other commercial or governmental products or deployments. This obligation should extend to any private entity that gains access to federal data through improper means, whether formally contracted or informally granted access, to ensure that no private actor is permitted to benefit from or perpetuate the misuse of unlawfully obtained government data.

No agency, government partner, private contractor, or private entity should be allowed to avoid deletion requirements by claiming ignorance or indirect access as a defense to evade responsibility. In short, no participant in the federal data pipeline—whether governmental or private—should be permitted to retain or benefit from unlawfully obtained or improperly used data, and accountability must comprehensively apply to all parties involved.

To ensure that deletion obligations are meaningful and cannot be easily undermined, federal law must also prohibit the re-collection or reacquisition of the same data following a finding of illegality. Agencies should be barred from re-collecting substantially similar data for a defined period of time unless they can demonstrate that the new collection fully complies with all statutory requirements and has undergone additional oversight review. This safeguard is essential to prevent agencies from circumventing deletion mandates by re-collecting the same information under a new procedural pretext. All deletion actions should be documented and subject to verification by independent oversight bodies. Public transparency should be required wherever feasible, including through the publication of deletion orders, redress efforts, and a timeline for full compliance. As part of any deletion order, agencies must also submit a full disclosure of all nongovernmental entities that received or accessed the unlawfully collected, shared, or used data.

Federal agencies and third parties must be explicitly required to delete algorithmic systems that are:

• Trained or fine-tuned on data collected in violation of statutory limits
• Built using data that were shared unlawfully between agencies or third parties
• Used to generate outputs or decisions based on data that were collected or used in violation of statutory limits

Just as federal agencies and third parties must be required to delete unlawfully obtained data, they must also be required to delete any AI models or systems developed using that data. When agencies and third parties train, fine-tune, or deploy AI models using data collected or shared in violation of federal law, they should not be permitted to retain, operate, and benefit from those tools. The law must require mandatory algorithmic disgorgement once a violation is confirmed. This includes not only the deletion of the affected model but also the removal of any outputs or decisions generated using the unlawful data. Agencies and third parties must also immediately suspend any systems or applications that rely on the compromised model or its outputs to prevent further misuse. A narrowly defined de minimis exception may be appropriate where the unlawfully obtained data use is truly incidental, immaterial to model development or outputs, and occurred despite good faith compliance efforts. In such cases, agencies and third parties should be required to promptly isolate and remove the data and demonstrate that their inclusion did not materially influence the system. Each of these requirements must be supported by independent verification. The burden must rest entirely on the agency or third party to prove that the violation was de minimis, that prompt corrective action was taken, and that no material benefit was derived from the unlawfully obtained data.

As discussed in the data deletion section, only in extremely limited circumstances where full model removal would significantly impair the continuity of essential public services should any delay in decommissioning be permitted. Agencies must secure approval from an independent oversight authority and provide clear and convincing evidence that deletion would cause immediate and serious harm, and that no lawful alternatives exist. Administrative inconvenience, operational difficulty, and cost efficiency must never justify continued reliance on systems built or deployed using unlawfully obtained data. Any permitted delay must be strictly limited in duration and purpose and should not exceed 90 days, and agencies must develop and implement a concrete plan for full removal or lawful remediation of the affected model. If an agency seeks an extension, it should be limited to one additional 90-day period and subject to approval and enforcement by a federal judge who would ensure compliance with the revised timeline and impose consequences for failure to meet the deadline.

Algorithmic disgorgement must apply across all federal uses of AI. Any agency that develops or fine-tunes models in-house using unlawfully obtained data should be required to delete the model itself along with any components or outputs derived from the affected data. To verify that violations are fully addressed, agencies should be required to submit documentation of their corrective actions to external oversight bodies, which may include inspectors general, dedicated privacy or AI oversight authorities, or Congress. Agencies that do not build models themselves but that deploy or rely on such systems must be held equally responsible. If an agency acquires a model that was developed or fine-tuned using unlawfully obtained data, it must promptly decommission and delete the model, cease to use it in any automated or assisted decision-making, and conduct an audit of internal systems or outcomes that relied on it.

Although accountability must extend to third-party actors, disgorgement obligations must be carefully tailored to clarify vendors’ and contractors’ responsibilities and to avoid discouraging them from working with federal agencies. Federal privacy law should require vendors and contractors to delete models that are developed, fine-tuned, or deployed using unlawfully obtained data, but these obligations must account for the varied technical and contractual settings in which such models are built. Models or AI systems approved for federal contracting—such as those offered by major cloud providers—may include contractual and technical safeguards that prevent incoming government data from being used for model training, and those requirements should be included or adopted in future legislation or regulation. The situation becomes more complex when government data are uploaded in an unauthorized way to a model that is designed to train on all user inputs. For example, if a government user uploads unlawfully obtained data to a publicly accessible AI model and the service provider is unaware that the data are illegal, the key question becomes whether the vendor knowingly accepted and used the data for training or whether the data submission and subsequent training occurred beyond the vendor’s control. In such cases, requiring a vendor to delete an entire algorithmic model due to a single illegal input may not be appropriate, depending on the nature and amount of data involved. In these cases, the law should include provisions allowing a judge to determine the proper course of action that balances the equities involved based on a fact-specific inquiry into the surrounding circumstances.

The risk also extends to scenarios in which vendors or contractors build custom models based on data that the government provides and illegally certifies to be legal. Unfortunately, given that DOGE has reportedly ignored existing laws,³² this is not an impossible scenario. If the government certifies that the data are authorized but does so knowing they are not, and a contractor relies on that assurance in good faith, deletion may still be required, but the contractor should have a path to seek recourse to address the harm and ensure fairness. Any vendor or third party that manages or develops systems on behalf of the government should also be subject to audit requirements to ensure affected models are fully decommissioned and not reused in future deployments in either government systems or any external or commercial application.

As a condition of accessing or receiving public data, private entities should be required to certify that they understand the applicable deletion and algorithmic disgorgement obligations and agree to comply with any future disgorgement orders. Private entities should enter into contracts knowing that deletion may be required if the data use is later found to be unlawful, and they should architect their services accordingly to prepare for any resulting impacts on their business. Similarly, given DOGE’s reported trend of noncompliance with existing law,³³ the government should be required to certify that any data passed to contractors are in full compliance with the law.

To ensure that systems and models built or deployed on unlawfully obtained data do not resurface under new names or reappear in future deployments, federal privacy law must include safeguards that prohibit agencies and third parties from rebuilding models using data already deemed unlawful. Like the data deletion requirements outlined above, these obligations must be mandatory, verifiable, and designed to prevent evasion. All deletion actions should be documented and subject to verification by independent oversight bodies. Public transparency should be required wherever feasible, including through the publication of deletion orders, redress efforts, and a timeline for full compliance. Penalties should apply not only when agencies or third parties refuse to dismantle models, but also when they knowingly continue to use outputs from these models or attempt to redeploy them under a different label or in a different context. Agencies and third parties that resist oversight, conceal the sources of training data, or fail to fully decommission affected systems should face heightened legal consequences.

Robust enforcement is essential to ensure compliance with both data deletion and algorithmic disgorgement requirements. In addition to establishing clear obligations, federal law must impose meaningful consequences for agencies and individuals that fail to comply.

As demonstrated by agencies’ exploitation of the Privacy Act’s “routine use” exception to justify broad data sharing,³⁴ government actors cannot always be trusted to enforce privacy obligations against themselves. To ensure accountability, individuals must be granted a private right of action to seek injunctive relief and civil penalties for violations of data deletion or algorithmic disgorgement requirements, not just to order an agency to amend records or to allow an individual access to their records. Additionally, federal law should authorize private attorney general actions, allowing individuals to bring suits on behalf of the public to enforce deletion and disgorgement obligations. Modeled on the False Claims Act’s qui tam provisions,³⁵ this mechanism would empower private individuals to serve as essential enforcers of federal privacy protections, ensuring accountability even when agencies or officials attempt to evade oversight.

Civil penalties should apply to federal agencies that unlawfully collect, share, or use personal data, or that fail to delete such data or the systems developed from them once a violation is identified. Criminal penalties should apply to agency employees, contractors, vendors, or other individuals who knowingly engage in unauthorized access, internal system probing, or efforts to extract protected information without lawful approval. Even if they are unsuccessful, these actions represent serious breaches of trust and institutional safeguards. Penalties should be scaled based on both the sensitivity of the data and the institutional power of the actor. They should be significantly higher when the data concern areas such as health, immigration, or financial records, where misuse poses serious risks to individuals. The most severe penalties should apply in cases involving senior officials or centralized entities, such as DOGE, where the potential for harm is greater and the abuse of authority most corrosive to democratic accountability.

Although the enforcement framework should apply consistently across deletion and disgorgement mandates, it must also recognize the unique challenges of enforcing algorithmic disgorgement. AI systems are often built on complex data pipelines, making it difficult to determine whether unlawfully obtained data were used and whether that use influenced downstream systems or decisions. Recent AI guidance from the Office of Management and Budget³⁶ begins to address this by requiring agencies to maintain AI inventories for current and planned use cases and to document high-impact systems, which may support future enforcement. Yet key questions remain around what standards or evidence will be sufficient to trigger disgorgement, how data provenance should be tracked, and what audit powers are needed.

The current federal privacy framework does not adequately respond to the risks posed by digital data collection, interagency sharing, the transfer of data to private entities, and the use of artificial intelligence in government decision-making. While the Privacy Act of 1974 laid the foundation for privacy protections in an earlier era, the law lacks the structural tools needed to stop data misuse once it occurs. It also fails to account for the growing role of algorithms in shaping how public services are delivered, how benefits are determined, and how individuals are monitored or assessed by the federal government. Efforts to reimagine and modernize the Privacy Act, including those led by Rep. Trahan and Sen. Wyden, are critical to addressing these gaps. As part of that effort, legally mandated data deletion and algorithmic disgorgement must be recognized as essential tools for curbing unlawful data use and preventing government and third-party actors from continuing to benefit from it.