Discover more from Thoughts on AI Policy
Aleksander's Testimony to the Senate Subcommittee on Human Rights and the Law
Senate Committee on the Judiciary's hearing on "AI and Human Rights"
On June 13, 2023, Aleksander testified in front of the Subcommittee on Human Rights and the Law of the Senate Committee on the Judiciary in a hearing on AI and human rights. Below is (an expanded and slightly edited version of) his written statement. (The original written statement can be found here, and a video recording of the hearing can be found here.)
Chairman Ossoff, Ranking Member Blackburn and Members of the Committee, thank you for inviting me to testify. Much has already been said and written about how AI may transform society, both about the opportunities and risks—from AI's potential to enhance our productivity, creativity, and overall quality of life to its ability to perpetuate discrimination, drive economic inequality, and pose an existential risk.
I will not reprise those conversations here. Instead, I will focus my testimony on one issue that I find particularly salient, time-sensitive, and worrisome: how AI could erode central tenets that enable our society to function, including our ability to carry out democratic decision-making.
Specifically, I will discuss how AI is poised to fundamentally transform mechanisms for the dissemination and understanding of information, and the unsettling implications of those changes. I will also sketch out what we can do to mitigate these emerging risks.
Thanks for reading Thoughts on AI Policy! Subscribe for free to receive new posts and support our work.
The information ecosystem
As the saying goes, information is power. Information influences people’s perceptions, beliefs, and, ultimately, their actions. It can shape micro-decisions (such as what foods people choose to eat) as well as macro-decisions (such as how people choose to vote). Whether it is delivered en masse or in a structured, targeted way, information can affect broader trends and even issues of geopolitical magnitude. As a result, how people consume, look for, and above all trust information can shape—and sometimes upend—societies.
Digital technologies, when introduced, transformed the information landscape to an extent that is rivaled only by the invention of the printing press. Tools like email and chat enabled us to communicate with friends online. When we wanted answers, we turned not to libraries or schools, but to sites like Wikipedia. We would even solicit opinions from strangers, following their blogs and tracking debates on Reddit.
It is important to recognize that changes in information technologies—whether the invention of the printing press or the advent of e-mail—do not just make information more accessible. Rather, they fundamentally change the dynamics of information sharing and acquisition. A change of significant importance, for example, was the fact that by using such tools we opened ourselves up to strangers, people beyond those we already knew. We allowed ourselves to be influenced, even persuaded, by members of the online community that we did not know personally.
One of the earliest problematic instances of such an unwanted influence was through email. Spam and scams became rampant in the late 1990s and early 2000s.
Fortunately, although our adjustment was tumultuous, we were eventually able to (mostly) adjust. Notably, this took a mix of technological remedies—better spam and scam identification—as well as policy ones—namely, regulations that protected consumers, like the CAN-SPAM Act. It also required having us develop “email literacy,” i.e., an understanding of what email is and is not and, in particular, how it differs from a letter from an actual acquaintance.
A particularly disruptive instance—and one which society has not (yet) been able to fully adjust to—is social media. Beginning as a way to connect people to their friends and family online, social media quickly became a primary information source for billions of users, with enormous ramifications. At the root of social media platforms’ success was that they drastically reduced barriers to content creation, making it possible for nearly anyone to post media in the form of their choosing. Perhaps most importantly, they introduced personalized information delivery—platforms hand-deliver content specifically chosen for each user’s tastes and interests (as inferred from the incredibly vast amount of user data they collect).
These developments opened users up to manipulation and persuasion at scale. Indeed, mis- and dis-information grew rampant on social media.1 It became difficult for users to distinguish credible sources from less credible ones (and even from bots). Content creators themselves were not incentivized to provide high-quality content because they had to compete with other, often low-quality, creators. Plus, due to the sheer number of such creators, removing “bad” or “toxic” users did nothing to change the health of online discourse.
Personalization adds an extra dimension to this problem too. Under personalization, users are exposed to different “versions of reality,” making it difficult to identify the “truth” and causing filter bubbles and echo chambers to thrive online. Targeted content delivery also allowed third parties (e.g., advertisers) to figure out which users are most influenceable and what messages would be most effective for them, as the Cambridge Analytica case laid bare.
In comparison to our adaptation to challenges posed by email, our response to issues on social media has been much less effective. On the one hand, “social media literacy” has improved, with many learning how to identify bots, to avoid clickbait meant to foster outrage, and to avoid trolls. However, on both the technical and regulatory fronts, our approach has been more timid. For the most part, platforms have resorted to mild measures, such as adding features that flag unreliable information and improving bot detection algorithms. A number of regulatory solutions have been proposed, but none of them have become the law so far—in no small part because of the complex structure of our freedom of speech legislation. All in all, social media and its many impacts on our society’s information ecosystem remains largely without oversight, leaving us maybe a bit more social media literate but still very much vulnerable.
AI will be (another) transformation in the information ecosystem
While we are still dealing with the transformations in this space brought to us by email and social media, there is already a new transformation afoot—a transformation fueled by AI—that may be more consequential than anything we have experienced recently.
With the advent of AI (especially the newest wave of generative AI) anyone who can use a chatbot is in a position to become a “trusted source”—and in particular, a highly personalized trusted source. Indeed, as more of what we see becomes generated and disseminated by AI, the lines between humans and bots are becoming blurred. We need to start to be more wary than ever about how information reaches us, its trustworthiness and its ability to persuade us.
More precisely, AI is changing the information-delivery landscape in (at least) three key ways:
It enables the creation of a wide variety of extremely realistic content, namely written text, audio, photos and, soon, videos.
The language produced by Large Language Models (LLMs) like ChatGPT or Google Bard can seem natural and highly persuasive, in no small part since we are wired to believe that such speech can come only from humans.
It makes the creation of such content cheap and broadly accessible—even to parties with little if any technical expertise—making it frighteningly easy to deploy it at scale.
We are already seeing early adoption of generative AI in our information sphere, from art, to copywriting, to political ads, but these are just a tip of the iceberg. We will see much, much more very soon. The onset of this technology brings with it a whole spectrum of risks and potential harms. I will highlight just a few of them below.
Enhancing “traditional” cybercrime
One immediate impact of the newest wave of generative AI is that “traditional” spam and phishing campaigns are even easier to conduct. What previously required careful photo editing and writing (as well as some non-trivial human involvement) now only requires a few clicks. The recent use of an AI-generated fake image of a fire near the Pentagon is just one illustration of that.
Also, the fact that generative AI can convincingly impersonate a human online poses a fundamental challenge to our existing mechanisms for protecting our information infrastructure, public discourse and governance. After all, the bot detection and moderation algorithms that our online discussion platforms use—whether they be Internet forums or newspaper comment sections—tend to rely on some kind of “prove that you are human” test. How will these platforms cope with malicious parties that can deploy swarms of AI-driven bots that breeze through such tests?
“Spear-phishing” and personalized blackmail
The enhancement of the “traditional” deception is, however, just the beginning. AI's unique ability to create content that is both convincing and personalized means that, for example, phishing will no longer need to involve generic emails sent out to thousands of recipients, hoping someone will be duped. Instead, we will have “spear-phishing,” where both the message and the whole conversation that ensues are fully automated and customized to each individual recipient.
In fact, there is a very real possibility that a new kind of blackmail scheme will emerge. In such a scheme, a malicious actor edits someone's photo from social media (using AI) to depict them in a compromising situation, and then threatens to make the edited photo public unless the victim pays up. How many of us would not pay to simply make the problem go away? Thanks to AI, these kinds of schemes can now be executed (again) fully automatically, cheaply, and at scale.
Another variant on this theme, one that a fellow witness has experienced first-hand, is (again, automatic and highly targeted) fake blackmail. Indeed, AI’s ability to impersonate the voice of just about any person enables a whole new array of scams.2 As the ability to generate video with AI improves, other troubling possibilities such as targeted AI-generated explicit content will become an even more acute problem too.3
Personalized persuasion at scale
This expansion of the cybercrime toolkit is hardly the only worrisome consequence of the next generation of AI tools. Indeed, AI is bound to transform how we think about any information campaign, be it ideological, political or commercial. Specifically, such campaigns will no longer need to rely solely on the promoted message to go viral. Instead, campaigners can incorporate generative AI, and have promoted messaging reach its intended audience individually and in a highly personalized manner.
For example, in the near future, an advertisement may not be just some post that comes across your social media timeline. Rather, it may be a Facebook friend that you met online—a “friend” who is actually an AI-driven agent impersonating a human. A friend that (only subtly) weaves political commentary, or product endorsements, or any other messaging, into your engaging conversations about sports, movies or favorite food.
Similarly, instead of trying to corral a critical mass of people to campaign for a cause—whether on social media, via direct calling, or letter-writing—a single actor can field a campaign by themselves, using generative AI-driven bots in place of people. A campaign that is equally effective (thanks to the sophistication of these bots) but needs neither any buy-in from the broader population nor even comparable resources. As far as I know, as of now, this would all be legal too.
Automated creation of addictive content
Another possibility is that interacting with AI becomes not only attractive and persuasive but also addictive to us (or to some subset of the population). After all, loneliness and an unmet need for intimacy are growing problems in our society.45 The kind of focus, “fit” and “care” that AI-driven “friends” might seem to exhibit could be extremely alluring—potentially, to the point of becoming addictive. Indeed, we can see inklings of this trend even now, with AI-enabled chatbots like Replika drawing millions of dollars in monthly subscriptions fees (with a reported 60% of its chats being intimate in nature).6
This aspect of AI could (and, I hope, will) play a positive role too.7 But imagine the power someone who is able to deploy such AI-powered agents could have over us, especially at scale. What if that power gets abused? What if these capabilities are harnessed to hijack the “attention economy” that already drives much of our social media and online commerce? What would this mean for our productivity and long-term happiness? How do we feel about having our children exposed to all of that?
Eroding trust in information and written (or audio-visual) records.
Thanks to AI, we are entering an era when any record might plausibly be faked. How does this affect our collective discourse as well as the legal and governance system? After all, we are a society whose foundations rely on the veracity and binding of such records—think contracts, deposition recording, or video evidence in criminal cases—and this reliance will only increase as more of our critical interactions occur in the digital sphere. How does our society adapt to such a tectonic shift?
What can we do?
The concerns I have outlined above may paint a rather bleak and, potentially, daunting landscape. But there is much we can do here. Specifically, we need a combination of technical solutions and policy actions that reinforce each other. In particular, policy can help drive the development and implementation of technical remedies, and technical innovations can, in turn, unlock new policy solutions. Let me describe some of these below.
On the technical front, we need tools that can help humans judge the authenticity of content—to understand the extent to which it was generated by a human and/or AI. These tools can take a variety of forms (and for many of them we already have proof-of-concept prototypes):
Watermarking and deepfake detection tools
One promising idea for ensuring the authenticity of content is “watermarking”—that is, placing an imperceptible “signature” in generated content that makes clear AI was used. Content consumers can then look for this watermark can to detect AI-generated content. Researchers have developed prototypes of watermarking systems, both in the context of large language models8 and image generations models.9 Much more work is needed, however, to make them sufficiently robust, and even then we may need policies to drive their adoption too (in particular, the providers of generative AI tools must volunteer to adopt such techniques for them to be effective). Also, like all such technologies, there will likely be an “arms race”—malicious actors will develop tools to evade the watermark system, and we will need improved techniques to respond.
Watermarks need to be placed in documents directly by the AI providers, but there is also a line of work on detecting AI-generated content in the absence of cooperation from the developers of a given AI model.10 Of course, this lack of cooperation makes it easier for malicious actors to thwart these detection techniques, making the corresponding “arms race” much more challenging.
Protection against unauthorized AI-powered content editing
Another problem that technology can help address is unauthorized AI-powered content editing—that is, the ability to use AI-powered editing tools to manipulate content against the wishes of its creators or people depicted in it. (Think, for example, of the personalized blackmail scheme described earlier, which involved a malicious party manipulating photos the victim had published on social media.) Could we develop a way for users to protect the photos they put online, to make it much harder to modify using AI? It turns out that such an “immunization” capability is a possibility11 but, again, much more work is needed to make it ready for the real world.
Provenance certification techniques
Beyond detecting AI-generated content, we may want tools to prove the authenticity of content. This could involve, for example, leveraging cryptography to provide automatic certification of the authenticity or provenance of a given document by tracing it to the exact primary source that created it (e.g., the person who took a given photo). Such tools are less likely to spur the same sort of “arms race” as watermarking and editing-prevention tools, but may also induce a shift in mindset. In particular, when such a technology is broadly available, we might presume content to be fake unless verification proves it to be real.
There is also a risk of a negative feedback loop with verification. For example, if a journalist from a reputable newspaper is fooled by AI-generated content and writes about it, the same (incorrect) fact can later be traced (correctly) to the newspaper, giving it unwarranted authenticity.
However, just to reiterate: no matter how work on such tools proceeds, these tools will not be a panacea. They will be neither perfect nor foolproof. Nonetheless, they can provide the necessary “friction” that makes undesirable use of AI that much harder to execute. They will also give us effective “footholds” for policy action.
As I noted above, technological approaches will need to work hand-in-hand with policy. Here are some possible policy approaches to pursue.
AI-generated content disclosure requirement
One relatively straightforward step would be to require that any consumer-facing AI-generated content be labeled as such. This kind of mandatory disclosure would, for example, likely hamper an AI-powered persuasion campaign we described above—at least, as long as everyone abides by this rule. (Even without full adherence, such a mandate would at least make the aforementioned mass persuasion campaigns illegal).
Of course, deciding the exact level of AI involvement that would trigger such a mandate, as well as the form such labelling would need to take, requires careful deliberation. And the rules would have to be updated as the technology and the use of it evolved. In particular, it would be important to avoid the “user desensitization” effect, in which the users stop paying attention to the corresponding disclosures due to being bombarded with them at every occasion (and for trivial reasons). (Such desensitization seems to have occurred, for example, in the context of the web cookie usage disclosure and consent requirements imposed in the European General Data Protection Regulation (GDPR).12)
Accelerating the use of content authenticity tools
As discussed earlier, content authenticity tools such as watermarking, deepfake detection, protection against unauthorized AI-powered editing, or provenance certification can be very useful but their effectiveness is hardly guaranteed. Even leaving aside technical questions, the efficacy of these solutions will critically depend on how broadly adopted they are. We need here a broad cooperation of the industry players that develop the relevant AI systems, so as to establish consistent expectations and standards. Policy can accelerate this process and broaden the use of such techniques, through incentives and/or mandates. After all, we don't know if market incentives will ever be sufficiently strong to drive the development and deployment of these technologies; they certainly are not enough at this point.
Client identification and suspicious activity reporting mandates
One possible approach to deterring rogue actors could be adapted from anti-money laundering laws. It would require providers of sufficiently capable AI services to implement adequate client identification mechanisms. These AI providers would then be expected to monitor the usage of the tools they supply to flag (and, potentially, block) suspicious activity as well as to report it to appropriate governmental agencies (such as FBI) or other organizations.
Advance “AI literacy” efforts
Of course, no technical solution or set of regulations will ever suffice to fully mitigate the risks AI now poses. It is thus crucial that, in addition to “email literacy” and “social media literacy,” we think about promoting “AI literacy.” The public needs to understand how to judiciously interact with AI systems—and how to be on the lookout for when they are interacting with AI in the first place. This includes helping the public avoid the natural tendency to anthropomorphize AI systems. We also must go from assuming that content is authentic until proven otherwise to assuming that content is fake until proven otherwise—or at the very least discounting the value of unverified content.
Overall, there is a need for a shift in the public mindset to accommodate how AI is changing the world. We thus need decisive policy thinking on how to advance AI literacy more intentionally, instead of relying on our society learning it the “hard way.”
To conclude, let me reiterate that I am excited about the positive impacts that AI can have, but I also want to be clear about and mindful of the risks it gives rise to. Today, my aim is to highlight one family of such risks. I am optimistic that we can mitigate these risks, but this will require work. It cannot be left to chance. And we need to get started now.
Thank you and I look forward to your questions.
I am grateful for invaluable help from Sarah Cen, David Goldston, Andrew Ilyas, and Luis Videgaray.
Allcott, H., Gentzkow, M., and Yu, C. (2019) Trends in the diffusion of misinformation on social media. Research & Politics 6.2: 2053168019848554.
Verma, P. (2023). They thought loved ones were calling for help. It was an AI scam. The Washington Post.
Hao, K. (2021). Deepfake porn is ruining women’s lives. Now the law may finally ban it. Technology Review.
Klinenberg, E. (2018). Is loneliness a health epidemic?. International New York Times.
Murthy, V. (2023). Our Epidemic of Loneliness and Isolation. The U.S. Surgeon General’s Advisory.
Huet, E. (2023). What Happens When Sexting Chatbots Dump Their Human Lovers. Bloomberg Businessweek Technology.
Balasubramanian, S. (2023). AI Offers Promise And Peril In Tackling Loneliness. Forbes.
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., & Goldstein, T. (2023). A watermark for large language models. arXiv preprint arXiv:2301.10226.
Wen, Y., Kirchenbauer, J., Geiping, J., & Goldstein, T. (2023). Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust. arXiv preprint arXiv:2305.20030.
Mirsky, Y., & Lee, W. (2021). The creation and detection of deepfakes. ACM Computing Surveys, 54(1), 7.
Salman, H., Khaddaj, A., Leclerc, G., Ilyas, A., & Madry, A. (2023). Raising the cost of malicious ai-powered image editing. arXiv preprint arXiv:2302.06588.
Kulyk, O., Gerber, N., Hilt, A., & Volkamer, M. (2020). Has the GDPR hype affected users’ reaction to cookie disclaimers?. Journal of Cybersecurity