Annie Kuo Becker (AKB): Welcome to Discovery, a podcast at the University of Washington School of Law. I'm your host, Annie Kuo Becker, today we're talking about a growing challenge in AI governance.
What happens when people want their data removed from systems that have already learned from it? Our guest is Jevan Hudson, director of our Technology Law and Public Policy clinic, a J.D. alum from the class of 2020, and incoming assistant professor at the UW School of Law.
Welcome Jevan.
Jevan Hutson (JH): Thank you so much for having me. It's great to be here.
AKB: We'll be discussing a recent article that Jevan co-authored with Ph.D. student Jay Conrad and Cedric Whitney in the Columbia Science and Technology Law Review. It's called “Forget Me Not: Machine Unlearning's Implication for Privacy Law.”
This article explores whether emerging techniques for machine unlearning could help make privacy rights, like the right to delete personal data and be forgotten, meaningful in the age of large AI models. So, here's the first question, Jevan. For listeners who may not be technical experts like myself, what exactly is machine unlearning and how is it different from simply deleting data from a database?
JH: Large language models are very different than traditional data architectures. It's not a filing cabinet where you can find a file and put it in the shredder, or going to a row in an Excel sheet and simply deleting that row.
When we're dealing with artificial intelligence models, particularly large language models, data that's trained on is baked into billions of parameters, which makes deletion or even data mapping incredibly hard.
And so, we dove into the concept of machine unlearning, which reviewers who aren't necessarily technical, is a suite of techniques targeted to remove or suppress data in a trained AI model. nd that data could be personal data, which is data protected by privacy and data protection laws, but also extends to data like intellectual property, which many in the audience may have read news stories about large AI companies training on textbooks or published written works.
And in our work, we divide machine unlearning into three families. The first family is what we call structural removal methods. And there's sort of a spectrum here. On one end of that spectrum is pretty aggressive, right? Start from scratch. Retrain the model, and when you retrain that model, it will behave as if it was never trained on that data. Right? Starting from scratch is a pretty straightforward way of removing the data of concern, but there are some trade-offs here, right?
On one level, you've completely removed the data, you started from scratch. The risk isn't there. The problem is that's incredibly expensive. Retraining a model, particularly large language models can be computationally expensive, and for organizations, there really isn't an appetite to start from scratch.
And so, foundational work in machine unlearning in the structural method space starts with SISA training. And CISA stands for sharded, isolated, sliced and aggregated. In effect, before training a large language model, for example, you partition your data. So, maybe in one part of that data is biometric data, maybe in another it's health data. Or one could imagine intellectual property having similar partitions.
And in lieu of training on all of that data at once, you train a variety of sub models, right? So, a sub model trained on biometric data, a sub model trained on insert XYZ types of high-risk data. And in the event of a need to retrain, instead of retraining the entire model, you retrain on one of those sub models.
This reduces costs compared to throwing out the thing and starting from scratch, but can still be quite expensive.
And so, that's what leads us to our second family, which we refer to as approximate removal methods. And here, in lieu of throwing out the model or retraining sort of compartmentalized parts of a model, researchers and practitioners work to effectively de-influence the role of particular data across parameters on a model.
So, as we talked about at the beginning, there are billions of different parameters that data might particularly impact. And in approximate removal weapon methods, we effectively, sort of, down weight the influence of the data we're seeking to remove. And so, here, you know, we trade fidelity for cost, right? So, the cost of approximate removal methods is markedly less than structural removal methods.
A great example is that the Harry Potter experiment, this was Eldon and Racinovich who wanted to think about like, how would we remove the entire corpora of Harry Potter from a trained model, and if we were to retrain it from scratch, right? It's going to cost us, like, 184,000 GPU hours, which is incredibly expensive and time consuming.
And so, they leveraged a technique called Fisher scrubbing. This falls in this approximate retraining methods bucket, and it took them one GPU hour. And so, here, it's cheaper, potentially more scalable. But the problem for privacy law and potentially for intellectual property, is that this removal is probabilistic. We cannot, for certain, know if that data has been fully removed. We can know to a good percentage degree that it certainly trades the fidelity away from the structural removal methods.
And then there's the third bucket, which is output suppression. And this technique may be probably what the audience is most familiar with. For folks who've ever used a public facing large language model like Claude or ChatGPT, there's sometimes you might ask it to do something, and it's going to respond with, I can't do that or that violates particular rules, right? I wouldn't suggest anyone in the audience go to ChatGPT and ask it to build you a bomb, but it's going to give you a pretty predictable response.
And output suppression ultimately doesn't deal in deletion or removal. It simply layers censorship on the top. So, maybe a more acceptable experiment for members in the audience is to ask ChatGPT for your social security number. You could imagine it's going to respond with, I can't do that, or something like, I don't actually have access to that.
And here, while we are trading away the ability to remove, say, personal data from the model, we do prevent issues of leakage, right? So, that if there is personal data in a training data set that ultimately doesn't get exposed in outputs.
The benefits here is scale. Output suppression techniques can be layered across tons of different models per jurisdiction. It can be customized, and it is extremely inexpensive, but when it comes to some of the normative principles that underlie privacy law, such as the right to deletion or the right to be forgotten, ultimately, the organization still has that data. Especially you could imagine if they collected that data illegally, suppressing it at the surface doesn't really address that initial illegal collection, and they're ultimately still benefiting from that collection by developing a trained model.
So, across these is a hierarchy, right? When we think about structural removal methods, there's stronger guarantees of removal, but it's incredibly expensive. And as of right now, there really aren't the incentives for organizations to pursue these types of methodologies. Approximate retraining, still not cheap, but certainly more scalable, and then output suppression techniques are very doable, and we're already seeing this play out, particularly in trust and safety contact.
AKB: This is a really cool topic because as we are more and more utilizing artificial intelligence for queries. Like I use it to ask questions about volleyball, but then it remembers my daughter's name. Some people use it as, like a personal therapy.
You know, it's absorbing all this information, personal information that sometimes consumers or users don't want to live in the ether. And even when there's like a removal method applied, there's still that, you use this term, something like a ghost shadow or information shadow that's like a residue that's left behind.
There's a lot of privacy regimes that promise individuals a form of deletion right. How do those rights become particularly difficult to enforce once personal data has been incorporated into a trained AI model?
JH: So, I'd say two points. I think the first is just the challenge of deletion itself.
So, when we look at those three families of methods, one of those guarantees deletion, but organizations, I would say, are not incentivized, and are really unlikely to throw out their entire model because one person wants to delete that information. And so really, then, they move into that second bucket of approximate removal methods, and for consumers, that might go pretty far in terms of reaching their goals of removing their information.
But there's an open question as to whether regulators treat that as compliance with, say, the general data protection regulation, which, at a normative level, requires full deletion.
But then there's a second problem, just setting aside the deletion challenge, which is the ways in which personal data can move from observed to becoming latent to the model, the ways in which data is abstracted into parameters in ways that we can't de-influence this. And this gets to what you pointed to at the beginning, which is the algorithmic shadow, which Professor Tiffany Lee has written quite a bit about, which is that even if we remove the observed data or engage in that sort of structural removal, in the case of SISA retraining, there's still good probability that some of that information has been absorbed by parameters in ways that we are not able to observe. Which may surface later or may influence particular outputs, even if it doesn't result in a direct leakage or output of the personal data in particular, which can raise important questions specifically when these systems are used in critical decision-making.
But when we get back to the question of our data subject rights under privacy and data protection law, even if you engage in structural removal, we have a pretty good chance that the observed data is gone. What about that other data that's been abstracted in these systems? And I think here is just a fundamental tension between the development of AI systems and privacy and data protection law.
AKB: What are some of the protections that are offered to us by privacy law?
JH: So, in theory, it depends on your jurisdiction, and varying organizations may expand protections to folks in jurisdictions that don't have privacy and data protection law. But we can take, for example, the general data protection regulation in the EU, which provides a variety of protections that we write about in our article.
AKB: First and foremost, is lawful basis. In the European Union in order to process and collect personal data, organizations need to have a lawful basis to do so. Some examples could be contractual necessity, right? Others could be consent.
For example, if an organization wants to collect and process your sensitive data, say that's information about your sex life, your sexual orientation, ethnicity, race, membership in a trade union. Organizations need consent in order to collect and process, and here that is an important sort of legal basis.
So, one could imagine, in the context of what we write about in machine unlearning, an organization collects a bunch of data and does not have a legal basis to do so. In that instance, that's potentially a body of data that could be subject to removal.
Additional obligations that are important, which we have talked about are data subject rights. We talked in particular about rights to deletion. Individuals also have other data subject rights that are complicated when it comes to large-scale AI models.
These include the right of access, right? What information or personal data does an organization have about you? And again, the stuff that's in the Excel sheets, that's in the files that might be able to be reproduced pretty easily, it becomes harder for organizations to track the personal data they have once it's baked into large language models.
Additional rights within data subject rights include rights to correction or rectification. What happens when the information that an AI is trained on is incorrect, that has potential, you know, reputational implications for an individual. How do we correct the model, which is a potentially different consideration in the machine unlearning context that requires not only the removal of the incorrect data, but then retraining with the correct data?
So, just some examples of the protections, I think the last one I touch on, which is growing in state privacy laws, particularly out of the state of Maryland, are general requirements of data minimization, that in order for an organization to collect data, they need to collect sort of the minimum amount of data necessary in order to achieve a particular processing task.
And this is a fundamental tension with large scale AI models, where organizations are effectively scraping the entire web, collecting as much as possible in order to develop a smarter or capable AI model. And that, in many ways, flies in the face of requirements for substantive data minimization when we're in a mode of collect as much as possible, to do as much as possible.
AKB: I also saw in your article that it talks about how data offered for a particular purpose has to, kind of, stay in that lane.
JH: Exactly. So, that is what we would call purpose limitation, right? For folks who maybe have unfortunately read privacy policies, they're important legal artifacts. I do recommend that you read them as a practicing privacy attorney. But these policies will lay out not only what data that organizations are collecting, but what purposes they're going to use it for.
And when an organization collects data for one purpose. They can't pick it up and then use it for an entirely different purpose that they weren't transparent about. And here, when we meet large scale AI systems, the problem is, is many of these tools are general purpose. They can be used for a variety of different purposes, and so depending on what those disclosures are up front, there are countless ways in which an organization may use your personal data for purposes unfamiliar to you or unavailable to you at the point of notice, which is again a fundamental tension when we're building general purpose technologies that might leverage our personal data for a thousand different ways, which is ultimately hard to capture at the point of a privacy policy or a notice that you read on an app or a website.
AKB: Wow. It almost feels like where we've entered this like new frontier for privacy law and all the protections that we are accustomed to, is almost like the Wild West has opened up.
JH: I don't think you're wrong. We've, you know, studied this. I have some other colleagues in the Department of Computer Science, and we recently engaged in a data set audit of data comp common pool. So, this is the largest publicly available data set available. These are a bunch of images mapped to a, sort of, a caption or a text.
In here, we took a small slice of that data set, and we were curious what personal information is in there, and we found tons. In our estimates, upwards of 187,000 real resumes linked to real people, credit cards, social security numbers, images of children. And these data sets have been downloaded millions of times and used to train countless models.
And so, there are real questions about the contours of our privacy and data protection laws in the EU and the US, particularly some of the exceptions around publicly available data, which, in my view, have really been stretched beyond imagination, such that you can really drive a truck through them.
But as you say, we're really at an inflection point where the protections provided and guaranteed by privacy and data protection law are being stretched. I would argue, are being abused. And it really requires regulators and enforcers to step in to meaningfully police those contours, such that organizations process data legally. Unfortunately, we really haven't seen that.
The one place I would point to would be the Khan administration of the Federal Trade Commission. And there they have deployed a rather important remedy that we write about in the paper, which is algorithmic disgorgement or model deletion.
And in those instances, the FTC is now deployed, I think, upwards of nine times, starting in 2019 and effectively there for organizations that collected data illegally — say, for example, they say one thing in a privacy policy, but end up using that data for another purpose or collect more data than the one they disclose — generally, a violation of Section Five of the FTC Act. In a number of different instances, the Federal Trade Commission has required organizations delete all technical work product tied to that data.
So, when we talked about structural removal methods, that's the most aggressive. You collected data illegally, it has to be deleted. We've seen this in cases against Everalbum against Kurbo, which was Weight Watchers children's dieting app, which as maybe a disclaimer to listeners, if you're interested in developing a children's dieting app, I would recommend you maybe reconsider. But that's a serious remedy to contend with that if you collect data illegally, it results in the deletion of full stack of technologies that you've invested in countless time, hours, people power.
And for us, that's where machine unlearning maybe plays potentially more of a surgical role or creating different incentives for organizations, where, look, if regulators are threatening full deletion of an entire model, right? Deploying these particular design methodologies puts you in a better spot to maybe avoid that, right? If you have a technical pathway to say, look, we understand, this data was collected legally. Here is our verifiable pathway to remove that information.
Again, require some design decisions up front, but I think for us, when we think about machine un learning as another remedy in the toolkit for regulators, it's in the same way a proactive compliance function for organizations who want to do things right.
And for me, as a practitioner in the privacy law space, this is something I would love to have operationalized in any organization I'm advising or work for, because when we get either a request from a data subject or an inquiry from a regulator, we're able to really tell that story of how we know the data we've collected. We have pathways to remove it, if required to remove.
And I think lastly was, a lot of the inspiration for this work is in chatting with state attorneys general's offices who are enforcing new state privacy laws, as well as just state consumer protection statutes, is they often hear the refrain, you know, it's technically impossible. We can't remove that data. It's been baked into the cake, and we can't remove that grain of flour.
And while I understand that sort of retort from tech companies, our synthesis of the computer science literature says, no, there are a variety of methods available. You know, different questions of scale and cost, but organizations can't simply throw their hands up and say it's an impossible task when there's a growing body of methods that show it is possible,.
AKB: Just to piggyback on that piece about policymakers, in your article you talked about the silver bullet, how policymakers might treat machine unlearning as a silver bullet for AI privacy issues, but it may not fully solve the problem. Is there that danger?
JH: 100%. I mean, as we argue quite clearly, it's not a silver bullet. It's not a panacea, but it is a tool in a toolkit that needs to sit across a variety of other privacy preserving interventions, both on the compliance front as well as on the enforcement front.
So, I would certainly warn lawmakers, this is not your silver bullet. We have not solved these tensions in privacy and data protection law, but it is certainly an important tool that shouldn't be forgotten in enforcement of privacy and data protection law, but also as a proactive compliance function. And as an example there, you know, when we're thinking about, say, for example, SISA training or compartmentalized machine learning that requires you to have a good understanding of the data that you've collected. That itself, right, the data mapping and data provenance is itself a very large privacy and data protection task that organizations, I would argue, required to do, but is an important function.
And so, you know, we could throw machine unlearning against the wall, but if there is no data provenance or data mapping, it can be quite hard to understand the removal task itself, and so certainly not a silver bullet, but I think, an important tool in the arsenal of enforcers and an important tool for AI developers in working to sort of de-risk their AI portfolio.
AKB: As a final question, I want to, kind of, like meld a couple of them together. When touching on the limitations of machine unlearning. And then a looking forward question, do you see machine unlearning likely becoming a standard tool for AI governance, or is privacy law going to need deeper structural changes to address AI systems?
JH: So, I'll start with your last question, and I would say both. I think we will see machine unlearning emerge both as a tool for regulators and enforcers of privacy and data protection law, but also an affirmative tool for internal compliance at any tech company that is working to develop AI technology.
But we still need meaningful, not only enforcement of privacy and data protection law, but also potential substantive revisions of those laws to account for some of these issues. I previewed earlier that study that we had done as a data audit of data comp common pool, and one of our recommendations out of that work is to narrow the exemptions to privacy and data protection law that target publicly available data. Their state privacy laws effectively say if an individual made data public and a controller or an organization has no reasonable basis to determine that that data is public, it doesn't constitute personal data, and therefore isn't subject to privacy and data protection law.
Part of our arguments there is, you collect 1.5 or 12 point 5 billion pieces of personal data, you have no way of knowing each and every data point whether an individual made it public. And so in that example, closing the publicly available data exemption could meaningfully constrain indiscriminate data scraping and training on as much data as possible.
Similarly, passing privacy laws, or amending privacy laws to require substantive data minimization could really help reduce this sort of Wild West of vacuuming as much data as possible. So, back to your question. It's a tool in the toolkit, but we also need to attend to the laws that we have in place, both in terms of enforcing them as they're written, but also thinking about how we might expand substantive red lines.
I think one of the things that I'm concerned about as a scholar and as a practitioner is when it comes to privacy and data protection law, we put a lot on the individual. You need to submit this data subject request. And even as a privacy lawyer, there are a ton of different AI companies that might have trained on my data that I don't know about. And now I'm facing, you know, an army of a thousand different companies that I need to now file a thousand different deletion requests.
And that really is, to me, a problematic asymmetry that could be better addressed by, you know, substantive red lines and privacy laws that sort of stop these practices in the first place, rather than requiring individuals to sort of police the universe of technology companies and AI first companies that are that are growing.
So, machine learning has a variety of limitations. So, to your point, still not a silver bullet, but let's maybe dive into what those limitations look like. As we previewed at the top, there are key decisions between cost and scale. The cost of exact unlearning or structural removal methods is quite high, and we're unlikely to see organizations do that affirmatively.
Well potentially, and we have seen this happen in pretty severe regulatory actions, but it remains to be seen whether the political will exists for state attorneys general or data protection authorities to sort of take AI companies to task and require them to delete.
And so, when we go down to the other methods, proximate removal and output suppression, again, we're making some trade-offs in terms of scalability, but also whether we're honoring the normative principles and privacy data and protection law that require full deletion or things like correction.
But there are also some other limitations for machine unlearning. The one that we talked about earlier was the algorithmic shadow that even if we engage in machine unlearning, there is still latent influence of that personal data across the model that could function to shape particular outputs or shape particular decisions that a particular AI model may be making.
And then the last, and this is a particularly perverse consequence of machine unlearning, and particularly in the sort of structural removal method space, is what we would call a membership inference attack. And here, effectively, if you have an unlearned model, you're effectively able to figure out, well. Who's been removed from that model, which in some ways is your perverse way of figuring out who's in training data, even if it's been removed.
In the same way that you know if your entire family has provided DNA to a genetic company to sort of track their ancestry, you can really figure out who has not submitted that data, which can be particularly important. And so when we think about limitations, there's questions of cost, there's questions of latent data that can't be observed and potentially removed, and then there's some of the perverse implications of what potential privacy risks do we create by engaging the unlearning in the first place.
But setting these aside, or even considering them in relation to utility, any privacy preserving mechanism has a trade off, and simply because there are trade-offs doesn't mean that we again, throw our hands up and just say it's technically impossible.
And this would be my call to practitioners and to researchers, which is to dive into the space. Are there better ways of doing this work, of removing data? Because it's in the best interest of organizations to have those pathways, and it's in the best interest of principles of privacy and data protection law that organizations are able to remove, or at minimum, suppress personal data where an individual or an enforcer has requested to do so,
AKB: That call to action is a great note to end on. Thank you so much. Thank you for being here. And by the way, I wanted to mention that Jevan was the student body president when he was at the law school.
JH: Full Circle. Full Circle moment from student body president to an incoming professor. It's certainly a dream come true.
AKB: Thanks so much for being here. We appreciate your telling us about this paper that you co-authored: “Forget Me Not. Machine Unlearning's Implications for Privacy Law.” The link will be in the show notes. Thank you so much for joining us here at Discovery.
JH: Thank you so much.
RSS feed
Apple iTunes
Spotify
Amazon Music