Bản viết lại dựa trên nguồn: https://news.mit.edu/2026/new-method-keeps-kids-safe-from-illegal-ai-generated-content-0713
New method aims to keep kids safe from illegal AI-generated content | MIT News | Massachusetts Institute of Technology Skip to content ↓ Massachusetts Institute of Technology MIT Top Menu↓ Education Research Innovation Admissions + Aid Campus Life News Alumni About MIT More ↓ Search MIT Search websites, locations, and people See More Results Suggestions or feedback? MIT News | Massachusetts Institute of Technology – On Campus and Around the world Subscribe to MIT News newsletter Browse Enter keywords to search for news articles: Submit Browse By Topics View All → Explore: Machine learning Sustainability Startups Black holes Classes and programs Departments View All → Explore: Aeronautics and Astronautics Brain and Cognitive Sciences Architecture Political Science Mechanical Engineering Centers, Labs, & Programs View All → Explore: Abdul Latif Jameel Poverty Action Lab (J-PAL) Picower Institute for Learning and Memory Media Lab Lincoln Laboratory Schools School of Architecture + Planning School of Engineering School of Humanities, Arts, and Social Sciences Sloan School of Management School of Science MIT Schwarzman College of Computing View all news coverage of MIT in the media → Listen to audio content from MIT News → Subscribe to MIT newsletter → Close Breadcrumb MIT News New method aims to keep kids safe from illegal AI-generated content New method aims to keep kids safe from illegal AI-generated content Researchers developed an auditing technique to test generative AI models for malicious capabilities, without prompting them for illegal outputs. Adam Zewe | MIT News Publication Date: July 13, 2026 Press Inquiries Press Contact: Abby Abazorius Email: [email protected] Phone: 617-253-2709 MIT News Office Media Download ↓ Download Image Caption: Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content. Credits: Credit: Christine Daniloff, MIT; iStock *Terms of Use: Images for download on the MIT News office website are made available to non-commercial entities, press and the general public under a Creative Commons Attribution Non-Commercial No Derivatives license. You may not alter the images provided, other than to crop them to size. A credit line must be used when reproducing images; if one is not provided below, credit the images to "MIT." Close Caption: Researchers developed an evaluation procedure that tests generative AI models for harmful capabilities without generating outputs. This could enable auditors to identify open-source models that have been adapted to produce illegal content. Credits: Credit: Christine Daniloff, MIT; iStock Previous image Next image With the exploding popularity of generative artificial intelligence, many open-source models are now available online for anyone to adapt for their task, such as generating product renderings in a certain artistic style. But these models also find their way into the hands of nefarious actors who may optimize them to produce illegal content, like hate speech or child sexual abuse material (CSAM). This is a growing problem — the National Center for Missing and Exploited Children received more than 1.5 million reports of AI-generated CSAM in 2025, an increase from 67,000 in 2024.Engineers usually test AI for harmful capabilities by prompting the model and inspecting its outputs, but this is impossible for CSAM, since it is illegal in the U.S to generate such content, regardless of intent.To avoid this dilemma and improve AI safety, Associate Professor Ashia Wilson and her graduate student, Vinith Suriyakumar, teamed up with researchers from MIT’s Healthy ML Lab, led by Marzyeh Ghassemi, and child safety non-profit Thorn to develop a new auditing approach that determines whether a model can produce CSAM without prompting it. Thorn is a child safety nonprofit whose mission is to transform how children are protected from sexual abuse and exploitation in the digital age.Their technique examines how the inner workings of a model have been adapted, but it never generates an output. By examining hidden representations, it can reliably infer whether a model has been specialized to produce harmful imagery.When tested, the auditing procedure identified model variations that had been specialized to generate CSAM with 100 percent accuracy. A hosting platform could use this technique to flag unsafe models and quickly remove them or prevent them from being uploaded in the first place.“This unlocks a new avenue for platforms that host open-source models and for law enforcement to actually test whether a model is capable of generating CSAM. Before, we had no way of measuring this. It was a huge blind spot that some people were taking advantage of. Now, we can address an AI safety problem that is having severe negative impacts,” says Vinith Suriyakumar, an MIT electrical engineering and computer science (EECS) graduate student and lead author of a paper on this technique.Suriyakamur and Wilson, the Lister Brothers Career Develop Professor in EECS, a principal investigator in the Laboratory for Information and Decision Systems (LIDS), and senior author, are joined on the paper by Lena Stempfle, an MIT postdoc; Ghassemi, an associate professor in EECS and a member of the Institute of Medical Engineering Sciences (IMES) and LIDS; and others at Boston University and Thorn. The paper was be presented as a spotlight at the “Trustworthy AI for Good” workshop at the International Conference on Machine Learning.Auditing adaptationsRecent techniques have made it easier for users to specialize a generative AI model for their task through a process known as fine-tuning. Rather than retraining the entire model on a task-specific dataset, individuals can utilize an algorithm called low-rank adaptation (LoRA) to specialize the model in a more efficient manner.This has led to a wave of new generative AI model variants for a variety of purposes, like producing watercolor images that mimic an artistic movement. But it has also enabled malicious actors to create models that can generate high-quality CSAM and other harmful imagery.To audit a model, engineers typically prompt it for harmful content and check its outputs, but this manual auditing procedure is not scalable. In addition, repeatedly generating heinous images can have negative psychological impacts on human evaluators. This evaluation method quickly falls apart when testing CSAM, which is illegal to generate for any purpose in the U.S. and many other international jurisdictions.“We are in this very difficult situation where, based on the law itself, we cannot use the de facto means of evaluation. We had to throw out the entire toolkit and take a different approach,” Suriyakumar says.After learning about this conundrum, the researchers joined forces with Thorn, to address this issue.A nongenerative solutionInstead of focusing on outputs, the researchers targeted the modifications a LoRA algorithm makes during fine-tuning. Their technique probes these modifications, called LoRA adaptors, to determine whether a model has been specialized for a harmful capability, without generating an output.Using a technique called Gaussian probing, the researchers feed the model a set of random data points and analyze how it manipulates those data within its multilayer internal structure. “We never run the model all the way to the end or prompt the model, so we never generate images,” Suriyakumar explains.The researchers capture those modifications at multiple time points within the model’s inner structure and average them to summarize how the LoRA adaptor changed the model’s computation. They found these responses to be a strong signal of how a model had been specialized.They tested their method on variations of three types of models, comparing the results to ground-truth data from LoRA adaptors known for generating CSAM, other harmful images, and safe content. Their method was 100 percent accurate in identifying models that had been adapted to generate CSAM. “There is a huge bucket of child safety concerns with AI, and these are real concerns that need to be addressed. A lot of children are being harmed by AI deepfakes. We’ve shown that Gaussian probing can be a very useful tool, and we hope the research community really pours more attention into this problem,” Wilson says.Importantly, their technique is scalable and would be relatively inexpensive to implement. Since thousands of model variations are published online every month, scalability is key to help auditors remove harmful adaptations before they are widely distributed.Gaussian probing is also more robust than some other auditing techniques, since a nefarious actor would need to carefully alter the inner workings of the base model to avoid detection.In the future, the researchers want to evaluate their technique on a larger set of model variations and explore whether Gaussian probing can detect harmful capabilities in base models before they are adapted.“Now we have a technological approach to partially address this concern. So much effort was poured into this collaboration, which enabled us to tackle a really hard problem that is harming so many children, nationally and around the world. Hopefully, we can have a transformative impact in this area,” Ghassemi says.This work was supported, in part, by the Bridgewater AIA Labs Research Fellowship. Share this news article on: X Facebook LinkedIn Reddit Print Paper Paper: “Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM” Related Links Vinith SuriyakumarMarzyeh GhassemiAshia WilsonLaboratory for Information and Decision SystemsInstitute for Medical Engineering and ScienceDepartment of Electrical Engineering and Computer ScienceSchool of EngineeringMIT Schwarzman College of Computing Related Topics Research Computer science and technology Artificial intelligence Machine learning Algorithms Human-computer interaction Ethics Public health Privacy Laboratory for Information and Decision Systems (LIDS) Institute for Medical Engineering and Science (IMES) School of Engineering MIT Schwarzman College of Computing Related Articles Personalization features can make LLMs more agreeable Study: AI could lead to inconsistent outcomes in home surveillance LLMs factor in unrelated information when recommending medical treatments Study shows vision-language models can’t handle queries with negation words Previous item Next item More MIT News The importance of indoor airflow patterns in spreading airborne disease Researchers explore how a key factor can mitigate or promote tuberculosis transmission. Read full story → Paving the way for greener ammonia production New MIT research could lead to better materials for a fossil-fuel-free process for making the chemical that's essential to fertilizer and other products. Read full story → MIT engineers design a better controller for operating construction diggers The new, more intuitive system could speed up the training process for excavator operators. Read full story → The game’s the thing MIT senior Kayode Dada has a passion for designing games, and a penchant for building community. Read full story → How the Toxoplasma parasite adapts to crowded conditions in host cells A new study by Whitehead Institute researchers identifies how the widespread parasite Toxoplasma gondii survives in crowded environments in infected cells, which helps it persist in long-lasting brain cysts. Read full story → Creating innovations that scale across borders A new book coauthored by Professor Amos Winter asserts that, to solve major challenges and deliver products of value to consumers in both developed and emerging markets, innovators must think and act globally. Read full story → More news on MIT News homepage → More about MIT News at Massachusetts Institute of Technology This website is managed by the MIT News Office, part of the Institute Office of Communications. News by Schools/College: School of Architecture and Planning School of Engineering School of Humanities, Arts, and Social Sciences MIT Sloan School of Management School of Science MIT Schwarzman College of Computing Resources: About the MIT News Office MIT News Press Center Terms of Use Press Inquiries Filming Guidelines RSS Feeds Tools: Subscribe to MIT Daily/Weekly Subscribe to press releases Submit campus news Guidelines for campus news contributors Guidelines on generative AI Massachusetts Institute of Technology MIT Top Level Links: Education Research Innovation Admissions + Aid Campus Life News Alumni About MIT Join us in building a better world. Massachusetts Institute of Technology77 Massachusetts Avenue, Cambridge, MA, USA Recommended Links: Visit Map (opens in new window) Events (opens in new window) People (opens in new window) Careers (opens in new window) Contact Privacy Accessibility Social Media Hub MIT on X MIT on Facebook MIT on YouTube MIT on Instagram