Skip to main content

Yahoo is open sourcing its deep learning model to identify pornography

Yahoo
Image used with permission by copyright holder
Looking to keep content that is not safe for work off your work screen? Artificial intelligence may be able to help you do that. On Friday, Yahoo research engineer Jay Mahadeokar and senior director of product management Gerry Pesavento published a blog post announcing the release of the company’s “deep learning model that will allow developers to experiment with a classifier for NSFW detection, and provide feedback to us on ways to improve the classifier.” In essence, Yahoo is open sourcing its algorithms for detecting pornographic images.

“Automatically identifying that an image is not suitable/safe for work (NSFW), including offensive and adult images, is an important problem which researchers have been trying to tackle for decades,” the Yahoo team wrote on Friday. “With the evolution of computer vision, improved training data, and deep learning algorithms, computers are now able to automatically classify NSFW image content with greater precision.”

That said, an open source model or algorithm for identifying NSFW images doesn’t currently exist, Yahoo pointed out. As such, “in the spirit of collaboration and with the hope of advancing this endeavor,” the company is filling the gap, providing a machine learning tool that focuses on solely identifying pornographic images. The reason for this specificity, Yahoo explained, is that “what may be objectionable in one context can be suitable in another.” But for these purposes, porn is decidedly unsuitable.

Yahoo’s system uses deep learning to assign an image a score between 0 and 1 to determine just how NSFW it really is. “Developers can use this score to filter images below a certain suitable threshold based on a ROC curve for specific use-cases, or use this signal to rank images in search results,” Yahoo said. But bear in mind that there’s no guarantee of accuracy here — really, that’s where you come in. “This model is a general purpose reference model, which can be used for the preliminary filtering of pornographic images,” the blog post concluded. “We do not provide guarantees of accuracy of output, rather, we make this available for developers to explore and enhance as an open source project.”

Lulu Chang
Former Digital Trends Contributor
Fascinated by the effects of technology on human interaction, Lulu believes that if her parents can use your new app…
A dangerous new jailbreak for AI chatbots was just discovered
the side of a Microsoft building

Microsoft has released more details about a troubling new generative AI jailbreak technique it has discovered, called "Skeleton Key." Using this prompt injection method, malicious users can effectively bypass a chatbot's safety guardrails, the security features that keeps ChatGPT from going full Taye.

Skeleton Key is an example of a prompt injection or prompt engineering attack. It's a multi-turn strategy designed to essentially convince an AI model to ignore its ingrained safety guardrails, "[causing] the system to violate its operators’ policies, make decisions unduly influenced by a user, or execute malicious instructions," Mark Russinovich, CTO of Microsoft Azure, wrote in the announcement.

Read more