Whether you are managing a social media platform or an e-commerce site, minimizing harmful content is critical to the user experience. Such harmful content can include everything from posts promoting violence to child abuse. In fact, the range and scope of potential harmful content has proven too broad for human moderators to comprehensively review. AI systems, designed to mirror the way humans think and process information, may be able to improve the speed and accuracy of this process. AI technology can take large data sets and teach machines to identify patterns or make predictions about certain inputs. Ultimately, this capability allows computers to recognize and filter certain words or images with more efficiency than humans can process this information. As an added benefit, this reduces or could potentially eliminate the need for human moderators to be directly exposed to harmful content.
While AI systems are promising, they are not without their own set of challenges. By some estimates, there are 2.5 quintillion bytes of data created each day. As such, while AI offers a way to more efficiently process large amounts of data, the volume of content at issue is now so vast that it has become critical that AI models perform with both speed and accuracy. And achieving optimal accuracy requires an AI model to not only be based on accurate data and imagery but also be able to appreciate nuances in the content reviewed to distinguish satire from disinformation. Further, questions have been raised regarding whether these models remove the inevitable biases of human content moderators, or if the AI models themselves actually introduce, entrench or amplify biases against certain types of users. One study, for example, found that AI models trained to process hate speech online were 1.5 times more likely to identify tweets as offensive or hateful when written by African-American users.
This tension demonstrates the difficult balance between designing models to address human inefficiencies and root out human error in content moderation while ensuring new systematic issues are not introduced into the models themselves. In fact, U.S. policymakers have conducted numerous hearings and floated legislative proposals to address concerns regarding bias within AI systems and the unintentional discrimination that could result from using such systems.
AI systems undeniably offer online platforms enhanced capabilities to effectively moderate user-generated content, but they present their own set of challenges that must be considered as these systems are designed and deployed as moderation tools.
RELATED ARTICLES
Facial Recognition, Racial Recognition and the Clear and Present Issues with AI Bias
Digitalized Discrimination: COVID-19 and the Impact of Bias in Artificial Intelligence