How Meta prioritises content for review

UPDATED 12 NOV 2024

Meta's technologies detect and remove the majority of violating content before it's ever reported. When someone posts on Facebook, Instagram, Messenger or Threads, our technologies check to see if the content goes against the Community Standards. In most cases, identification is a simple matter. The post either clearly violates our policies or it doesn't.

Other times, identification is more difficult. Perhaps the sentiment of the post is unclear, its language is particularly complex or its imagery too context-dependent. In these cases, we conduct further review using people.

Prioritising factors for human review

When determining which content our human review teams should review first, we consider three main factors:

SEVERITY

How likely is it that the content could lead to harm, both online and offline?

VIRALITY

How quickly is the content being shared?

LIKELIHOOD OF VIOLATING

How likely is it that the content in question does in fact violate our policies?

As we want to prevent as much harm as possible, our review systems use technology to prioritise high-severity content with the potential for offline harm and viral content, which is spreading quickly.

Human review teams help our technology improve

Our human review teams use their expertise in certain policy areas and locales to make difficult, often nuanced judgement calls. Every time that reviewers make a decision, we use that information to train our technology. Over time, across millions of decisions, our technology gets better, allowing us to remove more violating content.

Helping reviewers make the right calls

Technology helps human review teams do what they do best

Like many machine learning models, our technology improves over time as it receives more examples of violating content. This means that human review teams have been able to focus more on severe, viral, nuanced, novel and complex content – exactly the sort of decisions where people tend to make better decisions than technology.

Common questions about content review prioritisation

How has enforcement prioritisation changed over time?

Previously, human review teams would spend the vast majority of their time reviewing content reported by people. This meant that they were often spending too much time on low-severity or clearly non-violating content and not enough time on the severest content with the greatest potential for harm. It also meant that many human decisions weren't that useful for improving our enforcement technology.

Our current approach to prioritisation addresses these issues, allowing us to review the most potentially harmful content first and improve our technology faster.

Do human review teams review every user report?

Not necessarily. Both human review teams and technology play a role in reviewing user reports. In cases where our technology can analyse a given piece of content, it will automatically take action – or not – on the content in question.

How do you ensure that human decisions don't introduce bias into your technology?

To address fairness and inclusion concerns associated with the deployment of AI in Meta technologies, we created our Responsible AI team – a dedicated, multidisciplinary team of ethicists, social and political scientists, policy experts, artificial intelligence researchers and engineers. The team's overall goal is to develop guidelines, tools and processes to tackle issues of AI responsibility and help ensure that these systemic resources are widely available across Meta.