Oversight Board Selects a Case Involving a Cartoon Depicting Iran’s Supreme Leader Ayatollah Ali Khamenei

Meta should err on the side of issuing scaled allowances where (i) this is not likely to lead to violence; (ii) when potentially violating content is used in protest contexts; and (iii) where public interest is high. Meta should ensure that their internal process to identify and review content trends around protests that may require context-specific guidance to mitigate harm to freedom of expression, such as allowances or exemptions, are effective. The Board will consider this recommendation implemented when Meta shares the internal process with the Board and demonstrates through sharing data with the Board that it has minimized incorrect removals of protest slogans.

Our commitment: We aim to scale allowances in instances where content is not likely to lead to violence, when potentially violating content is used in protest contexts, and where public interest is high. However, we are continuing to refine what data we can commit to sharing.

Considerations: Our approach to newsworthy content takes context into account when weighing the public interest value of the content against the risk of harm. One of the examples in our Transparency Center is a newsworthy allowance we granted in connection with protests in Colombia. The protest context is directly relevant to the content’s public interest value, while the likelihood content may contribute to a risk of violence relates to the risk of harm.

To identify and track protest-related content trends that may pose on-platform risk, our teams deploy a wide-range of detection initiatives including automation tools like content classifiers and pipelines. Human reviewers with language and regional expertise work with policy subject matter experts to provide assessments on this content and take appropriate action in accordance with our Community Standards and identify areas for further mitigation or remediation. We also work closely with local teams to identify and protect at-risk users, including civic actors, that may be more likely to be targeted with abuse during protests or whose content may warrant additional consideration for policy allowances or exemptions.

Where possible, we also consider scaling these allowances so that they apply as broadly as appropriate under the circumstances. Criteria for scaling include considerations of the potential for harm weighed against the public interest value of the content. With these considerations top of mind, we may allow a newsworthy allowance at scale to apply more broadly to content across Facebook in certain instances. As explained in our response to recommendation #6, below, we will add further details about our approach to “narrow” and “scaled” newsworthy allowances to our Transparency Center.

Pending changes to the Violence and Incitement policy, Meta should issue guidance to its reviewers that "marg bar Khamenei" statements in the context of protests in Iran do not violate the Violence and Incitement Community Standard. Meta should reverse any strikes and feature limits for wrongfully removed content that used the "marg bar Khamenei" slogan. The Board will consider this recommendation implemented when Meta discloses data on the volume of content restored and number of accounts impacted.

Our commitment: As stated in our January 23rd response, we have updated our guidance to allow the phrase “marg bar Khamenei” in the context of ongoing protests in Iran. We have also run sweeps to identify and reverse strikes made on the basis of this type of content, and will continue to do so where possible.

Considerations: We appreciate the Oversight Board’s input on this case and the nuanced issue of moderating content that uses the phrase “marg bar Khamenei” (Death to Khamenei). While we’ve always understood that calls for death of heads of state could be rhetorical in certain contexts, given our limited ability to know someone’s intent, it’s difficult for companies like ours to determine when and where to allow these types of statements on our platforms. We accept the Board’s guidance that in Iran this slogan is an integral part of political speech during the ongoing protests, and that it alone is unlikely to pose risk of physical harm.

That’s why, on January 23, 2023, we fully implemented the Board’s recommendation to allow the phrase “marg bar Khamenei” in the context of the ongoing protests in Iran. We have also run sweeps to identify previous enforcement actions and reverse strikes made on the basis of this type of content and will continue to pursue further reversals where they are applicable and feasible. At this time, we are still assessing capacity for that work and determining which metrics we will be able to share. We will provide an update on our progress in a future Quarterly Update.

Meta should revise the indicators that it uses to rank appeals in its review queues and to automatically close appeals without review. The appeals prioritization formula should include, as it does for the cross-check ranker, the factors of topic sensitivity and false-positive probability. The Board will consider this implemented when Meta shares with the Board their appeals prioritization formula and data that shows that it is ensuring review of appeals against the incorrect removal of political expression in protest contexts.

Our commitment: Our internal optimization and appeals experiences teams have launched the first iteration of a solution for the ranking and automation of content takedown appeals jobs. This work will be implemented in a phased approach beginning with auto-restoring content takedown appeals where we have a high confidence, per a designated score, that an enforcement error has occurred. Content that receives a lower score will be reviewed in order of the ranking. We will continue to provide updates of the rollout in future Quarterly Updates.

Considerations: We continue working to understand the tradeoffs of prioritizing certain appeals over others. We have further developed this work since our commitments in the Armenian people and the Armenian Genocide case and the Post Requesting Advice on Pharmaceutical Drugs case. We have now determined integrity metrics that allow us to confidently determine the likelihood of false positives and quickly restore the highest ranking content where it has been erroneously removed.

We will initially rank content takedown appeals against various metrics and restore content where we have high confidence that the takedown was incorrect. The appeals ranking process takes into account a variety of metrics; as this work is evolving we will continue to refine that process.

We aim to further refine the appeal process throughout 2023. We will continue to share updates on implementation and expansion of our efforts in future Quarterly Updates, including whether it is feasible to share confidential data with the board to verify implementation.

Meta should announce all scaled allowances that it issues, their duration and notice of their expiry, in order to give people who use its platforms notice of policy changes allowing certain expression, alongside comprehensive data on the number of "scaled" and "narrow" allowances granted. The Board will consider this recommendation implemented when Meta demonstrates regular and comprehensive disclosures to the Board.

Our commitment: We will provide data on the number of total scaled allowances on a yearly basis.

Considerations: Due in part to previous recommendations from the Oversight Board, we collect data and information to track the number of newsworthiness allowances we issue. As described in recommendation #6, we will provide further clarity about the types of newsworthy allowances, which include both “narrow” and “scaled” approaches. We will also share the total number of scaled allowances with the board. We continue to mature our efforts in this regard and have committed to improving the scope and accuracy of our newsworthiness allowance tracking across regions and violation areas. As part of our commitment for this recommendation, we will work to provide updates on the total number of scaled allowances on a yearly basis. At this time, we do not plan to share the duration and notice of an allowance’s expiration; however, this is something that we may explore collecting and sharing in the future.

The public explanation of the newsworthiness allowance in the Transparency Centre should (i) explain that newsworthiness allowances can either be scaled or narrow; and (ii) provide the criteria that Meta uses to determine when to scale newsworthiness allowances. The Board will consider this recommendation to be implemented when Meta updates the publicly available explanation of newsworthiness and issues Transparency Reports that include sufficiently detailed information about all applied allowances.

Our commitment: We will share additional details about scaled as well as narrow approaches in our Approach to Newsworthiness page in the Transparency Center.

Considerations: We will add further details about our approach to “narrow” and “scaled” newsworthy allowances to our Transparency Center. Currently, “narrow” newsworthy allowances apply to a single piece of content, while "scaled" newsworthy allowances apply more broadly. For instance, if we see a phrase being used in a way that meets our newsworthiness criteria, such as a protest slogan, we may allow it. We recognize that it may be useful to include these details in our existing article and plan to make these changes before the end of the year.

Meta should provide a public explanation of the automatic prioritization and closure of appeals, including the criteria for both prioritization and closure. The Board will consider this recommendation implemented when Meta publishes this information in the Transparency Centre.

Our commitment: Our progress in automatic prioritization and closure of appeals is newly developed and quickly transforming. Given the nature of this work, we believe that providing ongoing updates of our implementation effort will suffice as the criteria involved are evolving. Building, testing and strengthening automatic prioritization and closure of appeals remains our priority, and we will continue to report on the implementation progress as the criteria matures.

Considerations: As shared in our response to recommendation #4, we will further refine the appeal process throughout 2023. We plan to expand this solution to reporter appeals for simple objects, complex objects and demotions appeals in subsequent rollouts.

Our content review prioritization processes are publicly available on the Transparency Center where we explain that we primarily consider severity, virality and likelihood of violation in determining which content our human review teams should review first. This framework is embedded in our automatic prioritization framework.

Since Q1 2022, we have undergone a multi-stage process to identify key drivers of trust in appeals in order to improve their overall effectiveness. In our Q4 2022 Quarterly Update to the Oversight Board, we reported that we have launched the first iteration of a new appeals prioritization system, which ranks appeals based on potential impact of an enforcement error. Further development of solutions targeted at appeals ranking based on the severity of enforcement decisions and/or a specific policy exception remains a long term priority for our teams. As this dynamic process continues, we will surface new insights on how to effectively prioritize appeals and the resultant implications on our processes, accuracy and fairness. With this in mind, we will continue to iterate on our appeals ranking processes and embedded criteria through rigorous testing and prioritization. At this time, we are not planning on publicly disclosing a criteria for both prioritization and closure, as doing so would be premature. However, once these new processes have reached maturity, we will reassess the best way to increase transparency around the new system.