How moderation works

A person decides

Every removal, suspension, strike, ban and decision on a report or an appeal is made by a person on our team. Automated tools never take those decisions, and never stop the clocks the law sets for us (2 hours for the most urgent reports, 36 hours for others).

What automated tools do

We use two classifiers to read new and changed posts, comments, community descriptions, profiles, Showcase listings and reports: Jev, from TypeSafe, which answers questions we write in the words of our Community Guidelines, and omni-moderation, from OpenAI, which scores text and pictures for common kinds of harm. They help us sort the queue so the most serious things are seen first.

• They may publish a post or comment that was held for review (because the account or its links are new) when every check calls it clean and its author has no active strike.
• They may hold a published post or comment for a person to look at when it is very likely spam, harassment or hate. It is out of sight until a person approves or removes it, and its author sees it as waiting for review.
• They may mark something urgent, so a person looks at it first.

What they read is the content itself and the report about it, never an email address, an IP address or other account details. A report that says a child is at risk is never sent to either service: it goes straight to a person.

The sites names point to

We check every live name’s page every two days, and more often while a name is new or reported. Most checks only compare the page with what we saw last time. When a page has changed, we read it for signs of phishing, scams and malware, and may open it in a walled-off browser on our own server, take screenshots and have them screened.

A page that looks like phishing or malware is put under review and shown to a person. Only Google Safe Browsing listing a page suspends a name on its own; anything else waits for a person.

When a page passes every check, a picture of its first screen may be shown as a preview, for example on its owner’s profile. Owners can turn previews off for any name; the checks run either way.

Reports

Anyone can report a post, a comment, a community, a profile, a listing or a name. Reports go to a person; the automated tools only add their reading of the report beside it. How reports work and what happens next is in the Community Guidelines and on the abuse page.

What we publish

Our transparency report lists, for each period, how many items the automated tools screened, how many held posts they published, how many they held and a person later approved, how often a person agreed with them, and what they cost.