Content moderation rules used to be a question of taste. Now, they can determine a service’s prospects for survival.
6 min
Every public communication platform you can name—from Facebook, Twitter and YouTube to Parler, Pinterest and Discord—is wrestling with the same two questions:
How do we make sure we’re not facilitating misinformation, violence, fraud or hate speech?
At the same time, how do we ensure we’re not censoring users?
The more they moderate content, the more criticism they experience from those who think they’re over-moderating. At the same time, any statement on a fresh round of moderation provokes some to point out objectionable content that remains. Like any question of editorial or legal judgment, the results are guaranteed to displease someone, somewhere—including Congress, which this week called the chief executives of Facebook, Google and Twitter to a hearing on March 25 to discuss misinformation on their platforms.
For many services, this has gone beyond a matter of user experience, or growth rates, or even ad revenue. It’s become an existential crisis. While dialling up moderation won’t solve all of a platform’s problems, a look at the current winners and losers suggests that not moderating enough is a recipe for extinction.
Facebook is currently wrestling with whether it will continue its ban of former president Donald Trump. Pew Research says 78% of Republicans opposed the ban, which has contributed to the view of many in Congress that Facebook’s censorship of conservative speech justifies breaking up the company—something a decade of privacy scandals couldn’t do.
Parler, a haven for right-wing users who feel alienated by mainstream social media, was taken down by its cloud service provider, Amazon Web Services, after some of its users live-streamed the riot at the U.S. Capitol on Jan. 6. Amazon cited Parler’s apparent inability to police content that incites violence. While Parler is back online with a new service provider, it’s unclear if it has the infrastructure to serve a large audience.
During the weeks Parler was offline, the company implemented algorithmic filtering for a few content types, including threats and incitement, says a company spokesman. The company also has an automatic filter for “trolling” that detects such content, but it’s up to users whether to turn it on or not. In addition, those who choose to troll on Parler are not penalized in Parler’s algorithms for doing so, “in the spirit of First Amendment,” says the company’s guidelines for enforcement of its content moderation policies. Parler recently fired its CEO, who said he experienced resistance to his vision for the service, including how it should be moderated.
Now, just about every site that hosts user-generated content is carefully weighing the costs and benefits of updating their content moderation systems, using a mix of human professionals, algorithms and users. Some are even building rules into their services to pre-empt the need for increasingly costly moderation.
The saga of gaming-focused messaging app Discord is instructive: In 2018, the service, which is aimed at children and young adults, was one of those used to plan the Charlottesville riots. A year later, the site was still taking what appeared to be a deliberately laissez-faire approach to content moderation.
By this January, however, spurred by reports of hate speech and lurking child predators, Discord had done a complete 180. It now has a team of machine-learning engineers building systems to scan the service for unacceptable uses, and has assigned 15% of its overall staff to trust and safety issues.
This newfound attention to content moderation helped keep Discord away from the controversy surrounding the Capitol riot, and caused it to briefly ban a chat group associated with WallStreetBets during the GameStop stock runup. Discord’s valuation doubled to $7 billion over roughly the same period, a validation that investors have confidence in its moderation strategy.
The challenge successful platforms face is moderating content “at scale,” across millions or billions of pieces of shared content.
Before any action can be taken, services must decide what should be taken down, an often slow and deliberative process.
Imagine, for example, that a grass-roots movement gains momentum in a country, and begins espousing extreme and potentially dangerous ideas on social media. While some language might be caught by algorithms immediately, a decision about whether discussion of a particular movement, like QAnon, should be banned completely, could take months on a service such as YouTube, says a Google spokesman.
One reason it can take so long is the global nature of these platforms. Google’s policy team might consult with experts in order to consider regional sensitivities before making a decision. After a policy decision is made, the platform has to train AI and write rules for human moderators to enforce it—then make sure both are carrying out the policies as intended, he adds.
While AI systems can be trained to catch individual pieces of problematic content, they’re often blind to the broader meaning of a body of posts, says Tracy Chou, founder of content-moderation startup Block Party and former tech lead at Pinterest.
Take the case of the “Stop the Steal” protest, which led to the deadly attack on the U.S. Capitol. Individual messages used to plan the attack, like “Let’s meet at location X,” would probably look innocent to a machine-learning system, says Ms Chou, but “the context is what’s key.” Facebook banned all content mentioning “Stop the Steal” after the riot.
Even after Facebook has identified a particular type of content as harmful, why does it seem constitutionally unable to keep it off its platform?
It’s the “prevalence problem.” On a truly gigantic service, even if only a tiny fraction of content is problematic, it can still reach millions of people. Facebook has started publishing a quarterly report on its community standards enforcement. During the last quarter of 2020, Facebook says users saw seven or eight pieces of hate speech out of every 10,000 views of content. That’s down from 10 or 11 pieces the previous quarter. The company said it will begin allowing third-party audits of these claims this year.
While Facebook has been leaning heavily on AI to moderate content, especially during the pandemic, it currently has about 15,000 human moderators. And since every new moderator comes with a fixed additional cost, the company has been seeking more efficient ways for its AI and existing humans to work together.
In the past, human moderators reviewed content flagged by machine learning algorithms in more or less chronological order. Content is now sorted by a number of factors, including how quickly it’s spreading on the site, says a Facebook spokesman. If the goal is to reduce the number of times people see harmful content, the most viral stuff should be top priority.
Companies that aren’t Facebook or Google often lack the resources to field their own teams of moderators and machine-learning engineers. They have to consider what’s within their budget, which includes outsourcing the technical parts of content moderation to companies such as San Francisco-based startup Spectrum Labs.
Through its cloud-based service, Spectrum Labs shares insights it gathers from any one of its clients with all of them—which include Pinterest and Riot Games, maker of League of Legends—in order to filter everything from bad words and human trafficking to hate speech and harassment, says CEO Justin Davis.
Mr Davis says Spectrum Labs doesn’t say what clients should and shouldn’t ban. Beyond illegal content, every company decides for itself what it deems acceptable, he adds.
Pinterest, for example, has a mission rooted in “inspiration,” and this helps it take a clear stance in prohibiting harmful or objectionable content that violates its policies and doesn’t fit its mission, says a company spokeswoman.
Services are also attempting to reduce the content-moderation load by reducing the incentives or opportunity for bad behaviour. Pinterest, for example, has from its earliest days minimized the size and significance of comments, says Ms Chou, the former Pinterest engineer, in part by putting them in a smaller typeface and making them harder to find. This made comments less appealing to trolls and spammers, she adds.
The dating app Bumble only allows women to reach out to men. Flipping the script of a typical dating app has arguably made Bumble more welcoming for women, says Mr Davis, of Spectrum Labs. Bumble has other features designed to pre-emptively reduce or eliminate harassment, says Chief Product Officer Miles Norris, including a “super block” feature that builds a comprehensive digital dossier on banned users. This means that if, for example, banned users attempt to create a new account with a fresh email address, they can be detected and blocked based on other identifying features.
Facebook CEO Mark Zuckerberg recently described Facebook as something between a newspaper and a telecommunications company. For it to continue being a global town square, it doesn’t have the luxury of narrowly defining the kinds of content and interactions it will allow. For its toughest content moderation decisions, it has created a higher power—a financially independent “oversight board” that includes a retired U.S. federal judge, a former prime minister of Denmark and a Nobel Peace Prize laureate.
In its first decision, the board overturned four of the five bans Facebook brought before it.
Facebook has said that it intends the decisions made by its “supreme court of content” to become part of how it makes everyday decisions about what to allow on the site. That is, even though the board will make only a handful of decisions a year, these rulings will also apply when the same content is shared in a similar way. Even with that mechanism in place, it’s hard to imagine the board can get to more than a tiny fraction of the types of situations content moderators and their AI assistants must decide every day.
But the oversight board might accomplish the goal of shifting the blame for Facebook’s most momentous moderation decisions. For example, if the board rules to reinstate the account of former President Trump, Facebook could deflect criticism of the decision by noting it was made independent of its own company politics.
Meanwhile, Parler is back up, but it’s still banned from the Apple and Google app stores. Without those essential routes to users—and without web services as reliable as its former provider, Amazon—it seems unlikely that Parler can grow anywhere close to the rate it otherwise might have. It’s not clear yet whether Parler’s new content filtering algorithms will satisfy Google and Apple. How the company balances its enhanced moderation with its stated mission of being a “viewpoint neutral” service will determine whether it grows to be a viable alternative to Twitter and Facebook or remains a shadow of what it could be with such moderation.
Copyright 2020, Dow Jones & Company, Inc. All Rights Reserved Worldwide. LEARN MORE
What a quarter-million dollars gets you in the western capital.
Alexandre de Betak and his wife are focusing on their most personal project yet.
No trip to Singapore is complete without a meal (or 12) at its hawker centers, where stalls sell multicultural dishes from generations-old recipes. But rising costs and demographic change are threatening the beloved tradition.
6 min
In Singapore, it’s not unusual for total strangers to ask, “Have you eaten yet?” A greeting akin to “Good morning,” it invariably leads to follow-up questions. What did you eat? Where did you eat it? Was it good? Greeters reserve the right to judge your responses and offer advice, solicited or otherwise, on where you should eat next.
Locals will often joke that gastronomic opinions can make (and break) relationships and that eating is a national pastime. And why wouldn’t it be? In a nexus of colliding cultures—a place where Malays, Indians, Chinese and Europeans have brushed shoulders and shared meals for centuries—the mix of flavours coming out of kitchens in this country is enough to make you believe in world peace.
While Michelin stars spangle Singapore’s restaurant scene , to truly understand the city’s relationship with food, you have to venture to the hawker centres. A core aspect of daily life, hawker centres sprang up in numbers during the 1970s, built by authorities looking to sanitise and formalise the city’s street-food scene. Today, 121 government-run hawker centres feature food stalls that specialise in dishes from the country’s various ethnic groups. In one of the world’s most expensive cities, hawker dishes are shockingly cheap: A full meal can cost as little as $3.
Over the course of many visits to Singapore, I’ve fallen in love with these places—and with the scavenger hunts to find meals I’ll never forget: delicate bowls of laksa noodle soup, where brisk lashes of heat interrupt addictive swirls of umami; impossibly flaky roti prata dipped in curry; the beautiful simplicity of an immaculately roasted duck leg. In a futuristic and at times sterile city, hawker centres throw back to the past and offer a rare glimpse of something human in scale. To an outsider like me, sitting at a table amid the din of the lunch-hour rush can feel like glimpsing the city’s soul through all the concrete and glitz.
So I’ve been alarmed in recent years to hear about the supposed demise of hawker centres. Would-be hawkers have to bid for stalls from the government, and rents are climbing . An upwardly mobile generation doesn’t want to take over from their parents. On a recent trip to Singapore, I enlisted my brother, who lives there, and as we ate our way across the city, we searched for signs of life—and hopefully a peek into what the future holds.
At Amoy Street Food Centre, near the central business district, 32-year-old Kai Jin Thng has done the math. To turn a profit at his stall, Jin’s Noodle , he says, he has to churn out at least 150 $4 bowls of kolo mee , a Malaysian dish featuring savoury pork over a bed of springy noodles, in 120 minutes of lunch service. With his sister as sous-chef, he slings the bowls with frenetic focus.
Thng dropped out of school as a teenager to work in his father’s stall selling wonton mee , a staple noodle dish, and is quick to say no when I ask if he wants his daughter to take over the stall one day.
“The tradition is fading and I believe that in the next 10 or 15 years, it’s only going to get worse,” Thng said. “The new generation prefers to put on their tie and their white collar—nobody really wants to get their hands dirty.”
In 2020, the National Environment Agency , which oversees hawker centres, put the median age of hawkers at 60. When I did encounter younger people like Thng in the trade, I found they persevered out of stubbornness, a desire to innovate on a deep-seated tradition—or some combination of both.
Later that afternoon, looking for a momentary reprieve from Singapore’s crushing humidity, we ducked into Market Street Hawker Centre and bought juice made from fresh calamansi, a small citrus fruit.
Jamilah Beevi, 29, was working the shop with her father, who, at 64, has been a hawker since he was 12. “I originally stepped in out of filial duty,” she said. “But I find it to be really fulfilling work…I see it as a generational shop, so I don’t want to let that die.” When I asked her father when he’d retire, he confidently said he’d hang up his apron next year. “He’s been saying that for many years,” Beevi said, laughing.
More than one Singaporean told me that to truly appreciate what’s at stake in the hawker tradition’s threatened collapse, I’d need to leave the neighbourhoods where most tourists spend their time, and venture to the Heartland, the residential communities outside the central business district. There, hawker centres, often combined with markets, are strategically located near dense housing developments, where they cater to the 77% of Singaporeans who live in government-subsidised apartments.
We ate laksa from a stall at Ghim Moh Market and Food Centre, where families enjoyed their Sunday. At Redhill Food Centre, a similar chorus of chattering voices and clattering cutlery filled the space, as diners lined up for prawn noodles and chicken rice. Near our table, a couple hungrily unwrapped a package of durian, a coveted fruit banned from public transportation and some hotels for its strong aroma. It all seemed like business as usual.
Then we went to Blackgoat . Tucked in a corner of the Jalan Batu housing development, Blackgoat doesn’t look like an average hawker operation. An unusually large staff of six swirled around a stall where Fikri Amin Bin Rohaimi, 24, presided over a fiery grill and a seriously ambitious menu. A veteran of the three-Michelin-star Zén , Rohaimi started selling burgers from his apartment kitchen in 2019, before opening a hawker stall last year. We ordered everything on the menu and enjoyed a feast that would astound had it come out of a fully equipped restaurant kitchen; that it was all made in a 130-square-foot space seemed miraculous.
Mussels swam in a mushroom broth, spiked with Thai basil and chives. Huge, tender tiger prawns were grilled to perfection and smothered in toasted garlic and olive oil. Lamb was coated in a whisper of Sichuan peppercorns; Wagyu beef, in a homemade makrut-lime sauce. Then Ethel Yam, Blackgoat’s pastry chef prepared a date pudding with a mushroom semifreddo and a panna cotta drizzled in chamomile syrup. A group of elderly residents from the nearby towers watched, while sipping tiny glasses of Tiger beer.
Since opening his stall, Rohaimi told me, he’s seen his food referred to as “restaurant-level hawker food,” a categorisation he rejects, feeling it discounts what’s possible at a hawker centre. “If you eat hawker food, you know that it can often be much better than anything at a restaurant.”
He wants to open a restaurant eventually—or, leveraging his in-progress biomedical engineering degree, a food lab. But he sees the modern hawker centre not just as a steppingstone, but a place to experiment. “Because you only have to manage so many things, unlike at a restaurant, a hawker stall right now gives us a kind of limitlessness to try new things,” he said.
Using high-grade Australian beef and employing a whole staff, Rohaimi must charge more than typical hawker stalls, though his food, around $12 per 100 grams of steak, still costs far less than high-end restaurant fare. He’s found that people will pay for quality, he says, even if he first has to convince them to try the food.
At Yishun Park Hawker Centre (now temporarily closed for renovations), Nurl Asyraffie, 33, has encountered a similar dynamic since he started Kerabu by Arang , a stall specialising in “modern Malay food.” The day we came, he was selling ayam percik , a grilled chicken leg smothered in a bewitching turmeric-based marinade. As we ate, a hawker from another stall came over to inquire how much we’d paid. When we said around $10 a plate, she looked skeptical: “At least it’s a lot of food.”
Asyraffie, who opened the stall after a spell in private dining and at big-name restaurants in the region, says he’s used to dubious reactions. “I think the way you get people’s trust is you need to deliver,” he said. “Singapore is a melting pot; we’re used to trying new things, and we will pay for food we think is worth it.” He says a lot of the same older “uncles” who gawked at his prices, are now regulars. “New hawkers like me can fill a gap in the market, slightly higher than your chicken rice, but lower than a restaurant.”
But economics is only half the battle for a new generation of hawkers, says Seng Wun Song, a 64-year-old, semiretired economist who delves into the inner workings of Singapore’s food-and-beverage industry as a hobby. He thinks locals and tourists who come to hawker centers to look for “authentic” Singaporean food need to rethink what that amorphous catchall word really means. What people consider “heritage food,” he explains, is a mix of Malay, Chinese, Indian and European dishes that emerged from the country’s founding. “But Singapore is a trading hub where people come and go, and heritage moves and changes. Hawker food isn’t dying; it’s evolving so that it doesn’t die.”
