Illustration by Emily Scherer for Forbes; Photos by Getty Images
Click to save this article.
You'll be asked to sign into your Forbes account.
Aug 26, 2024,06:30am EDT
It’s been more than eight years since content farms overseas started “American” fake news pages on Facebook. Their business, now fueled by AI, is still going strong.
By Emily Baker-White, Forbes Staff
The idea of America is big business on Facebook. The social network has hosted more than a hundred pages that have adopted American patriotism as a theme, boasting names like Proud American, Proud To Be An American, American Story, and We Are America.
But a large swath of those pages — despite their names — aren’t American at all. Instead, they’re run by foreign click farmers, many of whom are based in Macedonia, who use AI to pump out a near-endless ocean of clickbaity soup. Posts sharing prayers for American soldiers, rewritten tweets, memes and pictures of old Hollywood pin-up girls link out to AI-generated articles, against which the click farmers can sell advertising. Headlines like “Dedicated Firefighters Risk Their Lives To Save Others” and “A Father’s Heroism: The Tragic Story of Phil Dellegrazie And His Son Anthony” tease short, uninformative articles on websites plastered with often sexual advertisements. The pages promoting them fake Americanness because they get paid every time someone clicks on one of their links, and in the advertising world, American clicks are some of the most valuable.
A Forbes review identified 67 Facebook pages — now taken down — that identified themselves as champions of American news, culture or identity, but were actually based overseas. As of August 20, they had more than 9 million followers combined — more than the Facebook pages of the Wall Street Journal or the Washington Post. Thirty-three of them were run from Macedonia, with others spread out across 23 different countries, including Canada, France, Morocco, Venezuela and Vietnam.
Click farmers, especially those from Macedonia, have a long history on Facebook. During the 2016 presidential election, teenagers in the small Eastern European country pushed fake news to millions of Americans on Facebook, making tens of thousands of dollars in ad revenue. In 2019, similar Eastern European pages ran the same playbook — this time, reaching nearly half of all Americans on the platform.
Now, AI has given those same operations the capacity to produce near-infinite volumes of low-quality (or outright fake) news — and in at least some cases, this AI-produced slop is breaking through. The pages have begun using generic AI-generated imagery (bald eagles, stars and stripes, camo soldiers and the occasional Statue of Liberty) to appeal to American Facebook users — and in at least some cases, it’s working. One post made last week by the Canada-based page American Patriots featured an AI-generated photo of an American soldier and his children, and received more than 100,000 likes and 35,000 comments. The American Patriots page, like most of the others, directed people from Facebook to click farms featuring low-quality articles.
Pages like We Are America, American Patriots and USA Army Is Love post a mix of real and AI-generated photography and memes.
Forbes fed three of the American Patriots articles through an AI text detector called GPT-Zero, which found that they were 79%, 85%, and 100% likely to have been generated by AI. The detector also found that stories linked from We Love America, a page from Spain, and American Story, a Macedonian page, had a 100% likelihood of being generated by AI. (Disclosure: In a previous life, I held content policy positions at Facebook and Spotify.)
“Every platform has incentives … and they provide a window into what is at the heart of Facebook, what makes it tick," said Jeff Allen, co-founder of the Integrity Institute and a former Facebook data scientist who tracked networks of spammy page administrators from the inside. To him, click farmers are a “great magnifying glass … into the more reptilian parts of our brain.”
Meta spokesperson Margarita Franklin told Forbes that all 67 pages violated Meta’s rules on inauthentic behavior, because they misrepresented where they were based; all were taken down. It’s not necessarily a violation of Meta’s rules to make a page about one country while based in another, but the pages cross a line when they deceive people about where they’re from. Franklin said the pages had only been active for a little more than a week when Forbes flagged them.
Franklin also said that while AI does make content generation easier for spammers and scammers, their primary challenge has always been getting eyeballs on their pages, whether they’re made with AI or not. A recent Meta Threat Report found that generative AI has “provide[d] only incremental productivity and content-generation gains” to “threat actors,” because the cost of creating low-quality clickbait articles has always been pretty low.
When Macedonian content farms first became big on Facebook in 2016, they leaned hard into hyper-partisan rage bait focused on divisive issues like immigration, trans rights, race and policing. The theory was simple — write about what people were most likely to engage with. And at the time, posts about those issues often topped the charts of Facebook engagement.
But Facebook’s algorithm has shifted away from politics in the eight years since then. The company began aggressively demoting political posts after the January 6, 2021 capitol riots, which were organized in part on Meta platforms.
Some of the American patriotic pages still featured political topics, with recent posts on topics including critical race theory and trans rights. In the aggregate, though, the pages didn’t focus on politics. More often, they featured formulaic tabloid stories, like tales of cheating spouses (“You won’t believe what he did next!”) or disrespected blue collar workers who get revenge on the elitists who snubbed them. Oddly, ever-present across the pages were memes featuring the television personality and America’s Got Talent judge Simon Cowell. Along with changing their content to echo the Facebook algorithm’s shift away from politics, the pages also showed other telltale signs of adapting to the platform’s ever-changing rules and incentives. For instance, Facebook has reduced the reach of “spammy” links but prioritizes a page admin’s comments on their own posts; as a result, these pages often posted a meme or other image that summarizes the gist of an article, and then posted the link as a comment.
Patriotic Warriors, a Facebook page that had 141,000 followers before it was taken down, is run out of Macedonia.
Some of the pages also used other engagement-juicing tricks that have long been popular. One page based in Kosovo, called Animals News America, featured clickbait posts similar (and in some cases, identical) to those on other, non-animal themed pages. But it also posted a regular stream of kittens and puppies, using a strategy previously employed by notorious misinformation spreaders like the COVID- denying doctor Joseph Mercola and NTD News, a Falun Gong-affiliated sister brand of the Epoch Times.
After Forbes reached out for comment, Meta removed every page.
Even if these pages weren’t intentionally being used to shape people’s political views, click-farmers will sometimes shift their pages into deliberate geopolitical influence operations, Allen said. While still at Facebook, he observed one Thailand-based operation that targeted pages about politics to audiences in Myanmar. They would "pop in and out of being guns for hire for political campaigns," he said. "But when it wasn't political campaign season, they'd run the exact same operations, just making the money themselves."
That makes these pages less innocuous than they might seem. “I bet there are plenty of foreign influence operations that would like to buy these Pages when the time is right. So, there are times when ‘click farms’ can become much more nefarious," Allen said. After the original Macedonian click farmers were exposed, Facebook launched a feature to enable users to find out which country a page is run from, if it has at least 5,000 followers or has run political ads. But the country of a page administrator’s origins is often hidden in an obscure panel called Page Transparency, and comments on the foreign America-themed pages’ posts suggest that many people engaging with those posts did not know that the pages are run by foreigners.
The America-themed pages themselves were also deliberately misleading. One post made last week by a page called America Today reads: “Not another cent to nations that disrespect our flag and values! 🇺🇸” The page was managed from Macedonia.
Franklin noted that in certain cases, Meta now displays the location of certain pages’ page managers directly in the Facebook News Feed.
Accounts that pretend to be American when they’re not may be a widespread issue on social media. Facebook, to its credit, is the only major social media platform reveals the country from which its large pages are managed. Other platforms, including YouTube and TikTok, allow users to self-declare a location if they want to, making it harder to detect accounts that are pretending to be American when they’re not. The incentives, however, are the same. Parveen Kumar Shah, who makes his living advising people about how to build audiences on YouTube and Instagram through his channel TubeSensei, recently suggested other page creators seeking to build an audience should pretend to be American. Why? You’ll make more money that way, he advised.
AI makes that even easier. In an interview, he told Forbes that now, “if you don’t want to show your face, everything can be done through AI.” On YouTube, Shah showed his followers how to make masculinity-themed pages for American teens with titles like Far From Weak and Sigma Male. He told Forbes: “Targeting that type of audience is very easy because a teenager’s brain is very easy to mold.”
There are many videos on YouTube that explain how to hide or spoof your country of residence on the platform, to make it look like your channel is based in another part of the world. For Shah, this is a simple economic calculus: YouTube pays channel managers based on the ads that run on their channel, and advertisers spend far more in Western markets than they do in India. On the TubeSensei channel, he explained: “Our channel is going to be for a U.S. audience, and as soon as they come to know that this is an Indian channel, or there is an Indian creator behind it, they stop watching the channel.”
YouTube did not respond to a request for comment.
Allen, the former Facebook data scientist, characterized engagement farming as a problem for platforms to fix — one that if they don’t address, regulators might eventually penalize them for. He compared the prevalence of inauthentic pages to defective tires on a car: "Your tires have been popping on the highways for the past ten years. At a certain point, there's going to be some regulatory teeth."
As long as the click farmers’ crimes don’t go beyond the proliferation of stale, low-quality memes, though, Allen doesn't think removing pages is the solution. Instead, he said, Facebook should move away from an algorithm that incentivizes people to post sensationalist slop in the first place.
"If a click farmer tries to farm on your platform, but doesn't get any clicks — does he do any farming?"
Rashi Shrivastava contributed reporting.
MORE FROM FORBES
Emily Baker-White is an investigative reporter and senior writer at Forbes. She joined Forbes from BuzzFeed News in 2022 and covers the way tech companies shape our discourse,
...