Spam, scams, and propaganda: The state of Twitter 15 months into Elon Musk’s reign

How Musk turned to conspiracy theories

By the end of 2021, Elon Musk was increasingly adamant that free speech was in retreat around the world. The culprit, according to the billionaire, was the “woke mind virus.” The billionaire then decided to “trace the path of the woke virus,” which according to him was “attempting to destroy civilization.” After all, if civilization ended, then “humanity will never reach Mars.” Musk's obsession with the spread of “woke culture” was one of the reasons the billionaire began buying Twitter shares in 2022, Walter Isaacson writes in his biography of the businessman. Musk was upset that the platform had blocked the account of former President Donald Trump and other members of the American far-right.

After joining the company's board of directors, Musk claimed to set his sights on increasing free speech on the platform, presenting himself as a champion of democracy. “Twitter should be moving toward free speech, at least to the extent that the law requires. And right now, Twitter is suppressing free speech more than the law requires,” he said in a meeting with friends and business partners Luke Nosek and Ken Howery. A few weeks later, Musk would call himself “a free speech absolutist,” and in a little more time, he would gain full power over Twitter — not just a popular social network, but an important tool of political influence.

Isaacson says that, for most of Musk’slife, the billionaire’s political views had been largely centrist. Many of his best friends from the business world, however, were ardent libertarians — notably Nosek, Howery, and PayPal co-founder Peter Thiel. The latter had been a critic of “woke culture” for years and even wrote a book in the 90s criticizing political correctness. According to Isaacson, Musk’s old friends, along with prominent podcast host Joe Rogan and blogger Jordan Peterson, had a significant influence on the entrepreneur's future political course. All of them are actively opposed to “cancel culture,” feminism, and anything considered close to left-wing political views. After meeting Peterson and Rogan at a party at Tesla's Gigafactory in Texas, Musk began actively speaking out on political issues and looking for new ways to influence politics. The change in Musk's views was also noticed by his ex-wife, Justine Wilson, and by the singer Grimes, Musk's ex-girlfriend and the mother of two of his children. Both were taken aback by his new political beliefs and fascination with far-right conspiracy theories.

Musk's control of Twitter became more of a political action than a financial investment. He decided to fight against a culture that he considers “communism rebranded” and a “death cult.”

Acquired for $44 billion in October 2022, the company lost almost half its value over the course of the next year, this according to Musk's own admission. However, Fidelity — one of the majority investors in the social network — estimates the loss of the company’s market capitalization at the end of 2023 to be even larger: 71.5%, which amounts to over $30 billion.

Part of the reason for the decline was the businessman's commitment to freedom of speech — at least as he professes to understand the concept. Policies implemented under Musk led to the increased spread of anti-Semitic statements, misinformation, and false data. As a result, half of the largest advertisers on the platform refused to use the company's services. For example, in November 2023, Apple and Disney stopped advertising on Twitter, as the companies were not satisfied with the fact that their ads appeared in the same feed with “pro-Nazi content.”

Interestingly, the site's loss of revenue does not appear to bother Musk in the slightest. When major advertisers announced a boycott of the social network over the distribution of Nazi content and anti-Semitic statements by Musk personally, the billionaire literally told them to “go f*ck themselves”:

“If somebody is gonna try to blackmail me with money… Go f*ck yourself.”

Free speech — without minorities

The billionaire's fight for free speech turned out to be a fight for the free speech of far-right bloggers, politicians, and publicists. Many of them, after being banned from Twitter, ended up on alternative far-right platforms. Now those banned from other platforms are beginning to settle back on the renamed “X.” Tucker Carlson, who was fired from Fox News for spreading slander and fakes, found a new home on Musk’s platform. Most of the 12,000 accounts “pardoned” by the billionaire in the first month of his ownership of the social network belonged to neo-Nazis, far-right activists, racists, and spammers, according to an analysis by The Insider. It became a haven for bloggers and politicians blocked on other platforms for their inflammatory rhetoric.

Among the unblocked accounts are many stars of the alt-right. Many of them had been demonetized or blocked for misogyny or hatred of transgender people before Musk's arrival. Twitter sheltered Matt Walsh, who had been banned from YouTube. Musk unblocked the previously banned ultra-conservative satirical resource The Babylon Bee, along with the account of blogger Andrew Tate, who argued before his arrest on human trafficking charges that women are responsible for the violence perpetrated against them. Rapper Kanye West, whose account was previously suspended for anti-Semitism, was also able to return to the social network. One of the first things West did after being unblocked was to say that he sympathized with Hitler. Musk also unblocked the account of Donald Trump. Scott Ritter, a columnist and Russia Today's favorite author, who denies Russian war crimes in Ukraine and glorifies the shelling of Ukrainians, was also unblocked. Jordan Peterson is back on the platform as well.

In late 2023, Twitter unblocked the account of conspiracy theorist Alex Jones after an interview on Tucker Carlson's show. In Russia, Jones is known for his appearances on the Tsargrad channel — as well as on programs hosted by propagandist Vladimir Solovyov. Like many other characters in this story, Jones is a known denier of Russia's war crimes in Ukraine. In January 2024, the platform announced a collaboration with Congresswoman Tulsi Gabbard, an opponent of U.S. sanctions against Russia who defended Vladimir Putin’s rationale for invading Ukraine. Gabbard sympathizes not only with the Russian regime, but also with that of Bashar al-Assad in Syria. In 2019, she denied Assad's use of chemical weapons against his citizens. She inclined her audience to believe that the attacks were staged with the intention of justifying U.S. involvement in the war in Syria. Gabbard’s claims had been debunked as unfounded conspiracy theories long before she made them, but the former Congresswoman continued repeating the arguments of war-crimes-denying Russian and Syrian propagandists nonetheless.

In the name of free speech, Musk also dissolved his new company's human rights advisory board, the Trust and Safety Council, which since 2016 had set policies for moderating the platform to keep it free of hate speech, child abuse, and messages promoting self-harm. A former employee of that group, an openly gay man, was targeted by one of Musk's hate groups for allegedly misrepresenting the billionaire’s views and statements. Unblocked far-right activists immediately began harassing and flooding complaints towards the accounts of human rights organizations that help LGBT and non-white people in various countries. Many of them were eventually forced to leave the site.

“Stop criticizing X. X is freedom. X is interaction. X is discussion. X is fun. Compliments to Elon Musk,” far-right Dutch MP Geert Wilders tweeted from his official account in 2024. The politician is known for his Islamophobia, hatred of migrants, and support for Russian aggression in Ukraine. In his election campaign, Wilders insisted, among other things, on stopping military aid to Ukraine, and he previously spoke out against the imposition of sanctions against Russia.

Musk guarantees free speech for politicians like Wilders. However, different rules apply to his critics and bloggers with different views, who are often subject to explicit or implicit blocking (the so-called “shadow ban”). Just two months after Twitter was acquired, the platform blocked the accounts of several tech industry journalists who had published investigative pieces about Musk's companies and the spread of conspiracy theories on social media. No explanation for the treatment of these journalists has been forthcoming, and no accusation has been made that they violated Twitter’s terms of use.

The account of another social platform — Mastodon — which was used by people leaving Twitter due to the innovations of Elon Musk’s changes, was also blocked. Accounts mentioning a new social network launched by Twitter founder Jack Dorsey — Bluesky — were blocked too. And in January 2024, Twitter again began blocking journalists en masse. Reporters from the watchdog publication The Intercept, the liberal investigative magazine Texas Observer, and the account of the left-wing podcast TrueAnon were all banned. A few hours later, the accounts were unblocked as suddenly and synchronously as they had been taken offline, and there was no explanation as to why journalists who disagreed with Twitter's policies or U.S. government actions were blocked.

Cosying up to bots and trolls

Even before buying Twitter, Musk pointed to “armies of bots” as one of the social network's biggest problems. He said he was ready to “defeat the spam bots or die trying” and promised to “authenticate all real users.”

Despite all these assurances, the number of bots on the platform has only grown, academics assert. University of Queensland professor Timothy Graham analyzed nearly a million tweets in August 2023 and concluded that fake accounts were banding together to spread conspiracy theories. Several of the messages being spread suggested that Donald Trump had actually won the 2020 U.S. presidential election, while others merely engaged in spamming, promoting content from media outlets, and spreading fake news (much of it also in support of Trump). Twitter moderation did not stop these flows of misinformation and manipulation in any way, the researcher notes. What’s more, the social network was literally filled with bots in February 2024 during the Super Bowl — the most-televised annual event in the United States. This conclusion was reached by researchers at CHEQ, a cybersecurity company. According to their data, three-quarters of the advertising traffic during the football game came from fake accounts. “I've never even seen half of the traffic [be fake], let alone 76%. I'm shocked, I've never seen anything even close to that in my life,” CHEQ founder Guy Tytunovic told Mashable.

The same viewpoint is held by other experts interviewed by The Insider. Recorded Future, an American company engaged in cybersecurity and network threat research, monitors bots and identifies related fake accounts in different social networks. According to Alexander Leslie, an analyst at the company, “accounts with inauthentic behavior” post a wide range of mostly right-wing political messages. “We see posts about Ukraine, Israel. These are the two main 'bait' accounts they are using right now. They're all very clickbait,” Leslie notes.

The same posts are monitored by the Bot Blocker / antibot4navalny project on Twitter, along with other social networks. These publications quickly disappear, but the project's analysts archive them. In November 2023, for example, they recorded mass mailings of pictures with fake demands to stop aid to Ukraine and attributed the sentiments to various celebrities.

The accounts that sent the messages were the same accounts that had previously distributed posts by the pro-Kremlin Doppelgänger / RRN bot network. IT companies National Technologies and the Social Design Agency («Агентство Социального Проектирования»), which are under European sanctions, are behind the spam attack, but Twitter allows such accounts to continue their Kremlin-sponsored disinformation campaign.

According to Der Spiegel, the German Foreign Ministry is conducting its own investigation into the pro-Russian and anti-Ukrainian bots. According to preliminary results, 50,000 inauthentic accounts participated in the disinformation campaign in one month alone. Starting from December 20, 2023, these inauthentic accounts posted up to 200,000 messages per day calling for an end to programs supporting Ukraine.

International Crisis Group analyst Alessandro Accorsi notes that within a few months of firing the staff responsible for Twitter’s moderation and security, changes to the privacy policy led to a rapid deterioration of the platform. ProPublica journalist Craig Silverman, who has spent more than a decade researching the spread of misinformation and manipulative content online, agrees. Journalist and conspiracy theory expert Mike Rothschild notes that problems with bots and moderation have been around for a while, but have become more pronounced over the past year.

“It's definitely connected to Musk's purchase and takeover of the site, though the problem was certainly there before. There seems to be no guardrails to creating fake accounts and using them to drive engagement, as Twitter had at least attempted to get the problem under control before Musk took over.”

In an attempt to push back against Musk’s company, he EU has decided to apply the Digital Services Act, which regulates the distribution of illegal content, intrusive advertising, and misinformation. This is due to the huge number of fakes on the platform. As Johannes Barke, a spokesperson for the European Commission, explained, “The opening of proceedings means that the [European] Commission will now investigate X's systems and policies related to certain suspected infringements. It does not prejudge the outcome of the investigation.” Before announcing the move, the European Commission examined the proportion of posts containing misinformation on major social networking websites. Twitter was found to be the leader in that dubious category. If, as a result of the investigation, the platform does not change its policies to comply with European law, it could theoretically face fines of up to 6% of its annual revenue or even be blocked across the EU. However, it remains unclear what action will actually be taken in practice.

“There’s no timeframe for the investigation, they can collect information for a long time and depending on its outcome, they can impose sanctions in the form of restrictions or a fine. Obviously, this case will be used as an example to develop the practice,” says media lawyer Galina Arapova.

An inconvenient truth

One of the means introduced in recent months to combat fakes on Twitter is the use of so-called Community Notes. If a post contains incorrect or misleading information, users in the Notes program can add context to help clarify the post. Other users can vote on the notes, collectively approving or rejecting them. Tweets that have community notes added to them are also not monetized — by design, this should discourage accounts that hope to profit from the platform from publishing fakes.

The system's source code is published on Github, allowing anyone to verify its reliability, but researchers say the measures taken are not sufficient. “The company shifted to a moderation model based on AI and community notes, while reducing transparency and access to its data for external researchers,” argues Alessandro Accorsi.

Musk may have already begun to regret this innovation. In December, the Twitter owner tried to draw public attention to the arrest of Gonsalo Lira in Kharkiv. Through his personal account, Musk asked what had become of the “American journalist,” hinting at restrictions on freedom of speech in Ukraine. A note to his post was later added saying that Lira was not a journalist, but a YouTube blogger who had been spreading pro-Russian propaganda and denying Russian war crimes. Musk reacted to the note by accusing its authors of being “gamed by state actors.” But the platform owner did not provide any proof of his claims, and the note under his tweet was soon deleted.

As things stand, it is becoming increasingly difficult for independent experts to assess the amount of misinformation on the social network. Moderation systems are becoming less transparent and effective, but by all appearances, fakes are published more frequently. The platform even pays users to post such content.

Cash incentives for spammers

Although researchers unanimously claim that the social network is full of bots, many of which are aimed at promoting conspiracy theories and hateful political views, Twitter management officially disagrees, claiming instead that the best way to combat the overpopulation of bots on the social network is through the use of a paid “verification check.” Before Musk took over the company, blue checkmarks were used to denote that an account claiming to belong to a celebrity or otherwise prominent person was in fact authentic. In order to be authenticated, users had to provide the firm with confirmation that they were who they said they were. However, starting in 2023, the only barrier to “verification” was a monthly fee. Unsurprisingly, the new system is being actively exploited by bots and scammers.

Popular users who buy a verification checkmark can even receive payments directly from the platform. The amount of the payments depends on the number of views the account attracts. That incentive has inspired some users to begin publishing more and more provocative content — including false information. The war in Gaza, which began after the October 7 Hamas terrorist attack in Israel, also contributed to the spread of such information. For example, a large number of fakes about the attack were published by Malaysian blogger Ian Miles Cheong, who is popular among Americans on the far-right. Cheong’s account is actively promoted by Twitter and by Elon Musk himself. This is not surprising, as in addition to the fakes, Cheong frequently publishes statements against the modern world order, and these posts are duly admired by his conservative audience.

And Cheong is not the only example. Pakistani TikTok star Hareem Shah (posing as a journalist from London), Danish doctor Anastasia Maria Loupis, showman-entrepreneur Mario Nawfal, and many other accounts — some supporting Palestine, others supporting Israel — have published hundreds of fakes about military actions in Gaza and politicians' statements. However, according to Alessandro Accorsi, both sides’ disinformation shares something very important in common: three-quarters of all Twitter fakes about the war in Gaza were published by verified accounts with a blue checkmark.

All of these accounts are verified, which means that the millions of views they collect turn into hundreds and thousands of dollars in payouts. In addition, the social network's algorithm promotes the tweets of verified accounts, showing them more often and higher than the posts of those who didn't buy a checkmark. All of this undermines any efforts to combat fakes on the platform, argues researcher Mike Rothschild:

“The monetization of blue checks is a huge problem [for Twitter], given that it now gives scammers and conspiracy theorists a cash incentive to spread the biggest and most believable lies. There's also no real trail for where the money comes from or where it's going, and many blue check accounts spreading pro-Russia conspiracy theories are clearly linked to Russia itself.”

American blogger Brian Krassenstein was one of the first to receive — and report — a large payout, posting on July 13, 2023 that Twitter had paid him “almost $25,000.” Krassenstein, already famous for his constant trolling of Donald Trump, was banned by Twitter in 2019 for organizing a network of fake accounts aimed at scoring likes. The blogger was able to return to the social network only after Elon Musk’s “amnesty” of numerous accounts.

Accorsi notes that the system that has developed in recent months also incites premium users to post more toxic content, which attracts even more views — and payouts. “Allowing premium users to monetize engagement incentivised rage-baiting and sharing of misinformation to have more interactions,” he points out.

Where do verified bots come from?

While it's now difficult to register multiple accounts at once, that doesn't stop scammers and bot farms from using hacked and long-abandoned accounts that can be bought on exchanges on the darknet. “These blue checks accounts are hacked. It’s like you give your password to [someone], nothing to do with ‘bot account’,” Fortune reporter Kylie Robison quoted the owner of one such commercial bot farm as saying.

The trend of selling hacked accounts has also been observed by analyst Alexander Leslie. “Now, phishing links and misinformation sent through verified accounts are focused on accounts that have already bought a blue check,” he told The Insider. In order to obtain such accounts — and then use them to distribute malicious or manipulative content — scammers use the same phishing techniques to trick verified users out of their usernames and passwords. Recorded Future, a company that researches the dark web, has seen an increase in the sale of these hijacked accounts on hacker forums.

It's also no problem to “gain” subscribers to create the appearance of popularity. A hundred thousand fake subscribers cost less than one thousand dollars on any subscription exchange. The social network itself does not limit encryption in any way.

“You can see documents being offered for verification on Twitter on hacker forums — selfies, passports, driver's licenses. Usually such documents are sold for verification at online banks or cryptocurrency exchanges, but now buyers specify that they can use them specifically for social media registration,” Leslie notes.

The use of stolen photos, bank cards, and documents for social media scams has become a growing trend over the past year, according to the analyst.

Propagandists and fraudsters

In December 2023 — just after the biggest advertisers left the social network — Twitter was filled with promotional posts with links to phishing sites. Of course, such posts had proliferated on the social network before, but they began to be massively promoted at the end of 2023, Leslie notes. For users from Russia, for example, the promoted posts most often showed photos of host Ksenia Borodina with clickbait headlines and links leading to data-stealing websites. Even after two months of constant complaints, not all of these posts were taken down.

Promotional posts with fraudulent links almost always come from accounts bearing the blue verification tick. Scammers' use of “verified” accounts is not only intended to create the illusion of trust among users, but also to ensure that their posts reach the widest possible audience, as Twitter automatically promotes the accounts that pay for the imprimatur. The larger the audience of a fraudulent post, the greater the chance that someone will click on it and lose their account or money.

Not only do scammers directly promote their services, they also seek out vulnerable users who can be scammed. To demonstrate this, in November 2023, Leslie published a deliberately absurdly worded post. Leslie’s “live-baiting” tweet twice noted that it was not a genuine post, but in addition to that disclaimer, it did contain several of the key terms bots look for — including the names of popular money transfer services and social networks in the U.S. and Europe, along with the phrase that his “account got hacked.”

Within 10 minutes, the entry had received 14 replies and five “hidden” reposts — all of which were from fake accounts. Half of the fake accounts were verified. “This is an impressive result,” notes Leslie, whose team has carried out tests like this before.

Another Twitter policy change that was criticized by information security experts occurred in October 2023, when the decision was made to hide headlines in posts featuring links. As a result, users began clicking on images and being redirected to fraudulent sites, unaware that the picture they intended to view had contained a hidden link.

“As soon as [Twitter] removed the headers on posts with third-party links, we immediately saw entire networks of bots spreading links with misleading captions and leading to stealer sites,” Leslie notes. Stealers are malicious programs that collect logins, passwords, and cookies on a victim's computer. This information is then sold on the black market. Victims of stealers can lose both their social network accounts and their associated bank accounts. Alexander Leslie notes that it's mostly verified accounts that spread these types of links.

“Phishing [on Twitter/X] has become much easier for hackers. We've seen hackers pretending to be a Sky News account. And it was very easy for them to do so because the link attached to the post is simply not visible. A person accidentally clicks on the link and is taken straight to the fake website.”

In fact, the hackers use the same attack methodology as the pro-Kremlin bots found by Bot Blocker / antibot4navalny. Last October, the project detected a disinformation campaign against Ukraine being promoted through fake clone sites of major Russian and European media outlets.

What does Twitter’s future look like?

The social network was and still is a source of news for many people. “People come to Twitter to discover #WhatsHappening,” read a slogan that once adorned the top of the social network's portal for advertisers. That messaging changed in 2023, and now businesses are promised access to a “Global Town Square” where people share their insights. Of course, the platform has served as a sort of digital town square for much of its history, with multiple examples of mass movements using it to coordinate anti-authoritarian protests: the “For Fair Elections” movement in Russia in 2011-2012, protests against the election results in Moldova in 2009, the revolutions in Tunisia and Egypt in 2011, and the Euromaidan in Ukraine in 2013-2014. At the same time, Twitter, like other social networks, had also been a place where plenty of misinformation was spread and where information campaigns organized with the help of bots flourished. The most famous of these is Wagner PMC founder Yevgeny Prigozhin’s “troll factory,” which aimed to interfere in the 2016 U.S. election. However, as a 2023 study showed, this campaign was largely ineffective, as it targeted a small group of people who would have voted for Donald Trump anyway.

EU officials are now investigating the spread of misinformation on social networks. At the same time, the data on the basis of which one could draw conclusions about the mass activity of bots on Twitter has become largely inaccessible for ordinary researchers: if earlier the automatic data collection was almost or completely free, now even the minimum user fee will cost scientists $5000. This was one of the first innovations of Elon Musk, who justified the introduction of fees for automatic data collection by portraying it as a necessary step in the fight against bots.

But again, all of the available evidence shows that the bot problem has only grown since Musk took control of the company. Purportedly as a result, at the end of 2023, the platform began testing a paid subscription model that would charge all users a cost of $1 a year, the idea being that bots will not pay the nominal fee. However, Dan Woods, head of research at the Internet company F5 Networks, believes that this will not solve the problem, as both scammers and government organizations have enough resources to buy fake accounts that the social network will deem authentic.

Since the end of 2023, the social network has been flooded with inauthentic accounts that generate posts using linguistic models like ChatGPT. They are used to promote accounts by increasing audience engagement. Regular users can easily recognize such accounts, as they often give themselves away easily. For example, in February 2024, accounts discussing non-existent photos or posting an automated response that says “I can't generate a response to this request” have gained millions of views. In addition, bots connected to ChatGPT also generate political posts. This was revealed by researchers at The Institute for Strategic Dialogue, which has been researching online disinformation campaigns for years. In December 2023, scientists discovered a network of accounts criticizing Ukraine and the Russian opposition. According to experts, their publications were generated with the help of neural networks. Elon Musk, however, accuses ChatGPT itself of spreading the woke virus. But so far, the social network has not prevented the registration or activity of such accounts.

“On the monetization side, the big change is how much they are trying to make the paid user model work. but that push plus Elon's favouritism of right wing users and targeting of news [organizations] and others means that the paid user base seems to have a political aspect. and because paid users get more visibility, that means more extreme political stuff seems to be even more present on the platform,” says Craig Silverman.

“That's not so good for the ad side of the business. so they have not been able to get a mainstream amount of adoption for the paid user offering and at the same time they have depressed ad sales with the chaotic and politicized nature of the company, and because the blue checks are sharing lots of questionable stuff that gets engagement.”

For his part, Musk has taken to accusing Twitter’s former advertisers of attempting to “kill the company.” In order to make up for lost revenue, the platform has expressed a desire to replicate the success of Facebook by attracting large numbers of small advertisers. So far, however, according to The Wall Street Journal, the platform doesn't have a stable technical solution for this type of collaboration. Still, small entrepreneurs are actively promoting their services on “X” — from the already described scam and phishing sites to stores selling fake Louis Vuitton bags.

“I probably spend too much time on Twitter. It’s a good place to dig your own grave. You get your shoulder into it, and you keep on digging,” Musk said during a dinner with friends in April 2022. A couple of days before that, the billionaire had announced he was joining the company's board of directors. In a few months, he would buy the platform, disperse the board, amnesty the far-right Influencers, and begin his crusade against the “woke virus” — that last item representing perhaps the only promise he made before buying the social network that Elon Musk was able to fulfill.

The text uses quotes from Walter Isaacson's biographical book “Elon Musk,” published in September 2023. An electronic copy of the book is available on the Internet Archive. Elon Musk changed the name of Twitter to X in July 2023, but for the sake of simplicity, the social network was referred to by its original name throughout this piece.