sign up log in
Want to go ad-free? Find out how, here.

Algorithmic oversight and reform needed for platforms in AI era, Mariana Mazzucato & Ilan Strauss argue

Technology / news
Algorithmic oversight and reform needed for platforms in AI era, Mariana Mazzucato & Ilan Strauss argue
Algorithm locking in users

In a new lawsuit in the United States against Meta, 41 states and the District of Columbia argue that two of the company’s social-media products – Instagram and Facebook – are not just addictive but detrimental to children’s well-being. Meta is accused of engaging in a “scheme to exploit young users for profit,” including by showing harmful content that keeps them glued to their screens.

According to one recent poll, 17-year-olds in the US spend 5.8 hours per day on social media. How did it come to this? The answer, in a word, is “engagement.”

Deploying algorithms to maximise user engagement is how Big Tech maximises shareholder value, with short-term profits often overriding longer-term business objectives, not to mention societal health. As the data scientist Greg Linden puts it, algorithms built on “bad metrics” foster “bad incentives” and enable “bad actors.”

Although Facebook started as a basic service that connected friends and acquaintances online, its design gradually evolved not to meet user needs and preferences, but to keep them on the platform and away from others. In pursuit of this objective, the company regularly disregarded explicit consumer preferences regarding the kind of content users wanted to see, their privacy, and data sharing.

Putting immediate profits first means funneling users toward “clicks,” even though this approach generally favours inferior, sensational material, rather than fairly rewarding participants from across a broader ecosystem of content creators, users, and advertisers. We call these profits “algorithmic attention rents,” because they are generated by passive ownership (like a landlord) rather than from entrepreneurial production to meet consumers’ needs.

Mapping rents in today’s economy requires understanding how dominant platforms exploit their algorithmic control over users. When an algorithm degrades the quality of the content it promotes, it is exploiting users’ trust and the dominant position that network effects reinforce. That is why Facebook, Twitter, and Instagram can get away with cramming their feeds with ads and “recommended” addictive content.

As tech writer Cory Doctorow has colourfully put it, platform “enshittification comes out of the barrel of an algorithm” (which may, in turn, rely on illegal data collection and sharing practices).

The Meta suit is ultimately about its algorithmic practices that are carefully constructed to maximise user “engagement” – keeping users on the platform for longer and provoking more comments, likes, and reposts. Often, a good way to do this is to display harmful and borderline illegal content, and to transform time on the platform into a compulsive activity, with features like “infinite scroll” and nonstop notifications and alerts (many of the same techniques are used, to great effect, by the gambling industry).

Now that advances in artificial intelligence already supercharge algorithmic recommendations, making them even more addictive, there is an urgent need for new governance structures oriented toward the “common good” (rather than a narrowly conceived notion of “shareholder value”) and symbiotic partnerships between business, government, and civil society. Fortunately, it is well within policymakers’ power to shape these markets for the better.

First, rather than relying only on competition and antitrust law, policymakers should adopt technological tools to ensure that platforms cannot unfairly lock in users and developers. One way to prevent anti-competitive “walled gardens” is by mandating data portability and interoperability across digital services, so that users can move more seamlessly between platforms, depending on where their needs and preferences are best met.

Second, corporate governance reform is essential, since maximisation of shareholder value is what pushed platforms to exploit their users algorithmically in the first place. Given the well-known social costs associated with this business model – optimising for clicks often means amplifying scams, misinformation, and politically polarising material – governance reform requires algorithmic reform.

A first step toward establishing a healthier baseline is to require platforms to disclose (in the annual 10-K reports filed to the US Securities and Exchange Commission) what their algorithms optimise for, along with how their users are monetised. In a world where tech executives descend on Davos every year to talk about “purpose,” proper disclosures will pressure them to do what they say, as well as help policymakers, regulators, and investors distinguish between earned profits and unearned rents.

Third, users should be given greater influence over the algorithmic prioritisation of information shown to them. Otherwise, the harms from ignoring user preferences will continue to grow as algorithms create their own feedback loops, pushing manipulative clickbait on users and then wrongly inferring that they prefer it.

Fourth, the industry standard of “A/B testing” should give way to more comprehensive long-term impact evaluations. Faulty data science drives algorithmic short-termism. For example, A/B testing may show that displaying more ads in a feed will have a positive short-term impact on profits without overly harming user retention; but this ignores the impact on acquiring new users, not to mention most other potentially harmful long-term effects.

Good data science shows that optimising recommender systems for long-term, delayed rewards (such as customer satisfaction, retention, and new-user adoption) is the best way for a company to drive long-term growth and profitability – assuming it can stop focusing primarily on the next quarterly-earnings report.

In 2020, a team within Meta determined that fewer intrusive notifications would be better for both app usage and user satisfaction over a longer period of time (one year). Long-term effects differed sharply from short-term effects.

Fifth, public AI should be deployed to evaluate the quality of algorithmic outputs, particularly advertising. Given the considerable harms arising from platforms lowering the standard of acceptable ads, the United Kingdom’s advertising watchdog will now use AI tools to scrutinise ads and identify those making “dodgy claims.” Other authorities should follow suit. Equally important, AI evaluators should be a feature of platforms’ openness to external auditing of algorithmic outputs.

Creating a digital environment that rewards value creation from innovation, and punishes value extraction from rents (especially in core digital markets), is the fundamental economic challenge of our time. Safeguarding the health of Big Tech’s users and the entire ecosystem means ensuring that algorithms are not beholden to shareholders’ immediate profit concerns. If business leaders are serious about stakeholder value, they should accept the need to create value in a fundamentally different way – drawing on the five principles above.

Meta’s forthcoming trial cannot undo past mistakes. But as we prepare for the next generation of AI products, we must establish proper algorithmic oversight. AI-powered algorithms will influence not just what we consume, but how we produce and create; not just what we choose, but what we think. We must not get this wrong.

Mariana Mazzucato, founding director of the UCL Institute for Innovation and Public Purpose, is chair of the World Health Organisation’s Council on the Economics of Health for All.

Ilan Strauss is a research associate at the UCL Institute for Innovation and Public Purpose.

Copyright: Project Syndicate, 2024.

We welcome your comments below. If you are not already registered, please register to comment.

Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.


This is a useful, if slightly depressing, commentary on how the attention economy operates.

For a techie's take on that next AI step:…

My own take: this may have already gone wrong, and a return to analogue, more secure and credible methods, might be the only way out. Think cash rather than hackable digital currency, passwords memorised and handwritten in a book rather than biometric security (I can change my stolen password, but my fingerprint?), written letters that stay private rather than email read by AI systems to sell you stuff you don't need and can't afford...


Interesting lawsuit. The algorithm certainly seems to have impacted some folk very negatively over the last couple of years, especially during COVID. I saw a couple of Facebook friends dragged down incredibly by the algorithm before I left Facebook. Crazy how much of this persuasive psychology has been targeted directly at children too.

Former CIA and NSA Director Michael Hayden's book The Assault on Intelligence provides an interesting view into how much American society has been affected by Russian and other informational warfare that's taken advantage of these too. Published in 2018, too, so before much of the strangest carry-on in Trumpian times.


I am sick of "algorithms" and "AI".

Search engines are becoming more and more hopeless to use. They won't look for what I type, rather what they think I should have typed/wanted.

Netflix is becoming pointless. You can't easily browse as it thinks it knows what you want to watch. I'm not in a comedy/war/action/documentary mood today, I want sci-fi, nope. No browsing, just know the title...

MS keep trying to focus my inbox, and push Co-pilot.

Best of all, they all requires 57-step verification, fingerprints or some other ludicrous unique identifier. Then most of the T&C's involve vairous organs, first born kids. While the privacy docs allow them to access to every single other piece of data on your device and sell it to whoever will chuck them a buck.

We get fatter, lazier, and dumber. Combine Wall-E with Idiocracy and you have the current future of humanity. It is like we are being farmed without realising.


You are not alone: Freakonomics take and the MIT techie point of view - searches are getting worse.


Also from Sergey Brin and Larry Page, who started Google. What they said earlier:

 It is clear that a search engine which was taking money for showing cellular phone ads would have difficulty justifying the page that our system returned to its paying advertisers. For this type of reason and historical experience with other media, we expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.

...This type of bias is very difficult to detect but could still have a significant effect on the market. Furthermore, advertising income often provides an incentive to provide poor quality search results. For example, we noticed a major search engine would not return a large airline's homepage when the airline's name was given as a query. It so happened that the airline had placed an expensive ad, linked to the query that was its name. A better search engine would not have required this ad, and possibly resulted in the loss of the revenue from the airline to the search engine. In general, it could be argued from the consumer point of view that the better the search engine is, the fewer advertisements will be needed for the consumer to find what they want. This of course erodes the advertising supported business model of the existing search engines. However, there will always be money from advertisers who want a customer to switch products, or have something that is genuinely new. But we believe the issue of advertising causes enough mixed incentives that it is crucial to have a competitive search engine that is transparent and in the academic realm.


Maybe governments should start charging users for access to offshore social media? [evil grin]


VPN (waves anarchist flag).