Social media and Purity Spirals


A “complete unraveling of the social fabric of society” is one of the primary dangers that led to the founding of The Center for Humane Technology by Tristan Harris et al. Social media platforms and their business models are uniquely primed to reinforce and even weaponise negative vicious cycles that speed up this unraveling. As we witness all around us.

Today I bumped into several Hacker News discussions on controversial topics that had turned into flamewars, people from opposite sides upvoting and downvoting others in quick succession, and a stream of insensitive comments being fired at the thread.

What struck me first of all, was - regardless of who posted them - how incredibly un-nuanced and even cruel many of these comments were. Posted by people who truly believed in the narrow viewpoints they expressed: black vs. white, good vs. evil, with me or against me. There was no place for the reasonable voice anymore. And this discussion would only lead to people getting further apart, digging in deeper, and more divided than before.

(Luckily on HN - being the best moderated social media platform I know of - such threads are rapidly demoted and disappear from the front-page real quick).

This way every participant, unless they were deliberate trolls, was not achieving anything from the discussion. Only validating they belonged to the tribe and engaged in a pointless fight. Wasting their time, and making society nastier.

Purity Spiral

Note: The article is only attributed for how it defines Purity Spiral as a concept, not because of its particular quality and contents.

But what stood out even more was the fact that most of these people didn’t see it that way. Their belief is that their involvement is right and proper, and comes from having the ‘moral high ground’. This is a phenomenon I witness very often, where people who feel they represent ‘the good cause’ may not only be counter-productive in their engagement to others, but also are they driving their own community apart.

Just now I found an article by journalist Gavin Haynes that addresses this in more detail:

A very interesting read. Some quotes from the article:

A purity spiral occurs when a community becomes fixated on implementing a single value that has no upper limit, and no single agreed interpretation. The result is a moral feeding frenzy.

But while a purity spiral often concerns morality, it is not about morality. It’s about purity — a very different concept. Morality doesn’t need to exist with reference to anything other than itself. Purity, on the other hand, is an inherently relative value — the game is always one of purer-than-thou. […]

It is a social dynamic that plays out across that community — a process of moral outbidding, unchecked, which corrodes the group from within, rewarding those who put themselves at the extremes, and punishing nuance and divergence relentlessly. […]

A purity spiral propagates itself through the tipping points of preference falsification: through self-censorship, and through loyalty tests that weed out its detractors long before they can band together. In that sense, once one takes hold, its momentum can be very difficult to halt.

The author comes to the conclusion that:

The phenomenon isn’t going anywhere. These are deep psychological truths about humanity, carved into the cliff-face of how we construct our societies. The cudgels of morality will always be a convenient lever for hidden competition — you can pretend to be socialising the private realm, when in effect you’re privatising the social realm for your own status gain.

The problem is that we tend to see the dynamic for what it is only in its aftermath. In the moment, the mesmerism of ideology fills the screen entirely.

In order to stop the spiral:

Takes a much larger critical mass, hundreds, recognising the purity spiral’s signs and saying so.

The simplest solution is to notice earlier, to notice better, and to call it out as something that has nothing to do with morality, and everything to do with purity — and to say why that’s different.

If you call its name, it flinches. After all, the best defence against witch-finders is a population that doesn’t believe in witches.

Photo credits Pexels by Pixabay

1 Like

As far as purity for the sake of purity then yeah, it causes social problems because the nature of life is that things which cause the most concern as are almost always in the realm of nuance. Ethical considerations are never a question of a binary; they are about balance and often fall into choosing the lesser of two evils - no mean feat.

But some truth does not protect this piece from the logical fallacies it relies on. It’s just another tired piece of projection from my POV because these disputes aren’t naturally occurring. More often than not they are engineered.

I’d urge you to note the comments - do they look inclusive to you? Does this article grow the things it claims to want to grow or is it, in fact, achieving the opposite?

I cannot think of a single person who is not a white male; that is a self-declared raving fan of Jordan Pietersen and that tells me something. Likewise the comments indicate this article is motivated by ideology/; it’s doing the very thing it’s pointing fingers at other people for.

Nice to see you on the forum again, @Birdie! You are quite right in stating your reservations above. The specifics of the article are not all-that-important. They are (or easily can be in current climate) divisive in themself and hence be part of a spiral that way.

But I wanted to highlight the concept of Purity Spirals, so people are more aware of them and the effects they have on public discourse.

When it comes to polarisation it’s often through disingenuous acts but the majority of the time it is probably more accurately described as default binaries ie a kind of social simplification shorthand which is mostly benign but can quickly metastasise into something aggressive. Some people are simply in thinking habits of interpreting debate or discussion as a personal attack and that suggests emotional responses which have bypassed logic - a ‘straight to the amygdala’ thing.

I wish the author had exercised some self-analysis before publishing but the article serves as a good example really. Hyperbole and an emotional morality can be seen in the following snippets:
It isn’t a robust debate it’s an ‘ethical gazumping war’
It isn’t that some people use moral posturing to access the privileges of power it’s a “bidding war for morality turned into a proxy war for power”
All the violent language serves to create an emotional sense of extreme aggression where there may actually have been very little or even none - that affects our perception of the validity of the arguments because how we label things matters. If a discussion about respectful language gets framed as some kind of violence then how do we recognise the extremely aggressive language? If we exaggerate for effect we turn a valuable ratio scale into a much less meaningful ordinal scale, or worse the binary ‘violence/not violence’ even though we all know that poking someone in the ribs is not the same as punching them in the ribs, is not the same of punching them repeatedly in the ribs, is not the same as stabbing them in the chest… hopefully you get the drift that scale is important.

That really is, for me the meat and bones of what is important here; nuance matters. The shades of grey matter. If we are to organise for social benefit then we really do need to actively put careful consideration into when we use binaries and why because it’s crucial that we know an ordinal scale from a ratio scale and use appropriate statistical tests when we try to find meaning in them.
For the record, Likert scales are not true ratio scales because their measures are subjective…

In the context of flame wars it is, as you say, a matter of group culture but there is a volume dynamic. If you look at social housing you’ll find that communal areas are a fairly good measure of how much community spirit exists in the lives of the tenants who share that space. In a small building where only a handful of households have to share a stairwell and a laundry things remain civil and mutually supportive in spite of a tenant or two with profound mental health or social problems. When that number becomes bigger the spirit of community can quickly fall victim to the coalescing of bullies. If 25 households use a shared stairwell suddenly you find that the lives of everyone is impacted by the behaviour of the worst. To be honest this is also true of private housing but to a lesser degree because if tenants in private housing experience the pressures if extreme poverty they just become homeless…

in social media terms the magnitude is different but similar rules apply. I would guess the numbers to be group size < 2000 = manageable. group size > 2000 = problematic. Whether problematic becomes catastrophic or not is the core problem. My guess is that the really important decision making is around exposure risk. A focus on personal data collection is entirely unethical if this focus exists next to a void where duty of care exposure risk mitigation should be occurring, This has been a problem on fb for a long time and is now a risk on LI. In open source contexts this dynamic seems to be understood. Nicknames are welcomed and they do serve to strengthen community in my view. In spite of this assumption that it’s all about hiding one’s identity to do devious things it’s actually much more about compartmentalising. It is human to play different roles in different contexts and that’s a part of why us humans value privacy, it’s not about deceiving, it’s about different modes of operating.
Oh my wandering mind… time for sleep.

1 Like

Ps I just had a look at some of the comments in your link… these related to Ivermectin. In medical/scientific circles it’s well known that the reason posts about Ivermectin are considered as fake news isn’t that it’s deemed there is no efficacy; it’s that while it may be helpful there are other pharmaceutical treatments which are found to be much more effective. There was an amazing collaborative effort last year. The Solidarity Clinical Trial really was ground breaking and did a phenomenal job of collecting and analysing treatment data. Those who promulgate conspiracy theories give no credit to the enormous amount of work done for entirely ethical reasons by people who just wanted to do what they could to help.

This crystallises the frustrations of the polarisation. If you have one group that is actively politicising and deceiving (have a read through the chat thread on the Barrington delusion page and you’ll see there is no effort to correct fallacious comments or to vet whomever signs the thing); and the polar opposite is people who are aware of the seriousness of this virus and are actively trying to dilute misinformation because they know that every person sharing fallacious claims is amplifying the workload and risk of nurses then it isn’t fair to call that a polarisation because there are some fundamental social standards that can be measured in relation to each group… classifying them as polarised is like watching someone get beaten up by a partner and then talking about ‘the fight’ they had. That is mischaracterisation. I will be honest with you, it was seeing pandemic denying comments which made me wary of this space. I know too many people whose lives are deeply affected by this. I have no doubts that the politicisation has always been the minimisation of the threat and the undermining of the public health effort.

An accurate and eloquent analysis, much better than I could do.

There are other ways that the author might have explained the concept of Purity Spirals. Ways that wouldn’t be controversial in and of themselves. And more self-analysis would’ve been better indeed. But I’ve read through all that to focus on the gist article explaining the concept itself.

Your quoted texts reminded me to mention something else. Namely that Purety Spirals are now among the many kinds of ‘guided weapon systems’ that can be applied by those seeking to set and manipulate the narrative in public discourse by weaponisation of social media.

And here we have another weapon that is wielded all over the place in online media.

1,000% agree, and nuance is a very delicate flower, easily crushed, and that is exactly the objective of those seeking to divide use for their own gain.

I have extracted these snippets from you text, because I too strongly believe that we are entering ‘solution space’ when we start to think about how to achieve that in social media. Solution-orientation related to humane technology innovation is the primary focus of this forum, and I would like to leave very specific examples from ‘problem-space’ for what they are, unless they are directly technology-related.

I strongly believe that the concept of Community and the Relationships between communities are crucial for social media to work at scale. You mention a group size of about 2,000 people as still manageable, and that may well be. It all depends on the kinds of moderation and (self)governance tools that are in place. Dunbar’s number is often used to indicate the number of relationships with others that a person can manage themself.

As it happens online we can build communities in better ways, that are more representative of how they exist in real life. I am investigating this in the context of the Fediverse in a paradigm I call “Community has no Boundary”. The full monty should combine group dynamics and ‘social-at-scale’.

I digressed a bit from the topic, but communities can help avoid purity spirals, because in a community after a time there is a common understanding between people and a shared culture. What is particularly interesting is that it has a decentralized infrastructure where you have informal communities (based on server instances), and it becomes easier to both study and manage the Purity Spiral phenomenon.

Social media should be avoided. Hn is the same thing as reddit. It’s just a sounding board for these people in their filter bubbles and the moderators side with the woke each time. It should be called anti social media. Not everything needs comments. Not everyone’s opinion has value. The op assumes moral equivalence which isn’t something that I think exists. There are moral high grounds and low grounds. The problem comes from censorship, so these points can’t flush out naturally. The fact that people can’t be civil, it seems to be how social media is designed, it’s on purpose.

Like when Facebook finds out depressed teen girls look for makeup online, so they are pitched make up ads. It’s not designed to foster good dialog about anything serious (sex, religion, politics, things that make people upset).

I took a big step back in 2016 and I don’t try to understand the mentality of people who call names or are steeped in identity politics or social justice warriors.

Life is to precious to waste on this. I agree with the speech Sacha Baron Cohen made about social media, I don’t have a link but it’s spot on everyone should hear it / him.

Fascinating. I read a little quickly through the knitters article. First, it was badly written, too much hyperbole and pretentiousness in the tone of the article author. Instead of what I wanted (a run-down of what happened in a community I know nothing about), I got an article giving me all the conclusions along the way, telling me what to think about what happened (and before I really got the basic facts of what happened).

I’m not saying the author is wrong (but he might be somewhat, I dunno), but I was forced to grapple with his assertions before he presented evidence, which was itself sparse. I imagine he wrote it in order to preach-to-the-choir for an audience he imagined was already on board with a pretty complete rejection of anything that looks like “virtue signaling” and so on.

From what I can tell of the facts, the concerns are real. I mean, I know of and don’t deny the problems with simplistic over-the-top misguided forms of well-intended (at least by many) social justice online discussions. But I didn’t quite follow the narrative in this case.

There’s something about “selling patterns” in knitting, and IMO, this is already a pitfall that the community is liable to. This is like proprietary software and copyright in other ways. Selling access to artificially-restricted concepts like a knitting pattern is itself already introducing power dynamics. There’s going to be all these tensions around whether a pattern is truly “original” (none of them are, of course).

Hi @Broodwich, thank you for your feedback. I agree that most social media - as it is currently designed - should probably be best avoided. On the other hand if we’d like to learn from people from all corners in the world and discuss in groups, then the internet is best-suited and social networking is what we do.

I have to disagree with you about Hacker News. This maybe shows you are slightly mis-informed, or somewhat biased. In any case it is very different than Reddit. I know you have some strong anti-HN feelings, but maybe they stem from the early days of the network, when its members mostly reflected Silicon Valley mindset and culture. That is no longer the case, and HN has a broad and world-wide audience of techies now.

  • HN as you know is a link aggregator. The topic is basically anything that may interest the audience of technologists.
  • HN knows no sensorship. Downvoting & flagging might make things invisible, but with “showdead” setting you can see it again, and you can vouch to bring things back.
  • HN has a very simple and comprehensive set of guidelines, that forms the basis upon which moderation takes place.
    • It says for instance “Please don’t use Hacker News for political or ideological battle. It tramples curiosity.”
  • HN has a single active moderator, the user dang, who is the most humble, balanced and reasonable guy.
  • HN’s member base is quite diverse, with people from all opposite ends of the spectrum, and different backgrounds.
    • Is the audience dominantly male? Yes. This is unfortunately still the reality in tech, but things are rapidly improving.
  • HN discussion is based on providing reasonable arguments. Insubstantial reactions are either mostly ignored or downvoted.
    • Moderation algorithms detect flamewars and many other dark patterns (e.g. voting rings) and quickly downranks these threads.

No social networking app is without its flaws, and there are always people unhappy with them no matter how they are set up. I don’t agree with many comments I encounter, while others are greatly appealing to me. I find things relatively well-balanced, and for me most importantly, HN surfaces both most interesting and/or important topics that play out in the tech world. It does this better than any other source I am aware of.

Last but not least… what is funny is that many ‘old’ members of HN complain about the ‘anti-tech’ stance getting too dominant. There is a lot of criticism on Big Tech, for instance. I think this reflects the broader sentiment in society with people getting truly aware of the harms of technology. Participating on HN is a means to add substantive feedback, from a humane technology perspective, and bring it to the attention to many of the SV ‘hot shots’ that are also still member of the network. In other words: If you are an activist for humane technology, then membership of HN is a good way to act on that.

The reason I included the article was purely to highlight the concept of Purity Spirals. It was where I first heard the term, but the phenomenon was familiar to me. I see it all around me. As for the specific details and the way it was written I agree with you, but I couldn’t discuss the concept without attributing the source.

I have very deliberately quoted from the article to just distill the concept definition, how it manifests, what it usually results in, and how it may be stopped.


I have one example of purity spirals I see that relate to humane technology proponents. On the Fediverse most people are very aware of harms of Big Tech, and actively involved with activism, opposition and building better alternatives (mostly as free software). It is a David vs. Goliath fight, but with a common cause and “united we are stronger” credo, there’s a good basis for collective efforts and strong community, right?

But there are Purity Spiralists, factions that only serve to corrode the unity. They are the holier-than-thou ethics crusaders. So you may be a free software developer that worked tirelessly and investing your own savings to create some software for the common good. Then out of the blue a crusader come in and shouts “You use a third-party service that uses Cloudflare. Change this immediately or you are the devil”. And if you don’t comply they blocklist you and start an active PR campaign denouncing all your work.

Luckily these folks and their purity spiral do not get much of a foothold on the fediverse, as they are not well-liked with their overly radical approach. It is a good example where any shade of grey is not allowed to exist. You are either good or bad.

Very unproductive. As most of the world isn’t fully aware of the harms of technology and the urgency to bring change, we have to offer people a gradual path that allows them to ultimately “see the light”. The crusaders only create an extremist group, that turns most people off. Nothing much would change if we all followed that approach.

@wolftune I added a note to the first post to highlight why I attributed the article (i.e. definition of the concept of Purity Spirals)