Lies, damned lies, statistics, and developer experience rhetorics
Watching Rich Harris’ recent talk on “Frameworks, the web, and the edge” reminded me that I had this blog post in drafts. So here it goes. My apologies to everyone I might wrong here.
A while back, Andy Bell offered some thoughts on silent majorities and whether you should feel technology FOMO from “a very loud minority.” Specifically, to what extent do you, as a (web) developer, need to feel the pressure to learn React, Vue, or TypeScript? Because if you log on to Twitter and follow the trades, it seems everyone is using those nowadays, right?
Andy’s argument seems to be that you can safely ignore the pressure of learning these tools since, according to W3Tech’s statistics, React, and Vue only make up 4% of the web, the majority being WordPress, with a booming at least 43.6% of the top, publicly available, websites of the web.
In other words, if you’re following the discourse on web development on Twitter, you’re not hearing from most who are building the Web, which is a way to say that you don’t need to put that much weight on their opinions.
In Andy’s own words:
I’m not sure we can use these stats like this, and I’m not sure this is the way to fight gatekeeping either.
But first, we need to set the stage
I think it’s great and important that we have professionals like Andy, who advocate for the web platform as an accessible (in the broadest sense of the word) place to do work. But I also think it’s great to have educators like Kent C. Dodds (one of the people quoted without attribution in Andy’s blog post) who have done a lot to introduce people to the discipline of professional web development. Andy and Kent have different approaches and preferences regarding tooling and focus, and this diversity ultimately a great thing that enables the conversations that bring us forward.
And a little bit about me
I have been around on the web long enough that I was first introduced to style attributes and presentational HTML (looking at you
<center>) as the way to design a web page. I have used clear fixes and tables for the layout as the idiomatic way to do exciting stuff that got you noticed because it pushed the web beyond a boring document viewer for CERN engineers.
But more importantly, I have met thousands of developers who use these technologies to solve problems for their clients and customers and to earn a living. The kind of developers, like Andy points out, who aren’t necessarily blogging about bundle sizes or being loud on Twitter. So having established my ethos, let’s get into it.
Statistics are just numbers with a narrative
Nothing makes you doubt statistics as studying statistics does. You’ll find this quote by Benjamin Disraeli (made popular by Mark Twain) in every other statistics introduction book: “There are three kinds of lies: lies, damned lies, and statistics.” Statistics can help us understand landscapes and connections that are usually beyond what we can intuit, but numbers and graphs can be incredible rhetoric persuasive devices. Building your argument on statistics requires you to trust how these numbers were produced. One usually does that in the academic world by being very transparent with the data collection, methodology, caveats, and degree of uncertainty we’re dealing with.
My statistics professor used to say that a “proven” (by falsifying the null hypothesis with a mutually accepted degree of repetitive uncertainty) correlation is only interesting if it comes with a reasonable explanation for why it can occur (yes, I know this can be discussed, he was being didactic). I think the same can be said for sampling and framing of stats. If you want to use statistics to build your stance in an argument, you have to make a reasonable explanation for why the statistics are relevant and prove your point. I don’t feel Andy and similar takes do this. Yes, that goes for lighthouse scores too.
To be honest: I’m unsure what to conclude from W3Tech’s stats when choosing what technologies to recommend and prioritize when creating education and making topics for discussion and critique. If the W3Tech stats proves anything, it’s the adoption inertia of web technologies. That goes for both WordPress and React.
What is the W3Tech stats representative for?
According to W3Tech, they crawl the web by downloading the web pages and running them through static analysis using regular expressions and DOM traversal. They have built-in rules that allow them to assume server-side technologies as well. For example, if you can identify a WordPress site, you can safely assume that it’s using PHP on the server. This seems reasonable, although we don’t get much more insight since W3Tech is a commercial and proprietary operation that sells more fine-grained insights. In fact, it’s a 2-person company called Q-Success operating from Austria.
I think it’s reasonable to assume that their numbers are representative of the technologies that are used for the accumulative publicly available web. This means that these numbers tell a story about technology choices that were made up to several years ago, and only for the stuff that isn’t hidden behind login forms or is running in browsers but not on the open web. It’s still important, but it doesn’t tell the whole story.
These stats don’t tell us what’s expected from web developers when it comes to what proficiencies are mentioned in web developer job descriptions, everything that runs behind a login screen in the ever-growing personalized web, the wealth of internal web applications, and the aspirations of web developers looking what technologies they want to learn and put to use.
And if we chose to use these numbers to anticipate the technology choices of a silent majority, then we absolutely should be teaching and talking about React. Since it's become a core technology in WordPress’ block editor, Gutenberg. Or new WordPress adjacent technologies like Faust.js (also React). Of course, it can be discussed if web developers who use WordPress need to use it headless or build/customize the block editor, but you see where the arrows point.
Stories told by other types of statistics
What I find confounding with how the W3Tech’s stats were used is that they tell a different story than most of the annual surveys on web technologies trends. If you look across surveys like State of JS, Stack Overflow, GitHub, etc. Generally, they are all telling the same grand story:
- TypeScript is growing a lot, and people want to learn it
- React-based frameworks are dominating, but other frameworks are relevant too
Now, these surveys come with their own challenges regarding sampling biases, but collectively it’s reasonable to assume that they represent a significant portion of web developers. At least relevant enough not to be outright dismissed and ignored when thinking about what frameworks to discuss and make education for.
Again, I’m drawing on these numbers to make a point. The above stats are nothing to the 4-year-old introduction to Python with its 38mill views. Which tells a different story about what developers and programmers are into and what demand we see in the market. Does that mean that we all should be talking about Flask and Django instead?
Fighting gatekeeping with gatekeeping
Even though Andy isn’t mentioning it explicitly in the blog post, it can be read as an entry in a conversation about gatekeeping and heightening the bar to web development. I understand and support the desire to speak out against gatekeeping, but I often find that it’s done in a way that outright dismisses some legitimate and empathetic advice.
Beware of The Ideology of The Holy Web Trinity
If you want to make a strong case against popular technologies, it’s probably more effective to communicate an understanding of why these are used, and then provide viable alternatives. And not to undermine your audience: In most of my conversations with developers, I see curiosity towards things that are “hyped” co-exist with a healthy skepticism for what technology to put in production.
Having done developer marketing for five years, I know that the most efficient marketing is to show developers how to solve real problems without hyperbole. You’ll find much of that in the React ecosystem (take the marketing of TanStack). Even Vercel’s Apple-like marketing of Next.js is grounded in narratives of solving actual problems (and their style doesn’t escape critique either).