skip to content
Krishna Sundarram

Please stop citing TIOBE

/ 6 min read

tech

What is TIOBE?

The TIOBE Programming Community index is an indicator of the popularity of programming languages. The index is updated once a month … It is important to note that the TIOBE index is not about the best programming language or the language in which most lines of code have been written.

The index can be used to check whether your programming skills are still up to date or to make a strategic decision about what programming language should be adopted when starting to build a new software system.

Source: tiobe.com

A programming language index that helps us choose which language we should use. Sounds useful!

So how does TIOBE calculate this index? You might not believe this, but they count the number of search engine results for each programming language. Not the number of people querying, not the number of queries they’re making, not sentiment. It relies entirely on that useless number that search engines report. (Source: TIOBE Index Definition)

To give themselves a veneer of being data driven they don’t check just one search engine (google.com), they check several (bing.com, google.de, google.co.in and a few others). But 25 useless sources isn’t any better than 1 useless source.

XKCD programming language

Although it has 24.9 million Google results I would not recommend programming in this language.

Absurdities in TIOBE

So what kind of output do we see with this garbage input?

Let’s look at broad trends. Maybe that will allow us to make strategic decisions.

TIOBE 2016-2018

In 2016 and 2017, the trend lines for the top 2 languages in the index Java (green) and C (black) collapsed. Java dropped by 42%. C was even worse - 62%. What was happening, was the demand collapsing or the supply? No other language was gaining at their expense, so maybe we were seeing a software industry recession? Should we prioritise rewriting our Java apps?

Turns out, C and Java both rose dramatically in 2018, regaining their 2016 level. I guess Google was tinkering with their search algorithm. I hope no one made any decisions based on this data.

Maybe instead of trends we could look at the ranking of specific languages.

  • On any passable metric like Github PRs, or StackOverflow questions, or developer surveys or job postings JavaScript is the #1 language. According to TIOBE, JavaScript (#7) is smaller than Visual Basic (#6).
  • Now you’re probably wondering, why is Visual Basic so high? It didn’t used to be. March 4, 2020 - 0.77%. But when the world was entering lockdown, VB exploded in popularity. 29 days later it had grown by 6x, reaching 4.72%. It has remained ascendant since. (By the way, no other source is able to back this up. Not Google Search trends, or StackOverflow questions or anything else. Only TIOBE thinks this happened.)
  • To put that absurdity in context, Visual Basic is more than twice as large as Swift (1.27%) and Objective-C (0.94%) combined. The entire iOS, iPadOS, watchOS, macOS ecosystem is apparently half the size of the mighty Visual Basic ecosystem.
  • Assembly language (#8) is either too high because no one programs in Assembly or too low because isn’t all programming ultimately Assembly? Either way, the ranking is wrong.
  • Sadly, the market for Logo (#48) programming seems way down. Back in it’s heyday, it was as high as #21 on TIOBE. This is the programming language that involves moving turtles across the screen.

We could go on and on. We could ask why TypeScript languishes at #37, well behind Delphi/Object Pascal (#14) and Scratch (#21). But you get the idea, none of this makes sense except by accident.

Is it possible that even with completely wrong output, they’re still lucking into correct predictions? Again, no. In the TIOBE report from March 2012 they’re pessimistic about Go, a popular language in 2022. They also ranked Python low, a top 3 language today by any metric.

Curiously, you can’t find old reports with these bad predictions on their website.

Evaluating languages for projects

Fortunately there are better ways to evaluate languages.

For projects, look at technical fit.

  • Can developers be hired/trained to use it?
  • Will they be happy using it?
  • What will our development velocity be on this language? What will be the operational burden of keeping it running in production?
  • Do we need to rely on third party code? Does that code already exist and what is it’s quality?

Only your team can answer these questions. It depends on your business problem, your local jobs market, your engineering budget. If you’ve hired smart people, they can figure this stuff out.

To a lesser extent, your team needs to account for long term trends. The language must remain a good technical fit in future as well. It’s no good if you can find developers today but end up having to beg programmers to come out of retirement in a few years.

Predicting the future is hard. But you can take a look at

Evaluating languages as a developer

Unlike a project that needs to pick a language and stick to it, you don’t need to. If you attach your identity to a single language, drop that. You’re a developer, not a $language developer.

Grow your toolbox over time. Your favourite programming language can’t solve every problem you’ll encounter in future, so learn new ones periodically. If your toolbox only contains a hammer, everything starts to look like a nail.

If you’ve worked with garbage collected languages, try a language that uses manual memory management and vice versa. If you’ve mostly worked with imperative languages, try a language that uses functional constructs and vice versa. Analyse the languages you know and try one that offers something different.

But whatever you do, don’t make decisions based on the number of Google results.

A comment on Reddit pointed out previous articles criticising TIOBE. It also explained why an unknown language like Delphi is so high (#14) in this index. It’s because the Delphi community put a lot of effort into manipulating TIOBE rankings. Good for them.

Thanks to Chandra Sekar and Kushagra Sinha for reading drafts of this.


Check out posts similar to this one