Krishna Sundarram

Please stop citing TIOBE

Please stop citing TIOBE

What is TIOBE?

The TIOBE Programming Community index is an indicator of the popularity of programming languages. The index is updated once a month … It is important to note that the TIOBE index is not about the best programming language or the language in which most lines of code have been written.

The index can be used to check whether your programming skills are still up to date or to make a strategic decision about what programming language should be adopted when starting to build a new software system.

Source: tiobe.com

A programming language index that helps us choose which language we should use. Sounds useful!

So how does TIOBE calculate this index? You might not believe this, but they count the number of search engine results for each programming language. Not the number of people querying, not the number of queries they’re making, not sentiment. It relies entirely on that useless number that search engines report. (Source: TIOBE Index Definition)

To give themselves a veneer of being data driven they don’t check just one search engine (google.com), they check several (bing.com, google.de, google.co.in and a few others). But 25 useless sources isn’t any better than 1 useless source.

XKCD programming language

Although it has 24.9 million Google results I would not recommend programming in this language.

Absurdities in TIOBE

So what kind of output do we see with this garbage input?

Let’s look at broad trends. Maybe that will allow us to make strategic decisions.

TIOBE 2016-2018

In 2016 and 2017, the trend lines for the top 2 languages in the index Java (green) and C (black) collapsed. Java dropped by 42%. C was even worse - 62%. What was happening, was the demand collapsing or the supply? No other language was gaining at their expense, so maybe we were seeing a software industry recession? Should we prioritise rewriting our Java apps?

Turns out, C and Java both rose dramatically in 2018, regaining their 2016 level. I guess Google was tinkering with their search algorithm. I hope no one made any decisions based on this data.

Maybe instead of trends we could look at the ranking of specific languages.

We could go on and on. We could ask why TypeScript languishes at #37, well behind Delphi/Object Pascal (#14) and Scratch (#21). But you get the idea, none of this makes sense except by accident.

Is it possible that even with completely wrong output, they’re still lucking into correct predictions? Again, no. In the TIOBE report from March 2012 they’re pessimistic about Go, a popular language in 2022. They also ranked Python low, a top 3 language today by any metric.

Curiously, you can’t find old reports with these bad predictions on their website.

Evaluating languages for projects

Fortunately there are better ways to evaluate languages.

For projects, look at technical fit.

Only your team can answer these questions. It depends on your business problem, your local jobs market, your engineering budget. If you’ve hired smart people, they can figure this stuff out.

To a lesser extent, your team needs to account for long term trends. The language must remain a good technical fit in future as well. It’s no good if you can find developers today but end up having to beg programmers to come out of retirement in a few years.

Predicting the future is hard. But you can take a look at

Evaluating languages as a developer

Unlike a project that needs to pick a language and stick to it, you don’t need to. If you attach your identity to a single language, drop that. You’re a developer, not a $language developer.

Grow your toolbox over time. Your favourite programming language can’t solve every problem you’ll encounter in future, so learn new ones periodically. If your toolbox only contains a hammer, everything starts to look like a nail.

If you’ve worked with garbage collected languages, try a language that uses manual memory management and vice versa. If you’ve mostly worked with imperative languages, try a language that uses functional constructs and vice versa. Analyse the languages you know and try one that offers something different.

But whatever you do, don’t make decisions based on the number of Google results.

A comment on Reddit pointed out previous articles criticising TIOBE. It also explained why an unknown language like Delphi is so high (#14) in this index. It’s because the Delphi community put a lot of effort into manipulating TIOBE rankings. Good for them.

Thanks to Chandra Sekar and Kushagra Sinha for reading drafts of this.