The Cycles of Technology

We believe that internet platforms are too dominant to fail. But this is not true when you think of the evolution of internet search and curation - suggesting that technology evolves in cycles. We may be on the cusp of a major cyclical transformation - in which case we may challenge the notion that Big Tech is too big to fail.

This article was first published in The Mint. You can read the original at this link.

There is a widely held belief that internet platforms are too big to be dislodged from their current positions of dominance. Many believe we have no option but to apply the full force of competition regulation to protect customers from harm arising from their size and market power.

But are these platforms really too big to fail? Or are we in the midst of a cycle that has already repeated itself and will continue to do so in the future?

When I first accessed the internet, it was through VSNL’s Gateway Internet Access Service that offered a 9.6 kilobytes per second connection over notoriously spotty dial-up lines. Even under those constraints, it was evident to early adopters like me that the internet was a vast storehouse of extraordinarily diverse and useful content.

At that time, the World Wide Web was little more than a collection of ‘html’ pages loosely linked to each other. This meant that it was really hard to find information you needed unless you knew the URL of the specific internet page on which it was stored.

I have previously written about Archie, the world’s first internet search engine, that attempted to solve this problem by indexing webpages by title. Early internet browsing involved trawling through Archie indexes, hoping to find relevant information by decoding the contents of a page from its title.

The limitations inherent in this approach prompted organisations like Yahoo and Alta Vista to invest heavily in curation. Armies of librarians were hired to organise the internet. They personally visited hundreds and thousands of websites to manually sort them into a hierarchy that could lead to more effective search results. This was how we navigated the web when I first went online. But it was already evident that given the rate at which the internet was growing, it would very soon be impossible for human curation to keep pace.

In 1996, news began to filter through the academic grapevine of a magical new search engine that could generate highly relevant results by ranking pages based on the importance of its back-link data. When I first used Google, it was as magical as I had been promised it would be. I was able to access more relevant information than Yahoo or Alta Vista had ever been able to suggest. So confident was this upstart search engine in the accuracy of its need-satisfaction that it had a button on its search page that bypassed search results and took you straight to the website of its top-ranked listing. And you were rarely disappointed.

Since then, search has been our primary means of accessing the internet. But even though it has served us well for over two decades, I have of late begun to sense something lacking in its quality and completeness. In the face of diverse and often contradictory sources of information, I have begun to find it increasingly difficult to locate content I can trust.

Curation

This has forced me to turn, once again, to curation. WireCutter is now my first port of call for product reviews—though I often head straight to even more specialised websites for things I am passionate about—DPReview.com (for photography equipment), HeadFi.org (for high-end headphones) and wholelattelove.com (for coffee).

As good as algorithmic recommendations often are, I prefer to find new artists from playlists manually assembled by friends who share my taste in music. I have stopped relying on online book recommendations, unless they are from reading lists of people I admire or have been featured on a podcast I subscribe to.

It’s got to the point where I am more confident of getting a sufficiently diverse range of relevant viewpoints on a given issue from authors on Substack and the discussions on Reddit than from generic search results. Benedict Evens summed this all up perfectly in a tweet… an idea that many others have since expanded upon.

Centralisation

It is an inescapable truth that technology evolves in cycles, often swinging like a pendulum between extremes. If it feels as if the technology behemoths of the day are unassailable, it is only because we cannot yet see the upstarts lurking just around the corner who are going to be their downfall.

Take the internet itself. Even though it was originally designed to be open, our access to it is today almost exclusively facilitated by services that determine what it is we get to see. All the content, commerce, entertainment and social connections we consume has been pre-packaged into endlessly scrolling feeds of information in an online experience that is a far cry from the original open vision of the internet.

I mention this not to denigrate those platforms or decry the current state of the internet, but because I believe the pendulum has already begun to swing in the opposite direction. As much as the last two decades have been about the centralisation of the once-open internet in the hands of a few, over the past five years, it is impossible to ignore the decentralised solutions—blockchain-based services and decentralised autonomous organisations—that have sprung up as a counterpoint to this narrative of extreme centralisation.

While I broadly agree that we must have appropriate regulations to protect consumers from harm, I am not yet convinced that we need to do so because Big Tech has grown too big to fail. To the contrary, if history is anything to go by, we are probably on the cusp of a major cyclical transformation.

After all, current technologies have always been replaced by the next best thing.