Predicting the Future
The fictional science of psychohistory is predicated on the proposition that while human behaviour is erratic in isolation, when aggregated to population scale it becomes predictable. Today we use big data to predict customer behaviour and third wave economics uses real-time data to solve real world problems. In time this will allow us to understand the steps we need to take to shape desirable outcomes.
This article was first published in The Mint. You can read the original at this link.
As a lifelong fan of science fiction, I was thrilled to learn that Apple TV+ was bringing Isaac Asimov’s classic Foundation series to the small screen. I have to admit that my excitement was tinged with just a soupçon of trepidation, unsure as I was that anyone would be able to do justice to the vast multi-generational span of the storyline. But it was great that even an attempt was being made.
Psychohistory
Much of why the Foundation trilogy is so timeless has to do with psychohistory—the fictional science around which the plot of the entire series revolves. Using a combination of mathematics, history and sociology, psychohistory makes it possible for the protagonist, Hari Seldon, to predict the flow of future historical events with fine-grained accuracy. The science is based on the premise that, while no one can ever expect to predict what a single human is going to do, if you are modelling a large enough population, it is possible to describe the sequence in which future events will take place with unerring accuracy. The Galactic Empire in which the Foundation series is set has a population in excess of a quintillion people, allowing Seldon to accurately foresee its downfall and develop a plan to shape the course of these future events so that its worst effects could be mitigated.
The appeal of psychohistory lies in its believability. Even though it was conceptualized 80 years ago — well before big data and the miracles of modern data-driven innovation — Asimov intuitively zeroed in on the fact that while human behaviour might be erratic in isolation, when aggregated at population scale, it can become predictable.
Science fiction has always been prescient. Jules Verne wrote about space travel and submarines before anything even approaching the technology required to make this a reality existed. Arthur C. Clarke predicted satellite communication well before the first satellite was placed in geosynchronous orbit. Even Douglas Adams’ vision of a universal translator (the babelfish—a live fish that lives in your ear) long predated Google Translate.
Seeing how science fiction got all these things right, I’d like to think that it is only a matter of time before psychohistory becomes a reality.
Third Wave Economics
A few weeks ago, The Economist had a feature on a new revolution that it called the third wave of economics. Unlike the first wave that was largely driven by individual thinkers who wrote books and papers about a singular big idea, or the second wave that was slightly more experimental with new assertions supported by empirical studies, the third wave of economics is almost entirely powered by the voluminous availability of real-time data.
Tech companies have long been able to draw on the data they generate to predict customer behaviour. E-commerce firms use this information to promote products and ensure their private-label brands are hawked alongside items that are likely to be in strong demand. Streaming media companies use this data to green-light movies and TV series based on what is most likely to find favour with global audiences.
Third-wave economists use similar techniques, drawing on vast amounts of real-time granular data to explain real-world problems. By studying granular mobility data obtained from social media companies and telecom service providers during the pandemic, they were able to understand the impact of lockdown restrictions on disease transmission. By studying live data on the day-to-day movements of ships, they were able to figure out where the bottlenecks lay in supply-chain logistics. In the US, economists analysed live data from restaurant booking sites to gather evidence in support of a stimulus package for the industry.
Real-time Big Data
The availability of real-time granular data is only going to increase. Apart from the fact that more and more people are going online every day, a rapid acceleration in the volume and variety of wearable and Internet of Things devices has resulted in an exponential increase in the availability of new information. This real-time data is available on the cloud in easily accessible, inter-operable formats that are well suited for cross-platform analysis. And, as the available volumes of data increase, the accuracy of early trends that are identified is only going to improve.
While this may not yet be anything like Seldon’s psychohistory, we might already be able to see how, given time, it could develop into something along those lines. If economists can model the data generated by commercial transactions to forecast the behaviour of markets, it can’t be long before this data is used to predict social outcomes. With real-time data at their disposal, it should be possible to build tight feedback loops that constantly refine and recast these models on the basis of how they perform in the real world, with constant iterations helping them improve accuracy.
When we have all the data required to predict outcomes as well as the models needed to do so accurately, we will also have all the tools it takes to shape those outcomes in desirable ways. Even though, at present, predictions from these models are relatively short term, it seems entirely within the realm of possibility that with more data and improved models, it will soon be possible to see further into the future, predicting not just an immediately proximate response but also the sequence of events that will take place. Once that happens, it will only be a matter of time before there is little to distinguish third wave economics from psychohistory.
