Inside The World¡¯s Best Weather Forecasting System And How It Unknowingly Impacts All Of Us

It won't be completely wrong to call Cameron Clayton the world¡¯s foremost weatherman even if it¡¯s half in jest. However, what he does is no joke, and his message is certainly no laughing matter.

Behind a cheerful face and mild-mannered persona, you wouldn¡¯t imagine the magnitude of power and responsibility concentrated in Cameron¡¯s hands. It¡¯s mind boggling -- knowing what he knows, doing what he does, and its impact on millions of people¡¯s lives around the world. There¡¯s no mistaking the fact that he takes his work -- his mission -- very seriously.

¡°When it comes to weather forecasting, it all begins with data. You have to collect as much data as you can possibly collect of the atmosphere, that 100 kilometre high air bubble which surrounds our planet,¡± explains Cameron Clayton, General Manager, IBM Watson Media and Weather.

The insane logistics of weather data

It all starts with data. Bits and bytes, zeroes and ones. Cameron makes it sound so casual, so simple. It isn¡¯t until you dive into the different types of data being collected, the devices it¡¯s collected from, how they¡¯re scattered all around the world, and the sheer logistics involved in all of it that you start feeling your head hurt and jaw drop at the same time.

¡°We get information about our atmosphere from satellite networks hovering above the earth. On the ground, there are 200,000 personal weather stations all over the world that send their weather data to us. We partner with governments for their government sourced weather data. Our partnership with Boeing and Airbus allows us to receive sensor data from over 50,000 flights in the air flying around the planet every single day,¡± Cameron pauses to catch his breath.

¡°From weather balloons in the sky to buoys in the oceans, innumerable shipping containers fitted with sensors give us temperature readings both inside as well as outside the container. Trucks, trains, and cars all have sensors, and they become our data source.¡±

Try imagining it this way: wherever you're sitting right now, the exact latitude and longitude position, there¡¯s a cone going straight up into the air for 100 kilometres to the edge of space. And there's 2.5 billion such cones that The Weather Company tracks round the clock on and around the earth, all of which are interconnected in a giant web, according to Cameron.

In short, anything that moves across the land or water on earth¡¯s surface and floats through the atmosphere is a sensor network that Cameron wants to partner with. And he tells me how IBM¡¯s about to launch a tool to sense the weather around us through our own smartphones. You see, our smartphones have a barometric pressure sensor in them, and pending our permission, Cameron and his team will soon be able to collect the barometric pressure readings from all our smartphones -- with our consent, of course, he¡¯s quick to emphasize to allay any privacy fears. This will add a few hundred million sensor beacons into the mix, making weather predictions even more accurate than present.

¡°We have the largest set of observations in the world at any given time about the entire planet. And that's our strength, it¡¯s what separates us from others who try to forecast weather, as we're looking at this with a truly global perspective,¡± Cameron mentions proudly about The Weather Company.

Herculean exercise of data crunching

Collecting as much data as you can is just one half of the puzzle, when it comes to weather forecasting. If you can¡¯t crunch all that data quick enough, in a matter of hours if not minutes, all that data is as good as useless.

Most existing weather forecasting services outside the United States and Japan approximate weather over an area of 10-15 km every 6 or 12 hours, but Cameron¡¯s company is the only one in the world that can do global weather reporting for smaller 3 km blocks every hour, thanks to the exhaustive data they collect and a special system that crunches all of it in record-breaking speed.

¡°In order to pull this together and make all this work, our friends at IBM Systems helped us build a brand new, first-of-its-kind graphical supercomputer that's GPU based, called the IBM Global High-Resolution Atmospheric Forecasting System (or IBM GRAF),¡± says Cameron. ¡°It¡¯s something like the highest end gaming machine that any gamer can ever dream of on super steroids. Basically, it takes all this data, performs trillions of computations and puts it into a model over the whole world of what the atmosphere looks like,¡± he says.

On an average day, where there aren¡¯t a huge number of severe weather events around the world, ¡°We'll do about 35 billion data requests on the IBM hybrid cloud in just one day,¡± according to Cameron. ¡°These 35 billion requests -- which come from all kinds of gadgets ranging from smartphones, cars, air conditioners, and more -- leads to about 12 trillion transactions being processed on the IBM Cloud and GRAF for timely and accurate weather information for any location in the world.¡±

And this entire chain of events is only about informing us of the present, of weather conditions right here right now -- since it¡¯s based on observations near your location, and what's happening to the weather in that location at that time. What about predicting the future? It all depends on the quality and quantity of the present data, according to Cameron.

¡°The better our present parameters are with respect to data, the better we're able to predict what's going to happen in the future,¡± he says. This involves using complex mathematical and physics models. ¡°There's over 200 different weather models in the world, and we actually have all 200 of them, and we do what's called ensemble modelling on top of them all. So we run all the models all the time, and then we take the best one for your location to deliver the answer (weather update) to you.¡±

So IBM GRAF, at any given time, uses the hour before and the previous 48 hours to decide which model is the best model to use for your location for the next weather forecast. Then the forecast basically goes out from 0 to 72 hours in an hourly time frame and from 72 hours to 15 days in a daily time frame. ¡°But it's updated every hour,¡± Cameron explains. ¡°And the beauty of GRAF¡¯s weather modelling is it runs the model for you when you request it -- that¡¯s what¡¯s called on demand forecasting, as opposed to off the shelf forecasting done by others which isn¡¯t very useful. If everybody in your building made a query about the weather at their location, it would run the model each time for each person. And this is done to make weather reports hyper specialised and specific to you.¡±

Why all of this matters

From the air we breathe to the food we eat, from the clothes we wear to where we hang out, weather affects all of us -- and that¡¯s putting simply, of course. And increasingly it seems the only certainty we have about our planet¡¯s weather and climate is its growing uncertainty. ¡°Climate change is real,¡± says Cameron. ¡°Every scientist who¡¯s studied this agrees our planet¡¯s climate is changing in a very volatile way.¡±

Our earth¡¯s warming at an unprecedented rate, greenhouse gas emissions are at an all time high, polar ice caps are melting at a much faster rate than expected, with imminent sea level rise impacting cities from Miami to Mumbai. Here in India, we can¡¯t say we haven¡¯t witnessed a change in the climate -- a change for the worse. Summers are getting stretched, with each year hotter than the last one, and monsoon showers are getting erratic -- with the rising frequency of highly torrential, extremely concentrated rainfall occurring in short bursts which leads to flooding more often. All of this directly impacts how we live, whether you realize it or not, but that¡¯s not all.

According to the international non-profit Climate Disclosure Project¡¯s June 2019 report, over 200 global companies worth $17 trillion in market capitalization have valued the climate change risks to their businesses at $1 trillion within the next five years. As many as 16 Indian companies -- Infosys, TCS, State Bank Of India, Tata Motors, Wipro and others -- are part of this report, voicing similar concerns about the detrimental impact of climate change to their businesses. They mention everything from water shortage, crop failure due to floods, unseasonal rains, cyclones and other natural disasters as immediate risks that need to be dealt with.

¡°We also commissioned a survey on climate change in India, and a couple of things stood out to me,¡± Cameron tells me. ¡°As many as 94% of Indian business leaders believe that climate change is going to have a material impact on their business in the next couple of years, and 7 out of 10 Indian citizens told us that they had personally experienced more severe weather as a result of climate change.¡±

By bringing IBM GRAF and its unrivalled on-demand, high-precision weather mapping service to India and other parts of the globe, Cameron thinks they¡¯re at least taking a step in the right direction towards combating climate change. Having said that, he¡¯s also acutely aware of his own limits as an individual and IBM¡¯s influence as a company.

¡°We are in the trust business, we do get tested every day. We're not perfect, but we are the best of the breed,¡± Cameron acknowledges. ¡°Getting from 80% accuracy in weather prediction to an average of 90% is going to take public private partnerships. It's going to take, for example, the cube satellite companies putting thousands of satellites up into space; it's going to take Mahindra and others building automated drone tractors and getting them across farms; it's going to take so many actions and innovations along the way. We need entrepreneurs, government leaders, and the private sector to come together to help solve this problem. IBM can't do all of that.¡±

Apart from GRAF¡¯s weather prediction and forecast, another way IBM¡¯s leading by example is they want governments around the world to tax them with a carbon tax -- similar to Apple, Google, and Microsoft¡¯s recent commitments -- which Cameron believes to be a step in the right direction. It¡¯s what Cameron feels more companies need to do, ¡°Lock arms and do something as opposed to just be on the sidelines.¡±

Doing something about it all

Predicting the weather accurately still remains one of the hardest science and math problems in the world, given its applications and variables in play. Even if we reach a point where we get near perfect weather predictions for days and weeks in the future (how we will ever achieve that is beyond me, because climate change will increasingly put a spanner in the works), it won¡¯t amount to anything if we don¡¯t use that information to make better decisions that benefit all of us. And Cameron is acutely aware of this fact more than anyone else.

¡°What we are doing is predicting the future, which is really, really hard to do. Based on that, our whole mission is to help people -- individuals, businesses, governments -- make better decisions,¡± says Cameron. ¡°We've got pretty good at forecasting, and we¡¯ll continue to improve our odds, but what we need to do a better job of is recommending related actions or decisions. And we see this fairly often across the world. We've gotten pretty good at predicting flooding, for example, but what do you do when there's flooding forecasted? How can you change your behaviour? How do you change the school¡¯s or local company¡¯s behaviour? How do you change the state government¡¯s behaviour? What do you do differently as a decision maker, as a leader, to ride the storm and stay unaffected? No doubt there's a huge opportunity to keep doing better in terms of science and forecasting, but there¡¯s an even bigger opportunity out there in educating people how to make better decisions related to weather and climate.¡±

Cameron talks about climate activism and how whether it¡¯s US or India, ¡°there isn¡¯t a single young person¡± out there that doesn¡¯t feel strongly about climate change as an important issue to tackle urgently. He remains extremely bullish that we can still make a difference regarding climate change provided we all act. Act now.

¡°Hopefully, if we do a carbon tax credit, if we plant more trees than we cut down, if we put big regions of the ocean as global parks which prohibits commercial fishing -- if we do these three things, it will fundamentally change the entire planet for the better,¡± argues Cameron.

While we¡¯re doing that, Cameron feels we also need to ensure that access to high-quality, precise weather forecasts is something that everyone has -- much like the internet. ¡°I genuinely believe that every person in the world deserves accurate, high resolution information about their environment almost as a human right, so that they can make better decisions based on it,¡± he says, suggesting how historically it¡¯s been something that only the richest people in the world have enjoyed and benefited from.

¡°Because the poorest people in the world are the ones that are disproportionately impacted by climate change and by severe weather, and we want them to have a chance to get out of the crazy cycle and better their lives.¡±

Can¡¯t argue with that logic, as I end my conversation with Cameron, hoping against hope that his forecast on our climate change fortunes ends up being 100% accurate.