The Threat of Big Data for Risk Management

We’ve all heard that due to climate change, humanity is on a collision course with the planet. Today I want to talk about why humanity is on a collision course with the data that could actually help us save lives, and what can be done to fix it.

• Every day, the U.S. generates more than a quarter of a billion weather data points [1].
• This rate is only increasing as private companies, public agencies, and universities deploy their own sensor networks at an accelerating rate, each capable of collecting even more data more precisely than before.
• What this means is the gap between data generation and data utilization — or, that data being put to good use, is broadening. Today, thousands of companies and millions of citizens already do not understand or take advantage of the data that’s available to them.
• Private companies are outpacing the government in terms of mapping and web technology, both of which are foundational to communicating risk data in the 21st century.
• This strain will only increase as private industry moves into space with nano-satellites that provides weather data that rivals government satellites.
• With the rise of Big Data and these sensor networks, humankind is attempting to funnel a waterfall of privately- and publicly-gathered information into a public consciousness that’s already jammed full of messages and events.

These trends threaten to cause more confusion, not less. Our capitalism has cultivated a highly fragmented market, but decision-makers from individual citizens to corporate executives to the President of the United States will always need holistic views that excel in brevity, clarity, and relevance.

Therefore the greatest advances in the next 10 years with regards to public safety in the face of severe weather and climate change will be those that merge a proper understanding of severity with a proper management of psychology. To continue to depend on current systems for relating this information to the public is quite literally a disaster waiting to happen. You don’t have to go far to find damages that could have been avoided if people had understood what was at stake.

But people are overwhelmed. They get too many messages already. Too many alerts. Too many false alarms. Too much email. And so we as citizens are on an endless quest for simplicity.

Unfortunately, that quest for simplicity can cause us to put on blinders, or tunnel vision, and fixate on the wrong things. Specific pieces of data — the line in the hurricane forecast, how I remember the last storm — or sound bytes that cross over into my radio or TV. Then despite all the available data, people still make “gut decisions” — they’re going on a feeling based on whichever data they think is true.

So we see a parallel between severe weather and climate change: it isn’t enough to just say data is available. No; the greatest disasters happen when people don’t _understand_ the data.

What’s the solution?

We’ve got to innovate in the areas of data comprehension and information perception. We need systems that can ingest, tame, synthesize, and communicate this flood of data at the same rate it’s being created. It means newer, more visual ways of communicating. Fewer, well used words over jargon and noise.

It also means new business models. How often do citizens depend on life-saving information from news and media sources that thrive on ratings?

And that brings me to Stormpulse.

Stormpulse started out as a personal quest to create a site which could serve the needs of citizens without the hype and sensationalism or commercial breaks of the national media, but in highly visual and web-enabled ways.

Fast-forward 7 years, and we’ve built a company with a mission of serving businesses with the best available decision-support tools for their daily operations. During Hurricane Arthur, the owner of several hundred rental properties in the outer banks was able to use our enterprise product, Cygnus, to instantly visualize, assess, and download the latest data on risks to his beachfront homes.

Not everyone needs that level of sophistication. But everyone needs clarity. Which makes me very excited to offer our new free service on Stormpulse.com, fueled by official government data, analyzed and presented by our technology, free of charge to emergency managers, scientists and researchers, and the general public.

There are many weather maps out there, but there are several things going on with ours which are uniquely aimed at solving the problems caused by Big Data.

First is what we call the risk score. The score ranges from 1 to 25 and comes in three colors: green, amber, and red. As conditions increase in severity, the score rises. This score is unique for each city on the Stormpulse map.

How does it work? For every location, we’re analyzing an ever-increasing firehose of severe weather and hurricane data from official, public sources. And this analysis goes beyond the edges of the county, state, and rectangular outlines used for the watches and warnings issued by the National Weather Service. It takes into account the probabilistic forecasts for each location whether you’re in one of those zones or not. Because of this, we can show people that not all storms are the same – the storm on the horizon may contain destructive hail or high winds in contrast to the one that passed over yesterday. It’s the same technology used by our enterprise clients to assess risk to their facilities, now offered on a single, interactive map for over 100 major cities in the U.S. and soon the Caribbean.

The risk score is a major step towards solving the Big Data problem because it acts as the focal point of a very large funnel of data that is already impossible for any person to digest and growing by the day. If you’d like to see where the risk score could go in the future, you can check out one of our developer’s experimental projects here: U.S. Risk Index. The potential is enormous.

Finally, because our business depends on the success of our subscribers in the face of these threats, our organization will succeed or fail based on the honesty of our messages. If we create hype for the sake of hype, everyone suffers, including us. Therefore, along with the launch of free Stormpulse we are making a public commitment to honest forecasting: Stormpulse will always be a 100% No Hype Zone.

Our team continues to improve our products on a daily basis but we know we have a long way to go. You can send us your feedback at any time by emailing us at contact@riskpulse.com or tweeting us at @_Riskpulse.

1. This is actually very conservative. In reality it’s a much larger number which likely no one knows.