Data Deluge — Striim and NonStop Manage Overwhelming Volume, Variety and Velocity of Data
The time for speculation is long gone. Predictions about the likely future challenges facing IT have become a reality, at least in one aspect. Simply put, we are facing a wall of data coming, rushing towards us and there are few places to hide. The pundits tell us that wisely used it is this data that will separate winners in business from losers. But is it possible to extol the virtue of water to a drowning man?
In Sydney’s famed Opera House, spread across a long curved wall in the vestibule at the rear of the concert hall, facing an amazing view of the harbor is a mural called Five Bells. However, to the locals it’s simply called the last thoughts of a drowning man as it drew inspiration from a poem of many years before by another artist that fell into the harbor and drowned. A nearby war ship evidently sounded five bells at the time the artist was drowning – hence the mural’s official name.
This past week I was driving to the ATMIA US Conference in New Orleans when the skies east of the city opened unleashing a deluge of water, I have not experienced anything like that in many years. Visibility evaporated and traffic was reduced to a crawl and it was a stark reminder of just how great the volume of data heading towards us has become. And the land quickly became water-logged!
Take just a few industries where the increase volume of data is becoming very apparent. This past holiday season ecommerce really took it to bricks and mortar retailers and from the data deluge there’s rising water where great waves are forming that show no signs of breaking any time soon. To the contrary, it is continuing to climb in height and the shoreline is nowhere to be seen. In the January 20, 2016, article Why brick-and-mortar retail faces a shakeout to the publication, Retail Dive, came the hews that, “‘We are right now in the middle of the biggest, most profound transformation in the history of retail,’ Robin Lewis, CEO of the Robin Report and a former executive at VF Corp. and Women’s Wear Daily.”
Furthermore, according to Retail Dive, “‘We’ve now gone to a business where your best customer can be standing in your best store and with three touches of their thumb to a piece of glass, they can buy from your biggest competitor,’ Fred Argir, Chief Digital Officer for Barnes & Noble, told Retail Dive in an interview. ‘That’s changed everything.’” To be fair, the publication also quoted Steve Barr, a partner and the US Retail and Consumer Sector leader at PricewaterhouseCoopers. “The great news is the retail store is not dead,” Barr said. “But the retail store that does not have a meaningful relationship with the consumer is dead.”
Meaningful relationships? Yes, a much better understanding of the behavior of consumers even as we determine the trends these consumers will likely embrace. This means tapping into data as its being generated and picking up the information gems we need as they pass by. And one sure data source clearly is the byproduct of the transactions customers initiate particularly should it involve your competition. I admit, I did hit “Purchase” button on amazon.com while looking at a book at Barnes & Noble, not only because the price was a tad lower but, more importantly, because the site provided the book’s reviews and more information!
When it comes to thumbing a piece of glass, the change under way from simply having a device from which to make a phone call to where communication is visual and global, is sapping resources from all over the digital footprint surrounding each and every smartphone. But the usage patterns that can be derived not only helps mobile phone operators to better tailor messages to individual subscribers but allows them to accumulate metadata on an unprecedented scale and it’s becoming extremely valuable to not just retailers or even bankers, but to industry as a whole.
Whether it’s the purchase of a new car or just an old pair of shoes, ensuring the right product is presented to a consumer motivated to buy it is paramount, but even so, making sense of so much data coming from each and every one of us makes the data deluge all the more difficult to process. Clearly, the former model of capture, store and process antiquated when the challenge lies in integration with the real time world of transaction processing.
With so much being written about the Internet of Things (IoT) and even the Internet of Everything (IoE) there is recognition that as more and more sensors come online the data deluge is going to become even greater. However, much closer to home for all of us – the human body – as we are beginning to hear about, is one giant collection of almost infinite sensors. Imagine the plight of the medical profession, including researchers of every discipline, as steps are taken to mandate real time processing of everything our body generates from its vast array of sensors?
The new model involving capture, process and store offers the only real way to make sense of it all. This is not to say that there remains little of value for data scientists to exploit when data is finally stored but rather, by industry and markets, data by necessity needs to be parsed with only pertinent data ever making it to storage. For the HPE NonStop community this is of growing importance, as NonStop systems are home to many of the most important real time transaction processing business applications on the planet.
Striim is beginning to gain a small foothold within the NonStop community, with its first deployments taking hold. And this is good news for everyone in the NonStop community as even NonStop users are finding it necessary to fight back to keep consumers engaged with their products and services. The data deluge is upon us and the waters are rising fast – bigger and bigger waves are breaking over companies old and new.
We may be in the biggest and most profound transformation in history and it’s not going away and it’s not a time to simply redirect the waters to lakes hidden from view. To survive we need to know our customers and we need to be sensitive to their ever changing behavior and doing so, even as they instigate a transaction, is critical. Drowning is simply not a viable business option!