For the last half-dozen years, the buzz about Big Data has been pervasive, as it has been touted as a crucial part of any data analytics- dependent enterprise, be it in the commercial, government, or even consumer sectors. It’s not all useless hype or empty promises, as Big Data’s sheer volume of information is invaluable in making informed decisions and creating realistic, rational policies.
But all of that information requires a means of processing, and that’s how come any decent discussion of Big Data must include Hadoop. Which begs the question: is Hadoop now part of the mainstream?
A Little Refresher
Though most people know what Hadoop is by now, let’s review anyway. Hadoop was created by Apache Software, and is a free software framework that consists of four modules. It is written in Java and is used to process huge data sets (in other words, Big Data) in a distributed computer environment.
Hadoop’s Increasing Popularity
According to figures pulled from Information Week, as of March 2015, only 4% of companies surveyed use Hadoop on a regular basis, and an additional 18% use it in a limited capacity, a modest increase above last year’s 3% and 12% respectively. Furthermore, another 20% say they plan on using Hadoop in the future.
So yes, there are gains being made, but it’s hard to argue convincingly that Hadoop is going mainstream yet. Even if the full 20% of the “I was gunna..” respondents do in fact follow through by sometime next year, that still means that almost 60% have no real interest in using Hadoop in any way, shape, or form.
That’s hardly inspiring.
What’s The Delay?
Here’s a good reason, brought to light in the article “Dell Teams Up with Syncsort to Make Hadoop Adoption Faster, Easier, and Cheaper”, ” … getting Hadoop started can be enormously expensive and time-consuming, mostly because it takes a lot of skill, much of which isn’t readily available on the open market.”
In other words, sure, Hadoop is great at processing this geyser of information that, in and of itself, is also great. But how best to monetize it?
Furthermore, there’s still the unshakeable impression that Hadoop still has no definitive identity. It has no real boundaries, no standardization, and no overall, unifying image. This lack of a clear picture tends to make mainstream businesses, which are notoriously slow to embrace change, even less willing to drink the Hadoop Kool-Aid.
What’s To Be Done?
Aside from the obvious “take some time to define yourselves!” advice, what can Apache do to increase adoption rates? Reducing the degree of risk of money, time, and resources that comes with adoption is a smart strategy, and that can be achieved in part by alliances with big names in the tech and software industry. Such partnerships can help increase Hadoop’s exposure, hopefully in a form that businesses don’t find intimidating.
Lastly, Apache must just have to reconcile itself with the idea that maybe Hadoop just isn’t for everyone, and will never get that near-universal acceptance enjoyed by so many of the other tech innovations of the past decade.