background preloader

Big Data Right Now: Five Trendy Open Source Technologies

Big Data Right Now: Five Trendy Open Source Technologies
Big Data is on every CIO’s mind this quarter, and for good reason. Companies will have spent $4.3 billion on Big Data technologies by the end of 2012. But here’s where it gets interesting. Those initial investments will in turn trigger a domino effect of upgrades and new initiatives that are valued at $34 billion for 2013, per Gartner. Over a 5 year period, spend is estimated at $232 billion. What you’re seeing right now is only the tip of a gigantic iceberg. Big Data is presently synonymous with technologies like Hadoop, and the “NoSQL” class of databases including Mongo (document stores) and Cassandra (key-values). But there are new, untapped advantages and non-trivially large opportunities beyond these usual suspects. Did you know that there are over 250K viable open source technologies on the market today? We have a lot of…choices, to say the least. What’s on our own radar, and what’s coming down the pipe for Fortune 2000 companies? Storm and Kafka Why should you care? Drill and Dremel

What is Big Data, Hadoop and HDFS: 3 Must Watch YouTube Videos I received great response to one of my previous post titled 5 Must Watch YouTube Videos on Big Data and many of you reached out to me requesting for more such blog posts. Well, your request is granted. Here are three great videos on what is Big Data, Hadoop and HDFS. Hope you enjoy them and do let me know if it is useful. Based on your comments and feedback, I will share more such great resources on Big Data. So either leave a comment in comments section of this post or better still, tweet your comment, mentioning my twitter ID @HKotadia and I will surely reply to you. Enjoy and learn from these videos (double click on videos for full screen view). 1. 2. 3. Salary/Income of Analytics/Data Mining/Data Science professionals The big demand for analytics, data mining, and data science professionals led to a significant jump in their salaries in 2013. Salaries are highest in US, Australia/NZ, and among self-employed and industry data scientists. See details. The 2013 KDnuggets Annual Salary Poll asked What is your annual salary/income range (in USD) and employment ? The big demand for data scientists led to a significant jump in overall compensation in 2013, including 12% increase globally, 13% increase in US/Canada, and 29% increase among self-employed. About 80% of the respondents were from companies or were self-employed. US and AU/NZ lead in salaries, with Western Europe and Middle East forming a strong second tier. The following graph shows the overall landscape of analytics compensation: Graph 1: Analytics Salary/Income by Region and Employment type excluding students and unemployed. Table 1: 2013 Annual Income/Salary by Employment type In Table 2 we break down the data by region.

Top Big Data VC Investors Companies in the Big Data industry have received close to $5B in VC financing in the last few years. We look at the top VC firms that invest in Big Data. Companies in the Big Data industry have received $4.9 billion in funding since 2008, with the bulk of financing in 2011 and 2012. According to CB Insights blog, here are the top 10 venture companies that invest in Big Data, and the number of their unique investments in Big Data related companies: SV Angel, 14 Big Data companies, including Quora, Dropbox, bitly Sequoia Capital, 13 Big Data companies, including Meebo, LogLogic IA Ventures, 13 companies, including Datapop, Datasift, MemSQL, Metamarkets, Recorded Future New Enterprise Associates, 12 companies, including 10gen, 23andMe, AddThis, AeroSpike, Coursera. First Round Capital, 10 companies, including AsterData, Curalate, Gigya, Gnip, memsql, StumbleUpon, and the ClimateCorporation. Khosla Ventures, 8 companies, including Ayasdi, ClearStory Data, Kaggle, Metamarkets, bitly

Big Data Vendor Revenue And Market Forecast 2012-2017 Contributing authors: Jeff Kelly, David Floyer, Dave Vellante, Stu Miniman Original publication date: February 19, 2013 Introduction The hype surrounding Big Data, which showed no signs of abating in 2012, now has big dollars backing it up. Factory revenue generated by the sale of Big Data-related hardware, software and services took a major step forward in 2012, growing by 59% over 2011(a). The total Big Data market reached $11.59 billion in 2012, ahead of Wikibon’s 2011 forecast. Growth Drivers and Adoption Barriers The growth rate of Big Data revenue in 2012 was due to a number of factors, including: In the enterprise space in particular, the combination of a better understanding of the use cases for Big Data and more mature product and service offerings resulted in a significant percentage of Big Data early adopters graduating from small, proof-of-concept projects to large-scale, production-level deployments. Big Data Vendor Revenue The complete list is below: Methodology

Forty Vendors We re Watching: 2012 For a third year we bring you our list of up and coming vendors on our radar that are doing their part to shape the groundswell in information management technology in the 21st Century. All Information Management articles are archived after 7 days. REGISTER NOW for unlimited access to all recently archived articles, as well as thousands of searchable stories. Registered Members also gain access to: Full access to information-management.com including all searchable archived content Exclusive E-Newsletters delivering the latest headlines to your inbox Access to White Papers, Web Seminars, and Blog Discussions Discounts to upcoming conferences & events Uninterrupted access to all sponsored content, and MORE!

Open Source 'Critical' for Big Data For All | Hadoop | Intel | EMC | IBM | Red Hat Intel might shake up the insides of the computer yet again. Technology vendors have backed a community-developed software platform as the critical piece to bring big data to the masses. EMC and Intel have joined IBM and Red Hat in releasing their own flavours of an open-source software that manages how hardware in data centres accesses and processes information. The Hadoop open-source software by the Apache Software Foundation was released in October. It helps extract business insights from huge amounts of unstructured data, a trend commonly referred to as big data. While it is free to download, its complexity has meant only large players, such as Google, Amazon, and Facebook, had the manpower, scale and incentive to customise it. The new commercial releases are designed to make it easier, faster and safer to deploy. Advertisement big data. ''The risk for big data is that it becomes over-hyped when tech vendors just use it as a means to sell more of their gear,'' Anderson said.

Big Data: Where the Opportunities Are! by Harish Kotadia, Ph.D. I am often asked the question as to which industries offer the best potential for application of a Big Data solution and I have always said that Banking and Financial Services, Insurance, Telecom, Media/Information Services and Manufacturing offer the best promise to begin with. And I have based my judgement not only on the potential impact of a Big Data solution on the top and bottom-line numbers but also on “relative” ease with which Big Data can be used for reducing costs or improving productivity. My thinking on this subject was validated recently when I came across following excellent visualization on opportunities for Big Data across several industries prepared by Brett Sheppard based on Gartner Intelligence. According to Brett, this visualization is based on “Gartner’s comparison of eleven industries across seven dimensions using data from Gartner forecasts, market statistics and input from dozens of vertical industry experts”. (Source: See this)

NICTA - News Home | News | Jobs | Research Topics | People | Projects | Publications | Resources | Blogs | Students | Seminars | Visiting | Contact April 2014 Accepted Paper The paper "Gaussian Process Factorization Machines for Context-aware Recommendations" by Trung V. May 2013 Hanna Suominen short-listed for the 2013 L'Oréal Australia and New Zealand For Women In Science Fellowship A big congratulations to Hanna Suominen who has been shortlisted for a 2013 L'Oréal Australia and New Zealand For Women In Science Fellowship. Bob Williamson was invited to present on Machine Learning at the DBCDE meeting Prof Bob Williamson presented a talk on Machine Learning at the Department of Broadband, Communications and the Digital Economy meeting on 16 May. Results for the first two tasks of CLEFeHealth have been published The CLEF evaluation lab is a benchmarking activity aiming to develop processing methods and resources to enrich difficult-to-understand health text as well as their evaluation setting. April 2013

Big Data Market Size And Vendor Revenues By Jeff Kelly with David Vellante and David Floyer This is the 2011 report, originally published on February 15, 2012. See Big Data Vendor Revenue and Market Forecast 2012-2017 for the 2012 update. The Big Data market is on the verge of a rapid growth spurt that will see it top the $50 billion mark worldwide within the next five years. As of early 2012, the Big Data market stands at just over $5 billion based on related software, hardware, and services revenue. As explained in our Big Data Manifesto, Big Data is the new definitive source of competitive advantage across all industries. Below is Wikibon’s five-year forecast for the Big Data market as a whole: Figure 1 - Source: Wikibon 2012 Of the current market, Big Data pure-play vendors account for $480 million in revenue. Wikibon considers Big Data pure-plays as those independent hardware, software, or services vendors whose Big Data-related revenue accounts for 50% or more of total revenue. Figure 2 - Source: Wikibon 2012

Big Data Sees Venture Capitalists Invest Over $1 Billion Q2 2012 sees financing and deals up 150% and 304%, respectively. Is big data another buzzword or the next big thing? Big data, whether you believe it be this year’s VC buzzword (remember hyperlocal?) or truly the next big thing in IT, has clearly captured the hearts (and wallets) of VCs with big data financing eclipsing the $1 billion mark since Q2 2011. In fact, $1.15 billion has been invested across 90 transactions over that period. Although funding dropped off significantly in Q1 2012 (see chart below), deal activity has been steadily increasing over the last five quarters and shows little sign of abating.

Big Data Is Getting Bigger As a Startup Opportunity - The Huffington Post Even though "Big Data" has now been around for a few years, the opportunities for startups seem to keep growing, just as the amount of data keeps growing. According to IBM, companies have captured more data in the last two years than in the previous 2000 years. This data comes from sensors, social media posts, digital pictures and videos, purchase transactions, everywhere. Every day, we create 2.5 quintillion bytes of data -- much of it unstructured and far beyond the capability of conventional databases. Hence one segment of the opportunity is the need for new database technologies, like Hadoop, a distributed file system originally designed for indexing the Web. Data capacity is measured in petabytes (1000 terabytes), or soon even yottobytes (1024). By any definition, the opportunities from Big Data have the potential to create a next wave of successful technology companies that could change the way we all live and work. Targeted marketing.

Related: