Unstructured Data Processing – Why Textual ETL? by Krish Krishnan. Until the last decade, organizations relied on legacy systems, enterprise applications and market data gathered by analysts to make decisions for the business. To make any and all detailed, operational, up-to-the-second decisions, the systems that were in place worked just fine. To take care of any and all detailed analysis and reporting, the data warehouse and data marts were implemented. As time progressed, analytics and key performance indicators (KPIs) were needed by organizations – not only on data across all of these systems, but also beyond these from sources including the Internet, content management platforms and more. We discovered that we cannot simply process unstructured data (text, documents, policies, PDFS, contracts), semi-structured data (email, forms) or content management systems (Sharepoint, Documentum) with processing techniques from the existing systems.
Structured ETL Enter Unstructured Data Nearly all legacy data is structured. Recent articles by Krish Krishnan. ‘Unijam’ strikes mass chord at University of South Australia. New v-c aims to crowdsource strategy via online meet-up Source: Alamy I’m with the band: alumni, staff and students will all be encouraged to share their opinions on South Australia’s strategic direction When a fresh face enters a vice-chancellor’s office, a revised strategic plan invariably emerges from under the door a few months later.
But the University of South Australia’s new vice-chancellor David Lloyd has decided to break with this closed-door tradition and instead host a giant 48-hour online brainstorming conversation about university strategy with any and all of the institution’s staff, students and alumni who care to take part. South Australia will thereby become the first university in the world to hold what it has dubbed a “Unijam”, which will use technology developed by IBM. Professor Lloyd, who took over as vice-chancellor and president in January, said he had first seen the technology used in-house at IBM while he was working in the pharmaceutical industry in 2004.
Projects 2014 | Global Service Jam. The Lousy Linguist: IBM SPSS Text Analytics - STAS vs. Word Clouds. This is the fourth and final post in a series about IBM's SPSS Text Analytics platform STAS (first intro post here, second on a linguist's perspective here, third on Shakespeare's sonnets here). As I wrote in my third post, I want to run a bake off between the word frequency analyses of the President's State Of The Union (SOTU) speech last night and STAS's more in-depth tools.
I am no fan of word clouds and simplistic word frequency counts (see my discussion here or Mark Liberman's discussion of word count abuse in political punditry here), but I'm trying to put myself into the shoes of a novice with no NLP, text analytics, or linguistics background. Someone who wants a quick and simple way to analyze language in an objective but meaningful way. First, let's look at a word cloud of President Obama's 2013 SOTU (text here; note I deleted all instances of the string "(Applause.)
"). Jobs, America, people, new, work, now, get, like... There are other word count options for the lay reader. 100 - A Global Innovation Jam. Watson: The Technology. Hypothesis Generation When asked a question, Watson relies on hypothesis generation and evaluation to rapidly parse relevant evidence and evaluate responses from disparate data. Natural Language Watson can read and understand natural language, important in analyzing unstructured data that make up as much as 80 percent of data today. Dynamic Learning Through repeated use, Watson literally gets smarter by tracking feedback from its users and learning from both successes and failures. Watson is a cognitive technology that processes information more like a human than a computer—by understanding natural language, generating hypotheses based on evidence, and learning as it goes.
And learn it does. How IBM innovates dec 2013 - the front end of innovation in IBM. Influence and correlation in social networks. In many online social systems, social ties between users play an important role in dictating their behavior. One of the ways this can happen is through social influence, the phenomenon that the actions of a user can induce his/her friends to behave in a similar way. In systems where social influence exists, ideas, modes of behavior, or new technologies can diffuse through the network like an epidemic.
Therefore, identifying and understanding social influence is of tremendous interest from both analysis and design points of view. This is a difficult task in general, since there are factors such as homophily or unobserved confounding variables that can induce statistical correlation between the actions of friends in a social network. Distinguishing influence from these is essentially the problem of distinguishing correlation from causality, a notoriously hard statistical problem. In this paper we study this problem systematically. Influence and correlation in social networks. IdeaJam documents - Elguji. Com/ideajam/elguji/elguji.nsf/Images/BELT-8J8ULS/$File/ETSACaseStudy.pdf. IdeaJam Idea and Innovation Management Software Overview - Elguji. Idea JamTM Idea and innovation management software for the enterprise From the lifeblood of a company, its customers; to its most valuable resources, its employees; we help companies learn what matters most.
Our IdeaJam software drives innovation by helping companies understand which ideas are worth pursuing and which ones are not, and most importantly why. The concept is amazingly simple, yet extraordinarily powerful: with IdeaJam, people post their ideas on a topic, and others can vote on their agreement or disagreement with the idea by "promoting" or "demoting" it. Additionally, comments can be provided to elaborate ones thoughts on the matter at hand. Crowdsource Good ideas rise to the top, bad ideas fall to the bottom. How can an organization maximize their return on investment of intellectual capital?
IdeaJam is the fuel you need Who should use IdeaJam. IdeaJam - Idea Management and Innovation Software. IdeaJam - Idea Management and Innovation Software. Google Cloud Platform Blog: Performance advantages of the new Google Cloud Storage Connector for Hadoop. Our guest blog post today comes from Mike Wendt, R&D Associate Manager at Accenture Technology Labs, who recently published a study detailing the real world performance advantages of Hadoop on Google Compute Engine. His team utilized the recently launched Google Cloud Storage Connector for Hadoop and observed significant performance improvements over HDFS on local filesystems Hadoop clusters tend to be deployed on bare-metal; however, they are increasingly deployed on cloud environments such as Google Compute Engine. Benefits such as pay-per-use pricing, scalability and performance tuning make cloud a practical option for Hadoop deployments.
At Accenture Technology Labs, we were interested in proving the value of cloud over bare-metal and devised a method for a price-performance-ratio comparison of a bare-metal Hadoop cluster with cloud-based Hadoop clusters at the matched total-cost-of-ownership level. In comparison, the recommendation engine workload has only one input file of 5 GB.
Apache Hadoop Solutions — Google Cloud Platform. Storing Unstructured Data – From file servers to cloud services - Jose Barreto's Blog. After joining the Storage Solutions Division at Microsoft, I got exposed many challenges that were not so close to me before. One of them is how we store and manage unstructured data, including things like file servers, NAS devices, document management systems and blob storage solutions. Here’s my initial attempt to cover a little of this area’s history and summarize its main issues. As the personal computer gained popularity, a lot of data started being stored in files, like text documents and spreadsheets. Those personal computers eventually were networked together and started sharing those files.
In the 80’s, file servers stored those files for a small group of computers, usually organized under a folder hierarchy. Unstructured data existed side-by-side with database servers, which stored data as sets of data tables connected by key fields. The personal computer evolved to store more complex documents, like presentations, long manuals, diagrams, messages, pictures and video. Www.managementlab.org/files/u2/pdf/case studies/ibm.pdf.
Innovation collaboration academic publication. An Inside View of IBM’s ‘Innovation Jam’ IBM brought 150,000 employees and stakeholders together to help move its latest technologies to market. Both the difficulties it faced and the successes it achieved provide important lessons. IBM Research is the world’s largest corporate research organization, with eight labs and 3,200 researchers in six countries. Every year Sam Palmisano, IBM Corp.’s chairman, visits its headquarters in Yorktown Heights, New York, to review progress. When Palmisano toured the labs in early 2006, enthusiastic scientists showed him all manner of newly developed capabilities. After the demos, IBM’s Paul Horn, chief scientist, and Cathy Lasser, research chief information officer, met with Palmisano.
The executives conceived the idea of a “Jam” to promote innovation. IBM Client Experience Jam. IBMers are always thinking about how to deliver the best possible experiences for our clients- whether it's innovative cloud computing solutions, big data analytics, social business for industries or making the most of mobile for the enterprise—IBMers want to bring the most value to our clients. One way IBMers address great challenges is to "jam" together in massive online collaborative experiences. An IBM Jam is a guided online discussion with thousands of trusted collaborators from which we extract insights, discoveries and decisions. Thousands of IBM employees jammed on how we create the best experiences for our clients from March 12–15th, 2013.
The Jam is over and in read–only mode now -- it will be available for several weeks. This Jam is for IBM employees only. You will need a valid IBM email address in order to read the jam. The history of Jams Jams are not restricted to business. Follow us: Not an IBM employee? Not an IBM employee, but interested in IBM Jams? A decade of Jamming. Service Jam - Overview. IBM believes that a company culture based on core values not only helps our business, but also defines the role that we can and should play in society.
We identify and act upon new opportunities to apply our technology and expertise to societal problems.We scale our existing programs and initiatives to achieve maximum benefit.We empower our employees and others to serve their communities.We integrate corporate citizenship and social responsibility into every aspect of our company. Corporate citizenship IBM has developed a thoughtful, comprehensive approach to corporate citizenship that we believe aligns with IBM’s values and maximizes the impact we can make as a global enterprise. We focus on specific societal issues, including the environment, community economic development, education, health, literacy, language and culture. Environment IBM is committed to environmental leadership in all of our business activities.
Visit the IBM and the Environment site Supply chain. Strategic action plan 2013-2018 - About UniSA - University of South Australia. By 2018, UniSA will be a university which engages fully with the professions and industry globally, whose research is informed, leading edge and relevant, and whose graduates are the new professionals driving the national and international economy through their skills, capabilities and innovation potential. Latest news UniSA launches Reconciliation Action Plan UniSA is the first university in South Australia to launch its own RAP. Read more... UniSA becomes smoke free The University of South Australia will officially become smoke free from 31 May 2014.
A smoke free campus means that smoking is prohibited on all university owned grounds. Students add a splash of colour to street life Students are about to put paint to pavement in a street art project that is set to add vibrant colour to the West End precinct. Campus Connector UniSA is trialling a free bus service for students and staff between Magill campus and Mawson Lakes campus from March 2014. New UniSA clothing Key Enablers and Supports. Unijam - Jam rules. IBM Jam Events. Unijam - Jam over.