Simple machine learning: bot detection Big data, BI, digital preservation and the cloud » Martin's Insights There is a very interesting relationship between big data and digital preservation. Preservationists work with very similar data types – equally complex, variable, voluminous and sparse. However, preserving big data introduces its own challenges. What of it is necessary to hang on to? Some memory institutions, as libraries and museums are called, have started using big data technologies to store and maintain their digital collections and their metadata, some of it in the cloud. Digital preservation Digital preservation is way more involved than you may initially think. The goal of digital preservation is to render authenticated content accurately over time. So who uses digital preservation? Digital preservationists face many challenges, the two most pressing being the decay of storage media and the constant change in access mechanisms. So digital preservation is about a lot more than merely technology. Big data and preservation BI and preservation Preservation and the cloud
Introduction to Strategic Thinking Starts June 22, 2015 This is a short interdisciplinary course on strategic thinking and some of its most powerful tools. Strategic thinking is not exclusive to business or military applications. The skills taught in this course can be used by everyone. Young professionals can use the knowledge to effectively plan their careers, stay-at-home mothers can use it to improve how they communicate with their children, and entrepreneurs can use it to better position their business in the marketplace. We will draw lessons and use concepts from economics, game theory, scenario planning, behavioral sciences, and futures studies, as well as philosophy and linguistics. This course is designed to be a starting point for people who are just beginning to learn about strategic thinking. "Fascinating course. "The course was very enlightening! "Thank you for the stimulus and providing a completely different approach from what I was expecting from this class." Sandjar Kozubaev Economist & Strategist
BIG BI: Running with the Red Queen: BI is Here to Stay They would go something like this: Train A leaves the Station at 1:15PM, gradually accelerating for 7 minutes until reaching a speed of 52mph for a distance of 76 miles. Train B leaves 12 minutes later, accelerating to a speed of 47mph in 11 minutes. After one hour, how far behind Train A was Train B? This sort of problem reminds me of the Business Intelligence (BI) business, alternately known as Decision Support, Analytics, Reporting, Data Discovery, etc. In biology, this is called the ‘Red Queen Effect’. named after poor Alice in 'Through the Looking Glass', where the faster she runs with the Red Queen, the faster the landscape moves with them so they have to go as fast as they can to merely keep up. Historically, most information technology emerges faster than organizations are able to adopt and implement it. Why is this? Challenges of the BI Red Queen Effect BI has provided some amazing innovation over the years. The Red Queen and Big Data In Summary ~ Neil Raden.
A Visual Introduction to Machine Learning Finding better boundaries Let's revisit the 73-m elevation boundary proposed previously to see how we can improve upon our intuition. Clearly, this requires a different perspective. By transforming our visualization into a histogram, we can better see how frequently homes appear at each elevation. While the highest home in New York is 73m, the majority of them seem to have far lower elevations. Your first fork A decision tree uses if-then statements to define patterns in data. For example, if a home's elevation is above some number, then the home is probably in San Francisco. In machine learning, these statements are called forks, and they split the data into two branches based on some value. That value between the branches is called a split point. Tradeoffs Picking a split point has tradeoffs. Look at that large slice of green in the left pie chart, those are all the San Francisco homes that are misclassified. The best split Recursion
BIG BI: Data isn’t a Process. It’s an Asset. So let me repeat. Data. It isn't a process. How many layer-cake data-flow, architecture and process flow diagrams have you seen? That’s one of those questions that has one simple answer and a million more complicated ones, but the simple one is...to inform decision-making. What’s Missing? What is missing from virtually every diagram I’ve ever seen is just how all of the diagrams of information technology actually inform decision making. What Really Happens When the triple threat of Big Data/Cloud/Hadoop hit the world like a tsunami, attention to ‘how’ data informs decision making was inundated with options. Business Intelligence is actually a terrible term in my opinion, because it isn’t always about business, and intelligence is a term that implies something passive. OLAP. OLAP is a term coined in 1993 as an acronym for On-line Analytical Processing. If you want to earn the disdain of a big data poser, just mention OLAP. The magic of OLAP is that it works the way people think.
A Tour of Machine Learning Algorithms In this post, we take a tour of the most popular machine learning algorithms. It is useful to tour the main algorithms in the field to get a feeling of what methods are available. There are so many algorithms available that it can feel overwhelming when algorithm names are thrown around and you are expected to just know what they are and where they fit. I want to give you two ways to think about and categorize the algorithms you may come across in the field. The first is a grouping of algorithms by the learning style.The second is a grouping of algorithms by similarity in form or function (like grouping similar animals together). Both approaches are useful, but we will focus in on the grouping of algorithms by similarity and go on a tour of a variety of different algorithm types. After reading this post, you will have a much better understanding of the most popular machine learning algorithms for supervised learning and how they are related. Algorithms Grouped by Learning Style 1. 2. 3.
BIG BI: Business Intelligence in Digital Transformation Digital transformation is a broad term that has various meanings by application, but in general, it means that more and more of what organizations, people, governments do is happening in computers, mobile devices and networks. As a result, the way things are done is changing, especially in the way things are connected. So in this new world of data flying everywhere, being generated and consumed, where does one stop for a second to take a look at what’s going on? In the old days, maybe 5-10 years ago, the obvious answer was analytical tools, such as BI or Excel, pulling data from a data warehouse. Instaology ≠ Real-Time I like to call it a new theory of instatology. Now in the latter case, that really isn’t quite so bad as there are still a multitude of applications that are served by what we call classic Business Intelligence. The Two Forms of Big BI Actually Big BI comes in two forms. This first form, however, can be as wild as you can imagine. Until next time... ~ Neil p.s.
Start Here Get Started and Get Good at Applied Machine Learning Hi, Jason here. I’m the guy behind Machine Learning Mastery. My goal is to help you get started, make progress and kick butt with machine learning. I teach a top-down and results-first approach designed for developers and engineers. Access my best free tutorials on the blog or take the next step with my paid training material. You may be feeling overwhelmed. Take your time. Table of Contents What do you need help with? How Do I Get Started? The most common question I’m asked is: “how do I get started?” My best advice for getting started in machine learning is broken down into a 5-step process: For more on this top-down approach, see: Many of my students have used this approach to go on and do well in Kaggle competitions and get jobs as Machine Learning Engineers and Data Scientists. Applied Machine Learning Process The benefit of machine learning are the predictions and the models that make predictions. Machine Learning Algorithms Deep Learning
ConvNetJS: Deep Learning in your browser ConvNetJS is a Javascript library for training Deep Learning models (Neural Networks) entirely in your browser. Open a tab and you're training. No software requirements, no compilers, no installations, no GPUs, no sweat. Description The library allows you to formulate and solve Neural Networks in Javascript, and was originally written by @karpathy (I am a PhD student at Stanford). Common Neural Network modules (fully connected layers, non-linearities) Classification (SVM/Softmax) and Regression (L2) cost functions Ability to specify and train Convolutional Networks that process images An experimental Reinforcement Learning module, based on Deep Q Learning. Head over to Getting Started for a tutorial that lets you get up and running quickly, and discuss Documentation for all specifics. Code The code is available on Github under MIT license and I warmly welcome pull requests for new features / layers / demos and miscellaneous improvements. Discussion Group