background preloader

Bayesian Methods for Hackers

Bayesian Methods for Hackers
An intro to Bayesian methods and probabilistic programming from a computation/understanding-first, mathematics-second point of view. Prologue The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chapters of slow, mathematical analysis. The typical text on Bayesian inference involves two to three chapters on probability theory, then enters what Bayesian inference is. Unfortunately, due to mathematical intractability of most Bayesian models, the reader is only shown simple, artificial examples. This can leave the user with a so-what feeling about Bayesian inference. After some recent success of Bayesian methods in machine-learning competitions, I decided to investigate the subject again. If Bayesian inference is the destination, then mathematical analysis is a particular path towards it. Bayesian Methods for Hackers is designed as a introduction to Bayesian inference from a computational/understanding-first, and mathematics-second, point of view. Related:  Data ScienceRandom thingsBookmarks

Think Bayes Bayesian Statistics Made Simple by Allen B. Downey Download Think Bayes in PDF. Read Think Bayes in HTML. Order Think Bayes from Read the related blog, Probably Overthinking It. Description Think Bayes is an introduction to Bayesian statistics using computational methods. The premise of this book, and the other books in the Think X series, is that if you know how to program, you can use that skill to learn other topics. Most books on Bayesian statistics use mathematical notation and present ideas in terms of mathematical concepts like calculus. I think this presentation is easier to understand, at least for people with programming skills. Think Bayes is a Free Book. Other Free Books by Allen Downey are available from Green Tea Press. Download Precompiled copies of the book are available in PDF. Other Free Books by Allen Downey are available from Green Tea Press.

New AWS Command Line Interface (CLI) Graphical user interfaces (e.g. the AWS Management Console) are great, but nothing beats the expressiveness of the command line! Today we are releasing the AWS Command Line Interface (CLI). The AWS CLI provides a single, unified interface to a very large collection of AWS services. After downloading and configuring the CLI you can drive Amazon EC2, Amazon S3, Elastic Beanstalk, the Simple Workflow Service, and twenty other services (complete list) from your Linux, OS X, or Windows command line. Download and ConfigureThe Getting Started page contains the information that you need to know in order to download and configure the AWS CLI. You can set your AWS credentials for the CLI using environment variables or a configuration file. I recommend that you create an IAM user (I called mine awscli) so that you have full control of the AWS operations that can be performed from the command line. Running CommandsThe AWS CLI commands take the form: $ aws SERVICE OPERATION [OPTIONS] -- Jeff;

The Art of Lisp & Writing Ignorance more frequently begets confidence than does knowledge. –Charles Darwin Lisp is the language of loveliness. With it a great programmer can make a beautiful, operating thing, a thing organically created and formed through the interaction of a programmer/artist and a medium of expression that happens to execute on a computer. Taught that programming—or the worse "developing software"—is like a routine engineering activity, many find difficulty seeing writing as a model or even a metaphor for programming. Writing is creative, it is self-expression, it is art, which is to say it isn't a science and unlike science and engineering, it isn't a serious activity. Artists make things and have always done so, gathering knowledge. As people need or want to do things with materials and the world, people with special skill take the fore and devise or discover how to manipulate the physical world to make those things. Not really. Perhaps mapmakers are liars, too. –Christopher Alexander | Cam Davidson Pilon Introduction To Calculus With Derivatives Written February 18, 2018 Suppose you need to calculate 1012 but you don't have a calculator handy. How would you estimate it? Or suppose you had to estimate 4.12. What if I told you that 4.12 is 16.81. Derivatives will help answer these questions. Estimating squares 42 is 16, and 4.12 is 16.81. 52 is 25, and 5.12 is 26.01. Lets look at some other increases: At 4.1, the increase was around 8. We know 72 is 49, and now we guess that there's an increase of 14 happening here. 7.12 is actually 50.41, so we were really close! Going back to the original question, what is 1012? These numbers seem to magically tell you how to calculate values like 4.12, 7.12, or 1012: Here's the big reveal: those numbers are the derivative at those points. We know the value of f(x) for different values of x: Here x is the input, and f(x) is the output. The derivative of f(x) (written f′(x) -- note the apostrophe) is 2x. That last column has all the ratios we were just using! So our estimate is 5.12=26. P.S. Limits

Announcing SparkR: R on Spark Join us at the Spark Summit to learn more about SparkR. Use the code Databricks20 to receive a 20% discount! I am excited to announce that the upcoming Apache Spark 1.4 release will include SparkR, an R package that allows data scientists to analyze large datasets and interactively run jobs on them from the R shell. R is a popular statistical programming language with a number of extensions that support data processing and machine learning tasks. However, interactive data analysis in R is usually limited as the runtime is single-threaded and can only process data sets that fit in a single machine’s memory. SparkR, an R package initially developed at the AMPLab, provides an R frontend to Apache Spark and using Spark’s distributed computation engine allows us to run large scale data analysis from the R shell. Project History The SparkR project was initially started in the AMPLab as an effort to explore different techniques to integrate the usability of R with the scalability of Spark.

Learn Go in Y Minutes Go was created out of the need to get work done. It’s not the latest trend in computer science, but it is the newest fastest way to solve real-world problems. It has familiar concepts of imperative languages with static typing. Go comes with a great standard library and an enthusiastic community. The root of all things Go is the official Go web site. The language definition itself is highly recommended. You can play around with the code on Go playground. On the reading list for students of Go is the source code to the standard library. Another great resource to learn Go is Go by example. Got a suggestion?

Eigenfaces, for Facial Recognition This post assumes familiarity with the terminology and notation of linear algebra, particularly inner product spaces. Fortunately, we have both a beginner’s primer on linear algebra and a follow-up primer on inner products. The Quest We are on a quest to write a program which recognizes images of faces. The general algorithm should be as follows. Get a bunch of sample images of people we want to recognize.Train our recognition algorithm on those samples.Classify new images of people from the sample images. We will eventually end up with a mathematical object called an eigenface. God has given you one face, and you make yourself a vector Most naturally, we think of an image as a matrix of pixel values. image corresponds bijectively to a matrix with integer entries between 0 and 255. Representing an image as a matrix reminds us of the ubiquitous applicability of linear algebra. In other words, we want a face space. matrix with entries Now we have our face space: . pixels. Mean Faces Covariance and .

DDI On my first day of physics graduate school, the professor in my class on electromagnetism began by stepping to the board, and wordlessly writing four equations: He stepped back, turned around, and said something like [1]: “These are Maxwell’s equations. Just four compact equations. With a little work it’s easy to understand the basic elements of the equations – what all the symbols mean, how we can compute all the relevant quantities, and so on. Alan Kay has famously described Lisp as the “Maxwell’s equations of software”. Here’s the half page of code that Kay saw in that manual: What we’re going to do in this essay is understand what that half page of code means, and what it means that Lisp is the Maxwell’s equations of software. That perhaps sounds over-ambitious, but the good news is that it’s easy to learn the basic elements of Lisp. But I won’t stop with just showing you how to write some Lisp. Some elements of Lisp tiddlylisp> tiddlylisp> (+ 2 3) 5 tiddlylisp> (* 3 4) 12 As for atom?

Markov Chains explained visually Explained Visually By Victor Powell with text by Lewis Lehe Markov chains, named after Andrey Markov, are mathematical systems that hop from one "state" (a situation or set of values) to another. A simple, two-state Markov chain is shown below. With two states (A and B) in our state space, there are 4 possible transitions (not 2, because a state can transition back into itself). Of course, real modelers don't always draw out Markov chain diagrams. If the state space adds one state, we add one row and one column, adding one cell to every existing column and row. One use of Markov chains is to include real-world phenomena in computer simulations. One way to simulate this weather would be to just say "Half of the days are rainy. Did you notice how the above sequence doesn't look quite like the original? We can minic this "stickyness" with a two-state Markov chain. You can also access a fullscreen version at

Related:  BidData And NeuralNetworks