background preloader


Facebook Twitter

Profiling and benchmarking. “Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered.”— Donald Knuth.

Profiling and benchmarking

Optimising code to make it run faster is an iterative process: Find the biggest bottleneck (the slowest part of your code).Try to eliminate it (you may not succeed but that’s ok).Repeat until your code is “fast enough.” This sounds easy, but it’s not. Even experienced programmers have a hard time identifying bottlenecks in their code. Instead of relying on your intuition, you should profile your code: use realistic inputs and measure the run-time of each individual operation.

It’s easy to get caught up in trying to remove all bottlenecks. Outline Prerequisites In this chapter we’ll be using the lineprof package to understand the performance of R code. Devtools::install_github("hadley/lineprof") Measuring performance Then h(): Exercises. Untitled. A Community Site for R – Sponsored by Revolution Analytics.

DataCamp: The Easiest Way To Learn R & Data Science. Home Page. Finding and removing duplicate records. Regular Expressions with grep, regexp and sub in the R Language. The R Project for Statistical Computing provides seven regular expression functions in its base package.

Regular Expressions with grep, regexp and sub in the R Language

The R documentation claims that the default flavor implements POSIX extended regular expressions. That is not correct. In R 2.10.0 and later, the default regex engine is a modified version of Ville Laurikari's TRE engine. It mimics POSIX but deviates from the standard in many subtle and not-so-subtle ways. What this website says about POSIX ERE does not (necessarily) apply to R. Older versions of R used the GNU library to implement both POSIX BRE and ERE. The best way to use regular expressions with R is to pass the perl=TRUE parameter. All the functions use case sensitive matching by default. Finding Regex Matches in String Vectors The grep function takes your regex as the first argument, and the input vector as the second argument.

> grepl("a+", c("abc", "def", "cba a", "aa"), perl=TRUE) [1] TRUE FALSE TRUE TRUE Replacing Regex Matches in String Vectors. R Programming/Text Processing. This page includes all the material you need to deal with strings in R.

R Programming/Text Processing

The section on regular expressions may be useful to understand the rest of the page, even if it is not necessary if you only need to perform some simple tasks. This page may be useful to : perform statistical text analysis.collect data from an unformatted text with character variables. In this page, we learn how to read a text file and how to use R functions for characters. There are two kind of function for characters, simple functions and regular expressions. = "character", package = "base") However, their name and their syntax is not intuitive to all users. Keywords : text mining, natural language processingSee CRAN Task view on Natural Language Processing[2]See also the following packages tm, tau, languageR, scrapeR. Reading and writing text files[edit] R can read any text file using readLines() or scan().

Character encoding[edit] Example[edit] Statistical Analysis with Open-Source R and RStudio on Amazon EMR. Running R on AWS - AWS Big Data Blog. Untitled. Type the email address or phone number of the account you want to sign in with.


We're having trouble locating your account. Which type of account do you want to use? Converting a list to a data frame. There are many situations in R where you have a list of vectors that you need to convert to a data.frame.

Converting a list to a data frame

This question has been addressed over at StackOverflow and it turns out there are many different approaches to completing this task. Since I encounter this situation relatively frequently, I wanted my own S3 method for that takes a list as its parameter. I should note that it only works with atomic vectors (i.e. logical, integer, numeric, complex, character and raw).

If any one of the elements in the list are of some other class type, the function will call NextMethod. Running R on AWS - AWS Big Data Blog. Running R on AWS - AWS Big Data Blog. SparkR on EC2 · amplab-extras/SparkR-pkg Wiki. SparkR (R on Spark) - Spark 1.6.0 Documentation. SparkR is an R package that provides a light-weight frontend to use Apache Spark from R.

SparkR (R on Spark) - Spark 1.6.0 Documentation

In Spark 1.6.0, SparkR provides a distributed data frame implementation that supports operations like selection, filtering, aggregation etc. (similar to R data frames, dplyr) but on large datasets. SparkR also supports distributed machine learning using MLlib. A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R, but with richer optimizations under the hood.

All of the examples on this page use sample data included in R or the Spark distribution and can be run using the . Starting Up: SparkContext, SQLContext.