background preloader

Programming Languages

Facebook Twitter

Hadley (Hadley Wickham) Tutorials Tagged with R (#rstats) – NEON Work With Data. Courses/R Stuff. Courses > R Stuff More R… useful material compiled by the community Guides & reference cards websites basics Data visualisation Gallery of graphics done with RWebsite of Hadley Wickham, useful packages for data visualisation Time series Quick introGrafiken und Statistik in R (Andreas Plank, in German) Using netCDF-files Mapping with R powered by PmWiki. R useful functions. Just a (very incomplete) list of R functions. builtins() # List all built-in functions options() # Set options to control how R computes & displays results ? NA # Help page on handling of missing data values abs(x) # The absolute value of "x" append() # Add elements to a vector c(x) # A generic function which combines its arguments cat(x) # Prints the arguments cbind() # Combine vectors by row/column (cf.

Log(x),logb(),log10(),log2(),exp(),expm1(),log1p(),sqrt() # Fairly obvious cos(),sin(),tan(),acos(),asin(),atan(),atan2() # Usual stuff cosh(),sinh(),tanh(),acosh(),asinh(),atanh() # Hyperbolic functions union(),intersect(),setdiff(),setequal() # Set operations +,-,*,/,^,%%,%/% # Arithmetic operators <,>,<=,>=,==,! = # Comparison operators eigen() # Computes eigenvalues and eigenvectors deriv() # Symbolic and algorithmic derivatives of simple expressions integrate() # Adaptive quadrature over a finite or infinite interval. sqrt(),sum() ? Help(package=stats) # List all stats functions ? UsingR. Data-wrangling-cheatsheet.

Python

R and Matlab programming. Forecasting: principles and practice. Welcome to our online textbook on forecasting. This textbook is intended to provide a comprehensive introduction to forecasting methods and to present enough information about each method for readers to be able to use them sensibly. We don’t attempt to give a thorough discussion of the theoretical details behind each method, although the references at the end of each chapter will fill in many of those details.

The book is written for three audiences: (1) people finding themselves doing forecasting in business when they may not have had any formal training in the area; (2) undergraduate students studying business; (3) MBA students doing a forecasting elective. We use it ourselves for a second-year subject for students undertaking a Bachelor of Commerce degree at Monash University, Australia. For most sections, we only assume that readers are familiar with algebra, and high school mathematics should be sufficient background. Use the table of contents on the right to browse the book. rCharts. Using R for Time Series Analysis — Time Series 0.2 documentation. Reading Time Series Data The first thing that you will want to do to analyse your time series data will be to read it into R, and to plot the time series.

You can read data into R using the scan() function, which assumes that your data for successive time points is in a simple text file with one column. For example, the file contains data on the age of death of successive kings of England, starting with William the Conqueror (original source: Hipel and Mcleod, 1994). The data set looks like this: Age of Death of Successive Kings of England #starting with William the Conqueror#Source: McNeill, "Interactive Data Analysis"604367505642506568436534... Only the first few lines of the file have been shown. The first three lines contain some comment on the data, and we want to ignore this when we read the data into R. In this case the age of death of 42 successive kings of England has been read into the variable ‘kings’. Plotting Time Series ARIMA Models. Home Page. AboutHydrology: R resources for Hydrologists.

R is my statistical software of election. I had hard time to convince my Ph.D. students to adopt it, but finally they did, and, as usually happens, many of them became more proficient than me in the field. Now it seems natural to use it for everything, but this was not always the case. A list of introductory material is here.

A nice series of video tutorial on R was just posted by the Google Developers Group, on youTube, here. Recently many resources were added for hydrologists, which I list below with a little comment.RWater: A Cyber-enabled Data-driven Tool for Enhancing Hydrology EducationHydroGOF and HydroTSM by Mauricio Zambrano-Bigiarini. In R there are also many toolkit for the analysis of extreme events. Finally, Mauricio Zambrano suggested also a series of other CRAN R packages that could be useful: If you arrived here and you never used R, you can start from here. Packages we like - The HydroFilesThe HydroFiles.

MATLAB scripts and programs. USGS Online Publications Directory. R Books. MATLAB DFS interface | Hydraulic and Environmental Engineering. DHI released the new MATLAB DFS interface library. This is a major extension compared to the previous version (2004). The new version supports e.g. read and write of flexible mesh data (dfsu 2D and 3D). From DHIgroup website, I see that th new MATLAB DFS interface provides different levels of support for different file types: dfs0: read, write, createdfs1, dfs2 ,dfs3: read only dfsu 2D and 3D: read and write mesh files and xyz files: read and create This Matlab toolbox is a good point for all DHI software and Matlab users who want to exploit Matlab built-in functions to manage and visualize multi-dimensional data.

The matlab code and an user guide can be downladed here and here. MATLAB® Recipes for Earth Sciences - Martin Trauth. Using_ml. MIT 18.S997 Introduction to MATLAB Programming Fall 2011. The Command Line Crash Course Controlling Your Computer From The Terminal. Learn Python The Hard Way. This exercise has no code. It is simply the exercise you complete to get your computer to run Python. You should follow these instructions as exactly as possible.

For example, Mac OS X computers already have Python 2, so do not install Python 3 (or any Python). Go to with your browser, get the Notepad++ text editor, and install it. From now on, when I say "Terminal" or "shell" I mean PowerShell and that's what you should use. Warning Sometimes you install Python on Windows and it doesn't configure the path correctly. > python ActivePython 2.6.5.12 (ActiveState Software Inc.) based on Python 2.6.5 (r265:79063, Mar 20 2010, 14:22:52) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.

>>> quit()> mkdir mystuff > cd mystuff ... It is still correct if you see different information than mine, but yours should be similar. A major part of this book is learning to research programming topics online. AOS 575: Climatological Analysis. Sample code for setting up a map axis to plot a polar-stereographic plot: plot_polar_stereo.m EOF Analysis eof_routine.m cosweight.m Suppose you want to perform EOF analysis on tropical Pacific SST from the NCEP reanalysis.

Here's a quick way to do that, assuming you've downloaded my matlab libraries, above: lims = [120 285 -30 30]; tim = get_time(1948,2002,1948); [sst, lat, lon] = getnc2('skt.mon.mean.nc', 'skt', lims, 1, tim); land = getnc2('land.sfc.gauss.nc', 'land', lims, 1, 1); sst = remove_anncyc(sst); sst = cosweight(sst, lat); [lam, lds, pcs, per] = eof_routine(sst(:,land==0)); % Get a scaled map of the leading EOF eof1 = repmat(NaN, [length(lat) length(lon)]); eof1(land == 0) = lds(:,1)*sqrt(lam(1)); eof1 = uncosweight(eof1, lat); chi2example.m: 1. A demo for using the Chi2 distribution to determine confidence limits on variance 2. Example2_16_Feb.m: This example demonstrates how to generate regression maps of SST or 500mb height onto the CTI. Visualization | Water Programming: A Collaborative Research Blog | Page 2. Have you done Part 1a yet? How about Part 1b? Go do those first, and come back with metrics.txt, which has all of the data in a single file.

Afterwards, please check out Part 2. We’re interested in Borg’s runtime metrics because they tell us interesting things about the problem we’re optimizing. Please consider this one released under the MIT license. Remember all of the text manipulation we had to do in Part 1a to deal with our data? Line 6 shows how I read in a table. 6 metrics = pandas.read_table('metrics.txt') metrics is now a pandas DataFrame object.

Line 8 identifies the models for which I expect to find data in metrics.txt, and Line 9 indicates what color to use for plotting, for each model. Lines 11 through 16 make a list of the columns I want to plot. A dictionary (dict) in Python is an associative data structure. Lines 39 through 60 make a plot for each of the columns we specified in toplot and save it to a file. Setting up the Axes An axes object belongs to a figure. Wrapping Up. Sensitivity Analysis | Water Programming: A Collaborative Research Blog. This post was updated on August 11, 2014 to update the SALib module structure. The Sensitivity Analysis Library (SALib) is an open-source Python library for common sensitivity analysis routines, including the Sobol, Morris, and FAST methods.

Step 0: Get the library The easiest way to install is pip install SALib, which will pull the latest version from the Python package index. Or, you can download at the above link and run python setup.py install. If you’re interested in contributing, you can clone the git repository: git clone . Step 1: Choose sampling bounds for your parameters First, you will need to create a simple text file with the form [parameter] [lower bound] [upper bound]. Cmax 0.0 1000.0 B 0.0 3.0 Alpha 0.0 1.0 Kq 0.15 1.0 Ks 0.0 0.15 The bounds are used to sample parameter values.

Step 2: Generate parameter sets using the Sobol Sequence Put your params.txt file in the same directory as the SALib folder. Step 4: Calculate Sobol Indices. How to add RMSE, slope, intercept, r^2 to R plot? R: Root Mean Square Error. Description Root Mean Square Error (RMSE) between sim and obs, in the same units of sim and obs, with treatment of missing values. RMSE gives the standard deviation of the model prediction error. A smaller value indicates better model performance. Usage rmse(sim, obs, ...) ## Default S3 method: rmse(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'data.frame' rmse(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'matrix' rmse(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'zoo' rmse(sim, obs, na.rm=TRUE, ...) Arguments Details rmse = sqrt( mean( (sim - obs)^2, na.rm = TRUE) ) Value Root mean square error (rmse) between sim and obs. If sim and obs are matrixes, the returned value is a vector, with the RMSE between each column of sim and obs.

Note obs and sim has to have the same length/dimension The missing values in obs and sim are removed before the computation proceeds, and only those positions with non-missing values in obs and sim are considered in the computation Author(s) References. R: Nash-Sutcliffe Efficiency. Description Nash-Sutcliffe efficiency between sim and obs, with treatment of missing values. Usage NSE(sim, obs, ...) ## Default S3 method: NSE(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'data.frame' NSE(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'matrix' NSE(sim, obs, na.rm=TRUE, ...) ## S3 method for class 'zoo' NSE(sim, obs, na.rm=TRUE, ...)

Arguments Details NSE = 1 - ( sum( (obs - sim)^2 ) / sum( (obs - mean(obs))^2 ) The Nash-Sutcliffe efficiency (NSE) is a normalized statistic that determines the relative magnitude of the residual variance ("noise") compared to the measured data variance ("information") (Nash and Sutcliffe, 1970). NSE indicates how well the plot of observed versus simulated data fits the 1:1 line. Nash-Sutcliffe efficiencies range from -Inf to 1. Value Nash-Sutcliffe efficiency between sim and obs. If sim and obs are matrixes, the returned value is a vector, with the Nash-Sutcliffe efficiency between each column of sim and obs. Note Author(s) References Nash, J. R4Hydrology – Communauté – Google+ MATLAB Source Codes. Analemma, a program which evaluates the equation of time, a formula for the difference between the uniform 24 hour day and the actual position of the sun, based on a C program by Brian Tung. Area_under_curve, a function which displays the area under a curve, that is, the points (x,y) between the x axis and the curve y=f(x).

Args, a program which shows how to count and report command line arguments; arpack, a library which computes eigenvalues and eigenvectors of large sparse matrices, accessible via MATLAB's built-in eigs() command; asa005, a library which evaluates the lower tail of the noncentral Student's T distribution, by BE Cooper. MATLAB Central - HBV Hydrologic Model - Education Version (MATLAB Code) - What are tags? A tag is like a keyword or category label attached to an entry in the Link Exchange. Tags make it easier to find or recall particular links of interest. They're also a good way to categorize your bookmarked items, so you can browse your bookmarks by subject. Tags benefit the community, too. What makes a good tag? A good tag helps people find the entry. How to Tag Tag your new submission Start typing in the Tag box, or click on a tag from the list that displays near the box.

Tag an existing link You can add a tag to an existing entry on the View Link page.