background preloader

Web_crawl

Facebook Twitter

Ruby From Other Languages. When you first look at some Ruby code, it will likely remind you of other programming languages you’ve used. This is on purpose. Much of the syntax is familiar to users of Perl, Python, and Java (among other languages), so if you’ve used those, learning Ruby will be a piece of cake. This document contains two major sections. The first attempts to be a rapid-fire summary of what you can expect to see when going from language X to Ruby. The second section tackles the major language features and how they might compare to what you’re already familiar with. What to Expect: Language X to Ruby Important Language Features And Some Gotchas Here are some pointers and hints on major Ruby features you’ll see while learning Ruby.

Iteration Two Ruby features that are a bit unlike what you may have seen before, and which take some getting used to, are “blocks” and iterators. Some_list.each do |this_item| # We're inside the block. # deal with this_item.end Everything has a value Everything is an Object. ScRUBYt! - Scrape. Shape. Integrate. Profit.

Brynary's webrat at master - GitHub. Watir. Jon Hart's Blog: Hawler, the Ruby crawler. Over the years I've thrown together various bits of code that have crawling functionality built into them. There was termite, used to find backup copies, renames or common temporary locations of your entire web site. There was indexfinder, used to crawl your site and find anything that looked like a directory listing. There was also htcomment, used to ferret out all of the comments found in your html. These tools were all fairly well tested and worked quite well, but everytime I dusted off the code and fixed a bug or added functionality, my CS books would scowl at me.

The core crawling code was literally cut and pasted between tools. The problem with this is obvious -- a bug or missing bit of functionality in the core crawler code had to be fixed in numerous places. Starting maybe a month ago I decided to fix this problem. As an example of Hawler's usage, I've put together two tools that I've found quite useful so far. . $ htgrep shot www.latimes.com -r 1 -s 1 -p |wc -l 43 Only 43? Using the Ruby MySQL Module. Paul DuBoispaul@kitebird.com Document revision: 1.06 Last update: 2007-05-26 Table of Contents Introduction Programs that access MySQL databases can be written in the Ruby scripting language by using Tomita Masahiro's MySQL module.

This module provides a Ruby client API; it is implemented as a wrapper around the MySQL C client API. Obtaining and Installing the MySQL Module To use the Ruby MySQL module, first make sure that you have the MySQL C client API header files and libraries installed. The Ruby MySQL module can be obtained from the following site: The module is distributed as a compressed tar file, which you should unpack after downloading it. . % tar zxf mysql-ruby-2.7.1.tar.gz % gunzip < mysql-ruby-2.7.1.tar.gz | tar xf - After unpacking the distribution, change location into its top-level directory and configure it using the extconf.rb script in that directory: % ruby extconf.rb Alternatively, tell extconf.rb where to find the mysql_config program.