Introduction to Load Balancing Using Node.js - Part 1 by Ross Johnson Introduction At Mazira, a lot of what we develop takes the form of web services. While most of these are only used internally, it is still important that they are high-performance and resilient. These services need to be ready to churn through hundreds of gigabytes of documents at a moment’s notice, say, for example, if we need to reprocess one of our document clusters. Horizontal Scaling When it comes to increasing the performance of websites and web services, there are only a couple of options: increase the efficiency of the code, or scale up the server infrastructure. The Example In order to demonstrate load balancing, we first need a sample application. Listing 1: pi-server.js To run this example, put it into a file pi-server.js, browse to the folder in a command line and then execute: $ node pi-server.js 8000 Now, going to localhost:8000 in a web browser produces the following result: Figure 1: Example Pi-Server Result Load Testing Figure 2: JMeter Test Setup EDIT: Part 2
Robots exclusion standard The Robot Exclusion Standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to advising cooperating web crawlers and other web robots about accessing all or part of a website which is otherwise publicly viewable. Robots are often used by search engines to categorize and archive web sites, or by webmasters to proofread source code. The standard is different from, but can be used in conjunction with, Sitemaps, a robot inclusion standard for websites. History[edit] The standard was proposed by Martijn Koster,[1][2] when working for Nexor[3] in February, 1994[4] on the www-talk mailing list, the main communication channel for WWW-related activities at the time. It quickly became a de facto standard that present and future web crawlers were expected to follow; most complied, including those operated by search engines such as WebCrawler, Lycos and AltaVista. About the standard[edit] A robots.txt file covers one origin. Disadvantages[edit] Alternatives[edit]
css - using background image for li Parallel Processing on the Pi (Bramble) Parallel processing on the Raspberry Pi is possible, thanks to the ultra portable MPICH2 (Message Passing Interface). I was keen to try this out as soon as I managed to get hold of two of these brilliant little computers (yes I'm a lucky boy). Here I'm going to show how I managed to get it all working and will display the results :)(Bramble was a name an ingenious Raspberry Pi forum member made up, not myself!) There are three ways which you can install MPICH2 (in case one doesn't seem to work for you), compiling and installing from source, my .deb package then following the rest of the tutorial, or the Python script file. Installing from source takes a while on the little Pi when not cross compiling. Install - Choose Method 1, 2 or 3 1) Simply download the script;wget and then run it as root with the command;sudo python install.py Then follow the instructions on the screen entering all of the necessary info.
Robots.txt Tutorial How to Create Robots.txt Files Use our Robots.txt generator to create a robots.txt file. Analyze Your Robots.txt File Use our Robots.txt analyzer to analyze your robots.txt file today. Google also offers a similar tool inside of Google Webmaster Central, and shows Google crawling errors for your site. Example Robots.txt Format Allow indexing of everything User-agent: * Disallow: or User-agent: * Allow: / Disallow indexing of everything User-agent: * Disallow: / Disawllow indexing of a specific folder User-agent: * Disallow: /folder/ Disallow Googlebot from indexing of a folder, except for allowing the indexing of one file in that folder User-agent: Googlebot Disallow: /folder1/ Allow: /folder1/myfile.html Background Information on Robots.txt Files Robots.txt files inform search engine spiders how to interact with indexing your content. When you block URLs from being indexed in Google via robots.txt, they may still show those pages as URL only listings in their search results. Crawl Delay
Pinterest Raspberry Pi Weather Station for schools When I first joined the Raspberry Pi Foundation, over a year ago now, one of my first assignments was to build a weather station around the Raspberry Pi. Thanks to our friends at Oracle (the large US database company), the Foundation received a grant not only to design and build a Raspberry Pi weather station for schools, but also to put together a whole education programme to go with it. Oracle were keen to support a programme where kids get the opportunity to partake in cross-curricular computing and science projects that cover everything from embedded IoT, through networking protocols and databases, to big data. The goals of the project was ambitious. Between us we wanted to create a weather experiment where schools could gather and access weather data from over 1000 weather stations from around the globe. If you’ve been on Twitter a lot you’ll have noticed me teasing this since about March last year. This seemed like a good enough spread of data. Below is my second attempt.
Using a robots.txt file Using a robots.txt file Posted by Vanessa Fox A couple of weeks ago, we launched a robots.txt analysis tool. What is a robots.txt file? A robots.txt file provides restrictions to search engine robots (known as "bots") that crawl the web. Does my site need a robots.txt file? Only if your site includes content that you don't want search engines to index. Where should the robots.txt file be located? The robots.txt file must reside in the root of the domain. How do I create a robots.txt file? You can create this file in any text editor. What should the syntax of my robots.txt file be? The simplest robots.txt file uses two rules: User-Agent: the robot the following rule applies toDisallow: the pages you want to block These two lines are considered a single entry in the file. User-Agent A user-agent is a specific search engine robot. User-Agent: * Disallow The Disallow line lists the pages you want to block. URLs are case-sensitive. How do I block Googlebot? Google uses several user-agents.
grep -r "word" . How to Create an "unkillable" Windows Process Download source - 24.91 KB The topic of killing Windows processes has been investigated by developers and users probably from the first day this operating system appeared. Besides the task manager where it is possible to kill (practically) any process, there are a lot of freeware and shareware programs that will do all the dirty job of ending any process you select. Once I came across this problem, I analyzed how several adware programs were running, such as Gator Adware, using methods making it possible to avoid being ended by the user. How It Works Since we cannot forbid the user to select our process in the task manager with the mouse and select the "End Process" command, let's create two processes that are the same - one of them will directly execute the code of the program, while the other one will only monitor whether the main program is running or not. Implementation - Main Code This is the main code of the program. Code of the AntikillThreadWaiter1 Thread Hide Copy Code
Basics - Webmaster Tools Help Crawling Crawling is the process by which Googlebot discovers new and updated pages to be added to the Google index. We use a huge set of computers to fetch (or "crawl") billions of pages on the web. The program that does the fetching is called Googlebot (also known as a robot, bot, or spider). Google's crawl process begins with a list of web page URLs, generated from previous crawl processes, and augmented with Sitemap data provided by webmasters. How does Google find a page? Google uses many techniques to find a page, including: Following links from other sites or pages Reading sitemaps How does Google know which pages not to crawl? Pages blocked in robots.txt won't be crawled, but still might be indexed if linked to by another page. Improve your crawling Use these techniques to help Google discover the right pages on your site: Submit a sitemap. Indexing Somewhere between crawling and indexing, Google determines if a page is a duplicate or canonical of another page. Improve your indexing