Crawl a website with scrapy - *.isBullsh.it In this article, we are going to see how to scrape information from a website, in particular, from all pages with a common URL pattern. We will see how to do that with Scrapy , a very powerful, and yet simple, scraping and web-crawling framework. For example, you might be interested in scraping information about each article of a blog, and store it information in a database. We will consider that you have a working MongoDB server , and that you have installed the pymongo and scrapy python packages, both installable with pip . If you have never toyed around with Scrapy , you should first read this short tutorial . In this example, we’ll see how to extract the following information from each isbullsh.it blogpost : title author tag release date url We’re lucky, all posts have the same URL pattern: . We create a Scrapy project, following the instructions from their tutorial . Now, let’s implement our spider, in isbullshit_spiders.py : Pretty neat, hm?
Box model Summary In a document, each element is represented as a rectangular box. Determining the size, properties — like its color, background, borders aspect — and the position of these boxes is the goal of the rendering engine. In CSS, each of these rectangular boxes is described using the standard box model. This model describes the content of the space taken by an element. The content area is the area containing the real content of the element. If the CSS box-sizing property is set to default, the CSS properties width, min-width, max-width, height, min-height and max-height control the content size. The padding area extends to the border surrounding the padding. The space between the padding and the content edge can be controlled using the padding-top, padding-right, padding-bottom, padding-left and the shorthand padding CSS properties. The border area extends the padding area to the area containing the borders. Specification See also
Ginger Tech Stack Posted by Peter in Django, Ginger For our latest product, Ginger, we wanted to marry the real-time functionality we needed with the traditional Django stack we know and love. After some false starts and falling on our faces in the beginning, we ended with a stack we’re happy with and think will serve us well moving forward. Back-end This is our bread-and-butter. Using Django, we were able to quickly build the nuts-and-bolts parts of our site that didn’t require any special real-time functionality. tastypie for building the REST APIhaystack backed by solr for searchcelery backed by Redis for task queueingmisaka and pygments for text formattingeasy-thumbnails and PIL for image thumbnailing Front-end Our front-end uses Backbone.js to handle most of the routing (using HTML pushState) and interactions. Realtime Push/Pull Our real-time layer probably gave us the most trouble as we got started. Our Node.js server is very small, weighing in at less than 200 lines of code. Other Bits
50 Helpful Typography Tools And Resources Advertisement We love beautiful typography, and we appreciate the efforts of designers who come up with great typographic techniques and tools or who just share their knowledge with fellow designers. We are always looking for such resources. We compile them, carefully select the best ones and then prepare them for our round-ups. And now it’s time to present a beautiful fresh dose of typography-related resources. To help you improve the typography in your designs, we’re presenting here useful new articles, tools and resources related to typography. You may be interested in the following related posts: Typography: References and Useful Resources The Taxonomy of TypeThis article’s purpose is to help us as designers to distinguish basic properties of types. Typedia: A Shared Encyclopedia of TypefacesTypedia is a resource to classify, categorize, and connect typefaces. Typeface Anatomy and GlossaryMany fonts have abbreviations in their names. Finding The Right Type Combining Type Type Design
Exploring Backbone: Part 1 | Blog :: The JavaScript Playground Backbone.js is a framework that lets us structure our applications using a pattern similiar to MVC (technically Backbone is not pure MVC as the C stands for "Collection"). However, Backbone is a powerful system to use when creating apps that are beyond very basic. When passing & manipulating a lot of data, you should consider using something like Backbone. Since launching this blog I've had a lot of people ask me about Backbone. Although there are a lot of very good resources out there, I have struggled to get to grips with it myself and from the requests I've had I'd suggest a lot of others have too. So, I sat down to create a sample application with Backbone, and in this tutorial - which will span at least 3 parts - we will create a very simplified shopping cart application, with Backbone. The first thing to do is set up our basic page and include our dependencies. <! The first thing we want to do is create a model. Here I set up the default values for my item. Don't panic! That's it.
Typography Is The Foundation Of Web Design Advertisement For years you have been searching for it. You hear the question being asked in your dreams as you go on an Indiana-Jones-type-crusade to find the answer. When the answer comes to you, you know that the confetti will fall from the ceiling and the band will start playing your favorite song. You might even get a kiss from that special someone. What is the secret to Web design? A tough question and one that might not have an answer. Of course typography doesn’t mean font selection. More toys means more fun though, right? I’m not being sarcastic, saying that is all you need to know for a majority of websites. Information Architects is based around strong typography. One of my all time favorite designs is A Working Library. A Working Library by Mandy Brown. Some people find design like this to be dull and boring, they feel that design should have more pop to it. Well That Isn’t Hard It’s possible to create a wonderful design without the use of images at all. (jvb) It's done.
VIM Adventures A working library / by Mandy Brown How to write a simple interpreter in JavaScript Download source - 2.9 KB Writing interpreters and compilers Writing an interpreter or a compiler is one of the most educational tasks in programming because you can become familiarized with the details of the code interpretation and evaluation process. You can obtain much deeper knowledge of what sorts of things are going on behind the scenes and gain some insights into the decisions behind language design. This article will perform a basic overview of this process by showing how to write an interpreter for a simple language that we can use for a calculator application. This article assumes that you have intermediate programming experience and basic familiarity with Javascript. Some Background The difference between interpreters, compilers and transpilers In this article, we will be creating an interpreter, as opposed to a compiler. A transpiler is similar to a compiler, except that the source language and target language are about the same level of abstraction. Lexing Parsing Evalutation
Slicker Show and Hide Last time I showed you how to make something appear and disappear on a web page. This time I'll show you how to do it with style. Like we did last time, we'll start with our $(document).ready() and put everything else inside of it. Adjust the Speed This time, however, we're going to adjust the speed at which our item shows and hides. JavaScript: $('#slickbox').show('slow'); Note: If you use a speed word, put it inside quotation marks (either single or double); if you use a number, omit the quotation marks: $('#slickbox').show(500); Attach Effects to Events The final step here is to attach the effects to events. So let's take a look at how we would tie the "onclick" event of one link with an ID of "slick-show" to the "show" effect. // shows the slickbox DIV on clicking the link with an ID of "slick-show" $('#slick-show').click(function(){ $('#slickbox').show('slow'); return false; }); Notice that we attach ".click()" to "#slick-show". Demo 1 Try it out for yourself. Newbie Tip Other Effects Demo 2
Beginning Node.js | Blog :: The JavaScript Playground Unless you've been living under a rock for the past 12 months or so, you've probably heard of Node.js. Simply put, Node is JavaScript on the server. Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices. Node takes JavaScript onto the server, which means it's possible to write your entire application in JavaScript, using it for both the server side, handling requests & rendering views, and then on the front-end as we always have done. Today we will look at installing Node & the package manager, NPM (really easy) and then the traditional "Hello World" tutorial. There's two ways to install Node. If you're a Homebrew user (a package manager for OS X) you can get Node with brew install node and then NPM with: curl | sh.
8 Simple Ways to Improve Typography In Your Designs I wrote this article for Smashing Magazine and it was published last Friday on their site. I’m re-publishing it here for your reading pleasure. Enjoy. Many people, designers included, think that typography consists of only selecting a typeface, choosing a font size and whether it should be regular or bold. These details give the designer total control, allowing them to create beautiful and consistent typography in their designs. Measure The measure is the length of a line of type. A simple way to calculate the measure is to use Robert Bringhurst’s method which multiples the type size by 30. Leading Leading is the space between the lines of type in a body of copy that plays a big role in readability. Many factors affect leading: typeface, type size, weight, case, measure, wordspacing, etc. This takes some finessing to get the right spacing but here is an example of what the code would look like: Hanging Quotes Vertical Rhythm Widows & Orphans Scale
Python Iteration Created 24 April 2012, last updated 19 March 2013 This is a presentation I gave at PyCon 2013. You can read the slides and text on this page, or open the actual presentation in your browser (use right and left arrows to advance the slides), or watch the video: A talk for PyCon 2013. This talk is billed as Beginner, and sounds like a beginner topic, but I prefer to think of it as Fundamental. Plenty of expert Python programmers aren't making enough use of the tools I'm going to talk about. Python has a nice model of abstract iteration which can be used to increase the expressiveness of your programs. My goal here is to show Python iteration in a light that would encourage programmers to explore more of its possibilities. All programs iterate over data, of various sorts. Let's say you have a list of numbers, and you want to print each of them. This while loop works, and is a least common denominator: this way of thinking about the loop can be written in almost any language. To summarize:
Pixel Perfect Monday, 13 August 2012 I’ve spent my whole life thinking about dots, largely in the form of on-screen pixels. I remember first seeing a Pac-Man coin-op arcade game, wondering how it worked, and deducing the basic gist: the screen was a matrix of dots, like animated graph paper. Dots were how computers rendered everything: pixels on screen, dots of ink/toner on paper. Print always has been ahead, at least with regard to dot-density. I can think of almost nothing I have done professionally over the last 20 years that has not been rendered as a matrix of dots. When I went to college in 1991, my parents bought me an inkjet StyleWriter. That gave way to 600 DPI (and eventually a magnificent 1200 DPI model) HP LaserJets at the student newspaper. We loved our Macs and loved designing with them, but what we saw on-screen wasn’t our design work. It was all just dots, and looked like dots. Today’s pre-retina Mac displays are excellent, especially when judged by historical standards. But now this.