Things I wish I knew about MongoDB a year ago I’ve used MongoDB for over a year at scale at both Heyzap and Bugsnag and I’ve found it to be a very capable database. As with all databases, there are some gotchas, and here is a summary of the things I wish someone had told me earlier. Selective counts are slow even if indexed For example, when paginating a users feed of activity, you might see something like, In MongoDB this count can take orders of magnitude longer than you would expect. There is an open ticket and is currently slated for 2.4, so here’s hoping they’ll get it out. Inconsistent reads in replica sets When you start using replica sets to distribute your reads across a cluster, you can get yourself in a whole world of trouble. This is compounded if you have performance issues that cause the replication lag between a primary and its secondaries to increase to minutes or even hours in some cases. Range queries are indexed differently I have found that range queries use indexes slightly differently to other queries. Profiler
Callbacks are polluting your code I have been hacking on a project in Node.js/Express.js for some time now. I am really impressed by how fast it is to code and prototype in Node, not much gets in your way. However, there is one thing I am still struggling with, the callback model of Node.js applications. It is not that it is conceptually hard to understand or use, but i feel that it keeps me from writing clean code. Let’s imagine that we are writing a small nonsense program. app.post(‘/someurl’, function(req, res) {var x = {..}if (some contidion) { someOperation(function(err, result)) { x.result = result x.save(function(err, result) { if (err) next(err) res.redirect(url) }) })} else { x.save(function(err, result) { if (err) next(err) res.redirect(‘/’) }} (Edit: To clarify, both someOperation and save is doing some kind of I/O) So is this clean code? Let’s instead pretend that we are using synchronous methods (which of course would be a very bad thing in Node.js, so don’t do it) and save are methods that would block:
Alexander Luksidadi's Blog » ExpressJS without Jade? Use Underscore template! Many of you must have felt like a burden knowing that Express recommended you to learn another template language (Jade). Don’t worry, you can code all your templates on HTML using underscoreJS! Oh yay? Let’s take a look on how you implement that on your express app. First install express package, create your express app: $ npm install -g express $ express . Install your underscore package $ npm install -d underscore If you edit Now, all you need to do is, to comment out 1 line and register underscorejs: Now, go to routesindex.js : $ vi routes/index.js Change the template name from ‘index’ to ‘index.html’: Next, go to views directory and create layout.html And last, still in views directory, create another file called index.html And there you go.. you can write your HTMl code in peace =)
The secrets of Node's success In the short time since its initial release in late 2009, Node.js has captured the interest of thousands of experienced developers, grown a package manager and a corpus of interesting modules and applications, and even spawned a number of startups. What is it about this technology that makes it interesting to developers? And why has it succeeded while other server-side JavaScript implementations linger in obscurity or fail altogether? The key factors are performance, timing, and focusing on a real problem that wasn’t easily solved with other server-side dynamic languages. Browser wars and JavaScript performance In the early 2000s, AJAX web development was coming into its own and placing increasing demands on browsers’ JavaScript engines. As JavaScript libraries and websites became more complex and users started to notice poor performance in their browsers, browser developers started to focus seriously on their JavaScript engines. Save 50% – JavaScript Ebooks and Videos Evented I/O
Botsikas' Blog: Node.js modules cross platform compilation using gyp Update: I have made a pull request where you can find the updated tools discussed in this article, located here Node.js has been using waf (node-waf) to configure and build modules up to version 0.4. From v0.6 and on, the team has moved on to gyp (Generate Your Projects) which seems to be a bit more promising when it comes to cross platform compilation. This post shows how to create a simply gyp file to build your own custom native node.js modules and provides some scripts to automate the project generation process. A bit of history Gyp is a google project that was created to support cross platform building of the opensource chromium project. Node-waf vs gyp Up to version 0.4 the node.js team offered node-waf (a waf 1.5.3 wrapper script) to configure and build modules for node.js. Node Module’s gyp file I have edited a simple gyp file (see the end of this post for source code) to compile the simple hello world native nodejs module I have used in my previous posts (here and here).
Node.js is Important. An Introduction - PavingWays Once in a while you come across a new technology and are just blown away by it. You feel that something like this should have been around much earlier and that it is (gonna be) a significant milestone, not just in your own live as a developer but in general. The last time this happened to me was when I dug a bit deeper into a project called node.js or just “node” as the binary is called. What is node.js? Node.js itself is a program that will have to be compiled and installed on your machine. What is Special about node.js? There are other implementations for this kind of stuff, but what Ryan Dahlet al do differently with node.js is to use JavaScript as the main programming language and the strict paradigm of exclusively working with callbacks and non-blocking I/O. A Practical Example: The Web Server Each of these threads uses memory. If it would be possible to get rid of all these threads and to put them all into a big pool this would solve some memory issues. Event Loop to the Rescue
style guide Opinions are like assholes, every one has got one. This one is mine. Punctuation: who cares? Punctuation is a bikeshed. This post is concerned with higher-order style. Be obvious Don't do something complex just to make your api simpler. Example, avoid chaining DSLs. This is bad: thing.when('something').then(doThing) It's not really obvious how when relates to dothing. Chaining where you simply return this is acceptable. Be idiomatic, or not. if possible, make your code follow the APIs in node core. If you don't do this, you need documentation. if your can't follow an idiomatic API precisely, do something completely different. ALWAYS PASS ERR IN CALLBACK, (an event listener is not a callback, so this doesn't apply in that case) If you have a function called createServer, it should return a server, and it should have a listen function. createServer should never start the server listening. my favorite API from node is Stream (more on that later) "all you need is lambdas" -- John Lennon now, stream it:
dominictarr/JSONStream V8 javascript VM and Node.js memory management options | O sNAp Memory management behavior is one of the first topics I wanted to understand in node. This will be part one of two articles in which I intend to explore: Memory management / gc options in the V8 VM that runs node.js applications. Debugging / memory leak analysis for running node servers. At a high level V8 uses a generational memory model with a copy collector and incremental mark and sweep. Configuring V8 heap sizes Out of memory errors? --max_new_space_size (in kBytes) Control the size of the new generation. --max_old_space_size (in Mbytes) Control the size of the old generation. --max_executable_size (in Mbytes) The code space size. Controlling when GCs occur in V8 By default, V8 will perform garbage collections on failed allocations, after every N allocations, and on a notification of idle condition by its embedder. --gc_global This controls whether V8 will do automatic garbage collection after every gc_interval allocations. --gc_interval –-nouse-idle-notification --expose-gc --trace_gc
Data mining local radio with Node.js More harpsicord?! Seattle is lucky to have KINGFM, a local radio station dedicated to 100% classical music. As one of the few existent classical music fans in his twenties, I listen often enough. Over the past few years, I've noticed that when I tune to the station, I always seem to hear the plinky sound of a harpsicord. Before I sent KINGFM an email, admonishing them for playing so much of an instrument I dislike, I wanted to investigate whether my ears were deceiving me. Perhaps my own distaste for the harpsicord increased its impact in my memory. This article outlines the details of this investigation and especially the process of collecting the data. If it ain't baroque... A harpsicord is in many ways similar to the piano. The harpsicord can sound tinny to modern ears. At the start of the 16th century, the newly invented fortepiano began to push both the harpsicord and its close relative, the clavicord out of favor. These eras are: One exception is opera. Collecting the data Cheerio
Screen Scraping with Node.js You may have used NodeJS as a web server, but did you know that you can also use it for web scraping? In this tutorial, we'll review how to scrape static web pages - and those pesky ones with dynamic content - with the help of NodeJS and a few helpful NPM modules. Web scraping has always had a negative connotation in the world of web development - and for good reason. In modern development, APIs are present for most popular services and they should be used to retrieve data rather than scraping. The inherent problem with scraping is that it relies on the visual structure of the page being scraped. Despite these flaws, it's important to learn a bit about web scraping and some of the tools available to help with this task. Note: If you can't get the information you require through an API or a feed, it's a good sign that the owner does not want that information to be accessible. Scrapers can be written in any language, really. Let's start with the simple use-case: static web pages.