IndexTank is now open source!
Realtime Search With Lucene at Twitter. Detect Stolen and Duplicate Tweets with Solr. We Recommend These Resources A new feature “duplication detection” is implemented for the open source webapp jetwick and seems to work pretty good thanks to the great performance of Solr.
To try it, go to the tweet about this blog post and click on the ‘Find Similar’ button below the tweet to investigate existing duplicates. With that feature it is possible to skip spam, identify different accounts of the same user, skip tweets with wrong retweet or attribution. Welcome to Apache Lucene! ElasticSearch - Open Source, Distributed, RESTful Search Engine.