Jeremie Agnes
Online tv/movies. Gratuit: cartes de visites avec code barres en code QR ! meCard/vCard/URL 2D Code. Veille. Cyrille.verdier. Erictaunais. Joomla. Référencement naturel. Outils Du Net - Accueil. Outils Du Net - Outils Webmasters - Generateur de robot.txt. Robots exclusion standard. The Robot Exclusion Standard, also known as the Robots Exclusion Protocol or robots.txt protocol, is a convention to advising cooperating web crawlers and other web robots about accessing all or part of a website which is otherwise publicly viewable.
Robots are often used by search engines to categorize and archive web sites, or by webmasters to proofread source code. The standard is different from, but can be used in conjunction with, Sitemaps, a robot inclusion standard for websites. History[edit] The standard was proposed by Martijn Koster,[1][2] when working for Nexor[3] in February, 1994[4] on the www-talk mailing list, the main communication channel for WWW-related activities at the time. It quickly became a de facto standard that present and future web crawlers were expected to follow; most complied, including those operated by search engines such as WebCrawler, Lycos and AltaVista. [citation needed] About the standard[edit] A robots.txt file covers one origin. Disadvantages[edit] Mobile Barcode Link Generator - Free Barcode Image for Your Site.
Joomla. Marketing. Marketing & Facebook. Marketing & Social Media. Facebook pour les entreprises. E-commerce. Marketing mobile. Secret-Internet. Audio Books. Awesome.