background preloader

Importing

Facebook Twitter

Feeds. Import or aggregate data as nodes, users, taxonomy terms or simple database records.

Feeds

One-off imports and periodic aggregation of contentImport or aggregate RSS/Atom feedsImport or aggregate CSV filesImport or aggregate OPML filesPubSubHubbub supportCreate nodes, users, taxonomy terms or simple database records from importExtensible to import any other kind of contentGranular mapping of input elements to Drupal content elementsExportable configurationsBatched import for large files Installation For requirements and installation instructions, refer to the included README.txt file. Documentation. Feeds QueryPath Parser. QueryPath. QueryPath is a library for searching and manipulating HTML and XML documents.

QueryPath

It's like jQuery for PHP. See for more information. Feeds XPath Parser. This project is in maintenance mode.

Feeds XPath Parser

No new features will be added. New installs should use Feeds extensible parsers instead. Feeds XPath Parser is a Feeds plugin for parsing XML and HTML documents. It enables site builders to leverage the power of Feeds to easily import data from complex, external data sources. Each element you wish to extract is setup using configurable mapping queries, saving time for developers who would otherwise have to code complex, specific-use modules. You may need this module if you would like to: Import XML or HTML documents into nodes, users, taxonomy terms, or regular database tablesScrape webpages like regular feed sources with scheduling, updating, and expiringExtract content from HTML documents to create a semantic data bank or mashup Features Notes If you’re not familiar with XPath, but know CSS or jQuery, you might be interested in Feeds QueryPath Parser which has the same features with a different query language.

Feeds REGEX Parser. Feeds JSONPath Parser. This project is in maintenance mode.

Feeds JSONPath Parser

Feeds OAI-PMH Fetcher and Parser. This is a Drupal module that fetches and parses OAI_DC (Dublin core) metadata records from OAI-PMH services, as defined by It depends on the Feeds module.

Feeds OAI-PMH Fetcher and Parser

Features Harvests from OAI-PMH repositories, respecting resumptionTokens, compression (but no deleted record support yet). Can map OAI_DC metadata into Feeds targets (CCK, taxonomy, etc.) (other metadata schemas can be supported by this or other modules in the future, today you can roll your own using existing modules) Can harvest from the entire repository or a single set. Available sets are loaded in via AHAH when creating the importer. Requirements You need the Feeds module and its dependencies (Ctools, etc.) Feeds Crawler.

Feeds Crawler is a Feeds fetcher plugin for paging through a site or a feed.

Feeds Crawler

It is useful for building powerful web scrapers using Feeds. Features Can paginate url patterns using a start index and number of max resultsCan automatically find the next link for RSS, Atom, and some HTML sitesCan fall-back to a user-provided XPath query for the next link Installation. Feeds Tamper. Feeds Tamper provides a small plugin architecture for Feeds to modify data before it gets saved.

Feeds Tamper

Several plugins are available by default and are described in the examples section below. Additional plugins can be added in separate modules or through the issue queue. Examples: Feeds Comment Processor. Feeds Image Grabber. Feeds Image Grabber (FIG) is successor of FeedAPI ImageGrabber, an addon for Feeds module, which grabs image for each feed-item from their respective webpages and maps it to an image field in the node created by Feeds (Node Processor).

Feeds Image Grabber

[Google Reader (with thumbnails) can be imitated on a Drupal site by using Feeds, Feeds Image Grabber, FileField, ImageField, ImageCache, ImageAPI, Views and Voting API module ( Features Automatically downloads and attaches image to the node created by Feeds module.Configurable XPath of the desired image location on the webpage of the feed item on per feed basis (using element's id or CSS class).Support for FileField Paths module.Configurable maximum image size.Configurable minimum and maximum image resolutions.Configure defaults for FIG from the settings page.

Roadmap Better error logging and reporting.Integration with Views to grab images for only selected feed items. Please do not open issues in the queue that request features already on the roadmap. Feeds Self Node Processor. A Feeds processor plugin that can be used to populate the fields of a content type using a remote source.

Feeds Self Node Processor

It uses Feeds' attachment ability to make Feeds-based import workflows more extensible. It allows you to use the Feeds module as an importer for single objects instead of only lists of items. In Feeds' terms, the Feed is also the Feed Item / Feed Node. In most cases, the Node Processor is configured with a type of node you want to create, and with this you don't need that option. You may need this is if your data source has remote data in both a list and in more detail in its main display. Use case: An API that allows you to get a list, and get an item, but not a list and its items with all their properties. Credits: Import. Add additional Ubercart fields to node_import (possible merge with issue #913406?)

How best to import Ubercart products and images? Update existing nodes on import. Views Importer. This module provides an easy way to import previously exported Views definitions.

Views Importer

Create 'sites/all/imports/views' folder.Export the Views you wish to import and paste the export code into separate text files with '.inc' extension.Place the newly created text files in the 'sites/all/imports/views' subdirectory.Do not forget to add '<? Php' line to the top of the files.Ensure all dependant modules are installed for the fields in the views.Install this module. Upgrade instructions 6.x-1.0 to 6.x-1.1 Move all your views' files to 'sites/all/imports/views' folder.Add '.inc' extension to each view file.Add '<? Php' line to the top of the files. The idea is that you could have a collection of commonly used views in a folder. Database-Diving in Drupal + Ubercart. Moving Data to Drupal / Ubercart. Update (August 2011): We’re now recommending that people try using the Migrate module ( first, before trying a custom-coded import solution.

Use with Migrate UI, and check their list of required modules carefully (depends on version of Drupal you’re using). The solution below is for situations that Migrate can’t handle well, or where you need to do imports that tie into custom modules that have their own data tables. We’ve also built a similar importer as a Drupal module (its highly customer-specific – if there’s demand we can look at generalizing it) which can work well in situations where you need a quick upload and import inside the admin panel with a well-defined data layout (i.e. a CSV product list).

The following blog entry describes the solution to an issue encountered by Jeremy and Nathan in moving data from an old website to a new one. We’ve recently had to convert data from an old custom-built shopping cart, to a new website based on Drupal and Ubercart. How it works: