Importing Files

The Imports page in Feedonomics allows you to import feed files which will be used to create your database. A database can be built from a single imported file or from a combination of multiple imported files of different sources and different file formats. All imports can be scheduled independently to ensure that your database always contains the most up-to-date information.

Import Name

Giving your import a descriptive name helps keep track of what function the import serves, especially if you have multiple imports in the same database. In particular, indicating whether an import is a product feed or a join file (see the Import Type section), the source of the file (Shopify, BigCommerce, other eCommerce platforms), and/or what type of information it contains (e.g. “UPC join”) makes for a good Import name.

Import Type

There are two import types that will be of primary concern: Product Feed and Join Info. The simplest distinction is this:

A Product Feed import creates rows.
A Join Info import adds new columns to existing rows.

Every database must have at least one Product Feed import, however databases can have an unlimited number of Product Feed and Join Info imports.

Join Info imports are common for adding information that comes from a source different than your Product Feed, like UPC codes, campaign performance data, or special lists like a list of best selling products.

File Type

There are two options for File Type: Delimited and XML. Feedonomics supports comma-, tab-, semicolon-, and pipe-separated delimited files.

File Location

Files can be imported from a URL or an FTP server. Files can also be imported locally from your computer. Local file imports should be reserved for information that is not expected to change. Dynamic information like availability or pricing should ideally be imported from an internet-based source, like a URL or FTP server, so that it can be updated automatically on a schedule.

Scheduling

Scheduling your Imports is a way to keep a feed information up-to-date. At an Import’s scheduled time, Feedonomics will download the file from the import source (URL or FTP) and rebuild the database with the new file. Schedules can be weekly, daily, or hourly. In most cases, a daily import is enough for merchants to keep their information current.

As a best practice, find out at what time your source file is updated and schedule your Feedonomics Import shortly after that time.

Product Feed and Join Info Imports can be scheduled at the same time. Feedonomics will wait for both files to finish downloading before rebuilding the database. Local files do not need to be scheduled.

Feedonomics Import times are based on US Eastern Time.

Be sure to click “Update Schedule” when you make changes to an Import schedule.

Using Feedonomics’ Import APIs

Feedonomics has many custom-built APIs that allow product information to be extracted from all of the most common eCommerce platforms. Available APIs include, but are not limited to, Shopify, Magento, BigCommerce, and WooCommerce.

Separate documentation can be provided on API import setup.

File Maps

Overview

After a file is imported into Feedonomics, it must be mapped in order to be incorporated into your database. Mapping a file is the process of choosing which columns (or XML tags) in your import file you want to be included in your Feedonomics database, and what name you want to assign those columns in Feedonomics.

Delimited Files

Using Auto-detect

The first time you go to the File Maps page of a newly imported delimited file, you will see a preview window of the file itself, as well as drop-down menus to select options for separator, enclosure, escape, terminator, and encoding. These values can be set manually, or you can use Feedonomics Auto-Detect feature to find the correct values automatically.

Click “Auto-detect”, and when detection is finished, “Submit Delimiters.” Now you can begin mapping your data into Feedonomics. See Mapping Columns From Delimited Files below.

You can always make changes to the File Maps settings, even after using Auto-Detect.

Aside: Separator, Enclosure, Escape, and Terminator Characters

Separator
The separator is the character that tells Feedonomics when to make a new column within a row. Separator characters should be easily determined by looking at the header row of your import file in a text editor. Here are some examples:

Enclosure
The enclosure character is the character that surrounds the data within a column. This is almost exclusively either “ or None. If you see an instance of a double quote following a Field Separator, it is likely that the enclosure character is “.

Example:
id,title,description

123,dress,"Designer abc says, and I quote: \"This Dress is awesome\""

456,"My Awesome Dress", "description for dress"

The title of the item with ID 456 is really My Awesome Dress, not “My Awesome Dress”. Similarly, the description of the item with ID 123 is actually:

Designer abc says, and I quote: “This Dress is awesome”

Escape Characters
If there is an enclosure character, there is likely, but not always, an escape character. Basically, the file reader needs a way to know whether a character like “ or , is meant to serve its special function (as a separator or enclosure character) or whether it is actually part of the data within a field. In the dress example above, the escape character is \. This differentiates between the field enclosure and an actual double-quote mark within the data.

Line Terminator
The Line Terminator is the character that appears at the end of a line, before a new row starts. Generally, this shows which kind of operating system generated the file, as Linux, MacOS, and Windows each use different Line Terminator characters.

In the rare scenario that you switch the operating system from which your import file is generated, you will need to update the Terminator setting in File Maps using Auto-Detect, or by making a manual selection, to allow your database to build correctly.

Mapping Columns From Delimited Files

Once your delimited file has been configured using Auto-Detect, you will see a list of the available columns in your import file and can now choose which of those columns to include in your Feedonomics database. Simply place a value in the “Your Name” column to map it into your database. You do not need to map in all available columns in your import file.

You may also rename the columns in your import file as you map them into Feedonomics. In fact, we recommend doing so in many cases. See the “Use Consistent Field Names” section of the Feedonomics Best Practices Guide.

You can return to File Maps at any point after your database is built to rename, add, or delete columns. Note that if a column is used in a transformer or in an export, you will not be able to modify it.

XML Files

Choosing a Context Node

In markup languages (where the ‘ML’ in XML comes from), a file and all of its data is organized by tags that encompass different levels of data.

When mapping an XML file into Feedonomics, the Context Node setting determines when to make a new row in your database. In other words, with this setting you are saying to Feedonomics, “Create a new row every time you see a _______ tag.”

In the example to the left, There is a <document> tag that encompasses the whole file.

Nested underneath <document>, the <item> tag is the level that the encloses a product. Inside an <item> tag is unique information for each product, like id, name, price, and shipping cost and region.

So the context node in this example would be <item>.

This would be entered into the file like so:

Defining XML Paths

After the context node is set, other XML tags or nodes can be mapped.

Map in a tag that is a direct child—<id>, <name>, and <price> in this example—by simply typing the tag name as the XML path. As with delimited files, the Variable Name column will determine what the column name will be in Feedonomics. Access nested tags—<cost> and <region> in this example—by navigating down the XML tree using slashes. Remember, you are pointing to each tag from the “context” or “starting point“ of the Context Node, <item> in this case.

XML structure can get even more complicated. Refer to the Documentation link on the File Maps page for more information. XML path values are based on the XPath syntax, for which there are many public resources available online.

Querying Data

Refer to the “If Conditions” section of the Transformer Cheat Sheet (found on the Transformers page in Feedonomics) for a comprehensive list of available query statements.

Here are a few tips you begin querying data.

• Query statements can be combined using AND and OR.
  

[title] contains ‘blue’
AND [color] not_contains ‘blue’

• Queries which mix AND and OR should use ( ) for clarity. AND operates before OR in Feedonomics. See “Combining Statements with AND, OR, and ( )” in the Feedonomics Best Practices Guide.
  

( [title] contains ‘boot’
OR [title] contains ‘loafer’ )
AND [google_product_category] not_ends_with ‘ > Shoes’

• You may query for fields existing within other fields. When doing so, make sure your first statement checks that the “search for” field is not empty. See “Common Transformer Mistakes” in the Feedonomics Best Practices Guide.
  

[size] not_equal ‘’
AND [title] not_contains [size]

• Quickly convert a query to its complementary query using NOT ( ). See “Build And Refine Your If Statement With Queries” in the Feedonomics Best Practices Guide.
  

NOT ( [title] contains ‘red’ OR [title] contains ‘blue’ )

• Make advanced queries with regular expressions using match( ).
  

match( ‘^\d{12,14}$’, [gtin] )

Joining Files With Join Info Imports

Imports Settings

When adding the join file on the Imports page, make sure to select “Join Info” from the “Import Type” drop-down menu.

Defining a Joining Key

In order to join in information to your database, there must be a common field to link the information in the Product Feed to the information in the Join Info file, called the joining key. In the join file, the joining key needs to match the name given in the product feed.

Product Row  →  Join Value  ←  Join Information

Consider an example in which UPC information is being joined to an existing database, where the products are uniquely identified by the variable named “id.” The join file would have, at least, a column with the product ids and a column with the UPCs. In the File Maps page, the column with the product ids would have to be mapped as “id” in order to join in the UPC information.

Adding New Joined Columns

Once the joining key is established, new columns can be mapped in. Select your Join Info import on the File Maps page and map in the additional columns just as you would do with a product feed.

Just remember to use unique names that don’t already exist in your database when mapping in new columns from a Join Info import. For example, if your database has a variable named ‘color’ and you want to join in an additional color column from your Join Info import, you will need to use a new name like ‘color_2’ or ‘color_join’.

Exporting Data

After all that work assembling the database, writing transformers, and optimizing the feed, it is time to connect your data to your advertisement and shopping channels and marketplaces.

Adding an Export

Export name
Give your export a meaningful name, like “Google Shopping”, “Google - US”, or “Amazon Product Feed”.

File name
This is the name of the feed file that is exported. Include a file extension like “.txt”, “.csv”, “.xml”, etc. It is a good practice to have file name without spaces.

Example: “merchant_name_google.txt”

FTP Setup
“FTP Protocol” can be set to FTP, SFTP, FTPS, or HTTPS. “Host” is the address of the FTP server, but here’s a pro tip: you can export your files into specific FTP folders by adding the folder name to the host field. Example: ftp2.feedonomics.com/folder_name

Export If
The Product Count condition is very important for protection against exporting files that may be significantly smaller than usual. Say for instance that a database has 10,000 products. If one day an error occurs and the number of products drops to 500, the Product Count export condition, if set above 500, will prevent exports from running until the usual product count is restored. We recommend a minimum threshold of around 85% of the total expected product count.

The text box in the Export If section is used to filter certain rows out of your export as needed. Some common export conditions are

[price] not_equal ‘’

AND [price] not_equal 0

Filters can also be set to export products in select categories or to export in-stock products only, for example.

Custom Exports and Templates

Feedonomics has a number of built-in export templates for common feed types such as Google Shopping, Google Dynamic Text Ads, Amazon, eBay, eBay Commerce Network, Bing, and many others.

You may also create a custom export without selecting a template.

Mapping Fields To An Export

Mapping fields to an export is similar to the process of mapping fields from your import file. On the Exports page, add all of the columns that you would like to export in the Variable Name list, and then write your desired output file header for those columns in the Export Name column. The Variable Name and Export Name do not have to be the same. For example, you may choose a Variable Name called ‘price_brackets’ but assign it an Export Name of  ‘custom_label_0’ so that it can be used in your Google Shopping campaign.

Other Export Options

Separator, Enclosure, and Escape
These values can be customized for each export. See the Aside: Separator, Enclosure, Escape, and Terminator Characters section for more information on these values.

By default, Feedonomics exports tab-separated .txt files without enclosure or escape characters. Newline and tab characters are stripped during export.

Compression
Feedonomics can create .zip and .gzip compressed files. When selecting either of these options, set your “Zip - Inner File name” to be something like “export_filename.txt” and set the “File Name” field in the Credentials section above to be “export_filename.zip” for best compatibility.

Deduplicate On
Use this field to remove duplicate rows in your export. The value in this field must match one of the Export Name values in your export map. For example, to export a parent-level product feed, you may choose to deduplicate on ‘item_group_id’, ensuring that only one row per item group is exported.