Building Vs Buying a PIM: Product Data Distribution (Part 3)
One of the most critical pieces of infrastructure for a scalable e-commerce platform is its product information manager (PIM). It acts as the source of truth for product data, allows team members to add and modify product information, and makes this product information available to various distribution channels.
This post is the third in a three-part series on the technical decisions that go into building a PIM (catch up on part one here and part two here). In this post, I’ll focus on how you can make your product data available to your distribution network.
In modern e-commerce businesses, it’s common for retailers to maintain a website, mobile application, third-party marketplace integrations, and possibly physical stores as well. As your product catalog grows, ensuring that each of these channels has the correct information becomes a bigger challenge, but that’s where a PIM can help.
In this post, I’ll share some of the major technical considerations your team should make. Whether your team builds its own PIM in-house or uses one from a third party like fabric, engineers must understand the channels, output formats, and technical considerations that go into building a PIM distribution system.
Distribution Channels and Methods
While your PIM stores product data, variations, and images, customers don’t make purchases through your PIM directly. They will access your catalog through one or more distribution channels that use data from your PIM, and you need to be consistent across each of them.
PIM Data Distribution: Channels
While consistency is essential to prevent confusion and unmet expectations, each channel you use will have unique distribution requirements. This makes the engineering task of creating an interface for your PIM quite challenging. Before you start building the distribution arm of your PIM, consider the channels you need to support and how your team will get data from your PIM to the channel:
- Online stores: Most online stores built as web applications use a direct connection to a product database or an HTTP API to request product data. Unless you use a third-party storefront, you’ll have the most control over this channel.
- Mobile applications: If mobile purchases are common or e-commerce is a component of a larger offering, you might need to support a distinct mobile application. These are typically backed by an API as well.
- Third-party marketplaces: Many online sellers want to reach customers where they already shop, so they integrate with marketplaces like Amazon or eBay. Most third-party platforms offer API access, but some of the less sophisticated might require you to upload your product data manually as a CSV or XML file.
- Social media: Instagram and Facebook are becoming more important for B2C retailers as advertising and sales channels. Strong visuals are important here, so make sure your PIM’s data includes relevant images and headlines for social selling.
- Print resources: Those of us in online retail tend to forget that direct mail is still a large and growing market. If you need to support print catalogs or direct mail, make sure you know how data needs to be sent to your printing partners.
- Point of sale (POS): Retailers with physical locations or who distribute their products through physical stores must ensure their PIM supports the format required by their point of sale systems.
- Distributors: Finally, if you contract with a third-party distributor or warehouse provider, they’ll likely need data from your PIM to track and catalog products. These providers might be able to use your API, or they may require you to upload product lists manually.
PIM Data Distribution: Methods
Once you know all the channels your PIM needs to support, you need to figure out how product data will be kept in sync between your PIM and the distributor. Typically, your team will use a manual, automated, or hybrid approach to data synchronization, but the method varies depending on the channel.
- Manual: Your team must manually add and update products individually. This is common in distribution channels that are not connected to the internet (as in legacy POS systems).
- Automated: Your engineers set up a data pipeline that automatically feeds product updates to the distribution channel as they happen. For example, your web application could make API calls directly to your PIM to get up-to-date product information on the fly.
- Hybrid: Some distribution channels require you to upload data in a specific format that your PIM might not support. In this case, you might export data from your PIM, manually adjust it in a spreadsheet, and then send it to the distribution channel.
PIM Output Formats (the Firehose)
Now that you’ve decided which distribution channels you need to support and how you can get data to each, you’re ready to start designing your PIM’s distribution system. Depending on whether you need to manually or programmatically distribute data from your PIM, you’ll likely have to support one or more of the following output formats: product feed, API, and manual data export.
Product Feed
A product feed is a single file or list of files hosted on the internet that contains all of your PIM’s product data. You can create a single feed for all your partners and distribution channels or create a unique version of your feed for each of them.
Application Programming Interface (API)
APIs are an integral part of most web and mobile applications, so while implementation patterns vary (REST, SOAP, GraphQL, etc.), it’s very likely that your PIM will need an API of some form.
Data Export
Finally, the most widely supported yet time-intensive method of marshaling data from your PIM to your distribution channels is via manual data exports. While most PIMs should support this method, it provides limited value for large retailers.
Technical Considerations
Whether you support a product feed, an API, manual data exports, or all three, there are a few specific things software engineers should consider. While these are always important elements in software development, each of them has unique implications when building a PIM.
Documentation
Creating good docs for your APIs and data feeds is hard. Keeping them up to date as your product catalog evolves is even harder.
Reporting, Tracking, and Logging
All software needs logging to ensure that developers can quickly respond to issues, but a PIM also requires internal reporting and usage tracking. This helps your team see which product data is being accessed most frequently and by which channels so they can update it accordingly.
Caching
Because speed is so important in distributing your product data, you’ll likely need to implement some form of caching. I’ve typically seen flat file caches in e-commerce product feeds, but APIs might need more complex caching mechanisms that handle various input parameters.
Maintenance
Every engineering team I’ve ever worked with underestimates the cost of building software. I’ve mentioned this in every part in this series because it accounts for up to 90 percent of the cost of running software.
Conclusion
Now that you’ve seen how to store your product data, how to let your team manage it, and how to make your product data available, you’re ready to decide whether building a PIM is truly worth the investment.
If you have an engineering team on hand and you’re ready to incur the cost of building and maintaining a significant new software product, then building a PIM in-house will give you a lot of flexibility. You’ll be able to pick the pieces that you need, optimize them for the performance you expect, and customize data feeds for each of your distribution channels. On the other hand, you’ll likely spend months developing the product.
Using an off-the-shelf PIM from a SaaS provider like Fabric can save you months of development time and untold monthly maintenance costs. With Fabric, you get a robust data storage and optimization system, an interface for your team to edit products manually or programmatically, and an API so you can distribute your product data to any number of channels.