Presented by David Deppner
Mage Titans, Austin, TX, October 26, 2018
Nearly every Magento site has a Google shopping feed. But many developers and digital marketers only do the minimum to get their products approved for shopping ads, leaving many products rejected for policy violations and data quality issues. In this presentation, we discuss some common reasons for product disapprovals and how to fix them. We cover the differences between the Magento data architecture and Google’s outlook on the product data. We then dive a little deeper and discuss how to enrich the data feed, drive better matches, reduce advertising cost, and increase profits for Magento merchants.
TRANSCRIPT
I’ve been working in the Magento ecosystem for about 10 years now, since 2008, mostly on the merchant side and more recently working for Psyberware. We focus on managing advertising for Magento merchants, mainly mid-sized companies with some more sophisticated challenges. And we take a broad view of that and we get involved in all aspects of anything relating to advertising, so that includes a lot of analytics and financial projects, understanding returns, also the technical issues of creating shopping feeds, tag manager, all that sort of stuff, a lot of conversion tracking issues.
Today I want to talk a little bit about the opportunity of improving profitability for merchants through data quality improvements, and then specifically address some of the issues that typical Magento merchants have with different product types in Magento. There are a lot of issues there. And then kind of follow a perspective of first looking at how to get products approved, how to make those products more findable in Google, and then how to ultimately get more clicks on those products so we can make more sales.
The opportunity here is vast. There are a whole lot of merchants out there where Google Shopping Ads are driving like 25 or 50 percent of their total revenue. Also the Google Shopping feeds are used in a lot of other platforms, you can use them for remarketing ads on Facebook, for example, and a whole lot of other marketplaces. So I think this is a good place to start with some of these things.
When an agency or SI sets up a new M2 site for a client, they frequently do a really good job of getting the products exported out of Magento, but then oftentimes there’s a little bit of a stumbling block there, where additional work doesn’t get done and the merchant might not have the technical sophistication to deal with some of the issues that come up. But you’ve got to think, in order to make money, we have to get the product exported out of Magento, we have to get the product approved — no data quality issues — then we have to get the product found by a user when they’re searching for something on Google, then they actually have to click the ad. And after all that, we stand a chance of making a conversion. And we focus a whole lot on that top line and we neglect a lot of things in the lower part of that funnel. So that’s the area we’re going to talk about the most today.
And right off the bat I’ll say, the product types in Magento are one of the major things that confuse people. I’m not going to go through all the product types – that could be its own presentation – but we’ll take a few examples. Configurable products are very common on merchant sites, but they’re basically a violation of a whole lot of policies on the Google product feed side. So we have this situation where you’re going to select a few things and configure the product and add it to your cart, and that seems intuitive to an end user. And what that ends up being in the Magento database – I’m sure just about everybody in the audience understands this – is you have a number of simple products with individual SKUs, and they’re sort of collected under a parent product. So once you make your configuration choices from the drop-down, it actually adds the simple product. That’s what you check out, that’s what you purchased. But on the Google side, Google has all these policy requirements that basically say if you have a product that varies on anything, you can’t add that to the feed. You actually need to add the individual product variants. If the shirt can come in several different sizes, every one of those sizes needs to be a separate product in the Google feed. You’re not supposed to include the parent configurable product. It causes all sorts of problems and all sorts of policy violations, and that’s where you get disapprovals. And then there’s a special field in the Google product feed specification, the item_group_id. That is the field where you’re tying all the simple products, all the individual variants, together under a common code so Google understands that they’re all variants of the same basic thing. So this is possible to set up in most of the feed extensions, but it’s amazing how many people don’t actually set this up correctly. As an example, in the shopping feed we have the individual SKUs set up as separate products, we do not include the parent configurable, but the only place the parent configurable is referenced is that item_group_id field that ties them all together. And then we need to use the extra optional attributes in the Google feed spec to list how the variants differ.
Landing pages are the other really big problem with configurable products, because if you’ve got the individual simple products set to be visible on your site, you can use those individual simple-product product pages as the landing pages for the ads. That’s easy, but a lot of sites actually don’t do that because they want all the “link juice” to flow to the parent configurable product or they have other reasons they don’t want them in the site map being visible and exposed to the rest of the world. There are a couple of work-arounds that you can do. Magento out the door does support the ability to add this #142=168&93=50 and that’s basically saying if you put that extension on the end of a URL you can basically pass some configuration information to that configurable product page and javascript on the page is going to then set the correct size based on the attribute ID and the option ID. And it’s going to pre-select the right product, theoretically it’s going to show you the right image, and that’s going to give the user that experience of actually seeing the thing that they clicked on in the ad on the configurable product page. One point I will note is that, if you’re using swatches, this breaks. There’s a little bit of a javascript bug right now that hasn’t been fixed yet. But if you have drop-downs, it works just fine.
So in a case like this, here’s your shopping feed, the links to the actual products would just look like that – you’ll have the pound sign and then those attribute codes. Obviously that’s a little hard for, say, somebody on the marketing side of the business to figure out what’s the correct product URL for an extra-small blue shirt, so there’s a couple of shortcuts for how to do that easily, and if you want to talk about that, ask me questions later. But you don’t need a developer to figure that out – it’s actually pretty easy.
Another product that causes people a lot of problems is grouped products. This is a product type where there are actually several simple products on the same page. None of them are really the main product. This causes a lot of issues, similar to configurable products, with the Google policies, but even some additional issues because Google needs to see the price of the product on the landing page and it needs to be the most prominent price on the page. Of those three, none of them are the most prominent, so we have the same issues as configurable products, where we actually need to list all the products separately but in this case we can’t even use this as a landing page. If you actually think about what a grouped product is, it’s a category. Google looks at this as a category – it’s a collection of three products. None of them are the most prominent, and it violates so many policies, you basically can’t use these in Google.
Bundled products have a lot of problems – except, not always. If you have a simple static bundle that always includes, say, the same three products, you can’t configure it, it’s always just three products, you can do that in the Google shopping ads. You have to set a special field “is_bundle” to “yes,” and the product image has to show all three products, but you can use a bundled product in a context like that as a landing page. That one price – the price doesn’t vary.
But if you’re doing any sort of complicated configuration like I just showed you out of the sample data on the Magento 2 default install, you’ve got something with all sorts of check boxes and the product pricing is not a static price all the time, there’s absolutely no way to use these inside of Google Shopping, and instead you have to include all the simple products and deal with them individually.
Most people are aware of all the issues with getting products approved or rejected in Google Merchant Center. Let’s talk about just a few of those issues that relate to Magento. One thing to understand is that the product feed specs have a lot of information in there, and only the requirements matter for getting products approved. There’s a lot stuff in there that talks about best practices or what you should do, but you read that specification very carefully and understand what is required. That’s all you need to do to get products accepted. Forget everything else at this point. Also in Merchant Center you’re going to see a lot of warnings and notifications on things where there’s something not quite right Google wants you to do something different – again, none of this matters for getting products approved so just skip it for now. One of the easiest ways to fix a lot of these data quality issues is actually with a new feature that Google released just over a year ago called Supplemental Feeds. The really easy thing here is that you can go in and create a Supplemental Feed and it’ll patch the data in your main feed. So sometimes it’s more complicated, there’s more departments involved and maybe you’ve got integrations with PIM systems, you’re trying to pull data from multiple sources, and making a change to the Magento feed extension is actually rather difficult, but you can go directly into Merchant Center, create a Supplemental Feed – it’s a Google Sheet – you’re just setting an ID column to your SKU and then whatever other columns you want to override, and you can just let a marketing employee go in and edit that Google Sheet and patch data that’s incorrect and very quickly fix things. Now, from a development perspective, it’s also very useful because your feed generation could take two hours in the middle of the night and you might not be able to wait till tomorrow all the time to test something out, but you can create a Supplemental Feed, patch an individual product and try to fix a data issue — it’s still going to take some time to refresh that data, maybe you’re going to go half an hour between iterations — but you can use these to really quickly iterate and solve problems, and then you can fix the main Magento feed later when you have some issue. I use these all the time with clients.
GTIN issues, aka UPC codes, this is probably the major reason for product disapprovals at this point with your average merchant. Google uses these codes to basically determine what is this product, and understand what it is in the context of all the other people advertising as well. They’re really easy to find usually, but they’re just labor intensive, and again, if you can just use Supplemental Feeds to de-bug these things and push them down to lower-level marketing employees or whoever you’ve got that’s maybe a little bit less dollars per hour, a whole lot of these can just be found by searching the same product on Amazon – somebody’s already got the UPC code listed. There are databases to find these, it’s just a little bit labor intensive. With a little bit of work, you can write a little code and just query a whole bunch of stuff very quickly against many of the API providers out there. But some of the really common errors are just caused by Microsoft Excel. So people pull in large spreadsheets with all these numbers, many of which have leading zeros, they pull it in to Excel and then export it as a CSV or tab-separated file. Two key issues – number one is leading zeros get dropped because Excel says this is a numeric field so we’re just going to chop off these leading zeros when we export it. That’s a problem, and most UPC codes in North America start with a zero, so we almost always lose data when we do that. The other thing is the checksum at the end – that’s an issue that comes up a lot when people in warehouses are keying in numbers, they see a barcode like this and they key in the ten digits in the middle and they don’t realize the zero at the front and the 4 at the end are also part of the UPC code. But that 4 at the end is a calculated field and if you don’t have it you can just calculate it, add it on the end. So between fixing the problem with dropped leading zeros and calculating checksums on the end, you can really quickly take 2000 GTINs that are in error and check them all. The other way Excel kills GTINs is they convert them to scientific notation, so if you ever see one of these, say 6.5 E+11, you know that somebody edited it in Excel. But that also means they got the data from somewhere else before they edited it in Excel, so you just have to go back to that original data source and don’t screw up this time and import it correctly. It’s really simple to fix these things.
Images too small – a major issue that’s really easy to fix on the Magento side. We’ve got this requirement that we have to have 100×100 pixel images, minimum, so here’s a lawnmower blade that only 86 pixels high. That’s a perfectly fine image. All we’ve got to do is a simple shell script command to re-center that on a 100-pixel high canvas with a white background, and save that image out. It can be that simple. It’s really easy to write some PHP code to scan every image on a site and fix them all — all, quickly, five minutes. Maybe it takes you a couple hours to write some code to do something like that, but it’s easy. You can do it from a command line, or if you’ve got a whole lot of labor and you want to throw it at designers, you can give them 500 images and a lot of Photoshop folks are really fast and they can go in and batch it up and get it done really quickly as well. But I do like the automated solutions a little bit better.
Missing attributes – Google gives you an error and says, Hey, you’re missing a color attribute, and the color is required for products that vary based on color. One thing you have to ask yourself is how does Google know that this product even has a color, because all they know about this product is what you told it. Frequently, Google’s machine learning algorithms are kind of like scanning through the product title and descriptions and they’re seeing things that suggest this varies by color, and they’re saying, Oh, I saw “red” show up in the title and also the description. Again, you can fix a lot of these very quickly just by using some regular expression pattern-matching, finding the products that have these errors, looking at what color words are in the titles and descriptions and just adding these optional fields to patch them. Very quick, very easy.
Landing pages – I already talked a lot about landing pages. I’m not going to go into it much more but Google basically wants to make sure the price, availability and condition of the product on the landing page match what you have in the feed, and that’s the ultimate problem with a lot of the different Magento product types because Google can’t parse one of those grouped product pages and figure out what the price is when you land three different things on that page. There’s also some issues sometimes with schema.org mark-up, the hidden fields on the page that sort of signal to Google what the different fields mean and which product they’re associated with. You get issues with, you know, there’s an invalid price and the price is set to zero and that’s almost always because somebody added a configurable product to the feed. What’s the price of the configurable product? It kinda doesn’t really have one. What’s the availability of it, this virtual parent product? What’s the condition of this virtual parent product? It just doesn’t exist. You can fix most of these issues with understanding the Magento product types.
Findability is something that is far more important than we give it credit for when we’re just looking at how to get products approved. Findability – we’re just talking about when a user types a search into Google, how do they find the right answer to their query, how do we match them to the right product. And this is where all the best practices that are in the product feed specs start to become very important, because Google gets a lot of clues about how they’re matching searcher intent with your data in the best practices and the recommendations that they have and also some of their policy documentation. In this particular case, when we’re talking about the link, what the landing page needs to be, they’re saying it’s best if you pre-select the correct variants. They’re saying things like that. Maybe that one doesn’t matter for findability, but recommendations in general – well, many of the recommendations on various Google product feed attributes – are very important to findability. The other thing is, I said before, those warnings in Merchant Center don’t affect product approval but a lot of them really do have to do with findability. Google’s going to give you some errors about “well, there’s some ambiguous GTINs, we can’t determine really what the right code is for this product based on what you told us.” Well, when you think about it, they’re using that GTIN to identify the product and match searches to it. Maybe the product isn’t disapproved, but if you resolve those warnings you’re going to get a lot more searches matching the right products, and a lot fewer of the wrong searches wasting money on clicks that you don’t want.
The Title Attribute is generally regarded as the most important — that’s just the name of the product. The Magento product name is probably not ideal because you probably didn’t write the Magento product name with an eye towards essentially its SEO use. People have different things they do for the title, the meta-titles and so on, what we do for product name on the page and how we’re putting it in an H1 or an H2 and so on. But that product name might not be right for a shopping ad. There’s kind of a whole different set of ideas. You definitely want the top keywords in there, you want the top keywords at the front of that, and you want to have attributes that vary, so if it’s a blue shirt, you want to say it’s a blue shirt. You’ve got to think about what matters most to searchers and put those most important words first. This is something you really can’t automate, obviously, but it’s crazy important and if you can get somebody to work on this you can at least follow the 80/20 rule and get people to work on the data for the top-selling products and make them even better.
Description Attribute is obviously very keyword-rich as well, but here we find a lot of issues with what people put in the description by default when they’re just exporting their Magento descriptions – things that kinda mess up Google’s algorithms. So think about if you’ve got a part for some machine and you’re talking about that other machine, Google might start to think that this is that machine. You have all these keywords about something else, or these things go together, or if you like this you might like that, right? So if that’s actually in your product descriptions, that can be a big problem and confuse the matching algorithms and you get a lot of inappropriate clicks. Also, all that fluffy marketing double-talk, just business jargon, talking about how great your company is – that’s not relevant to a searcher and that’s just actually going to make matching searcher intent to the products that much worse. There are a lot of things that are really good to include, but oftentimes it’s more important to figure out what to chop out of the descriptions before they’re exported to Google.
Findability – I’ve already hyped on GTINs and how important those are. There are a lot of products that don’t have UPC codes so if you can have a manufacture brand and part number, that’s the next best thing. Some Magento product feed modules will actually use the entity_id in the Magento product tables as the unique ID for products as they’re exported, and this is not optimal because you might actually have customers of a particular business searching for the exact SKU they bought last time and while the requirement for that ID field is that it’s just a unique value, it’s better to set that to your SKU rather than your entity_id because that actually improves your findability for that product in the future.
Optional Attributes – the thing about Optional Attributes is almost none of them are truly optional, they’re required in certain circumstances, like color is required for products that vary based on color. But you can still put something in the color attribute for all the products that don’t vary based on color, and you’ve got to think to yourself Google is using these fields – yeah, they’re going to do keyword matching on the product’s title if you’ve got the color in there, but how much better are you if you make it explicit by putting that color in the color attribute. It’s very clear. Google uses all of these fields to improve their search matches. I think it’s important to spend a little bit of time and actually try to hook up your data to get these guys exported.
I’m going to wrap up by talking just a little bit about getting more clicks. When the ads actually show on the screen, there’s not a whole lot of data shown. We’ve got the image, we’ve got the price, we’ve got two to five words of the product title maybe, if we’re lucky; there’s not a lot. We’ve got some extensions down here, star ratings at the bottom, returns, shipping, that sort of stuff. It’s important to think about those first two to five words of the title and how we specify the image and so on. One of the issues that we find is that oftentimes the image on a product page is not the best image for the feed. The feed image is going to be shown in a really small context, and maybe you’ve got like six images and you want to be able to specify which one gets exported. The quick thing to do in Magento 2 is just make a new image type, a media type attribute, you can add that to all your attribute sets and that’s gives you the ability to specify a feed image just like you can specify your base or thumbnail or small image in Magento. Really quick hack, really easy to do. You can configure your feed extension to use the base image by default but if there’s a feed image set on a particular product, use that one instead. That can really help.
Product title – since so much of the title isn’t visible, you sort of have to think about what are the most important words to display to the user. This is something that not a lot of Magento merchants actually put a whole lot of time into. That’s rewriting all of those titles to get those most important words right at the front.
A word about Merchant Promotions – I’m just shocked at how many Magento merchants go to all this trouble for all these different promotions they are advertising, blasting out on social media and so on, and then they don’t actually put most of those promotions into Merchant Center. It’s really easy to do, it’s not that complicated. Maybe there’s some that you don’t want to put in and there’s some that you do, so there’s a few tricks to how to configure that. But these really, really have an impact on people clicking on products and purchasing them.
And Product Reviews – again, this is a no-brainer. They’re pretty easy. The challenge with this is that there are so many different ways to implement product ratings in Magento, all these different third-party solutions, you’ve got some things you can do inside Magento, so there’s no simple one answer to how to configure this but it’s not that hard. You do have to get some approval for these from Google, but it’s easy to get set up and they really do have an impact on click-through rates.
The Payoff – every step along the way does something for your store. For sites that are getting an awful lot of traffic from Google Shopping, these data quality improvements really improve profitability. One thing to think about when you’re dealing with really tough issues, like you’ve got a thousand products rejected for invalid GTINs and you’ve got to sit there and have somebody slog through figuring out what they are, all of the other merchants that are using the same data set – maybe some spreadsheet from a manufacturer – they all have the same problem. They’re all using bad data, and if you actually put the work into figuring out what those GTINs are and get those products approved, you have a whole lot less competition on those products. The typical merchant out there isn’t actually fixing all those issues. Getting more products listed obviously gives you the opportunity to get a lot more clicks. A lot of these data quality issue improvements really improve the click-through rate, and if we’re doing a better job of enhancing that findability and matching the right search to the right product to the right page on your site, we’re actually really improving helping people find the right thing so your conversion rates actually go up significantly. It’s really easy to measure. And ultimately that’s what drives the profitability on these guys. And in particular over the last couple of years, the rise in traffic on Google Shopping Ads has just been through the roof and it continues to climb. That’s why it’s important.
I’d love to talk to anybody more. I know we just skimmed some top-line things, but feel free to reach out, feel free to chat. I’d love to help out in any way possible if you’ve got some particular issues.