Presented by David Deppner
MagentoLive UK, London, June 20, 2016
This presentation offers a business perspective on managing your Magento 2 upgrade project, and when to get started (spoiler alert: start now!). I speak from a merchant point of view to illustrate why you may want to move to Magento 2 early, how to evaluate the tradeoffs between risks and rewards, when to use outside developers, and managing the complexity of a project like this.
TRANSCRIPT
All right, thanks, it’s great to be here. It’s my first time in London, so I’m enjoying the weather. [Laughter]
So we already went through introductions. I’m going to go ahead and move past that right now.
The agenda today is… I really wanted to come to you and talk about the business perspective on migrating from Magento 1 to Magento 2. There’s been a lot of talk from developers about the technical issues involved in these migrations. There’s been a lot of talk from the sales and marketing people about how great Magento 2 is, and I want to talk a little bit from the merchant perspective and from the business perspective at a merchant about why we chose to move to Magento 2 early. Some things that you might be considering about when to upgrade, questions about what the risks are and the trade-off between risks and rewards, and then on a project like this, managing the incredible complexity involved, simplifying things so it’s not quite so complex, and then working with developers through your project.
So just a little bit about ClearBags. This is a holiday shot of a lot of our products, but what you see in this picture isn’t really what our products are. It’s what you don’t see. It’s all the clear plastic product packaging around these things, so all the bags, and the boxes, everything from archival quality packaging to package a lot of greeting cards to food-safe packaging for bread and cookies and chocolates.
So ClearBags, to give you an idea of our scale, we’re not a massive Fortune 500 company and we’re also not really tiny. We’ve got about 75 employees between two locations, one in California, one in Tennessee, so we can cover the U.S. and North America pretty easily with quick shipping. We have about 5,000 SKUs on our site, we do dozens of orders per hour, but we’re not one of these massive sites doing thousands of orders per hour.
We actually are old-timers with Magento – we’ve been here since the beginning. We started in 2008, when the first official version of Magento came out of beta. We started working with that at that point and migrating a previous, kinda home-brewed solution for ecommerce into this new open-source platform. We launched that site in February 2009 and we’ve been running with Magento 1 ever since. People ask me often-times if we’re using Magento Community or Enterprise, what did we launch with, and back then there was no such distinction. There was no Enterprise edition. So we’ve been on Community the whole time. In fact, there’s features in Enterprise that we didn’t have access to back then that we developed in-house.
So also, just to put it in context – no hosting companies back then, right? No certified developers, no documentation to speak of, so I will say, the code quality of what we’ve done over the years has improved, but we’re still running on code that we wrote in 2008 when we were flying blind.
So we decided to move to Magento 2 early. If you look at the timeline here, we started preparing for it in the second quarter of 2015. We stopped development on Magento 1 at that point and we started doing clean-up to make the transition easier. That’s before Magento 2 was even released. Now I’ll talk about why. In the fourth quarter we selected Creatuity as the agency to help us implement this site. We did most of the development in the first quarter of this year [2016]. We also had some lingering things, sort of a long tail of the remaining issues, bugs to fix, things to tweak, “oh we forgot about this…” so we’ve been doing some of that ongoing migration and clean-up.
So we obviously decided to migrate early, so I have some real thoughts about when you should upgrade, when you should start this process. I think you should start now! [Laughter] Let me tell you why. So right off the bat, we know that they announced even before Magento 2 got released that they were going to be continuing to support Magento 1 for three years after the general availability release of Magento 2, and so you just do the math — we’re looking at Magento 1’s end-of-life being the end of 2018, okay? Now any upgrade project is going to take you months to do. So stop and think. You want to have some padding in there, you don’t want to be running on a site that’s no longer being supported with bug fixes and security patches. You need to start a project like this by the end of 2017 at the latest, in my opinion.
Basic supply and demand – if there’s a stampede of companies rushing to upgrade, at that point costs will rise.
Another consideration is – are you actively developing on Magento 1? If you’re actively developing on Magento 1, you’re in a situation where you have a short useful lifespan, a short period to get any ROI on those projects, before you’re going to upgrade to Magento 2 presumably anyway. So if you can get to Magento 2 sooner, you’re going to cut out a lot of duplicate development time when that point comes.
A lot of key extensions haven’t yet been migrated to Magento 2, but a whole lot have, and if you don’t have all of the extensions that you use on your site available to you right now, that might be something making you think that you should wait. But what you need to think about isn’t whether they’re available right now, but whether they’re gonna be available in a few months when you’re actually ready to deploy your Magento 2 site. So you need to have a dialogue with any extension vendors, and I have this point here – Beware! When you talk to extension vendors, they’re very gung-ho. They will say “Yes, we have a Magento 2 extension.” But you need to dig a little bit deeper and ask some questions about the key features that you depend on from that extension and make sure they’re all there, because many extensions don’t have feature parity between the Magento 1 and the Magento 2 extensions. So if you’re giving feedback – feedback from paying customers drives development — if you’re giving feedback saying “I need this feature in three months,” it gets on the roadmap and maybe that feature will meet you at the point in time when you need it.
Couple comments about Magento 1 versus Magento 2 costs. I’ve seen so much from developers talking about is Magento 1 harder to develop for, or easier, does it take a little bit longer, a little bit less time. People are, you know, on both sides of that a little bit. I don’t know that that’s the right question. When we looked at the Magento 1 lifespan, how long we developed on this platform and how much useful life we got out of it, that’s seven years of life on Magento 1. I don’t know how long Magento 2 is going to be out there in the market, I don’t know long that development that we do now is going to be used for, but I know it’s going to be quite a few years. And the costs that we’ve incurred moving to Magento 2 are actually far less than what we actually spent over the last seven years on all the things we did on Magento 1. That point – if you migrate sooner, you have a longer useful lifespan and when you consider those costs amortized over – who knows? – five, seven, eight years – we don’t know – but you wait two years from now to do it, yeah, pay-back period is going to be shorter.
So the key objection I hear from people about moving now, or moving last year before it was even released, is what if something goes wrong, what if there’s an issue with Magento 2, a technical problem, what are the technical risks involved? So, you know, let’s tackle it head-on, what’s the worst case, what’s the worst that could happen? You get some catastrophic bug in Magento 2. You go and launch the site and there’s something that makes it so you can’t process orders, it’s down. The thing to think about a technical risk this catastrophic is that Magento 2 is not an in-place upgrade to Magento 1. You’re going to be deploying a whole new code base and a new database on a new server. And what launch actually looks like is, here’s your Magento 1 site over here, here’s your Magento 2 site over here. Your domain name is currently pointing at your Magento 1 server, and when you launch, you’re just going to switch it over here to point over here to Magento 2. You can even get more technically complex than that with some firewall rules to do a soft launch so you just send some of your traffic to Magento 2. But my point with this is that your Magento 1 server didn’t get affected by that. If something goes wrong, just point it back. The technical risk here is minimal. If you need to, you can just sort of hit the pause button, solve whatever the problem was, and then relaunch at a later date.
So I don’t think there’s huge technical risks, but I think there are huge financial risks to not moving forward. And I’ve basically already covered them. There’s probably going to be a stampede to upgrade in a year or year and a half, there’s going to be a lot of people and it’s probably going to drive up development costs, or you’re not going to be able to get on the schedule to get your project completed. You’ve got also an opportunity cost. There are new features coming out in Magento 2 now, okay. The Magento 2.1 release adds additional things. They’ve talked about quarterly release cycles. We’ve got a Magento 2 B2B module that’s coming out soon, and all of these are things that are opportunity costs you lose out on if you don’t move to Magento 2, but the big thing – the reason that we decided to move here quickly at ClearBags – was the last one: it’s the doubling of development costs, because we have been actively developing on Magento 1 and we don’t want to pay to redevelop that code again. So we need to just rip the Band-Aid off, move quickly and do our development on M2.
Moving to Magento 2 early, in my opinion, has been a better financial decision for us, and I think that’s the case for a lot of merchants, particularly if you are actively developing on Magento 1 right now.
So a couple comments about managing complexity, and the reason I’m talking about managing complexity is because complexity in a project like this means additional development work, and developers are expensive. This is what’s going to drive the cost of your migration. If you have a lot of features you need, it’s crazy complex, you’re going to pay a lot more money. So if you can simplify, you can drop the costs significantly.
What we did is we took a breath and we said “What is the minimum viable product, what is the minimum that we can launch with and have a working clearbags.com site?” We did a lot of work to try to decide what we could leave out. And sort of the key question we kept coming back to is: if it’s not already live on the site, we don’t need it to launch. We weren’t looking at this as a way to add more features and grow the complexity of the site. We decided to look at this as just we wanted to take our Magento 1 site and essentially get feature parity on Magento 2, and then all of the other things that we want to develop on, we’ll continue to work with and improve in future iterations.
On a site like ours, with eight years of history, there was no canonical documentation anywhere of all the customizations we’d ever done, so the site itself is the documentation of every customization. So we had to start this process long before we engaged Creatuity to do development work. We started this process by doing a full site audit of the obvious things, like the extensions, the modules that we programmed in-house, external integrations with our other systems, but, you know, there are also situations where, in the interest of time, a developer might go in and implement some core hack to do something quickly that’s not done right. So you should look at, you should compare the Magento source code with the current release and make sure that the version you think you’re running actually is what you’re running with no modifications in the core code. Another thing that slips in sometimes is putting business logic in the themes. I’ve seen this over and over again with different merchants. So auditing all your theme files for any PHP that’s implementing business logic is important. And then non-Magento PHP files are frequently on peoples’ file systems that are for external integrations or quick utility scripts to do something or other. Anyway, we went through all of this to identify our existing customizations, everything that had been done on the site. Next step was to basically try to sort out those customizations and determine what was really required for launch, and we split things into three different categories:
Not needed – so we had old legacy code we weren’t using anymore, that was turned off but the code was still in there. We also had things where we’d implemented a feature that we didn’t truly need; we thought it was important at the time, but, you know, on second thought, it doesn’t add anything to the site. So we actually just got rid of some of the code right there and decided not to bring it forward.
We also made that decision to defer certain things until later, and this would be things like a little user interface tweak that’s a nice-to-have but it’s not necessarily important to launch. So we took some of those things and put them on a deferred list that we’ll tackle in the future as future projects. We tried to figure out what is actually the minimum necessary required to launch. And the question to ask when figuring out what is the minimum necessary to launch is what’s the impact on revenue. If this is just a nice-to-have but it doesn’t budge the numbers, it doesn’t make the revenue go up by 1%, maybe that’s something that could be deferred until later.
A lot of the requirements that you come up with don’t actually require development work, and in our case we had implemented, like I said, a lot of features internally on our Magento CE instance that later on were added as features in Enterprise edition. We also had things where we implemented something, we had to write PHP code to do something in 2008-2009, that later on was added as a feature in the configuration. Now there’s just a check-box. So we were able to eliminate a lot of code just by understanding Magento 2’s options, by installing that demo store site and going in and really looking through all of the features available and all of the options. We also made the choice to migrate from CE to Enterprise edition at this point, and that choice was because we eliminated quite a bit of development.
So we’re also keeping our eyes on this upcoming B2B module. There’s not a lot of information about the actual features that will be released, but some of the things that are on our deferred list that we’d like to implement later are features that may be included there, and by talking to the people involved with that maybe we can influence what the future roadmap for that module contains.
And third-party extensions – this is a big area. Again, in 2008 when we were first getting started on this, there were no external extensions, okay, the Magento marketplace really didn’t exist. We were actually able to eliminate a lot of our custom code with one single module, and that’s Shipper HQ. So about half of the customizations that we had created on our site were around shipping. We had issues with multiple warehouses, and again, there’s all sorts of solutions for that now – there weren’t in 2008, so we just got off on that path of doing everything in-house and over time we just kept adding more and more and more — the U.S. Postal Service, dimensional shipping, all these sorts of things. We were able to work with Shipper HQ and there were a few things that their module, their extension, didn’t really provide, where there were a couple of things that didn’t quite work the way we needed it to work, and we were able to give them some feedback, get a few things on their roadmap, and they were very responsive and helped us out quite a bit. And when I say we had two key partners in this project – we worked with Creatuity for development, but we worked a lot with Shipper HQ and they eliminated half of our code for us.
Talk a little bit about working with developers, and again, this goes back – I’m gonna say, this really has to do with the financial side of things, because if you can work well with developers you can keep costs lower. One decision we had to make right off the bat was what do we do with our in-house staff and what do we outsource. We made the decision to keep the integration work between Magento and our other systems in-house, to do that ourselves, and to basically outsource all of the actual Magento, the pure Magento customizations, to an agency. And that just basically is because the in-house developers already know all of our systems, they know the plumbing, they know what’s good and they know what’s broken about the plumbing between our systems. We have some legacy systems that date back fifteen years, so trying to ramp up an external agency to work with that would add quite a bit of cost, and we do have some internal developers to do some work so that was the logical split for us.
So a couple points about what I looked for in an agency. So understand, I’m looking for an agency to work on a Magento 2 project in mid-2015, before Magento 2 was released. So I was going to conferences like this and I was sitting in developer talks and listening to different developers from different agencies as they presented, and then frequently talking with them and networking after the fact. We were really looking for a well-respected company with experience in Magento 2 – in 2015 – and a company that was the right size for us. I didn’t want our project to get lost in the shuffle – I don’t want to be a little fish in a big pond. I wanted to make sure that our project was strategically important to the owners of the company, and that “right fit” I think is really important in a lot of our business relationships. So we ended up, I ended up seeing some presentations by Josh from Creatuity, and I knew him already. I’m very impressed with his level of knowledge and – I gotta tell you – Creatuity, very invested in Magento 2, experienced in Magento 2, already in mid-2015, we decided to go forward with them.
What development is going to cost – who knows? — and everybody in this room has a different case. It’s impossible to say. So I think what is more important is understanding how you can control the costs. I’ve seen a lot of other merchants that I know to ask agencies for fixed price quotes on projects like this and with the number of unknowns with a new platform and moving forward and not knowing your business, the thing is they always have to pad their quotes when they give you a fixed quote, because you’re trying to put all the risk on them. So they have to raise the price. And if you can develop a good relationship with an agency that’s a win-win relationship, and you’re willing to accept a little bit of uncertainty in that, you can maybe have an understanding about how much work they’re going to do roughly and what they’re going to charge, understand hourly rates and so on, and the thing is if you can build a good working relationship like that, where you don’t need a final number up front, you’re probably going to end up spending less money over the course of a project like this. So that’s the kind of relationship that we tried to pursue with Creatuity.
So not every agency uses an agile methodology but, so this is maybe a little more specific to those that do, but — I don’t know everybody’s skill level in the room and knowledge of current development methodologies – but agencies that are using an agile methodology are typically making use of sprints. A sprint is a block of time from one to four weeks, typically two weeks, and in a sprint developers are going to carve off a set of requirements and they’re going to commit to finishing those by the end of the sprint, and they’re going to release working code that can be demo’d to you at the end of that. And also in that sprint they’re going to be planning what’s going to happen in the following sprint, what requirements are they going to tackle then and implement. So at any point in time, you’ve got what’s just been released, you’ve got what they’re working on now and what they’re about to work on. So think about how you can work with an agency that’s using this methodology. If you’re constantly checking in with them and building that relationship and saying “What are you working on right now?” and you’re also saying “What are you planning on working on in the near future?”, then if things slip in that aren’t quite right or there’s some added piece of the project that you don’t need done, or if your requirements change in the middle of the project — something you said earlier you needed done, now you realize you don’t actually need – you can give some feedback promptly and say “Take that out, don’t do it, don’t spend time on that, focus on these other things instead.” That can seriously help bring the costs down. The other thing I’ll say – regularly get demonstrations of the previous sprint’s work. If they are using this methodology, they are frequently releasing this code. You should be able to see that, test the things that they just did, so if there’s any problems or any changes in requirements or there was just a miscommunication, you can address that at that point and get it dealt with promptly.
So above all, avoid being the roadblock. You don’t want to be the person that causes the developers to switch gears and work on another project. Think about this. An agency by definition is not just working for you. They’re working for their other clients as well, and if their developers hit any roadblock that makes it so they can’t keep working on your project, they switch gears. There’s cognitive overhead involved every time you switch from one project to another. Keep them working on your project. So if the developers send you a question, answer it now – not three days from now. I have made this mistake before. Sometimes I get caught up in my busy life, and later on I get to it and I realize, wow, they could have been productive that week if I’d responded more quickly. Give prompt feedback on every release. Every time they release working code, get on it, review it, talk to them about it. And also, just keep the money flowing. You don’t want to have some situation where your accounting department failed to cut a check on time so the developers stopped work, right? Keep the money flowing. Well-paid developers are happy developers.
So how did the ClearBags project go, right? Well, most of the Magento 2 work got finished in January and February of this year. We have had some ongoing data migration issues that took a little bit longer to resolve. Again, you look at a Magento 1 database that was installed in 2008 and you think about all the upgrades and all the database modifications over that time, there’s a few issues in there sometimes to deal with. Also, we’ve had a few API bugs, frankly, when we’re dealing with the integration code, Magento 2.0, there are a few bugs in there that we’re eagering hoping that the Magento 2.1 release will fix. And, you know, I talked a little bit about that issue of feature parity with Magento 2 extensions between the Magento 1 version and the Magento 2 issue, we came up against that, which is why I’m bringing it up. So there were a few cases where we had to say, “Ohhhh, could you add this feature to your extension – we really depend on it.”
But overall, great experience with our partners. Creatuity has done a ton of work for us. Shipper HQ did a ton of work before we even came to them that really reduced the size of our project. And at this point, we’re just about ready to launch, we’re right on that cusp. So we’ve got a few final wrap-up issues and, I don’t know, a week or two or three or four, just sometime in this very near future after we get a little more testing done, we’ll be putting our Magento 2 site live.
So, a couple key takeaways. So, number one, I’m going to just throw this slide on there, this is just a visual depiction of the methodology that I just talked to you about simplifying things, going through those issues of site audit and figuring out what you really need to do and how you can avoid development by leveraging other resources. So this slide will be up on the website pretty soon, so I’m going to go ahead and advance.
So key takeaways – I really think you should stop your Magento 1 development now. That code has just such a short useful lifespan. At least, even if you don’t start your Magento 2 project right now, try to wrap up what you’re doing on Magento 1 and only do things that have a really high ROI. Definitely as you’re tackling a project like this, simplify. Try to find a way to cut out all the complexity. Avoid development wherever you can by leveraging what other people have already done. Focus your in-house teams on integration. I think that’s a good general way to structure things, but I understand every business is a little different there. If you’re going to use an agency, which I think really makes sense, really focus on that relationship, building a really good win-win relationship that’s based on trust, and really get to know them. And above all, don’t delay. I think you need to start now.
So I’d love it if you stay in touch. If you check out clearbags.com in the next few weeks, we’ll have our lovely new Magento 2 site up. I’m looking forward to that. I’ve got a personal blog where I write some articles about topics of interest to me, including Magento development, I’ve got some things on there about Magento security, actually, and I’m also very interested in topics about online advertising and the economics and financial side of all that decision making. So got my twitter, you can send me messages there or email me at any point.
And I think we’ve got a few minutes, so let’s do some questions if you have any.
Question: What was the biggest learning experience in the migration so far? The biggest mistake?
Answer: The biggest mistake? Oh, I think I’ve got to say my biggest mistake was not really staying up on all the communication, getting sometimes caught up in my daily life trying to run a department at a company and not getting comments back to developers right now. So there were a few times when I had questions from developers, “What do you really mean by this” or “How should we go forward with tackling this bug?,” where I let it sit in my inbox for a week or a week and a half and then I kinda got around to it eventually, and if I’d been faster about some of those things I know it would have shaved off more cost and it would have just made some of those things where we had to take a pause on the project, it would have moved it forward faster. That’s why I brought that up in here.
Question: Hi there, so I may be a bit technical, I apologize, but is it fair to ask what or how much of the code you took from Magento 1 over to Magento 2?
Answer: Well, so when you’re talking about how much of the code, understand that we redeveloped everything that we needed to on Magento 2 so it’s not just like, I don’t think of it as a code migration exactly, but I will say again, going back to the point about Shipper HQ — half our code we got to leave behind because we have better modules right now, and then when you add in a lot of these other things that are just built in to Magento 2 now and built in to Enterprise edition, I don’t know, I think maybe we moved forward a quarter or a third of our code. Probably something on that order.
Question: Hi, quick question – did you migrate all your customers and orders from the old system to the new one?
Answer: Yes.
Question: So you migrated literally everything, the whole history of all ….
Answer: Yes, and we felt it’s important because some of our customers don’t order from us very frequently. We’re selling this product packaging frequently to small companies, so I mean, think about a local candy shop and they need all these plastic bags to package candy and different things, but they’re not really high-volume users frequently. So, or like a wedding photographer, packaging photographs for giving to their clients. So they might buy a few boxes a year of packaging and then they might not order again till they run out. So in a case like that, people don’t remember the SKU or the exact dimensions of the product that they ordered but they want the same thing. So that product history is something that’s important to a lot of our customers, and we did have some questions around whether that was something we could put off until after launch. We talked around that and said, well we could live without it for a while but we’d still need to do it, so we might as well just do it.
Question: Roughly what sort of size was your teams, in terms of developers and external developers, testers, BAs?
Answer: Well, so the external development agency was Creatuity, so Victor is here from Creatuity and he might be able to speak to that. In terms of internal developers, though, we have a small IT and ecommerce team. So back in the day, I did a lot of the development myself. We’ve got one full-time developer besides myself on staff, and we have other people who do a little bit of part-time development with certain systems that they know well. So, Victor, you want to say something real quick about Creatuity development team size?
Victor: We have about 30 people right now, but on your project it was four developers, one QA person, one architect, and I think that was it.
Question: What are the most important features of the Enterprise edition – what was the reason that you chose for it? Or was it just the backup from the Magento team or were there certain features you really needed?
Answer: So, you know, I can’t even say off the top of my head that this was like the killer feature that we really needed. There were just little things, a lot of little things, like, you know, the ability to just quickly and easily add additional fields to customer records. Things like that, that back in the era of Magento 1, in the very beginning, we had to write custom code to do that. So, yeah, just the simple nuts-and-bolts stuff. It’s got extra things in there in a number of different areas that just helped us tick off a number of those requirements that we had.
All right, I guess we’re done, so thank you very much, everybody.