Let's stop and think for a few minutes.
For years we have been advising our partners on why you should really think hard and understand the challenges on parking your critical business services in public cloud.
For quite some time over the last 10 years or so, there has been a strong push for IT & MSP shops to eliminate as much as possible on-prem and push everything to the cloud. Everything from Exchange Servers being pushed to Office/Microsoft 365, file servers being moved to Azure, Amazon, Google Cloud Services, stand alone apps being moved to SaaS, effectively everything was on the table to move to a cloud-based offering.
The same considerations applied to new startups who have erected new software / SaaS / cloud based answers to common business needs. It's a lot more scalable to a point, especially with data intensive services, to start with virtual infrastructure rather than buy, install, maintenance, and depreciate hardware in a network rack somewhere.
The results of this strategy has many benefits not limited to: elimination of unexpected truck rolls, stocking of spare parts, dealing with unexpected premise based environmental issues (security, power, cooling, building access, etc), quickly scale up or down, maximum up-time, median time to resolution of an issue, and others.
Politics aside, and strictly from an operations standpoint, based on current events imagine if you or your customer's servers were being shut down and deleted by your service provider overnight? This is a scary situation with hard-hitting consequences. When you have a server in a network room or "closet" sure it can break down, get stolen, or the building could burn down, these scenarios were mediated by good planning, and backing up data both on and off-site. If you have a single cloud vendor you have chosen then both your primary and secondary options are likely with the same provider. If that provider pulls the plug on you or your customer with little or no notice then you are in a extremely problematic situation.
Another quasi-problematic situation is that a lot of companies have adopted tools, apps, and software from their cloud infrastructure providers that have long reaching tentacles into their implementation. If you need to move your systems elsewhere they may not function the same way without serious changes that may not be easy to make quickly.
Let's say you did have everything you needed to stand up your entire system elsewhere, how long would it take you to go from completely offline to completely online? Maybe you or your customer only had a server or two or maybe they have 100 or 200. Likely, if you go to a new vendor you have a blank slate and have to start from scratch. If you had not planned for this likely whatever your disaster recovery strategy is didn't account for this type of thing to happen. As a result, not only do you have to locate, negotiate, and take delivery of your new cloud infrastructure, you need to deploy, install, and restore systems which in many if not most cases is not instant in this case.
So, with all of this said what do you now do in your own case and also in the case of your advice to your end customers who have been convinced to go totally cloud or bust?
Here is some food for thought:
- Terms of Service: It's important that you really scrutinize your terms of service. Don't just agree to the stock verbiage if it doesn't make sense. Also, make sure that if either side does trigger a termination, that you have time to safely move elsewhere. 24 hours is likely not enough time to get the job done without serious heartburn.
- Who Owns the Data?: It's important that if your vendor pulls the plug on you that not only did you have enough time to get your data transferred away in time but will they purge the data once they have cut your access of if you didn't have enough time to do so? That data could contain intellectual property, confidential data, regulated data, etc.
- Governing Law: Where is your vendor located and by which country, venue, and body of laws apply to your services?
- Domain Registrar: Where are you domain names located? Do you have more than one registrar? Where are they located? Can you quick move to backup domains?
- Egress Fees: Depending on your vendor, many of bandwidth limits, egress fees, and overage baked into their pricing. If you had to make a wholesale move then understand it may be really expensive just to get your data out.
- Transfer Time: Always consider how much time it will take to transfer your data across providers. Just because you had 1TB or 10TB pipes doesn't make the route across the internet between service providers across multiple ISP's matches your throughput within the data center. Likely, whatever you think it will take will be dramatically different from the end outcome. You won't know until you are in it in most cases.
- Cost of Labor: Time has a real cost to it. Especially if you are forced into an off hours emergency situation that neither you or your end customer caused in the first place. This can quickly add up.
- Multi-Cloud: While it may not be the most economical way to go, you may have to spread your systems around across multiple cloud providers so that you don't get stuck. Some of those providers may have to be outside of the larger more popular names we have all come to know and love.
- Hybrid-Cloud / Private Cloud: It's a not to kept secret that once you get large enough the public cloud vendors out there start to cost more than doing things on your own gear. In this case, while it may cost you most to do it, you may need to run some more critical systems on your own gear rather than all on outside cloud vendors. It causes more things to deal with this way but again it's systems that you can touch and under your control.
- Agnostic Deployment: You need to make sure that your systems are not reliant on platform specific tools so that you can easily move systems across platforms without complexity. That may mean having to make an investment into tools on your own which may not cost you anything right now but will come as a cost by making sure those software packages are yours to use on any system.
- Multiple Backup Destinations: You might need to have a one-to-many approach when it comes to backing up your data. You might not be able to safely park your backups in just one place, one vendor, one backup technology. You may need to also consider making sure you have offline copies on your own equipment outside of cloud providers all together.
- FQDN rather than IP Based: The same goes for ISP's who are providing you IP addresses. Unless you have made an investment to control your own IP address space, most have not, then you need to make sure your systems are setup in such a way that you can easily move your systems to other places without have to re-deploy end user software, apps, and endpoints the hard way. If you are not setup in this way now then it's time to start down that road.
The purpose of this article is that current events now force these concerns. Nobody, or very few, forecasted the possibilities of someone just turning the lights off instantly on you with little to no recourse available to you and ultimately your end customers. If this topic was brought up previously it would probably be something that would be written off as far fetched and not really top priority. Maybe a lot of you who read this will say it's still not really a concern, until it happens to you or your customer. Some i'm sure will say, hey this is not a reasonable business conversation. I remind everyone that you don't really know who your end customers are ultimately serve in the grand scheme of things. If/when this situation could happen, it would probably be too late for you to prepare. Almost like, you should made sure you had extra auto insurance before that big accident. But you never expected you needed more coverage, right?
It's something to seriously consider!
As the CEO of bvoip, I surely am taking the time to consider how we handle this for our own operational consideration. If nothing other than to make sure that we have double checked where we stand with our own infrastructure vendors, system topology, and international data center footprint strategy and made sure that we have a realistic plan in place to reduce risk.
I would suggest you start doing the same too! It's a worthwhile exercise to prepare for and ultimately if you are the "trusted advised" for your end customers when it comes to their own technology, it's worth your time to do the same for your customer's too.
