We’re three years into the ChatGPT era, and you’re itching to move beyond customer service chatbot prototypes and proofs-of-concept chain-of-thought reasoning agents. You’re ready to bite the bullet and start deploying production AI. The big question you’re now facing is what’s the best way to run AI apps in the real world. We’re glad you asked!
Finding the right runtime environment for enterprise AI requires thoughtful planning to make sure your investment will pay off. You’re going to need plenty of processing capability, as AI apps are voracious consumers of GPUs, CPUs, and fast memory. AI apps also need a lot of data, which complicates the platform decision, since moving data is expensive. There are also privacy and security concerns to consider with AI apps. Finally, there’s long-term manageability considerations, since you’re going to be asking your already-strapped IT team to maintain this new, fast-evolving system.
Each enterprise AI app is unique. While you could use a shrink-wrapped AI app developed generically by a cloud giant, you’ll get more value from training an AI model on your private, sovereign data that best reflects your company’s intellectual property.
Here are five reasons why the best choice for running your enterprise AI model and realizing a return from your investment is likely to be a hybrid, containerized platform:
Provide Consistent Operations Across Public Cloud, Core, and Edge Locations
Deploying your new AI app on a containerized, hybrid platform gives you the option to deploy it exactly where you need it. That could be on-prem at your company headquarters, in the AWS or Azure cloud, or even at the edge, such as a retail outlet, a distribution center, or an offshore oil platform.
Your first instinct may be to run your enterprise AI apps in the cloud. After all, that’s typically where you’ve developed and tested them. Why buy expensive GPUs when you can rent them for a fraction of the purchase price?
That’s true during the prototype stage. It makes more sense to rent cloud capacity when you’re in the R&D stage. But when you’re in the deployment stage, it will often make more sense to deploy your AI app on an edge device that you own. This will not only be the more affordable solution in the long run, but it will provide for a better customer experience and improve manageability.
AI applications are hungry for data and GPU cycles. If your GPUs live in the cloud, then you must send data in real time to the cloud for AI inference. That is both time-consuming and expensive. Running the AI on edge devices close to the customer will eliminate the need to send large amounts of data over the network, thereby lowering response latencies and saving money.
And then there’s containers and VMs and containers in VMs. AI apps usually run in containers. In fact, a recent survey of 1,600 cloud, IT, and engineering executives indicates 87% of executives expect the level of application containerization within their organization to increase—largely due to AI. Running those containers inside virtual machines (VMs) is the most practical and efficient approach for most enterprises because it leverages existing skill sets and provides consistent resiliency with other business-critical workloads. AI isn’t a one-size-fits-all endeavor, however, and you may want the choice to deploy your AI on-prem, in the cloud, or on the edge. Building on a standardized platform gives you the option to run your AI workloads wherever it makes the most sense.
Deliver Low, Predictable AI Infrastructure Costs
AI has a new cost meter for many customers to figure out – the token. Public clouds charge by the token, and users haven’t learned how to optimize their usage to keep costs under control. So the cost of running your AI applications can suddenly skyrocket, just as many experienced in the early days of public cloud. If you remember, all of a sudden there was the cloud bill shock as customers were learning how to contain their usage. Over time they learned to rein that in and became cloud smart. I think there’s going to be a similar dynamic here, going from “AI Fast” to “AI Smart”: where AI Smart means having a lower, predictable cost for AI workloads.
Control Shadow AI Risks
Shadow AI poses a real threat to maintainability, compliance, and ROI of your company’s AI deployment. Your company may have some experience with shadow IT during the big cloud boom that started 15 years ago, with line-of-business departments that bypassed your IT department by direct purchase of software-as-a-service (SaaS) options in the cloud. They may have started out with good intentions. After all, it’s just a small marketing app or a minor productivity tool. What’s the harm with using a company credit card to experiment with some fresh new apps in the cloud?
Once those SaaS apps started taking off—and many of them did, as they offered services not available with the company’s primary ERP or CRM system—it became very difficult to rein them back in. Shadow IT not only complicates accounting for IT, but it damages security and privacy.
Shadow AI poses a similar threat today. While it’s impossible to completely eradicate the shadow AI or IT urge, having a hybrid, containerized platform that is centrally managed and well-positioned to meet your company’s needs can minimize the tendency and mitigate the potential damage.
Maintain Privacy and Security over Sovereign Endpoints
AI applications introduce a particular set of risks and responsibilities around security and privacy. If your company is contracted with a third-party large language model (LLM) to power AI apps, you likely need to share user data which is sensitive or customer data with the LLM. Now you may have obtained customer data under specific customer agreements, that may or may not permit you to share that data outside your company. In either case, sharing sensitive company information or customer data or valuable intellectual property with outside parties exposes you to risk.
The reputational risk of a security or privacy breach is too great. You could lose the goodwill of customers that took decades to build, and may never get it back. However, by running AI on a trusted and centralized platform, you can mitigate those risks, thereby improving the odds of success—not only with your first AI deployment, but every subsequent one.
Turn on a Resilient AI “Dial Tone”
If it feels like enterprise AI is uncharted territory, that’s because it is! Unless you happen to work at one of the cloud hyperscalers that invented a lot of this technology in the first place, you likely lack the experience and muscle memory for successfully running AI apps as reliably as your users want. Just like users expect to always have a dial tone on their phones, and Wi-Fi in their offices, they’re beginning to demand ‘always on’ connections to LLMs with great performance. Serving that to your users without breaking the bank can appear daunting while your IT staff is learning to run the demanding AI apps successfully and in an optimized manner.
A modern software platform allows your staff to gain experience with AI while preserving their expertise in serving up resilient, maintainable, enterprise infrastructure. A hybrid, containerized platform allows your teams to deploy AI with a predictable set of building blocks that minimize security and privacy risks.
AI is quite new for most organizations, and there are a lot of moving parts. That is why relying on a tested, pre-built platform can minimize a lot of the design decisions and work that you might normally encounter in building applications of such complexity. In a way, it’s a modern version of the trusty telephony dial-tone—now reincarnated as resilient infrastructure for your new AI workloads
Bottom Line: Treat AI as Your Next enterprise Workload
Modern software platforms can abstract new AI hardware, provide secure access to the latest AI models, and deliver low, predictable costs and sovereign controls over cloud alternatives. Your busy IT teams will appreciate the ability to leverage their current skills to deliver resilient AI containerized workloads across disparate hybrid cloud endpoints. Treat AI as your next enterprise workload and you’ll quickly find that a modern software platform can reduce risk and help ensure a fast path to success.
