At Nvidia GTC, Union.ai CEO and co-founder Ketan Umare makes the case that AI is forcing a fundamental rethink of how software is built, deployed, and maintained. He argues that traditional software stacks were designed for deterministic systems, while AI introduces non-deterministic behavior, rapid experimentation, and infrastructure instability that can easily derail production.
Against that backdrop, Umare positions Union.ai’s open-source Flyte platform as an orchestration system purpose-built for the AI era, one that helps organizations manage multi-step workflows, recover from infrastructure failures, and reduce the operational burden that slows engineers down.
Across the interview, his broader message is that the companies moving fastest in AI are the ones that combine clear problem focus, production-grade tooling, and control over their own data.
Core Takeaways
The nature of AI systems: Umare says the conventional software stack was not built for AI because AI systems are inherently non-deterministic, which changes how development, experimentation, and production deployment have to work.
Infrastructure reliability: He argues that one of the biggest hidden problems in AI is infrastructure reliability, with many workflow failures caused not by the model itself, but by breakdowns involving memory, networking, or GPU availability.
The Flyte platform: Umare positions Flyte as an infrastructure-aware, data-aware orchestration platform designed to support self-healing, multi-step AI workflows, rather than forcing companies to assemble a fragile stack from separate tools.
The importance of open source: He says open source is critical for trust and adoption in infrastructure software, while Union.ai’s commercial value comes from helping enterprises scale, secure, and operationalize Flyte in demanding production environments.
Key Quotes
AI changes the assumptions behind software
“All the current set of software technologies and stack that people use to build software was not designed for the AI era. And the reason is, if you think about it, traditional software is deterministic, while with AI, that core is non-deterministic. It changes everything because experimentation becomes key.”
Reliability is now one of AI’s central problems
“There’s a huge reliability crisis – everybody’s writing code now using coding models. Who’s deploying them? Humans. We are generating code faster than anybody can review, anybody can deploy. Most of these reliability problems are caused because of infrastructure issues.
“Sometimes you downloaded a lot of data, you need more memory. Some other times you were using some spot GPU somewhere, those go bad or they are pulled away from you. In some other cases there was a networking issue. And to solve that problem, you need somehow to self-heal.”
Tool sprawl slows developers and erodes ROI
“You could build a stack by hand threading everything, take an observability portion, take a training portion, take an inference stack, take orchestration as a separate thing, and best of luck putting it all together and making sure all of them work and talk to each other.
“Furthermore, if you put this all together in this hodgepodge way, it slows down the developer. The entire onus of thinking and making it work together falls on the poor person who’s already burdened by keeping up with the AI world.”
The AI leaders are doing three things right
“People who are consistently pulling ahead in AI are able to do three things well. One of them is they’re able to identify the problem that they want to solve and focus effort on it. Two, they are using a tool that is built for production grade and that cuts down the noise. And the third part is actually data is your key. You don’t want to put that data into another person’s product. So you have to build it internally. Once you build that flywheel, you can move really, really fast.”
