Governance Challenges Slow Down GenAI Adoption for Enterprises

Recent discussions around the adoption of Generative AI (GenAI) highlight significant governance issues that hinder its potential within enterprises. A report from the Massachusetts Institute of Technology (MIT) reveals that despite substantial investments of between $30 billion and $40 billion into GenAI, a staggering 95% of organizations are reporting zero return on investment. This phenomenon, termed the “GenAI Divide,” showcases a stark contrast between different types of companies, including enterprises, mid-market businesses, small and medium-sized enterprises, as well as startups and consultancies.

In an exclusive conversation with Digital Journal, Matt Carroll, CEO of Immuta, provided insight into the current landscape of GenAI adoption. He argued that the technology has not lost its potential, but rather that the complexities of governance are slowing its integration into everyday business operations.

Understanding the Hype and Adoption Challenges

When asked about the perceived decline in enthusiasm for GenAI, Carroll emphasized that the term itself carries a lot of weight. He noted that while influential firms like OpenAI and Anthropic have made headlines, widespread embedding of GenAI models across all enterprises takes time. He explained, “Adoption takes time. With platforms like Databricks or Snowflake, it took years to integrate them into enterprise workflows, and those were just data. GenAI is bigger; it involves unstructured and semi-structured data and new workflows, creating numerous potential use cases.”

Carroll highlighted the critical role of data governance in this context. He pointed out that GenAI transforms everyone into a data consumer, which raises challenges regarding access to sensitive or low-quality data. In many enterprises, data governance teams are composed of only 10 to 20 experts responsible for overseeing data access for thousands of employees, presenting a scalability issue.

According to Carroll, for GenAI to harness its full potential, it must penetrate the governance layer. Enhancing middleware for access controls, auditing, and security is essential. He stated that empowering governance teams with GenAI tools could enable them to manage data access more efficiently, simulating the effect of having significantly larger teams.

ROI and Future Prospects of GenAI

When discussing the perception of failed pilot projects and the resulting skepticism regarding GenAI’s effectiveness, Carroll disagreed with the notion that such failures were the main issue. Instead, he noted that enterprises are still in the preparatory stage, working to refine their data for large-scale GenAI applications. “Inside Immuta, for example, we’re deploying GPTs for revenue, messaging, finance — but not all data is equal,” he explained. Ensuring poor-quality or sensitive data is excluded from models is a significant part of the process, emphasizing the need for robust governance.

Carroll is confident that the perceived shortcomings of GenAI are not a reflection of the technology itself but rather of the pace at which enterprises are aligning their data governance with GenAI capabilities. He reiterated, “Until enterprises align their data governance and infrastructure with GenAI, you won’t see adoption at scale. Once that alignment occurs, we’ll witness an inflection point where ROI becomes apparent, and adoption will accelerate rapidly.”

Looking forward, Carroll expressed excitement about the integration of conversational AI into everyday enterprise tools. He likened this shift to the transition from complex software like Photoshop to more user-friendly platforms like Canva, which democratized design capabilities. “Instead of tickets, forms, and clicks, we’ll just describe what we want — access to data, infrastructure, or analysis — and the system will execute,” he explained. This advancement has the potential to fundamentally reshape workplace dynamics, removing bottlenecks and providing advanced capabilities to a broader range of users.

As the conversation around GenAI continues, it is clear that addressing governance challenges will be crucial for unlocking its potential and achieving widespread adoption across various industries.