To be effective, AI models require real-time data that flows continuously between applications, says Jay Kreps of Confluent.

Early data technology involved multiple databases storing and retrieving static information or “static data”. Instead, modern enterprises rely on interconnected systems and require seamless data flow across applications. “In today’s dynamic environment, companies need ‘data in motion’, which keeps flowing between confluences.” Mint In an interview on Wednesday in India. He spoke at a user conference in Confluent in Bangalore called “Current”.

Confluent enables “Data in Movement” through Apache Kafka, a popular open source platform created by Kreps himself. Originally developed on the professional networking platform LinkedIn, it is now managed by Apache Software Foundation, operates in real time using Kafka – retailers track sales, bank-managed transactions and logistics companies coordinate delivery.

AI applications also depend on real-time data integration, which makes streams essential for leveraging language models in business operations. According to Kreps, this shift has fueled the growth of convergence, positioning it at the heart of real-time data flows across the industry.

AI adoption: India leads

He believes that India is still at the forefront of technology adoption and leads in AI and real-time data flows. Confluent has partnered with companies such as the Indian Jio platform, Swiggy, Meesho and Viacom. “We have partnered with Jio Platforms, which provides cloud services and we offer Confluent on Jio Cloud in its Azure West region,” he said.

For example, Jio’s network and companies like Viacom are leveraging Confluent’s technology for analysis, such as the audience for the Indian Premier League (IPL) Open Day, with more than 590 million events. Swiggy uses Confluent in its aspects for data metrics, governance capabilities, and real-time delivery insights to help its data teams quickly simplify complex workflows and scale operations during peak demand.

Kreps stressed that the convergence platform and the connection portfolio are “fully managed by India’s engineering team (the engineering team built for the world by India”. He added that India is the “largest and fastest region of Confluent, with the team growing by 50% last year”.

“We have an engineering profile here that is comparable to our Bay Area team, driving major projects like Table Flow, a feature we’re launching today.”

Table Streams help converge users to transform their streaming data from Kafka to organized tables (in Apache Iceberg or Delta Lake). This makes it easier to store, manage and analyze real-time data in a structured way so that businesses can quickly access and use information for reporting, analyzing or AI and generate AI (Genai) applications.

Confluent plans to continue hiring in the Asia-Pacific region and increase it by 20%, especially in India. Kreps added that India’s investment in digital infrastructure positioned it as a global leader. “Here, payments use Kafka and confluence.

Missing works: high-quality data

What is certain is that with the rapid development of AI, integrating real-time data has never been so critical. However, he acknowledges that AI adoption presents challenges, especially for enterprises that manage legacy systems and provide multiple application programming interfaces (APIs). “AI models trained in general Internet knowledge lack company-specific insights – undata is key,” he explained.

In addition, AI development requires structured high-quality data, which many companies work hard on. “Many tech teams are still learning how to effectively integrate into their workflow,” he added.

Kreps, who is also a board member of the Human Committee of AI startups, developed the Claude LLM model, “There are incredible work happening to make these AI models smarter and smarter, which solves a lot of problems in the scope and scale and reliability of what they can do.”

However, he admits that there are still “some big lost works” besides the models. For example, many companies’ data is “distributed across 1,000 different databases and applications as well as SaaS (Software as a Service) systems”, making integration with these models a “big challenge.”

But the good news is that while “there are a lot of different models and there are a lot of innovations in the interface with the customer, there are quite a few similar numbers”. Kreps believes that the key is to feed the model’s real-time data from various databases and applications, which is particularly important for AI-driven proxy systems, which automatically works. These systems do not have accurate, latest data and cannot make informed decisions.

According to Kreps, as the model evolves, more AI proxy applications will become practical, reshaping enterprise workflows.

As the co-creator of Kafka, Kreps is naturally optimistic about open source. “Open source AI models drive innovation, and companies like Meta and DeepSeek have a strong impact on future models.”

Meanwhile, as AI changes the industry, its impact on work remains a topic of debate. “In the short term, AI improves productivity without reducing employment,” Kreps notes.

However, long-term impacts are difficult to predict. “While past innovations have not caused massive unemployment, they do undermine the widespread impact of AI,” he said. “Technology is developing rapidly, and keeping means being left behind.”