Orply.

AGI Has Arrived, but Enterprises Still Struggle to Put AI to Work

Ali GhodsiAlex LiebermanAll-In PodcastTuesday, November 17, 202615 min read

Databricks CEO Ali Ghodsi argues that AGI is already here by the definition he and his UC Berkeley colleagues used in 2009–10, but says the larger question is why companies have yet to put today’s capable systems to work across their operations. He points to gaps in organizational context, security, cost and control, and says broad adoption will require more than models that can code. Ghodsi also distinguishes those practical constraints from recursive self-improvement, which he says he sees no evidence of, while treating cyber risk as a more immediate concern.

AGI, in Ghodsi’s definition, is already here

Ali Ghodsi says artificial general intelligence has already arrived—not because current systems can do everything a person can, but because they meet the definition of AI he and his colleagues used when they were building a research lab at UC Berkeley in 2009 and 2010. He says he checked that interpretation with former lab colleagues, worried he might be misremembering, and they agreed: by that definition, the milestone had been reached. In his view, even if progress stopped now, today’s systems would provide enough intelligence to transform the planet.

That is not the same as the superintelligence he sees researchers pursuing. Ghodsi distinguishes today’s broadly capable systems from the far more extreme intelligence described by Nick Bostrom: something able to perform what would amount to a millennium of human thought in a minute. Reaching that level, he argues, would require more than models getting better at coding. He identifies four conditions: training the next model in much less time; using fewer resources to do it; improving accuracy broadly, rather than only in a narrow trained domain; and repeating all three improvements in a self-sustaining loop.

Ghodsi says he does not see that loop. Large training runs still take substantial time and require more hardware; data preparation and domain-specific reinforcement learning still involve considerable human work. Even if a model writes 98 percent of its own code, he says, that does not establish recursive self-improvement. What matters is whether the whole cycle of training becomes faster and cheaper while general capability continues to improve.

Just because 98% of the code is being written by AI, we don’t have RSI.
Ali Ghodsi · Source

Ghodsi’s interviewer, Jason Calacanis, is broadly aligned with his distinction, but presses on what it means for work currently done by people. The question is not only whether models can solve difficult math problems; it is whether their abilities can be distributed through organizations and put to use. Ghodsi’s answer is that this is the practical challenge. He says companies he meets are mostly using AI as a more capable search box or for coding, where they report speedups of perhaps 20 to 30 percent. He does not see large organizations running thousands of agents that reliably perform core work.

Databricks, Ghodsi says, is built around that gap between available model intelligence and its use inside established organizations. The company began in 2013, after Ghodsi and colleagues at Berkeley found that industry was not ready to adopt their work combining data processing and AI. He describes starting a company as an act of frustration: at the time, he says, there was little interest in AI, which was more often called machine learning, and “AI” commonly meant robotics.

The team had begun with a recommendation system intended to help Netflix suggest movies. It believed that data science and AI would eventually matter across industries, not only in advertising technology. But turning research into enterprise software meant confronting conditions the lab had not adequately addressed: security, governance, regulation, and the accumulated weight of decades of legacy systems.

Cloud computing helped make the underlying work practical. In Ghodsi’s account, companies running their own data centers had to buy machines that combined storage and compute. Cloud infrastructure separated the two, making it possible to store large quantities of data and draw on compute as needed. Databricks, he says, committed early to cloud, AI, and open source, even when those choices were considered risky. The company’s wager was that organizations would need to work with the data produced by their core products—not only the information held in standard business systems such as customer relationship or human-resources software.

The opportunity is still constrained by the way organizations operate. Ghodsi describes four requirements for deploying AI meaningfully inside a large company: give it the context to understand the work; secure its access to information; make its use economically sustainable; and preserve the company’s control over its own intelligence and intellectual property. Until those problems are addressed together, he says, powerful models will remain largely outside the processes where they could have the greatest effect.

Enterprise AI runs into context, security, cost, and control

For an AI system to help with a company’s decisions, Ghodsi argues, it must know what the company is doing. Much of the relevant context is not captured in databases or documents. It is in meetings, communications, decisions, and the knowledge people carry between teams. Ghodsi recounts a bank CEO telling him that the bank would never record its meetings, citing liability. If the system is absent from those discussions, he says, it cannot know what was decided or why—and cannot reliably help with the work that follows.

Access to context creates the next constraint: security. An agent connected to a company’s data could expose a larger attack surface or gain access to information it should not see. Ghodsi says large organizations are acutely concerned about cyber risk. The answer is not simply to give an AI system unrestricted access and hope it behaves. Companies need to govern what it can reach and monitor how it uses that access.

Then there is cost. As agents use more tokens, their operating expense can rise quickly. Ghodsi asks whether a company’s revenue is growing at a comparable rate. He says he does not know companies whose revenue has accelerated exponentially in step with their AI spending. Executives may urge teams to adopt more AI, only for finance leaders to question the growing operating costs. A system that works but cannot be used economically is not a viable enterprise deployment.

Finally, companies want to retain control over their own intelligence. Ghodsi calls this “intelligence sovereignty.” He worries that relying on outside platforms could expose a company’s information or leave it dependent on a provider that later competes with it. He praises the AI labs’ work, while arguing that companies still need ways to protect their data and retain control over what they build on top of it.

Calacanis suggests that organizational inertia is another obstacle: extra horsepower does not automatically give a company the capacity to pursue a new direction. Ghodsi’s response is that the operational bottlenecks are also opportunities. Databricks does not charge for dashboards as such, he says; those dashboards depend on governed, reliable data. When agents make more queries against that data, Databricks can benefit through consumption-based pricing. The company’s aim, in his description, is to make data available to agents while remaining the governed repository those systems depend on.

The change he wants is not simply more automation within existing departments. Most organizations, Ghodsi says, are arranged hierarchically, with decisions moving up and down management chains. But the work of serving a customer crosses those chains: feedback goes from customers to product teams, then to engineering, marketing, and sales, before reaching customers again. Each handoff can add delay. An AI system that understands the company’s people, responsibilities, and work could help reduce that latency.

Ghodsi says the system also needs to understand the company’s internal structure. It must know who is responsible for what, how teams relate, and where work is stuck. He says he has seen digital-native companies move more quickly, while older, larger organizations face greater obstacles from accumulated software, regulation, and security processes. At scale, he says, Databricks is doing this kind of work internally; he also points to Block and Square, where Jack Dorsey has used the phrase “company world model” for a similar idea.

Genie makes company knowledge useful—and raises questions about access

Ghodsi’s demonstration of Databricks’ Genie product put the company’s approach to organizational context in operational terms. He described Genie as building an “ontology”: a continuously updated graph of the company’s people, reporting relationships, work, meetings, decisions, email, Slack messages, and documents. The interface showed a chat prompt, but Ghodsi’s point was that the system does not operate only as a chatbot searching for answers on demand. It has a pre-existing model of the organization and can use information already assembled in that graph.

That model featured in a demonstration using a company-wide culture survey with more than 10,000 employee responses. Genie produced a dashboard of the results, including a measure called “Action,” concerning whether employees felt the company acted on their feedback. Ghodsi said he was troubled by the result because the score had been low for several years, despite his promises to improve it.

When his CFO suggested that poor work-life balance might be associated with the low Action score, Ghodsi asked Genie to compare the two measures. The displayed chart showed a correlation of 0.67 between Work & Life Blend and Action by leader organization. Ghodsi said Genie also surfaced examples that complicated a simple explanation: one manager’s organization had a work-life score of 44 and an Action score of 77 percent. He said the analysis would otherwise have taken his data team about a week. The chart displayed a correlation; it did not, by itself, establish what caused the survey scores.

He then asked for results by department. Genie identified different reported concerns: product employees cited too many stakeholders in decision-making, engineering employees cited tools and systems, and sales operations employees cited career growth. Ghodsi followed up by reframing the question and examining the results at a more useful level.

The demonstration also showed the system connecting a survey metric to its definition and author. Ghodsi said Genie could recognize that author’s other contributions, drawing on accumulated company knowledge rather than searching only for answers at the moment a question was asked. The visible graph and metric definition made some of that underlying material inspectable.

The same organizational context supported analysis of the company’s structure. Asked about span of control, Genie identified a group with multiple reporting layers and managers with only a handful of direct reports. Ghodsi said this kind of analysis followed from having information about the people and reporting lines already available.

Other examples focused on what the system could reveal about work and performance. Databricks had about 2,000 engineers, and a chart of merged pull requests showed an L-shaped distribution: a small group of engineers was shipping much more code than the rest. Calacanis described the top group as producing roughly ten times as much. Ghodsi said the leading engineers were using automated “factories” that others were not.

Ghodsi then asked Genie to add AI costs to the comparison. The displayed chart compared top engineers’ average merged pull requests per week with their annualized AI costs and cumulative share of merged pull requests. Some of the most productive engineers were not spending more than some of their peers, Ghodsi said. He used that comparison to argue that token use could be made more efficiently—for example, by routing simpler questions to simpler models or using open-source models.

A query about a recent customer meeting produced a response that began, “The meeting went poorly.” Genie’s screen showed a customer-frustration ranking, and Ghodsi said the system listed who said what and which features the customer wanted. He said it could collect feature requests from customer-meeting transcripts and pass them to engineering, where automated processes could help build the requested features.

Ghodsi’s examples make a specific management claim: records of meetings, survey responses, and work can help leaders investigate what happened without relying entirely on a series of verbal accounts. Calacanis described leadership as detective work—asking who spoke in a meeting, what a customer actually thought, and whether a team’s account matches the record. A system that surfaces those records could let a leader begin with evidence rather than repeatedly gathering accounts from people.

Ghodsi said Databricks tracks “human blockers”: points where work pauses for legal approval, a finance decision, marketing agreement, or coordination between teams. The goal, he said, is to reduce the number of those blockers. When work crosses departments, the delay may come not from the task itself but from waiting for a meeting, agreement, or escalation.

The same access that could reduce those delays also makes the boundaries of the system consequential. Ghodsi described building a bot that would join every Slack channel as him, read every Notion page whether or not he had been invited, collect every email, and report important decisions, participants, and unresolved issues. He says he was glad he did not share that version with the whole company. The example illustrates a tension in his approach: comprehensive context can help surface how decisions are made, but assembling that context means reaching into communications and material that may not ordinarily be visible to every employee or executive.

Ghodsi says employees have access to the information used in Databricks’ internal analyses. He uses it to ask which leaders follow the company’s stated principles, what culture is actually operating inside a team, and where practice diverges from the organization’s declared norms. A divergence is not automatically evidence of failure, he says. A team may be right and the broader organization wrong. The point is to make differences visible so the company can decide whether a group or the organization should change.

That visibility can also turn activity into a measure of contribution. Ghodsi described identifying people who participate in decisions and asking which leaders follow principles such as truth-seeking and first-principles thinking. Calacanis framed the system as “management Moneyball”: an attempt to measure who contributes and how, rather than relying only on impressions. Ghodsi says he wants employees to see the productivity gap and help shift the curve so more engineers can work like the high-output group.

The exchange leaves open how those measurements should be interpreted. A count of pull requests, participation in decisions, or adherence to stated principles can make patterns easier to inspect, but the demonstration did not establish that any one measure captures an employee’s full contribution. Ghodsi’s stated aim is to use the information to improve performance and competitiveness, not to treat every difference as a verdict. He says this ability is a competitive advantage while it remains uncommon; if it diffuses across companies, the advantage may disappear.

Staying private protects focus during a volatile transition

Asked when Databricks would go public, Ghodsi described the decision as a question of what best serves the company’s long-term success. Public markets would bring access to capital and a powerful recurring opportunity to tell the company’s story. A public listing would also replace the annual process of organizing employee tenders—a substantial undertaking, he says, given the amount of liquidity Databricks has arranged.

The private-market alternative has worked so far. Ghodsi says employees have had regular opportunities for liquidity and that the tenders have not used all the money set aside. He also says the company has been able to complete billion-dollar acquisitions, with liquidity helping make its equity attractive to prospective sellers. He noted that employees may want to sell, then reconsider how much to sell when they weigh that decision against the company’s growth.

The main reason to wait, however, is the volatility of the current AI transition. Ghodsi says public-market swings can affect employee morale: a falling share price can make employees feel that promised future compensation has evaporated, only for the valuation to recover months later. He wants to avoid that cycle while Databricks is changing its product mix and entering markets such as security and marketing. Public-market attention, he argues, could distract the company and its leadership from that longer-term work.

He says Databricks will go public, but he does not see a need to do it now. Remaining private is not presented as a rejection of public markets; it is a way to avoid what he sees as unnecessary volatility during a period of rapid change.

Ghodsi sees cyber risk as concrete and the extinction scenario as unsupported

Ghodsi traces much of the current fear about recursive self-improvement to a particular chain of reasoning. AI systems have become good at coding. People who have spent their lives programming see models writing code and wonder whether those systems will improve themselves, producing an accelerating cycle that eventually escapes human control. Ghodsi says that concern is understandable, especially for people who have experienced AI taking over work central to their identity. But he argues that coding automation alone does not demonstrate such a cycle.

He says current training still involves large, carefully managed runs. Labs do not conduct many such runs each year, and Ghodsi describes researchers as scrambling to assemble the GPUs, networking, and engineering needed to avoid a failed run. Improving a model still requires more compute under the scaling laws he describes, while data preparation and reinforcement learning involve human labor and repeated, domain-specific adjustments. In his account, that is not the accelerating cycle imagined in a fast takeoff.

Even if a self-improving loop emerged, Ghodsi says, it would not by itself prove that a conscious system would take over. He argues that a threat would still need a way to act in the world. Calacanis likewise distinguishes between superintelligence and less speculative cyber incidents, citing a reported vulnerability in poorly secured refrigerators as an example of a real problem that is not evidence of superintelligence.

On cyber risk, Ghodsi is less dismissive. Attackers can try many approaches and need only one to succeed; defenders must prevent a breach. He says AI can help scan newly written code for vulnerabilities, but much of the remaining danger lies in legacy code. Databricks offers tools for scanning and monitoring systems, he says, and older software can be rewritten or protected through other methods. He compares vulnerabilities to fruit that attackers can pick: AI may enable them to reach higher, but he expects companies to secure those weaknesses over time. Calacanis raises the possibility that people themselves will remain a vulnerability, even as software defenses improve.

The two also distinguish broad existential claims from risks arising in specific applications. Calacanis points to the effects of social platforms, algorithmic news distribution, and possible harms to children using conversational systems. Ghodsi says those concerns deserve attention, while rejecting the idea that the available evidence supports a near-term human-extinction scenario. He says he sees no evidence for recursive self-improvement and considers cyber risk more concrete and manageable.

Ghodsi also resists certainty about claims that AI will quickly cure major diseases. He says AI can accelerate research, but medical advances still have to pass through human trials. He cites Databricks’ work with Regeneron on identifying a genome associated with chronic liver disease, and a newer model developed with Merck called TEDDY—Transformer Enabled Drug Discovery. Rather than predicting the next word, he says, TEDDY models a gene regulatory network to help distinguish genetic causes of disease from genes that are reacting to it. He calls the approach a potential accelerator for drug discovery, but says he cannot predict whether cancer will be cured in six months, ten years, or much longer.

He gives other examples of AI in use: Crisis Text Line, which he says uses large language models and agents to detect when children may be considering self-harm; and Zipline’s use of Databricks to manage data related to its delivery drones.

Ghodsi expects enterprise adoption to take a decade or more. His reason returns to the context problem: if a major bank will not record meetings, an AI system cannot know what happened in them. The models may be capable, but the information, governance, and organizational changes needed to put them to work are not automatically in place.

The frontier, in your inbox tomorrow at 08:00.

Sign up free. Pick the industry Briefs you want. Tomorrow morning, they land. No credit card.

Sign up free