When trying to build a case for workplace AI, enterprises are finding value by embedding AI agents into business intelligence platforms, for example, and by experimenting with AI tools that can perform multi-source analysis and generate multimedia results.
Early enterprise testing of online AI services generated results that were disappointing to say the least, which is likely why Gartner says generative AI is in the dreaded “trough of disillusionment.” Most enterprises found the popular chatbot technology wasted more employee time playing with it than it saved in actual applications. Only about 8% of enterprises found the first-level tools could really make a business case.
One enterprise executive told me: “I don’t dispute that [an online AI tool integrated with document and email production] has saved maybe half an hour a day for each of a thousand workers, which is 500 hours a day overall. I do dispute that would mean we could cut sixty of those thousand workers. We didn’t cut any.”
But that was yesterday. Today, almost 85% of enterprises say that there is value in online AI, and more than half say that they’ve found applications they’re willing to pay for. What changed? And is it enough to save the major investments giants have made in AI tools? We’ll try to answer both questions based on enterprises’ own comments.
The “what changed?” part is simple. Enterprises realized that while roughly 60% of employees overall used computers and could thus use AI tools readily, only about 28% of these workers did things that AI could improve in a way that created business value. For the rest, AI might save some time or do some good, but not in a way that impacted the bottom line. For many in that majority, the minimal improvements were offset by mistakes, hallucinations if you like, that the workers weren’t able to catch. For the key 28%, who are all what we’d call “knowledge workers,” traditional prompts to online generative AI wasn’t the best answer either. So, enterprises started looking at AI tools for the 28%, and found value. How? The companies who commented had one common point and two different ones to make.
The common point is that simple questions to an online generative AI service, or the use of AI embedded in email, word processing, or presentation applications, really didn’t move the needle much. Simple interactions seem to increase the risk of AI-generated errors or misunderstandings without adding much in terms of quality and nothing in terms of insight. To get that, you had to move beyond the AI basics, to something more like the AI agent approach.
Among the 85% of enterprises who found online AI value, more than two-thirds said the value came from integrating AI with things like business intelligence tools, the “embedded” model of an AI agent. IBM has been a big promoter of this approach, which may account for its acceptance. In this mission, enterprises said that AI often found insights in data that even knowledge workers struggled to find, or missed completely. Both the time saved and the quality of the insights were seen as justifying their paying for AI support.
A second path enterprises like had only about 35% buy-in, but generated the most enthusiasm. It is to use an online AI tool that offers more than a simple answer to a question, something more like an “interactive AI agent” than a chatbot. Two that got all the attention are a tool to generate a detailed and almost scholarly report and one to do a multi-source analysis of documents, generating some sort of audio summary or podcast. The specific tools cited most often are Google’s Gemini Deep Research and NotebookLM.
The things all users like about the research report option is the depth of content and the abundance of references, but a bit over half said that these reports are often insightful in themselves. I tried out Deep Research, asking for a report on the network equipment market since 1980 and identification of the main factors driving the future. The result developed the evolution as I recalled it as an analyst through that period, and also identified AI and the HPE/Juniper deal as the most significant developments, which mirrors my own views. All this, over twenty pages with references, from a single-sentence prompt.
Only one-fifth of the users tried the audio output option, but this actually got the most enthusiastic comments. Sales, marketing, and product planning teams absolutely loved the ability to generate a two-party podcast analysis of multiple sources, using it as a means of comparing their own material with that of competitors, for example. A few also tried creating a market report like the one I described, then using a competitor’s material or even the transcript of their earnings call and asking for a “podcast” to compare the two.
What tended to get people excited about this is the value of audio material in training and explaining. One enterprise had a project to build a sales training program from the tools, and another had one to prep salespeople for calls by giving them literal talking points. Could you get something like this without an AI tool? Sure, but it would take a lot of time and effort, and some enterprises noted that the AI tool produced an “objective” output, where human-authored material often has a bias in favor of the company’s own positioning. Thus, the AI tool is also useful in uncovering material that doesn’t paint the company’s products/services in the best light. I tried this too, and found the results to be uncannily realistic and totally absorbing.
The only issue that users mentioned regarding these AI tools is that of copyright. About half said that they would be concerned taking the output public, because they believe legal views of the copyrighting of AI-generated material are skeptical, that they are concerned that their AI tool might be accused of infringing on someone else’s copyright, or both. However, this didn’t impact internal use and some said they had external uses their legal teams had accepted.
What about errors? Did AI turn out some trashy results in either or both examples? Not often. Less than 10% of those who tried either of these AI-tool applications said they had to discard the results in any situation, and about half said that when there was a problem, it came down more often to a careless wording of the prompt rather than to an error in AI analysis. I found no serious issues in my own tests, and I truthfully have to wonder whether the output couldn’t have been used as-is in place of the kind of thing you might ask an analyst to do.
So is AI going to steal jobs, including mine? I won’t use the output as my own work because I view that as dishonest, but I’ve posted some examples identified as AI experiments, with totally positive responses. I think that, as a means of researching for a project, the AI tools could be useful to me as well. I also think that I’m more insightful than those AI tools, but is that like craftsmen arguing that hand-made swords and plows are better than manufactured ones? Maybe; we’ve likely all read that about half of all new Internet content is now AI-generated.
Overall, I was surprised and challenged by enterprise comments here. I don’t think that AI is going to destroy the human race or my own job, but I do think it’s going to be as impactful as the industrial revolution. Craft workers hundreds of years ago might have been able to make a better sword or plow than machines could, but once the machines came along, you couldn’t be broadly successful in either space without them. I think harnessing AI will be just as essential for knowledge workers. It may be that already.
So, on to our last question. Will this be enough to justify all the early AI hype? Maybe, but not a continuous high level of new investment. You can’t be satisfied with impacting 28% of the 60% of office workers to keep the AI dollars flowing. I think OpenAI may be demonstrating the path to redemption here, with what seems a clear shift toward AI tools for the 28%, but AI agents for the rest. Google has tools already, and its recent enterprise announcement shows a commitment to AI agents, a commitment we also see from Amazon and Microsoft. Salesforce is making a big AI agent push. So, forget AI doomsday and other hype; reality, in the form of AI tools versus AI threats and sovereign agents versus pure AI clouds, is taking shape. The players who get this will win, but those who don’t see the shift need to quickly reconsider their positions. There’s still a lot of hype in AI, but AI isn’t all hype.
