IT managers believe generative AI can improve IT service performance, but they remain concerned about AI data quality and security, according to new research from EMA.
The rise of generative AI in recent years has accelerated the application of artificial intelligence across various business segments, especially in IT. According to recent research from Enterprise Management Associates (EMA), genAI offers promising efficiency and productivity improvements, although concerns about security and data quality remain.
EMA recently released its Applying GenAI to IT Operations report to shed light on how IT organizations are using the technology. For the report, EMA surveyed 151 respondents who demonstrated that their organizations were either testing or using genAI tools, having had firsthand experience with the technology. The survey results indicate that 94% of the IT organizations surveyed are currently utilizing generative AI in some capacity.
“We asked survey participants to tell us what they think is the chief benefit of applying this AI to IT management, and the big ones are optimized IT service performance and improved IT business alignment. Those are really attractive things to people who control budget,” said Shamus McGillicuddy, vice president of research covering network management at EMA, during a recent webinar.
GenAI can benefit IT teams in several ways, but respondents don’t expect it to replace personnel. According to the survey, the biggest benefits survey respondents reported include:
- Optimized IT service performance: 38.4%
- Improved IT/business alignment: 36.4%
- Proactive problem prevention: 31.1%
- Accelerated technology implementation: 30.5%
- Reduced security risk: 28.5%
- Improved collaboration/knowledge sharing: 27.8%
- Cost optimization: 27.2%
- Accelerated incident response/resolution: 26.5%
- Mitigation of skills gaps/personnel shortages: 21.9%
In addition, 96% of those surveyed said they believe genAI can make their IT personnel more productive, and 98% believe genAI can be helpful for summarizing insights from IT dashboards and reports to show IT staff which users are having a bad experience, describe current trends in bandwidth utilization, and determine how alerts are related.
“[Respondents] see the least potential with the mitigation of skills gaps or personnel shortages. [GenAI] is about boosting the productivity of the people you have, as opposed to replacing them or avoiding hiring when there is attrition,” McGillicuddy said. IT organizations are putting genAI applications to work both with in-house developed use cases as well as vendor-provided tools. Some 94% of those surveyed said their IT organizations were using general purposed genAI today, and the most common use cases include:
- Create documentation/procedures/knowledge base: 74%
- Operational data analysis: 69%
- Step-by-step guidance for tasks: 66%
- Research/learning: 64%
- Programming/scripting: 61%
- Configuration generation: 47%
Nearly three-quarters (73%) of respondents also said their organizations are using generative AI that IT vendors supply to:
- Query IT systems: 67%
- Recommend actions: 66%
- Query product documentation/validated designs: 60%
- Automate actions: 55%
Vendor-provided genAI can also cause concerns. For instance, when it doesn’t perform as expected, IT organizations don’t get the full value of genAI, and it raises cost concerns. Nearly one-fifth (18%) of those surveyed indicated cost as a challenge associated with genAI and IT management.
“Cost is a concern for those who feel their IT vendors’ AI tools are not delivering as promised. This suggests that if IT organizations invest in vendor-provided generative AI capabilities but find them to be of poor quality or not meeting their expectations, then the cost of those tools becomes a bigger issue and concern for them,” McGillicuddy said.
Despite the reported benefits, genAI still concerns IT organizations to some degree. Other areas that IT organizations found genAI challenging is in evaluating the quality of AI outputs. Some 63% of respondents said they are concerned about data quality and the ability to properly evaluate the accuracy and reliability of the content and insights generated by AI. Integrating AI tools with existing processes is also somewhat of a challenge for 30% of respondents.
Nearly 20% pointed to user acceptance as a challenge with implementing genAI. Specifically, getting IT personnel accustomed to relying on and trusting AI-generated content and recommendations can be challenging, according to EMA. When it comes to general-purpose tools, 63% of respondents are “at least somewhat concerned,” and 54% said they feel the same about using IT vendor tools. IT professionals have several concerns about security and compliance, and they are not fully unwarranted. For instance, AI tools could be exploited to extract sensitive data, so IT teams must remain vigilant, despite the reported genAI benefits. According to EMA, the top security and compliance concerns are:
- Data leakage via prompts: 52%
- Malicious code/filters AI generates: 44%
- Overconfidence in AI-driven security: 43%
- Compliance violations: 40%
- Bad changes that create vulnerabilities: 32%
“Users are struggling with evaluating AI solutions, especially the quality of content that comes out of them. And they’re worried about security and compliance risk; a majority of respondents see some risk there,” McGillicuddy said.
