“As AI-generated data becomes pervasive and indistinguishable from human-created data, a zero-trust posture establishing authentication and verification measures, is essential to safeguard business and financial outcomes, ” continues Wan Fui Chan.

AI-generated data heightens risks

Large language models (LLMs) are typically trained on “web-scraped” data and a variety of sources, including books, code repositories and research papers. Some of these sources already contain AI-generated content, and if current trends continue, nearly all will eventually be populated with AI-generated data.

According to the 2026 Gartner CIO and Technology Executive Survey, 84% of respondents expect their enterprise to increase funding for Gen AI in 2026. As organisations accelerate both adoption and investment in AI initiatives, the volume of AI-generated data will continue to rise. This means future generations of LLMs will increasingly be trained on outputs from previous models, heightening the risk of “model collapse,” where AI tools’ responses may no longer accurately reflect reality.

“As AI-generated content becomes more prevalent, regulatory requirements for verifying ‘AI-free’ data are expected to intensify in certain regions,” said Chan. “However, these requirements may differ significantly across geographies, with some jurisdictions seeking to enforce stricter controls on AI-generated content, while others may adopt a more flexible approach.

“In this evolving regulatory environment, all organisations will need the ability to identify and tag AI-generated data. Success will depend on having the right tools and a workforce skilled in information and knowledge management, as well as metadata management solutions that are essential for data cataloguing.”

Strategic management actions

According to Gartner, active metadata management practices will become a key differentiator, enabling organisations to analyse, alert and automate decision making across their data assets.  The company suggests that businesses should consider several strategic actions to manage the risks of unverified data, such as:

• Appointing an AI governance leader
• Fostering cross-functional collaboration
• Leverage existing governance policies
• Adopt active metadata practices