By Yazen Rahmeh, Cybersecurity expert at SearchInform.
Companies across the Gulf region are rapidly adopting AI tools to improve employee productivity, analyse large volumes of data, and automate routine tasks. However, such capabilities are also creating new security risks: employees increasingly upload confidential documents, source code, and customer records to AI systems.
As a result, AI is becoming not only a productivity tool but also a new channel for data leakage. In this article, we will discuss the main ways AI technologies create risks for confidential information.
Accidental data leaks
In 2023, Samsung experienced widely reported data leakage incidents. Following these incidents, Samsung restricted employees’ use of AI tools in the workplace. In one incident, an employee uploaded confidential source code for debugging; in another, an employee uploaded internal engineering data to ChatGPT. These stories illustrate how easily sensitive information can leave the corporate perimeter.
In many cases, people don’t intend to cause harm. They are simply trying to streamline repetitive tasks or increase productivity. However, once data is uploaded to external AI systems, it can be shared with other users. How? The answer is simple: user inputs can be included in training data sets.
Although most platforms provide settings that allow users to choose not to have their data used to train models, many users forget to enable this option.
How to protect:
Establish clear AI usage policies
Define what types of data employees are allowed to upload to external AI services and which AI tools are approved for corporate use.
Regulatory risks
One of the main reasons AI chatbots have become so popular is their convenience. Users only need to open a browser tab, enter a request, and receive a result within seconds. AI services are accessible from corporate laptops, personal devices, and mobile phones, while the computing infrastructure itself remains hidden from the end-user.
From a security perspective, however, this convenience creates additional risks. What appears to be a user-friendly service can become a headache for the security team. AI services are built on physical infrastructure located in data centers somewhere in the world.
This raises several important questions:
• How exactly is the uploaded data stored?
• Who has access to it?
• How long is it retained?
• Can it later be accessed by external parties?
How to protect:
Deploy AI on local infrastructure
Many governments prefer to adopt a cautious approach and apply strict regulations for cross-border data transfers, especially when personal or sensitive data is involved.
For this reason, many organisations consider deploying AI services within local infrastructure or private environments to be the preferred approach for protecting sensitive information. Keeping AI systems inside the corporate perimeter significantly reduces the risk of confidential data exposure to third-party infrastructure.
Intentional data leaks
One of the most serious threats comes from malicious insiders who deliberately expose confidential data. They commonly use various business collaboration tools, such as email, instant messengers, and AI chatbots, to leak sensitive files.
Insiders can use AI tools to bypass the security controls of protective solutions. For example, they can rephrase text to evade keyword-based detection rules while preserving the original meaning.
How to protect:
Strengthen monitoring and data protection controls
To protect against such evasion methods, companies should look for advanced DLP solutions. Such systems use AI-powered security policies to detect attempts to disguise sensitive information. Transitioning from rigid keyword detection to content-based detection significantly enhances the effectiveness of data protection measures.
AI Agent Threats
AI tools are constantly evolving. While AI chatbots are reactive and only act on users’ behalf, autonomous AI agents are rapidly becoming one of the most advanced technology applications. They are very different from chatbots because they are proactive and can act without a direct command from the user. For instance, they can interact with email, stored files, different applications, and services. On the one hand, they can improve the user experience and enhance productivity. On the other hand, these capabilities introduce major security risks. A recent study analysed how AI agents interact with people and other agents. It demonstrated that existing security guardrails cannot ensure reliable protection.
The study discovered that AI agents can be manipulated. They can be influenced by non-owner users, which can lead to significant risks. Researchers were able to manipulate an AI agent to share an email list and content, including confidential data, such as bank account info, national ID, and home address.
Another major flaw was that a non-owner user was able to convince an AI agent to follow external instructions and modify its long-term behaviour. They tricked it by creating a “Constitution for AI agents” placed on GitHub Gist. The caveat is simple: the constitution document was editable for anyone on GitHub.
Criminals can use such vulnerabilities to leak confidential data. Instead of relying on vulnerability exploitation and privilege elevation, hackers may only need to trick an AI agent. It may exfiltrate data, leveraging legitimate rights and permissions. Thus, various Living-off-the-Land techniques based on AI abuse may become more efficient and widespread.
How to protect:
Manage access rights for both human and non-human entities
Access control policies should apply not only to employees but also to AI agents and other autonomous systems. AI tools should only have access to the data and services necessary for their specific tasks.
Conclusion
These risks are no longer hypothetical. Companies around the world are rapidly integrating AI into business processes. Research conducted by our company demonstrated that more than half of companies in the Gulf region already use AI tools to process sensitive data. As AI tools become more widespread, new data leakage risks will continue to emerge. Companies that fail to invest in advanced information security tools may find themselves relying on the assumption that AI tools will process confidential data responsibly.
