Generative AI is moving into ordinary business processes: drafting customer responses, summarising documents, analysing spreadsheets, searching internal knowledge and assisting software teams. The security question is no longer whether employees will use these tools. It is whether the organisation can control what information enters an AI system, where that information is processed and what happens to it afterwards.
Putting an AI service inside a virtual private cloud, or VPC, can reduce exposure to the public internet and provide stronger control over network access. It does not, by itself, make business data secure. Security depends on the model provider’s data policies, identity controls, application design, logging, retention settings and the way the AI system connects to other company systems.
What happens to business data when it enters an AI system?
A typical AI request contains a prompt and, often, additional context. That context might include a document retrieved from a company knowledge base, a customer record, a support ticket or a section of a spreadsheet. The AI service processes this material to generate an answer. Depending on the product and configuration, several different things may then happen.
The request may be processed temporarily and discarded. It may be retained in application or provider logs for monitoring and abuse prevention. It may be stored by the company’s own AI application for audit, conversation history or retrieval. In some services, customer content may be excluded from model training by default or by contract. In others, training or product-improvement use may depend on the account type and settings.
Those distinctions matter. A model does not necessarily absorb every prompt into its permanent knowledge, but organisations should not assume that data disappears simply because an answer has been generated. The full information lifecycle should be documented: collection, transmission, processing, storage, retrieval, logging, human review, deletion and backup.
Provider documentation should be treated as a starting point rather than a substitute for due diligence. For example, organisations can review the data-governance materials for Amazon Bedrock, Azure OpenAI Service and Google Cloud Vertex AI. The relevant terms can differ by service, region, deployment model and account configuration.
Is AI inside a VPC secure?
A VPC can create a private network boundary around an AI application or a model endpoint. Private subnets, security groups, routing rules and private service connections can limit who is able to reach the system. This is valuable, particularly when the application handles sensitive data or needs to connect to internal databases.
However, a VPC controls network reachability, not the complete use of information. A model hosted or accessed through a private network may still expose data through an overly permissive identity, an insecure prompt pipeline, application logs, third-party connectors or an administrator with excessive access. A private endpoint can also be configured incorrectly.
The answer is therefore conditional: using business data with an AI service inside a VPC can be secure when the architecture, provider terms and operational controls are appropriate. It is not automatically secure merely because the traffic remains within a cloud environment.
There is also an important difference between two arrangements. In one, the company runs an AI application in its VPC but sends prompts to a managed model service through a private connection. In the other, the model itself is deployed within the company’s cloud environment. The first may offer less infrastructure responsibility but requires careful review of the provider’s handling and retention practices. The second can provide greater control over the environment, while creating additional responsibilities for model security, patching, access management, monitoring and capacity.
Controls that should be in place before BAU use
Security teams should begin with data classification. Employees need clear rules explaining which information may be used with approved AI tools, which information must be masked and which information is prohibited. Customer identity data, payment information, credentials, health information, unpublished financial results, legal advice and confidential intellectual property may require stricter treatment than routine internal material.
Technical controls should reinforce those rules rather than relying on employee judgement alone. A mature design typically includes:
- Approved services and accounts: Route business use through managed enterprise accounts, not personal AI accounts. Disable or restrict unsanctioned tools where appropriate.
- Strong identity controls: Use single sign-on, multifactor authentication, role-based access and short-lived credentials. Separate development, testing and production access.
- Data loss prevention: Detect or block sensitive fields before they reach the model. Redaction, tokenisation and masking can reduce exposure while preserving useful context.
- Private connectivity: Use private endpoints, controlled egress and network segmentation where the provider and architecture support them. Inspect outbound traffic from applications and connectors.
- Retention and logging controls: Decide what prompts, responses and metadata must be retained for audit, then prevent accidental duplication in application logs, observability platforms and support tools.
- Tool and retrieval restrictions: Limit the databases, files, APIs and actions that an AI agent can access. Retrieval should respect the user’s existing permissions.
- Human approval for consequential actions: Require review before an AI system sends an external communication, changes a record, approves a transaction or makes a decision affecting an individual.
Prompt design is also a security concern. Instructions should tell the application what information it may use, what it must not reveal and when it should refuse or escalate. These instructions are useful, but they are not a complete defence against prompt injection. A document retrieved by an AI system may contain malicious or conflicting instructions, so retrieved text should be treated as untrusted input.
How to govern retrieval-augmented AI
Many business applications use retrieval-augmented generation, or RAG. Instead of training a model on every internal document, the application searches a controlled knowledge base and supplies relevant passages with the user’s question. This can reduce the need to alter the base model, but it does not remove data risk.
Access checks must occur at retrieval time. A user who cannot open a confidential document in the source system should not receive its contents through an AI assistant. Document permissions need to be carried into the index, and stale files should be removed or re-indexed when access changes. Embeddings, metadata and cached passages can themselves contain sensitive information and should be governed accordingly.
Organisations should test the system with realistic attempts to retrieve restricted material, override instructions or combine harmless fragments into a confidential answer. The OWASP Top 10 for Large Language Model Applications provides a useful framework for reviewing risks such as prompt injection, insecure output handling and excessive agency.
Contracts, compliance and provider due diligence
Technical architecture cannot resolve a weak commercial arrangement. Before production use, procurement and legal teams should establish whether the provider uses customer content for model training, how long it retains prompts and responses, where processing occurs, who can access the data and how deletion requests are handled.
The review should also cover subprocessors, breach notification, audit evidence, regional processing, encryption, business continuity and the treatment of backups. A provider’s security certification may be relevant, but it does not confirm that a particular AI workflow is configured safely.
Regulatory obligations depend on the sector and the jurisdictions involved. The organisation should map AI use to existing privacy, records-management, financial-services, employment and information-security requirements. The NIST AI Risk Management Framework offers a voluntary structure for identifying and managing AI risks, while local laws and contractual duties may impose additional requirements.
A practical operating model for secure AI use
Start with low-risk BAU tasks that have a clear business owner, limited data scope and measurable review criteria. Record the approved use case, data categories, model provider, region, retention settings, connected systems and responsible team. Test the workflow with synthetic or redacted data before introducing live information.
Once deployed, monitor more than uptime and answer quality. Review access logs, unusual query patterns, sensitive-data detections, retrieval failures, prompt-injection attempts and actions taken by agents. Reassess permissions when employees change roles and remove access when a project ends. Keep a route for staff to report an unsafe output or accidental disclosure without discouraging legitimate use.
Internal guidance should be easy to find. A company’s technology and innovation coverage can help employees understand the broader operating context, while an AI governance checklist can provide a practical review point before a new tool reaches production. These resources should complement, not replace, formal security and privacy policies.
The safest approach is to treat AI as a new processing environment, not as a neutral interface. Put sensitive data through an explicit decision process, connect models only to the systems they need, and verify what the provider and the application do after each request. A VPC can be an important layer in that design, but the real protection comes from combining network isolation with disciplined data governance and continuous oversight.
