A private AI agent is one whose knowledge, tools, network access, and data retention are each deliberately scoped by the organisation running it, rather than left to whatever a platform or model provider defaults to. An agent earns the label "private" only when all four of these are controlled on purpose, not because a vendor states somewhere that its model doesn't train on customer data.
That's the short answer. The rest of this covers what each boundary actually means, why the two most common shortcuts people reach for, self-hosting and a no-training clause, are each necessary but not sufficient, and what to ask a vendor before you trust an agent with anything sensitive.
What actually makes an agent private, and what doesn't
Being private is a property of what an agent can reach and what happens to what passes through it, not a property of where it sits or what a vendor's homepage claims.
A few things that do not, by themselves, make an agent private: a privacy policy promising good behaviour (policies describe intent, not what the software can technically do), a "we don't train on your data" line from the model provider (one link in a longer chain, more on that below), running on your own servers (self-hosted software can log everything to a world-readable bucket just as easily as hosted software can), or the word "private" sitting in the product name.
What determines it is four boundaries, each controllable on its own: what the agent knows, what it can do, what it can reach over the network, and what gets kept afterward. A platform can get any one of these wrong while nailing the other three, so all four need checking, not just the one a vendor happens to lead with.
What are the four boundaries that define a private AI agent?
Four things, each worth checking independently, because getting three right and one wrong still leaves a hole.
Knowledge. Which documents, databases, or knowledge bases can the agent actually read? A private agent's knowledge is limited to what's explicitly bound to it, agent by agent, rather than a shared pool everyone on the account can query by default. Retrieval-augmented generation (RAG) is one way to give an agent knowledge, but it's optional: an agent with no knowledge base attached can still be private or not, depending on the other three boundaries.
Tools. What can the agent do, beyond generating text? Every connector, API, or function call available to an agent is a door out of the conversation, and often a door into another system: a CRM, a ticket queue, a shared drive. A private agent's tool list is a scoped allowlist someone deliberately chose, not "everything the platform happens to support." Ask what's on that list and who set it.
Network. Where can requests originate from the agent, and where can they go? This is the boundary people get wrong most often, usually by treating "no browsing" and "no network access" as the same thing. An agent with no general-purpose web-browsing or web-search tool cannot fetch arbitrary URLs on its own initiative. That's different from having zero outbound calls: a connector to Slack or Jira does make an outbound call, on purpose, to the one system it was wired to reach. The useful version of this boundary isn't "the agent can't talk to anything," it's "the agent can only talk to what it was told to talk to."
Retention. What gets kept after the conversation ends, and for how long? This covers message content, but also the less visible stuff: request logs, execution traces, tool-call records, anything an observability layer captures on the side. A private agent's retention is something the customer sets, not something the platform defaults to "forever, for troubleshooting." And deletion should mean the record is gone or irreversibly anonymised, not soft-deleted and still sitting in a backup someone can restore.
Is self-hosting the same as private?
No. Self-hosting is one way to get some of these privacy properties, and for a real subset of buyers it's the right way, but it isn't the only way and it doesn't automatically deliver any of the four boundaries above.
Self-hosting wins when the requirement is about who has physical or administrative custody of the infrastructure: a government agency, a defence contractor, or a bank operating under a rule that the workload has to run inside a boundary the vendor doesn't control. If that's the actual requirement, no amount of vendor promises about scoping and retention substitutes for it.
That's a different calculation from the one facing buyers without a custody requirement, which is most of them. For that group, what gets underweighted is the cost on the other side. Self-hosting an agent platform means your team now owns model updates, security patching, and scaling under load, on top of building the same knowledge, tool, network, and retention controls a hosted vendor would otherwise maintain, and getting those four right is the hard part regardless of who runs the servers. A self-hosted deployment nobody has staffed to keep its logging tight and its connector scope narrow is not more private than a well-run hosted one; it's just misconfigured somewhere your own team can't see either. For most teams without a standing platform security function, a hosted vendor that publishes and enforces the four boundaries above will beat a self-hosted setup with no one dedicated to hardening it.
Does "the model doesn't train on my data" mean the agent is private?
Not by itself. A no-training commitment from a model provider covers exactly one hop: the moment your prompt reaches that specific model. It says nothing about everything else that happens to your data on the way there and back.
Between a user's message and a model's response there's usually a platform layer: request logs, an orchestration or agent framework that may retain transcripts for debugging, whatever the agent's tools returned, analytics or tracing systems capturing payloads for observability, and any subprocessors sitting underneath. A no-training clause is silent on all of it. A platform can honestly say the model provider doesn't train on your data while still storing full conversation transcripts indefinitely in its own database, shipping tool-call payloads to a third-party logging service, or giving a wide internal team read access to customer traces.
Privacy is a property of every hop, not the last one. Treat a no-training commitment as answering one question, not the whole audit.
What should you actually ask a vendor?
Ask questions specific enough that a vendor without the right architecture can't dodge them.
- Which knowledge bases can this specific agent read, and can I see that list, not just the account-wide one?
- What tools or connectors can the agent invoke, and who chose that scope, me or your default?
- Is there a browsing or web-search tool available to agents on your platform, and if not, is that a setting or an architectural fact?
- Your connectors make outbound calls; where exactly do those calls go, and is that list fixed or something the agent decides at runtime?
- What message content do you store by default, and for how long, before I change anything?
- If I delete a conversation, is the record gone, retained and anonymised, or retained and just hidden from my dashboard?
- Where do request logs and execution traces live, and who at your company can read them?
- Does your "we don't train on customer data" statement cover your own platform logging and your subprocessors, or only the underlying model?
- If I self-host instead, what operational burden moves to my team, and have you sized it honestly?
A vendor with a genuinely private architecture can answer all nine specifically. A vendor selling "private" as a feature usually stalls around the third one.
What does this look like in practice?
Commt, an AI agent platform, scopes each of the four boundaries by design: an agent reads only the knowledge bases explicitly bound to it, with RAG optional rather than assumed; which tools and connectors an agent may reach is set by the customer, from a list of twenty connectors available at launch; no web-browsing or web-search tool exists on the platform for an agent to invoke, a fact about the architecture rather than a setting that can be switched off; and no message content is stored by default, with retention set by the customer and deletion meaning the record is deleted or anonymised. It's one example of applying the checklist above, not the only correct shape a private agent can take.