Forward deployed engineering, Part 3: Designing inside someone else's estate

In Part 2 we turned a vague ask at Acme Consumer Brands into a signed one-page brief: one first use case, “What did stockouts cost this campaign?”, read-only, with success metrics and a baseline. Now someone has to design the system.

This is where forward deployed work differs most from building your own product. On your own product, you choose the cloud project, the network, the identity provider and the security rules. In a customer’s environment, all of those already exist, other teams own them, and each change you need is a request with its own lead time. A design that ignores this looks good on a whiteboard and then waits weeks for approvals.

This post covers the upfront design in the order I work through it: where the data lives and who owns it, who signs in and how, how traffic travels, which compliance rules apply, and how the security review runs. It ends with the questions I ask in week one and a design review checklist.

If you prefer to watch first, here is the heart of this part as a short animated explainer.

Video: how a forward deployed engineer designs inside a customer's estate, in three minutes. The narration is a synthetic voice; the script and animation are mine.

Where does this show up in an FDE job description?

Public FDE postings describe this work directly:

Google Cloud’s Forward Deployed Engineer, GenAI posting asks you to “architect and code the connective tissue between Google’s AI products and customer’s live infrastructure, including APIs, legacy data silos, and security perimeters.” The same posting describes addressing “release blockers including solving integration hurdles, data readiness issues, and state-management issues.”
Google Cloud’s AI Outcome Customer Engineer, Forward Deployed Engineering posting says you will “architect how these technical assets actually integrate into the customer’s IT ecosystem (connectors, identity, data residency, legal constraints).”
OpenAI’s Forward Deployed Software Engineer posting describes “coding side-by-side to drive projects to completion on their infrastructure.”

The common thread is their infrastructure. The design has to fit what the customer already runs.

What is an estate, and why is it someone else’s?

An organisation’s estate is everything it already runs: its cloud organisations and projects, networks, identity systems, data platforms, security tools and the policies that govern them. When you arrive as an FDE, you design inside that estate.

A few more terms come up throughout this post:

A landing zone (Google Cloud also calls it a cloud foundation) is the pre-built setup of projects, networks, identity and policies that a platform team creates so new workloads land safely. If the customer has one, your design starts from it.
A data owner is the person or team accountable for a dataset and for deciding who may use it.
An identity provider (IdP) is the system that signs people in and tells other systems who they are.
A network perimeter is a boundary that controls which traffic may reach, or leave, a set of resources.
A compliance boundary is a rule from law, contract or policy that limits where data may go and how it is handled.
A security review is the customer’s formal check that your design meets their policies before it touches real data.

Renovating a rented flat is a fair comparison. The tenant (the FDE) can change the kitchen (the agent code), but needs the landlord’s permission (the security review), cannot move load-bearing walls (the identity provider and the network design), must use the building’s pipes and wiring (the network paths) and has to follow building regulations (compliance). Here is Acme’s estate:

the Acme estate the agents must live in: the corporate identity provider, the Google Cloud organisation with Gemini Enterprise, Agent Runtime and BigQuery, and a separate Databricks workspace; every box has an owner, and the FDE owns only the agent code

Only one box is the FDE’s to build: the agents on Agent Runtime, which run in a project the platform team has set up for us. Every other box belongs to an Acme team, and the most useful thing you can do in week one is find out who.

Where does the data live, and who owns it?

Start with the data, because almost every other decision depends on it. For each source in the first use case, I fill in a data map: what the data is, where it lives, who owns it, how sensitive it is and which identity will read it.

a data map for the first use case: for each source, where it lives, who owns it, how sensitive it is and which identity reads it; campaign spend is read by the agent, supply data only as the user, penalty figures only by the supply side, and chat history needs a location and retention decision

Where it lives has two meanings, and customers care about both:

At rest means where the data is stored.
In processing means where it is handled while being used, including where a model reads it.

Gemini Enterprise documents both. Its locations page lists a global location, us and eu multi-regions, and in-country regions such as the United Kingdom (europe-west2). As of October 2026, the in-country regions are generally available with an allowlist, so you request access through your Google account team. The page explains that for the multi-regions, machine learning processing stays within the us or eu multi-region when requests go to the regional APIs, subject to listed limitations. It also recommends the global location when you have no security or regulatory reason to choose another, because it has the newest models and features. That is a real trade-off to put in front of the customer, not a default to pick silently.

At Acme, the data map shows three things straight away:

Campaign spend sits in BigQuery in the EU, and the marketing agent reads it as itself, read-only.
Orders, OTIF and penalties sit in Databricks, and the supply data team owns them. The supply agent reads them as the signed-in user, and only the supply side may calculate penalties. Acme uses federation so neither team copies the other’s data, as Part 1 of the agents series explains.
Chat history is data too. It may contain personal data, so it needs a location and a retention decision like any other source. The Gemini Enterprise security overview states that user-requested data is deleted within 60 days. Part 8 of the agents series covers what an agent should and should not remember.

Who signs in, and how does the agent know?

Every request needs an answer to who is asking?, and the customer’s identity provider supplies it. Two standards do most of the work:

SAML 2.0 (Security Assertion Markup Language) is an XML-based standard in which the identity provider sends a signed assertion, a statement saying who the user is, to the application.
OpenID Connect (OIDC) is an identity layer on top of OAuth 2.0. The identity provider issues an ID token, a signed JSON document describing the user.

Gemini Enterprise then needs to know which users are which, so it can enforce each data source’s access rules. Its identity provider guide offers two options:

Google Identity, which Google recommends for new setups. It works with a third-party identity provider if that provider supports OIDC or SAML 2.0.
Workforce Identity Federation, which federates users from an external identity provider, so their accounts are not stored in, or synchronised to, Google Cloud. The guide notes that Microsoft 365 sources connected with data ingestion need Microsoft Entra ID with Workforce Identity Federation.

Two warnings from the documentation belong in every design. First, if you change identity providers later, users lose their existing chat history, because it is tied to their old IDs. Second, the security overview warns that administrators who can edit workforce pool attribute mappings could, by mistake or on purpose, enable user impersonation. It recommends granting those roles sparingly and auditing them with Privileged Access Manager.

Signing users in is only half of the identity design. The agents also need their own identities, and Sam’s identity has to reach Databricks without the model ever seeing his token:

how identity travels for one question: Sam signs in through the corporate identity provider, Gemini Enterprise knows who Sam is, the router calls the supply agent as itself with Sam's identity as metadata, and Databricks checks Sam's own token against Unity Catalog grants

Part 7 of the agents series builds this flow in code. At design time, the job is to get each decision agreed and written down. I use a short architecture decision record (ADR): a page that records one decision, the options, the reasons and the consequences.

an architecture decision record for one estate decision, how users sign in: the context in facts, the options, the decision and why, the consequences someone will live with, and when to revisit it

The ADR matters most months later, when someone asks why chat history disappeared after an identity project, or proposes ingesting Microsoft 365 data. The answer, and the condition for revisiting it, are already written down.

How does traffic get from A to B?

Next, trace every network path. A few basics first:

A Virtual Private Cloud (VPC) network is a private network inside Google Cloud. Resources in it talk to each other on internal IP addresses.
Cloud VPN and Cloud Interconnect connect a company’s offices or data centres to its VPC, over an encrypted tunnel or a dedicated link.
Private Service Connect (PSC) lets you reach Google APIs, or services in other networks, through an internal IP address in your own VPC, so the traffic does not cross the public internet.
VPC Service Controls draws a service perimeter around projects and Google services to reduce the risk of data exfiltration (data being copied out, by mistake or on purpose).

This is Acme’s design:

network paths in the Acme design: users reach Gemini Enterprise over the corporate network through a Private Service Connect endpoint, a VPC Service Controls perimeter surrounds the Google Cloud projects, and the agents reach Databricks privately through a Private Service Connect interface into Acme's VPC

There are three paths to agree:

Users to Gemini Enterprise. By default, users reach Gemini Enterprise over the public internet, with authentication required. Gemini Enterprise documents private UI access: users connect through Cloud VPN or Cloud Interconnect to a PSC endpoint, and internal DNS resolves the Gemini Enterprise domains to that endpoint’s IP address.
The perimeter. To secure a Gemini Enterprise app with VPC Service Controls, you add its project to a perimeter and restrict the Discovery Engine API (discoveryengine.googleapis.com). After that, the API cannot be reached from the public internet, the user interface is blocked except where ingress rules allow it, and creating new data stores is blocked until an administrator updates the related organisation policy constraints. The documentation recommends dry run mode first, which logs what the perimeter would block without blocking it.
Agents to private data. An agent on Agent Runtime runs in a Google-managed network with no access to your VPC. A Private Service Connect interface connects it to a network attachment in your project, so its traffic can reach private services. The documentation lists the details that catch people out: the network attachment needs a subnet of at least /28, and inside a VPC Service Controls perimeter the agent has no internet access at all unless you build an egress path, for example a proxy in your VPC with Cloud NAT. On the other side, Databricks on Google Cloud supports Private Service Connect too, and its documentation points out that enabling private connectivity does not by itself block public access to the workspace.

Every arrow in this diagram belongs to the network team or the Databricks administrator. My view: draw the diagram in your first week, before you write any agent code, and take it to the network team as a question, not a request.

Which compliance boundaries apply?

Compliance rules nest inside each other. I draw them as rings, from the ones you can change least to the ones you control fully:

compliance boundaries nest: law and contracts contain Acme's own policies, which contain the platform controls you configure, which contain what the agent reads, stores and shows; each ring has a different person who signs it off

Law and contracts. In the UK, UK GDPR requires a data protection impact assessment (DPIA) for processing that is likely to result in a high risk to individuals. Whether this agent needs one is the privacy team’s call, so ask them in week one. Sector rules and customer contracts sit in the same ring.
Acme’s policies. Data classification, retention periods and approved regions. The data owners apply them.
Platform controls. The settings that put the policies into effect: the location you choose, customer-managed encryption keys (CMEK), the perimeter, audit logs and Access Transparency (logs of actions Google staff take on your content). Its documentation lists limits, including that data stores and apps must be multi-regional rather than global to be fully covered, which connects back to the location trade-off above. One detail from the CMEK documentation is worth a line in the plan: CMEK applies only after the key is registered, and apps or connectors created before that remain unprotected. Set the key up first.
The agent. What it reads, stores and shows, and to whom. This is the ring the FDE owns, through tools, guardrails and evals.

How does a security review work?

A security review is the customer’s check that the design meets its policies before any real data flows. Reviewers usually want the same artefacts: an architecture diagram, the data map, the identity design, the network paths and a threat model (a list of the ways the system could be attacked or misused, with a control for each). For an agent, the OWASP Top 10 for LLM applications is a good starting list, because it covers risks such as prompt injection that a standard web review may miss.

a security review as a sequence across three weeks: the FDE sends a design pack, security asks threat-model questions, data owners approve grants, the platform team runs the perimeter in dry run, and security signs off with conditions before the pilot

Three things make reviews faster in my experience:

Send the design pack before anyone asks, using the customer’s own template if they have one.
Answer threat-model questions with controls you can point to, such as the read-only identity, the callback that blocks writes, and the human approval step from Part 9 of the agents series.
Treat conditions as tasks. “Approved, provided logs go to our security monitoring system” is a yes. Add it to the plan and close it.

The write-blocking callback in the second point is short enough to put straight into the design pack. This is the read-only guard from the agents series, which runs before every SQL tool call:

def sql_guard(tool, args: dict, tool_context) -> dict | None:
    """before_tool_callback: block anything that is not read-only."""
    if not tool.name.endswith("_sql"):
        return None
    refusal = guard_sql(args.get("sql") or "")
    if refusal:
        _log("sql_blocked", agent=tool_context.agent_name, tool=tool.name)
        return {"status": "blocked", "error": refusal}
    return None

root_agent = Agent(
    ...,
    before_tool_callback=[sql_guard, person_gate],
)

Part 3 of the agents series, Letting an agent touch data safely explains guard_sql and the layers behind it.

What should you ask in week one?

These are the questions I take into the first week, grouped by owner. Each unanswered one is a risk in the plan.

Data (data owners)
  - Where is each source stored, and in which region?
  - Where may it be processed by a model? Global, EU, UK only?
  - Which fields are personal or confidential?
Identity (IT identity team)
  - Which identity provider, and does it support OIDC or SAML 2.0?
  - Will any Microsoft 365 data be connected later?
  - Who approves a new OAuth app, and how long does it take?
Network (network and platform teams)
  - Is there a landing zone and a Shared VPC we must use?
  - Must users reach the app privately, over VPN or Interconnect?
  - Is there a VPC Service Controls perimeter, and who changes it?
  - How do services in the perimeter reach the internet, if at all?
Compliance (privacy, legal, security)
  - Does this need a DPIA? Who decides?
  - Are customer-managed keys required? Who holds them?
  - How long may conversations be kept?
Operations (platform team)
  - Who owns the system after handover, and who is on call?
  - How do changes reach production here?

What goes on a design review checklist?

Before the formal security review, I run my own design review against this checklist. It is short enough to read in a meeting:

[ ] Every data source has an owner, a location and a sensitivity
[ ] Processing location agreed, not only storage location
[ ] Identity provider chosen and written up as a decision record
[ ] Each data source states its identity: agent or signed-in user
[ ] No credential ever reaches the model's context
[ ] Every network path drawn, with its owner
[ ] Perimeter tested in dry run; egress path agreed if needed
[ ] Encryption keys set up before apps and connectors are created
[ ] DPIA decision recorded by the privacy team
[ ] Retention set for chat history and memory
[ ] Audit logs routed to the customer's monitoring
[ ] Owner after handover named, with on-call and change process

Try it: check a design for gaps

This script checks a design description against the required decisions and lists anything missing or still open. It only needs Python 3. Save it as check_design.py:

REQUIRED = {
    "data": ["location_at_rest", "processing_location", "owner", "sensitivity", "read_as"],
    "identity": ["idp", "protocol", "user_identity_to_data", "agent_identity"],
    "network": ["user_path", "agent_egress", "perimeter", "dry_run_done"],
    "compliance": ["dpia_needed", "encryption", "retention", "audit_logs"],
    "operations": ["owner_after_handover", "on_call", "change_process"],
}

design = {
    "data": {"location_at_rest": "EU", "processing_location": "unknown",
             "owner": "supply data team", "sensitivity": "confidential",
             "read_as": "signed-in user"},
    "identity": {"idp": "corporate IdP", "protocol": "OIDC",
                 "user_identity_to_data": "OAuth, per user", "agent_identity": "Agent Identity"},
    "network": {"user_path": "PSC endpoint", "agent_egress": "PSC interface",
                "perimeter": "VPC Service Controls", "dry_run_done": False},
    "compliance": {"dpia_needed": "ask privacy team", "encryption": "CMEK"},
    "operations": {"owner_after_handover": "data platform team"},
}

UNSURE = {"unknown", "ask privacy team", "tbc", ""}
gaps = 0
for area, keys in REQUIRED.items():
    for key in keys:
        value = design.get(area, {}).get(key)
        if value is None:
            print(f"MISSING  {area}.{key}")
        elif value is False or str(value).lower() in UNSURE:
            print(f"OPEN     {area}.{key} = {value!r}")
        else:
            continue
        gaps += 1
print(f"{gaps} items to close before the design review")

Run python3 check_design.py. When I ran it, the output was:

OPEN     data.processing_location = 'unknown'
OPEN     network.dry_run_done = False
OPEN     compliance.dpia_needed = 'ask privacy team'
MISSING  compliance.retention
MISSING  compliance.audit_logs
MISSING  operations.on_call
MISSING  operations.change_process
7 items to close before the design review

Then try these changes:

Fill in the design dictionary for a system you know. The open items are your week-one questions.
Add an owner for each area, and print who to ask next to each gap. The list becomes a plan.
Set processing_location to "global" and ask yourself whether the customer’s policy allows it. The script cannot answer that; the data owner can.

What can go wrong

Designing for the estate you expected. You assume a public endpoint and find a perimeter, or assume Google accounts and find a third-party identity provider. Ask before you draw.
Asking for changes too late. A new OAuth app, a firewall rule or a perimeter change can each take weeks. Ask in week one, even before the design is final.
Storage location without processing location. The data stays in the EU, but the model call goes elsewhere. Agree both.
Enforcing a perimeter without a dry run. Users lose the interface, data store creation stops, and the agent loses its route out. Dry run first and read the logs.
Changing the identity provider after launch. Users lose their chat history. Decide once, and record the decision.
Treating the security review as a gate at the end. It works far better as a conversation that starts in week one.

What I would do now

Draw the estate map for one customer or internal system, and write an owner in every box.
Fill in the data map for one use case, including processing location and chat history.
Write one decision record, for the identity provider, with at least two real options.
Trace every network path and mark which ones cross a perimeter.
Send the week-one questions to the owners before the design meeting, not during it.

What did we learn?

An estate is everything the customer already runs, and every box in it has an owner whose approval comes with its own lead time.
A data map records, for each source, where it is stored and processed, who owns it, how sensitive it is and which identity reads it.
Identity has two halves: how users sign in through the identity provider, and how the agents carry the user’s identity to the data without the model ever seeing a token.
Every network path needs an owner, and a VPC Service Controls perimeter should run in dry run mode before it is enforced.
Compliance boundaries nest from law and contracts down to the agent itself, and each ring has a different person who signs it off.
A security review goes faster as a conversation that starts in week one, with the design pack sent before anyone asks for it.

What comes next?

The design is agreed and the security review has its pack. In Part 4, Data readiness and RAG from zero, we turn to the data itself: how documents become answers through parsing, chunking, embeddings and vector search, and why answers go wrong.

Sources

Opinions are my own. Every fact in this post comes from the public documentation linked above. Acme Consumer Brands and its data are fictional.