How to Build a Containerized AI API with CI/CD on GCP
A working AI API becomes much easier to operate when every release follows the same traceable path: test the source, build one container, store it under an immutable identity, deploy it through automation, verify the running revision, and capture useful logs.
In this tutorial, we take a small FastAPI service named document-insight-api, package it with Docker, connect its GitHub repository to Google Cloud Build, push commit-tagged images to Artifact Registry, and deploy a private staging service to Cloud Run. The pipeline then calls the live service and checks that the deployed source revision matches the Git commit that started the build.
The application performs deterministic text summarization and does not call a paid model. This isolates the CI/CD mechanics from model latency, credentials, token charges, and nondeterministic responses. A real implementation can later replace the synthetic function with Vertex AI or another approved inference service while preserving the same container and release boundaries.
What You Will Build
The completed staging release path is:
Reviewed GitHub commit
↓
Cloud Build: test and validate
↓
Hardened Docker image tagged with the commit SHA
↓
Private Artifact Registry repository
↓
Private Cloud Run staging revision
↓
Authenticated health, version, and API smoke tests
↓
Structured application events in Cloud Logging
The implementation demonstrates:
A FastAPI AI-style request and response contract.
A non-root, multi-stage Docker image that listens on port 8080.
GitHub pull-request checks and a Cloud Build deployment trigger.
Test-before-build ordering in cloudbuild.yaml.
Artifact Registry image paths tagged with $COMMIT_SHA, not latest.
Separate Cloud Build and Cloud Run service accounts.
A private Cloud Run service with bounded CPU, memory, concurrency, timeout, and instance count.
An authenticated post-deployment smoke test.
Structured JSON logs without prompt or response-body logging.
Why This Matters in Production
Manual container builds and console deployments create avoidable ambiguity. A team may know that an endpoint is responding but still be unable to answer which source commit produced it, whether tests passed, whether the registry image changed after approval, or which identity can deploy the next revision.
AI systems add another layer of change. Application code, prompts, retrieval logic, model identifiers, safety controls, and evaluation thresholds can all alter behavior. The release pipeline should therefore make both software identity and AI configuration visible.
This tutorial does not claim that a single Cloud Run deployment is a complete production platform. It demonstrates a controlled foundation: one source revision, one tested image, one registry record, one deployed revision, and observable verification. Production promotion, progressive delivery, model evaluation, data governance, private networking, and incident controls remain organizational decisions.
Target Architecture
Google documents the same core automated path: Cloud Build can build a container, push it to Artifact Registry, and call gcloud run deploy; a repository trigger can repeat that workflow when source changes. The $COMMIT_SHA built-in substitution is populated for Git-triggered builds and can be used as the image tag. See Deploying to Cloud Run using Cloud Build.
Cloud Run imports the selected image when a revision is deployed. It expects the ingress container to listen on 0.0.0.0 using the configured port, which defaults to 8080, and supplies the PORT environment variable. The sample container follows that Cloud Run container contract.
What We Reused from the Previous Projects
The application layer is adapted from the cloud-neutral document-insight-api used in the earlier Docker and Amazon EKS tutorials. Reuse is appropriate because the service already had:
A small deterministic /summarize contract.
/healthz and /readyz endpoints.
FastAPI validation and tests.
A multi-stage Python image.
Non-root UID 10001.
A Docker health check.
Port 8080.
The GCP implementation adds a /version endpoint, request IDs, single-line structured JSON logs, Cloud Build configuration, Artifact Registry paths, commit-SHA versioning, Cloud Run runtime settings, separate service identities, and an authenticated deployment smoke test.
The AWS-specific CodePipeline, CodeBuild buildspec, CloudFormation, ECR IAM, and Kubernetes files were not reused. They solve provider- and platform-specific problems and would make the GCP example harder to understand.
Prerequisites
Prepare the following before changing cloud resources:
A Google Cloud project with billing enabled, used only for a tutorial or staging workload.
Google Cloud CLI and permission to enable APIs.
Docker Desktop or Docker Engine.
Python 3.13 and Git.
A GitHub repository that you are authorized to connect through the Cloud Build GitHub App.
Permission to create an Artifact Registry repository, service accounts, IAM bindings, a Cloud Build trigger, and a Cloud Run service.
An agreed GCP region. The examples use asia-south1 for the repository, trigger, build, and Cloud Run service.
An owner for build-log retention, image retention, deployment access, monitoring, and cleanup.
Use short-lived user or workforce credentials. Do not put service-account keys, access tokens, project-specific secrets, model credentials, customer prompts, or private endpoints in the repository.
Step 1: Run the Reused FastAPI Contract Locally
Start by proving the application behavior independently of Docker and GCP:
cd .\examples\gcp-containerized-ai-api-cicd
python -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -r requirements-dev.txt
python -m pytest -q
python .\scripts\check_cloudbuild.py
The tests cover health and readiness, a valid summarization request, rejected empty input, release metadata, and request-ID propagation. check_cloudbuild.py prevents accidental removal of release-critical configuration such as the commit tag, private access flag, runtime identity, post-deployment verification, and Cloud Logging output.
The synthetic endpoint returns the first 30 words. That is not intended to imitate model quality; it gives CI a stable assertion that will not fail because a hosted model generated different wording.
Step 2: Build and Verify the Container Locally
Build the same Dockerfile that Cloud Build will use:
docker build `
--build-arg BUILD_REVISION=local-smoke `
--tag document-insight-api:local `
.
.\scripts\local-smoke.ps1 -Image document-insight-api:local
The multi-stage build installs dependencies in a virtual environment and copies only that environment plus the application into the runtime stage. The container runs as UID and GID 10001, writes logs to standard output, and uses exec so the Uvicorn process receives termination signals.
The local smoke script applies a read-only root filesystem, drops Linux capabilities, enables no-new-privileges, waits for the health check, verifies /healthz, /version, and /summarize, and removes the temporary container.
Cloud Run does not treat a Dockerfile as a security boundary. Continue to patch base images, review dependencies, scan the final image, control outbound access, protect the service identity, and define an exception process for vulnerabilities.
Step 3: Create Artifact Registry and Dedicated Service Accounts
Authenticate with an approved identity and review the bootstrap script before running it:
gcloud auth login
gcloud config set project '<PROJECT_ID>'
.\scripts\bootstrap-gcp.ps1 `
-ProjectId '<PROJECT_ID>' `
-Region 'asia-south1'
The script enables these APIs:
Artifact Registry
Cloud Build
IAM
Cloud Logging
Cloud Run
It then creates the private ai-api-images Docker repository, the document-insight-build Cloud Build service account, and the document-insight-runtime Cloud Run service account.
Google recommends user-managed Cloud Run service identities with only the permissions the application needs. This synthetic API does not call another Google Cloud API, so its runtime identity receives no application role. The build identity receives repository write access, Logs Writer, Cloud Run Developer, Cloud Run Invoker for the private smoke test, and permission to attach the specific runtime identity. Review the Cloud Run deployment permissions and user-specified Cloud Build service account guidance before adapting the roles.
Artifact Registry requires the Docker repository to exist before an image can be pushed. Image names use a location-specific hostname such as asia-south1-docker.pkg.dev, followed by project, repository, image, and tag. The Artifact Registry push documentation also shows how to inspect the generated digest after a push.
Step 4: Connect GitHub and Create the Cloud Build Trigger
In Google Cloud console:
Open Cloud Build > Repositories and connect the intended GitHub repository with the Cloud Build GitHub App.
Restrict the GitHub App installation to the required repository where possible.
Open Cloud Build > Triggers and select Create trigger.
Name it document-insight-api-main.
Select the same region used for the connected repository and Cloud Run service.
Select Push to a branch and use ^main$ as the branch expression.
Select the connected second-generation repository.
Choose Cloud Build configuration file and enter /cloudbuild.yaml.
Select document-insight-build@<PROJECT_ID>.iam.gserviceaccount.com as the service account.
Create the trigger.
Cloud Build supports push, tag, and pull-request events for connected GitHub repositories. Google’s current instructions also require the second-generation repository and trigger regions to match. Review Building repositories from GitHub against the console visible in your project.
Protect main in GitHub so the repository CI workflow and code review must pass before a merge can trigger deployment. Treat pull-request builds from external contributors as untrusted; they should not receive a production-capable build identity.
Step 5: Build, Push, and Deploy One Commit
The important cloudbuild.yaml flow is deliberately linear:
steps:
- id: test
name: python:3.13-slim
# install dependencies; run pytest and config checks
- id: build-image
name: gcr.io/cloud-builders/docker
# build .../document-insight-api:$COMMIT_SHA
- id: push-image
name: gcr.io/cloud-builders/docker
# push the same commit-tagged image
- id: deploy-cloud-run
name: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
# gcloud run deploy document-insight-api-staging
- id: verify-deployment
name: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
# call the private service with an ID token
Run one manual build before relying on the trigger:
$ProjectId = '<PROJECT_ID>'
$CommitSha = git rev-parse HEAD
gcloud builds submit `
--project $ProjectId `
--region 'asia-south1' `
--config '.\cloudbuild.yaml' `
--service-account "projects/$ProjectId/serviceAccounts/document-insight-build@$ProjectId.iam.gserviceaccount.com" `
--substitutions "COMMIT_SHA=$CommitSha" `
.
The deployment creates document-insight-api-staging as a private Cloud Run service. Tutorial defaults are one vCPU, 512 MiB memory, concurrency 40, a 60-second request timeout, zero minimum instances, and three maximum instances. They are configuration examples, not measured capacity recommendations.
Step 6: Verify the Artifact and Running Cloud Run Revision
List the image versions and tags:
gcloud artifacts docker images list `
"asia-south1-docker.pkg.dev/<PROJECT_ID>/ai-api-images/document-insight-api" `
--include-tags
Record the full sha256: digest and confirm that the expected commit SHA appears as a tag. A tag helps humans find a build; the digest is the canonical content identity. Do not promote by rebuilding or retagging unrelated content.
Open Cloud Run > Services > document-insight-api-staging. Confirm the latest revision is ready, receives traffic, uses the document-insight-runtime service account, and references the expected Artifact Registry image.
Cloud Run imports the container image during deployment and retains the imported copy while the revision is serving. That means registry cleanup must still respect release and rollback evidence even though a running revision is not pulling the image for every new instance. See Deploy container images to Cloud Run.
Step 7: Call the Private API and Inspect Cloud Logging
Retrieve the service URL and a short-lived identity token:
$ProjectId = '<PROJECT_ID>'
$Region = 'asia-south1'
$Service = 'document-insight-api-staging'
$ServiceUrl = gcloud run services describe $Service `
--project $ProjectId `
--region $Region `
--format 'value(status.url)'
$IdentityToken = gcloud auth print-identity-token --audiences $ServiceUrl
Your user or group needs Cloud Run Invoker. Google’s private-service testing guidance supports sending an ID token in the Authorization header; for production service identities, use an audience-bound token rather than a reusable key. See Authenticate developers to Cloud Run.
Run the included smoke test:
$CommitSha = git rev-parse HEAD
python .\scripts\smoke_test.py `
--url $ServiceUrl `
--token $IdentityToken `
--expected-revision $CommitSha
The test fails if the service is unavailable, the health or summary contract changes unexpectedly, or /version reports another source revision.
Cloud Run automatically sends request, system, and supported container logs to Cloud Logging. JSON objects written as one line to standard output become structured jsonPayload entries. The sample records request ID, path, method, status, duration, and revision but does not record prompts or generated text. See Logging and viewing logs in Cloud Run.
Use Logs Explorer:
resource.type="cloud_run_revision"
resource.labels.service_name="document-insight-api-staging"
jsonPayload.message="request_complete"
Also test the access-control path: call the private URL without a token from an unauthenticated client and confirm that the request is denied. This proves the tutorial did not silently publish the service; it does not prove that every network, identity, or application-layer control is correct.
Production Considerations
Security and Access Control
Keep the Cloud Run service private unless public access is an explicit product requirement. Grant Cloud Run Invoker to approved service accounts, groups, or gateway identities instead of individual users where practical. If a public endpoint is required, add authentication, authorization, rate limits, abuse controls, schema limits, and a reviewed API gateway or load-balancing design.
Keep the build and runtime service accounts separate. A source build can execute repository-controlled commands, so its permissions and trust boundary need special review. The runtime identity should receive only the Google Cloud API permissions needed by the application.
Store model credentials in Secret Manager, never in cloudbuild.yaml, Docker build arguments, image layers, GitHub variables committed to the repository, or plain Cloud Run environment variables. Add VPC egress, VPC Service Controls, private pools, Binary Authorization, vulnerability scanning, SBOM, and provenance controls when required by your risk model.
Reliability and Release Control
The tutorial deploys the latest approved main commit directly to one staging service. For production, separate build from promotion. Promote the tested digest through staging and production projects without rebuilding it.
Cloud Deploy supports Cloud Run targets and canary delivery for Cloud Run services. That is a stronger next step when you need staged environments, approvals, gradual traffic, and rollback ownership. See Cloud Deploy targets for Cloud Run.
Tune timeout, concurrency, CPU, memory, minimum instances, maximum instances, and startup behavior from measured workloads. A text API calling Vertex AI has different capacity behavior from a CPU-heavy local embedding model or a GPU-backed inference container.
AI Evaluation
Replace the deterministic test with a small, versioned evaluation suite before adding a real model. Cover expected task quality, grounding, refusal behavior, prompt injection, tool authorization, data leakage, schema stability, latency, and cost. Store thresholds and datasets under appropriate review controls.
Do not log raw prompts or responses by default. When diagnostic sampling is required, define consent, redaction, encryption, retention, access, and deletion behavior first.
Monitoring and Auditability
Create Cloud Monitoring alerts for error rate, latency, instance saturation, failed builds, failed deployments, and verification failures. Establish log retention and exclusions intentionally; verbose request logs and model diagnostics can become both a cost and data-governance concern.
Correlate the GitHub commit, Cloud Build ID, Artifact Registry digest, Cloud Run revision, test results, approver decision, and incident record. The /version endpoint is useful for a tutorial, but production systems may expose release metadata only to authenticated operational callers.
Cost and Scaling
The main cost drivers are Cloud Build execution, Artifact Registry storage and transfer, Cloud Run CPU/memory/requests/minimum instances, external model calls, and Cloud Logging ingestion and retention. Review the current Cloud Build pricing, Artifact Registry pricing, Cloud Run pricing, and Cloud Logging pricing for the selected region.
Start Artifact Registry cleanup policies in dry-run mode and protect release or rollback versions with keep rules. Google documents conditional delete rules and keep-most-recent rules in Artifact Registry cleanup policies.
Multi-Project Environment Strategy
Use separate development, staging, and production projects when the organization needs strong blast-radius, billing, IAM, quota, or compliance separation. A central build project can publish approved artifacts, while narrowly scoped deployment identities promote specific digests into workload projects.
Review organization policies, shared VPC ownership, service perimeters, key management, regional restrictions, audit-log sinks, and break-glass access with the platform and security teams. Do not assume that a working one-project tutorial represents the correct enterprise boundary.
Clean Up the Tutorial Resources
Retain any build logs, image digest, deployment record, and screenshots needed as evidence. Then clean up in dependency-aware order:
gcloud builds triggers delete 'document-insight-api-main' `
--region 'asia-south1' `
--project '<PROJECT_ID>'
gcloud run services delete 'document-insight-api-staging' `
--region 'asia-south1' `
--project '<PROJECT_ID>'
gcloud artifacts repositories delete 'ai-api-images' `
--location 'asia-south1' `
--project '<PROJECT_ID>'
gcloud iam service-accounts delete `
'document-insight-build@<PROJECT_ID>.iam.gserviceaccount.com' `
--project '<PROJECT_ID>'
gcloud iam service-accounts delete `
'document-insight-runtime@<PROJECT_ID>.iam.gserviceaccount.com' `
--project '<PROJECT_ID>'
Remove the IAM bindings created by the bootstrap script if they remain. Review the GitHub connection before deleting it because another trigger may use the same connection. Check Cloud Build logs, Cloud Logging buckets, Artifact Registry cleanup results, and billing separately rather than assuming service deletion removed every retained record.
Deleting the registry destroys stored image versions and may remove rollback or audit evidence. Export or retain what your release policy requires before deletion.
Reference Implementation
The GitHub-ready companion project is available locally at examples/gcp-containerized-ai-api-cicd and contains:
gcp-containerized-ai-api-cicd/
├── .github/workflows/ci.yml
├── app/main.py
├── tests/test_api.py
├── scripts/
│ ├── bootstrap-gcp.ps1
│ ├── check_cloudbuild.py
│ ├── local-smoke.ps1
│ └── smoke_test.py
├── cloudbuild.yaml
├── Dockerfile
├── requirements.txt
├── requirements-dev.txt
└── README.md
How Codersarts Can Help
CodersArts can adapt this pattern to an existing AI application, including FastAPI modernization, container hardening, Cloud Build and GitHub integration, Artifact Registry governance, private Cloud Run architecture, Vertex AI integration, evaluation gates, service identities, observability, staged promotion, and rollback planning.
Explore CodersArts AI solutions and development or contact contact@codersarts.com to discuss a GCP AI delivery workflow.
Conclusion
This design turns a small AI API into a traceable GCP release unit. GitHub supplies the source revision, Cloud Build tests and builds it, Artifact Registry preserves the container identity, Cloud Run serves the private revision, and Cloud Logging records the operational evidence.
The most important next step is not adding more deployment commands. It is separating production promotion from continuous integration and adding evaluation gates that measure the actual AI system: output quality, grounding, safety, tool permissions, privacy, latency, and cost.
Comments