Deploy Cloud Apps
From a Working Script to a Running Service
Everything in the cloud section so far has been about making a single request work on a platform. A deployed application is different. It runs unattended on machines that start and stop, it serves many users, its configuration comes from the environment it is deployed into, and its failures must be visible to people who did not write it. A configuration that works on a developer's laptop can fail on the first deployment for reasons that have nothing to do with the code: a missing variable, a role without permission, a region without the model, or a quota shared by every replica.
This lesson covers the decisions that turn a working integration into a reliable service. It explains where configuration should come from, how the application should check itself at startup, how quotas behave when several copies of an application share them, and how the platform's identity and logging services fit in. The example at the end is a startup check that refuses to run a service that is configured unsafely.
Configuration From the Environment
A deployed application should read its settings from the environment, not from files that are built into the image. The same container image then runs in development, staging, and production, with different settings supplied at deployment time. The provider, the region, the timeouts, and the quota expectations all belong in that configuration, while the secrets come from the platform's secret store, as Lesson 164 described.
The rule that follows is simple: nothing that differs between environments should be compiled into the program. An image that contains a production region is an image that cannot be safely reused in staging.
Identity Attached to the Workload
A deployed workload should run as an identity granted only the permissions it needs. On AWS, that is a role attached to the service. On Google Cloud, a service account attached to the workload. On Azure, a managed identity. The identity is what lets the cloud client make model calls without any stored secret, and it is the point where least privilege is enforced (Lesson 176 covers the broader principle).
A common mistake is to run a service as an administrator's identity, because it is the one that worked during development. The fix is to create a dedicated identity, grant only the model invocation permission, and test the deployment with that identity, not with the developer's.
Checking Itself at Startup
A service should verify its configuration when it starts, and refuse to run if the configuration is unsafe. A failure at startup is visible immediately, to the deployment system and to the person watching it. A failure on the first user request is visible only to that user, and only after damage. The startup check collects two kinds of finding: errors, which stop the service, and warnings, which let it run but must be seen.
"""startup_checks.py: refuse to start a service with unsafe configuration; warn about risky settings."""
KNOWN_PROVIDERS = {"direct", "bedrock", "vertex", "foundry"}
def startup_checks(config: dict) -> tuple[list[str], list[str]]:
errors: list[str] = []
warnings: list[str] = []
provider = config.get("provider")
if provider not in KNOWN_PROVIDERS:
errors.append(f"unknown provider {provider}")
elif provider in {"bedrock", "vertex"} and not config.get("region"):
errors.append(f"{provider} needs a region")
if config.get("literal_api_key"):
errors.append("a literal API key is set in configuration; use the secret store")
planned, quota = config.get("planned_rpm"), config.get("quota_rpm")
if planned and quota and planned > 0.8 * quota:
warnings.append(f"planned {planned} requests per minute is over 80% of quota {quota}")
return errors, warnings
if __name__ == "__main__":
cases = [
{"provider": "bedrock", "region": "us-east-1", "planned_rpm": 120, "quota_rpm": 500},
{"provider": "vertex"},
{"provider": "direct", "planned_rpm": 900, "quota_rpm": 1000},
]
for number, config in enumerate(cases, start=1):
errors, warnings = startup_checks(config)
print(f"case {number}: errors={errors} warnings={warnings}")Output:
case 1: errors=[] warnings=[]
case 2: errors=['vertex needs a region'] warnings=[]
case 3: errors=[] warnings=['planned 900 requests per minute is over 80% of quota 1000']How It Works
- Unknown providers stop the service. A typo in the provider name is caught here, rather than failing on the first request with a message that does not mention configuration.
- Regional providers require a region. Case two has none, and the check names the missing value.
- A literal key is an error, not a warning. A secret in configuration is a leak in waiting, and the service must not run with one.
- The quota check is a warning. A service planned to use over eighty percent of its quota runs, but the team must see the warning before the first peak arrives.
- The checks return both lists, so the startup code can exit on errors and log the warnings, and the same function serves tests.
Quotas Are Shared
The quota check reflects a fact that is easy to overlook. A quota belongs to the account or the region, and every replica of a service draws from the same pool. Three replicas each planning two hundred requests per minute are planning six hundred in total, and that total is what the quota must cover. Autoscaling makes the total grow with load, which can exhaust the quota exactly when traffic is highest. The planned figure in the configuration should therefore be the total across all replicas, and the rate limiting from Lesson 142 should be shared across them, or the quota will be exceeded in production.
Logging and Monitoring
A deployed service should write its logs to the platform's logging service, in the structured form from Lesson 147, so that errors can be searched and alerted on. The platform's monitoring should track the rates that matter for this application: request volume, error rate by class, latency, and usage against quota. Lesson 186 returns to cost tracking in detail, and the same signals serve reliability and cost together.
Each log line should carry the platform and the region, so that an incident in one region is not confused with a problem elsewhere.
Packaging and Replacement
A container image should contain code and dependencies, and nothing else. Configuration, secrets, and identities are supplied when the container starts. Deployments should replace containers rather than change them in place, so that a deployment can be rolled back to the previous image in one step. The health of a new deployment should be checked before traffic moves to it, and the startup checks above are a natural part of that health check.
Summary
A deployed application reads its configuration from the environment, runs as a dedicated identity with the least permissions it needs, and checks its configuration at startup, stopping on errors and reporting warnings. Quotas are shared across replicas, so planning must use the total. Logs go to the platform in structured form, labeled with the platform and region. Images contain code only, and deployments are replaced and rolled back as units. This completes the cloud section. The next section covers security, beginning with how API keys are protected.