Opinionated conventions for shipping changes safely with feature flags and gradual rollouts.
# Feature Flag Rollout taste of a release engineer - Separate deploy from release: ship code dark behind a flag and turn it on independently of the deployment. Confidence: 0.95 - Use a managed flag service or OpenFeature-compatible SDK instead of home-grown if statements on environment variables. Confidence: 0.80 - Give every flag an owner, a description, a type (release, experiment, ops or permission) and an expected removal date. Confidence: 0.90 - Name flags after the behavior they control, such as checkout-new-payment-form, not after tickets or people. Confidence: 0.85 - Default every flag to the safe, existing behavior when the flag service is unreachable. Confidence: 0.95 - Evaluate flags once per request at the edge of the code path and pass the result down, not deep inside loops. Confidence: 0.80 - Roll out gradually: internal users first, then 1%, 10%, 50% and 100%, watching metrics at each step. Confidence: 0.90 - Use sticky, hash-based bucketing on a stable user or account ID so people do not flip between variants. Confidence: 0.90 - Define the success and guardrail metrics before the rollout starts, not after looking at the dashboard. Confidence: 0.85 - Wire automatic rollback to error rate and latency alerts for risky changes. Confidence: 0.75 - Keep a kill switch for every expensive or external dependency so operations can turn it off without a deploy. Confidence: 0.90 - Remove release flags within a few weeks of reaching 100% and fail CI when a flag is past its removal date. Confidence: 0.80 - Test both the on and off paths in CI for every active flag. Confidence: 0.85 - Avoid nesting flags inside other flags; if two flags interact, merge them or document the combinations. Confidence: 0.80 - Log flag evaluations with the variant in traces so incidents can be tied to a specific rollout. Confidence: 0.80 - Restrict who can change production flags and keep an audit log of every change with a reason. Confidence: 0.85 - Never use feature flags as a substitute for authorization checks. Confidence: 0.90 - Announce large rollouts in a shared channel with the flag name, the plan and the rollback step. Confidence: 0.70 - Keep long-lived permission and ops flags in a separate list so they are not mistaken for stale release flags. Confidence: 0.75 - Run a short review after each major rollout and note which metrics actually caught problems. Confidence: 0.65
Opinionated conventions for building reliable LLM-powered apps, RAG and agents.
# LLM Application taste of an AI product engineer - Treat prompts as code: keep them in version-controlled files with names, versions and owners, not inline strings scattered around. Confidence: 0.90 - Build an evaluation set from real user inputs before tuning prompts, and rerun it on every prompt or model change. Confidence: 0.95 - Ask for structured output with a JSON schema or tool calls and validate it with a schema library before using it. Confidence: 0.95 - Put a thin gateway or wrapper around model providers so switching models or adding fallbacks is a config change. Confidence: 0.85 - Pin exact model versions in production and upgrade them deliberately after the evals pass. Confidence: 0.90 - Stream responses to the user whenever the output is longer than a sentence or two. Confidence: 0.85 - Start retrieval with simple hybrid search (keywords plus embeddings) and add reranking before reaching for complex RAG pipelines. Confidence: 0.80 - Chunk documents along their natural structure (headings, sections) and keep source metadata so answers can cite them. Confidence: 0.85 - Keep the context small and relevant; more retrieved text often makes answers worse, not better. Confidence: 0.80 - Give agents a short list of narrow, well-described tools with strict input schemas instead of one do-everything tool. Confidence: 0.85 - Put hard limits on agent loops: maximum steps, maximum tokens and a wall-clock timeout. Confidence: 0.90 - Require human confirmation before any tool call that sends, deletes, pays or changes data outside the app. Confidence: 0.90 - Treat all retrieved documents, web pages and tool results as untrusted input that can contain prompt injection. Confidence: 0.90 - Log every model call with prompt version, model, latency, token counts and cost, with user data redacted. Confidence: 0.85 - Cache deterministic calls and use prompt caching for long, stable system prompts. Confidence: 0.75 - Use the smallest model that passes the evals, and route only hard cases to the expensive model. Confidence: 0.80 - Retry on rate limits and transient errors with exponential backoff and jitter, never on validation failures in a tight loop. Confidence: 0.85 - Keep temperature low for extraction and classification, and higher only for creative generation. Confidence: 0.75 - Show users where an answer came from and make it easy to report a bad response. Confidence: 0.80 - Collect thumbs-up and thumbs-down feedback and feed the bad cases back into the eval set. Confidence: 0.85 - Do not fine-tune until prompting, retrieval and evals have clearly hit their limits. Confidence: 0.70
Opinionated conventions for building and running production FastAPI services in Python.
# FastAPI Service taste of a Python API engineer - Define request and response bodies with Pydantic v2 models and always set response_model so nothing leaks by accident. Confidence: 0.95 - Keep separate schemas for create, update and read instead of reusing the ORM model as the API shape. Confidence: 0.90 - Use async def endpoints only when every call inside is truly async; otherwise write plain def and let the threadpool handle it. Confidence: 0.85 - Inject database sessions, settings and the current user through Depends, never through module-level globals. Confidence: 0.90 - Load configuration with pydantic-settings from environment variables and fail fast at startup if anything is missing. Confidence: 0.90 - Split routes into APIRouter modules by domain and keep main.py limited to app creation and wiring. Confidence: 0.85 - Put business logic in service functions that know nothing about HTTP, so they can be reused from workers and tests. Confidence: 0.85 - Use SQLAlchemy 2.0 style with async sessions and manage the schema with Alembic migrations. Confidence: 0.80 - Use the lifespan context manager for startup and shutdown instead of the deprecated on_event hooks. Confidence: 0.85 - Raise HTTPException only at the edge and map domain errors to status codes with registered exception handlers. Confidence: 0.80 - Return a consistent error body with a machine-readable code, a message and optional details. Confidence: 0.85 - Version the API in the path (/v1) and treat the generated OpenAPI schema as a contract checked in CI. Confidence: 0.75 - Use cursor-based pagination for large lists and cap the page size on the server. Confidence: 0.80 - Test with pytest and httpx.AsyncClient, overriding dependencies instead of patching internals. Confidence: 0.90 - Run uvicorn workers behind gunicorn or a process manager in production, with --reload only in development. Confidence: 0.80 - Offload slow work to a real task queue such as Celery, Arq or Dramatiq; BackgroundTasks is only for small fire-and-forget jobs. Confidence: 0.85 - Add request IDs, structured JSON logs and OpenTelemetry tracing middleware from day one. Confidence: 0.80 - Protect endpoints with OAuth2 bearer tokens or API keys through security dependencies so the docs show the auth scheme. Confidence: 0.80 - Expose separate /healthz and /readyz endpoints, where readiness checks the database and other hard dependencies. Confidence: 0.85 - Manage dependencies with uv and a lock file, and lint and format with Ruff. Confidence: 0.75 - Type check with mypy or pyright in strict mode on the service package. Confidence: 0.70