2026-01-17

10 Python Lines I Trust in Production (And the Ones I Don’t)

Python, Reliability, Security, Observability · Dorian Sotpyrc

Most production incidents aren’t dramatic crashes — they’re quiet failures: requests that hang forever, exceptions that lose their cause, “helpful” defaults that silently change behavior, and logs that don’t preserve the crime scene.

Below are 10 Python one-liners I trust in production because they bound time, bound blast radius, and leave evidence. Each one is paired with the deceptively “fine in a tutorial” version I don’t trust.

A production-minded checklist: timeouts, explicit errors, safe defaults, and observability.
Production reliability is mostly about preventing quiet failures: infinite waits, erased stack traces, and “helpful” defaults that become outages.
TL;DR

In production, the worst bugs don’t crash. They hang, swallow evidence, or quietly choose unsafe defaults. These are 10 Python lines I trust because they bound time, bound blast radius, and leave a trail.

  • Every I/O call has a timeout (network and subprocess).
  • Exceptions are handled with context or re-raised with the original attached.
  • Config and input validation fail at startup (no silent fallbacks for secrets).
  • Logging preserves the “crime scene” without leaking secrets.
  • Paths and temp files use safe primitives (pathlib, tempfile).

The rule: production lines must fail loud, fast, and explain why

What “quiet failure” looks like

Quiet failures are the ones that don’t page you immediately: a request that hangs forever, an exception that gets swallowed and returns “None”, a retry loop that DDOSes your dependency, or a default config value that accidentally points production at a dev system.

The 2am debugging test

If you’re woken up at 2am, can you answer: What did we try? How long did we wait? What failed? What input triggered it? If the code doesn’t leave that evidence by default, I don’t trust it.

  • Bound time: timeouts for anything that can block.
  • Bound blast radius: limits and backoff to avoid cascading failures.
  • Leave evidence: structured logs, stack traces, stable error shapes.

1–3: Networking lines I trust (and the timeouts I refuse to skip)

1) HTTP requests with explicit timeouts

Trust: bounded connect+read time. I treat connect and read separately so “can’t reach host” and “server slow” don’t look identical.

PYTHON
r = requests.get(url, timeout=(3.05, 10))

Don’t trust: the same call without a timeout. “No timeout” is functionally “infinite hang”, and your worker pool will quietly drain until everything is stuck.

PYTHON
r = requests.get(url)  # can hang forever
Timeouts prevent quiet hangs
Bounded (timeout) Hang risk (no timeout)
0s 5s 10s 15s 0 1 2 3 4 5 min Latency Time timeout bounded hang risk

2) Surface HTTP errors immediately

Trust: fail fast on non-2xx. It forces the caller to decide what “error handling” means instead of quietly parsing an error page as JSON.

PYTHON
r.raise_for_status()

Don’t trust: assuming success and doing a best-effort parse. This is how you end up with KeyError in random places (or worse: silently wrong data).

PYTHON
data = r.json()  # may be an HTML error page, not JSON

3) Retries with backoff (only when safe)

Trust: backoff that slows down under pressure. Retries are not “reliability”; retries are load multipliers unless they’re deliberate.

PYTHON
sleep_s = min(2 ** attempt, 30)

Don’t trust: tight loops that hammer a dependency. If the upstream is degraded, this makes it worse and hides the root cause behind “random flakiness”.

PYTHON
while True: do_request()  # no backoff, no cap, no stop condition

Production note: don’t retry non-idempotent operations unless you have an idempotency key or another correctness guarantee.

4–6: Exceptions and logging that don’t erase the crime scene

4) Catch specific exceptions, not everything

Trust: narrow exception handling. It’s honest about what can fail and avoids accidentally catching programmer errors (like AttributeError) and continuing in a corrupt state.

PYTHON
except (TimeoutError, OSError) as e:

Don’t trust: bare except. It turns “bug” into “mystery behavior” and makes outages longer.

PYTHON
except:  # catches too much, including KeyboardInterrupt in some cases
    pass

5) Log exceptions with stack traces (and context)

Trust: logger.exception(...) inside an except block. You get a stack trace by default, which is usually the difference between “5 minutes” and “5 hours”.

PYTHON
logger.exception("upstream call failed", extra={"request_id": request_id})

Don’t trust: print(e) (or worse, nothing). Prints get dropped, reordered, or lost in container logging pipelines — and they rarely include the stack trace you’ll need.

PYTHON
print(e)  # no stack trace, often no context

6) Re-raise with the original error attached

Trust: explicit chaining. It preserves the original exception while providing a stable, higher-level message.

PYTHON
raise RuntimeError(f"payment provider failed: {provider}") from e

Don’t trust: raising a new exception without chaining (or returning None). That deletes the cause and forces you to reproduce the issue under pressure.

PYTHON
raise RuntimeError("payment provider failed")  # original cause lost

7–8: Input and config validation that prevents mystery states

7) Validate shapes and ranges, not just “it exists”

Trust: defensive parsing that fails early with a clear error. When inputs are untrusted (HTTP, queues, files), ambiguity becomes data corruption.

PYTHON
user_id = int(payload["user_id"])  # raises loudly if missing/invalid

Don’t trust: “best effort” conversions that turn bad input into a default. It hides upstream problems and can create surprising security behavior (“guest mode” by accident).

PYTHON
user_id = int(payload.get("user_id", 0))  # silent fallback changes meaning

8) Environment variables: fail closed for secrets

Trust: os.environ[...] for required config. If a secret is missing, the service should fail at startup, not limp into production half-configured.

PYTHON
DATABASE_URL = os.environ["DATABASE_URL"]

Don’t trust: defaulting secrets or critical endpoints. The “helpful” default becomes a breach, a data leak, or a split-brain incident.

PYTHON
DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///dev.db")

9: Filesystem and temp paths — predictable or it didn’t happen

Paths via pathlib, temp via tempfile

Trust: composition that works across platforms and avoids accidental path traversal from string concatenation.

PYTHON
path = Path(base_dir) / filename

Don’t trust: string-joined paths. It’s easy to get double slashes, missing separators, or accidentally allow ../../ behavior to slip into “it works on my machine”.

PYTHON
path = base_dir + "/" + filename

Trust: unique, race-resistant scratch files managed by the OS.

PYTHON
tmp = tempfile.NamedTemporaryFile(delete=True)

Don’t trust: shared temp filenames. Under concurrency, you’ll get collisions, partial writes, or (worst case) security issues.

PYTHON
open("/tmp/output.txt", "w")  # collision-prone in production

10: Running commands safely (subprocess without foot-guns)

subprocess.run with checks, timeout, and no shell

Trust: explicit argv list, fail on non-zero exit, and a timeout. This turns “mysterious stuck worker” into a bounded, debuggable error.

PYTHON
subprocess.run(["git", "rev-parse", "HEAD"], check=True, timeout=5)

Don’t trust: shell=True with string interpolation. It’s fragile (quoting breaks) and can become an injection bug the moment any part of the command becomes user-influenced.

PYTHON
subprocess.run(f"git show {ref}", shell=True)  # injection risk, harder to reason about

A quick production checklist (what I scan for in code review)

If I’m reviewing a PR and I don’t have time to deeply reason about every edge case, I scan for these “quiet failure” triggers:

  • Timeouts everywhere: HTTP, DB, queues, subprocess. No exceptions.
  • Retries are deliberate: capped attempts + backoff; only for idempotent actions.
  • No bare except: catch the specific failure modes you can handle; otherwise fail loud.
  • Evidence is preserved: logger.exception in handlers; exceptions chained with raise ... from e.
  • Config fails closed: required env vars accessed via os.environ[...]; no silent defaults for secrets.
  • Secrets don’t leak: log messages avoid tokens/credentials; follow redaction practices.
  • Paths are safe: pathlib joins; temp files via tempfile.
  • Subprocess is bounded: argv list, check=True, timeout=, and default shell=False.

Related PLEX reading

References & further reading

  • Python docs: Exception context (exception chaining with raise ... from ...)
  • Python docs: logging (including Logger.exception)
  • Python docs: subprocess (timeouts, check=True, and shell)
  • Python docs: tempfile (safe temporary files)
  • Python docs: pathlib (portable path handling)
  • Requests docs: Timeouts