A proxy comparison should keep every scheduled task in its denominator. If a report drops timeout rows and ranks only requests that completed, it can make a provider look faster while hiding that fewer planned jobs produced an acceptable result by the deadline. Keep a completed-only percentile as a diagnostic, but make the buying decision on a separately stated on-time valid-completion measure.
This guide is for a data team comparing providers for an authorized public-data, regional-check, or ad-verification workload. It uses a reproducible fictional fixture, not IPHTML measurements or a provider ranking. First freeze the task list with the representative trial plan; then preserve the result classifications below.
What a timeout does—and does not—measure
A deadline miss means the job did not produce the defined good result by the job deadline. It does not reveal an unobserved completion time, so do not replace it with the deadline, an arbitrary “tail” number, or a guessed percentile. A socket/client timeout also need not be the whole-job deadline: queueing, connection, response reading and validation can all sit outside a single timer. Requests documents that its timeout is not a time limit for the entire response; a 504 has a different, gateway-specific meaning in HTTP. Preserve the raw observation and its measurement boundary.
| Observed record | What can be counted | What it does not prove | Action |
|---|---|---|---|
| Accepted result at or before deadline | One on-time valid completion; observed elapsed time may enter the completed-only percentile | Long-run provider performance | Keep task ID, acceptance rule and elapsed boundary. |
| Accepted result after deadline | Completed-request diagnostic only, if elapsed and validation are recorded | On-time completion | Count as a deadline miss for the scheduled-task decision. |
| Connection failure | Scheduled denominator and failure category | That the proxy alone caused it | Retain error layer and route evidence; do not invent a duration. |
| Still unfinished at the closed observation cutoff | Known noncompletion and deadline miss | An eventual completion time | Keep the cutoff evidence; never substitute it for a completed duration. |
| Known queue/not started | Scheduled denominator and known missed delivery | Proxy fault or a completion duration | Keep the scheduler evidence; this is not missing telemetry. |
| Missing telemetry | Scheduled denominator and unknown outcome | Whether work completed | HOLD the comparison until evidence is reconciled. |
Keep connection failures distinct from deadline misses. A deadline can be missed after a result arrives too late; a connection failure may occur earlier; a queued job may never start; missing telemetry may mean the outcome is unknown. None should silently disappear. The four timestamps in the latency-breakdown guide help localize observed time; they do not fill in censored tails.
Use two measures, with one decision rule
Before running a trial, write an acceptance contract: the same scheduled tasks, the same permitted destination and windows, the same deadline, and the same definition of a good result. “HTTP 200” alone may be insufficient if the expected fields are absent or stale. Then calculate both measures on the frozen denominator.
| Measure | Numerator / population | Useful for | Do not use it for |
|---|---|---|---|
| Completed-only p50 | Observed elapsed times of accepted completed requests only | Describing the completed subset | Ranking overall delivery when missed tasks differ. |
| On-time valid completion | Accepted results by deadline / all scheduled tasks | Procurement decision under the stated deadline | Claiming why a task missed without diagnostic evidence. |
| Evidence completeness | Tasks with classifiable outcome / all scheduled tasks | Determining whether the comparison is interpretable | Replacing the delivery measure. |
Report the completed-only percentile with its actual subset size beside the scheduled-task delivery rate. In the fixed six-task example below, nearest-rank p95 has rank ceil(0.95 × 6) = 6. Without six observed accepted completion times, that final ordered value cannot be reported as a measured finite all-task p95. A job may never produce an accepted result at all. NOT_IDENTIFIABLE means no numeric all-task p95 from these records, not a promise that the job would finish later. This is a six-task result: in larger datasets, whether a percentile is identified depends on its rank and the known bounds, not merely whether any record is censored.
Different task sets, deadlines, validity rules, timing boundaries or retry policies make the comparison incomparable; missing outcome evidence also requires HOLD. Different valid metric rankings alone do not. The reliability scorecard owns delivery scoring and completed-subset percentiles. This guide adds a paired ranking reversal, the six-task percentile boundary and an explicit follow-up selection rule.
A counterexample you can reproduce offline
Download the timeout-benchmark fixture. It uses Python’s standard library only and makes no network request. Six scheduled fictional tasks are assigned to each provider. A good result is accepted; the deadline is 2.0 seconds. It deliberately supplies no elapsed values for deadline misses, connection failures, or a job that never started.
These are six matched task IDs, not twelve unrelated samples. Each elapsed time starts at scheduled eligibility and ends when the first result passes the same item, required-field and region check. Both fictional routes use one attempt, no retries, the same declared client/region/load and an observation window closed at 4.0 seconds for each task. The business deadline is 2.0 seconds; these two clocks are different. A connection failure means the only attempt ended with no accepted result; a known unstarted job has scheduler evidence through the cutoff. Neither is silently treated as an observed completed latency.
| Same task | Route A: accepted elapsed or outcome | Route B: accepted elapsed or outcome |
|---|---|---|
| task-01 | 1.0 s | 1.4 s |
| task-02 | 1.1 s | 1.5 s |
| task-03 | 1.2 s | 1.6 s |
| task-04 | 1.3 s | 1.7 s |
| task-05 | Unfinished through cutoff; deadline miss | 1.8 s |
| task-06 | Connection failure, no accepted result | Known not started through cutoff |
A's completed-only median is (1.1 + 1.2) / 2 = 1.15 seconds, using four completions. B's is 1.60 seconds, using five. The fictional team predeclares at least five of these six tasks accepted within 2.0 seconds for follow-up. This illustrative gate is not a recommended universal threshold or enough evidence for a purchase; the tiny sample demonstrates the arithmetic only.
python3 proxy-timeout-benchmark.py
Its exact stdout is:
synthetic fixture only; no network requests
good_result=accepted; deadline_seconds=2.0; denominator=scheduled tasks; required_on_time=5/6
A: scheduled=6 completed_only_p50=1.15s on_time_valid=4/6 all_task_p95=NOT_IDENTIFIABLE deadline_miss=1 connection_failure=1 queue_not_started=0 missing_telemetry=0
B: scheduled=6 completed_only_p50=1.60s on_time_valid=5/6 all_task_p95=NOT_IDENTIFIABLE deadline_miss=0 connection_failure=0 queue_not_started=1 missing_telemetry=0
completed-only ranking: A faster (1.15s < 1.60s)
deadline-completion ranking: B stronger (5/6 > 4/6)
REVIEW: B eligible_for_followup at >=5/6 on-time valid tasks
Deleting A’s two non-completions makes A appear faster among completed requests. Keeping all six scheduled tasks shows B met more tasks by the deadline. Neither provider has an all-task p95 in this fixture: A has two tasks without a measured accepted completion time and B has one. That reversal and non-identifiability are the point of the fixture, not evidence that either product has those results. The predeclared 5/6 threshold makes B eligible for follow-up, while A fails it; it does not turn the completed-only percentile into a purchase rule. B’s known not-started task remains in its denominator but is not a completion sample. Missing telemetry, mismatched task sets, scope, deadlines, or retry policy instead require HOLD. Treat retries consistently across providers; use the retry-budget guide to set that policy before the trial, not after a disappointing run.
To explore the example, edit a copy of ROWS and run it again. Keep the six-task design: a missing or duplicate ID, unknown category, non-finite/negative time, or time beyond the closed window is rejected. An accepted time of exactly 2.0 seconds counts; 2.00001 seconds is a late accepted result. Setting B's task-06 to accepted, 2.5 yields an observed six-task p95 of 2.50 seconds but still only five on-time deliveries. Declared deadline, retry, target, window, clock and validation policies are checked against CONTRACT; mismatches HOLD. The program checks the declarations, not their truth: it cannot validate permissions, inspect responses, discover an omitted task outside the supplied schedule, or prove that both clients actually ran the same configuration. Preserve raw logs and have a person reconcile them.
Make the buying decision usable
- Use the same frozen task IDs and authorized workload for each provider; record whether jobs were started, accepted, late, failed, or unknown.
- Set one business deadline and a concrete validity check before seeing results. Retain both raw classes and the completed subset.
- Choose only if the on-time valid-completion comparison is interpretable and meets your own threshold. If it conflicts with the completed-only percentile, investigate the miss classes instead of buying on the lower percentile.
- Use an official API, dataset, manual verification, or a smaller controlled run where those better fit the allowed task. Do not change exits or retry indefinitely to bypass a destination’s controls.
Next step: if your permitted workload has a genuine regional public-observation gap after checking the official source and manual alternatives, compare its session, location, and acceptance constraints with residential proxy options. Bring the redacted schedule, deadline, success rule, retry policy, and missing-record count to the contact page for a fit discussion. This is conditional evaluation guidance, not a speed, availability, price, or suitability claim.
Sources checked 2026-11-21: Requests timeouts; RFC 9110, 504 Gateway Timeout; Python statistics.