Python Proxy Tutorial: Requests, SOCKS5 and Errors

To use a proxy in Python, give your HTTP client the proxy address and port, then add authentication if the service requires it. This tutorial configures a client with Requests; it does not build a proxy server. Start with one request, verify the route, and add concurrency only after the connection works.

Configure an HTTP proxy with Requests

Install requests, then set PROXY_URL in your local environment to the endpoint supplied by your provider. For example, its shape may be http://username:password@proxy-host:port. Use your actual endpoint and URL-encode reserved characters in credentials. Keep the value out of shared code and logs.

import os
import requests

proxy_url = os.environ["PROXY_URL"]
proxies = {"http": proxy_url, "https": proxy_url}

try:
    with requests.Session() as session:
        response = session.get(
            "https://example.com/",
            proxies=proxies,
            timeout=(5, 20),
        )
        response.raise_for_status()
        print(response.status_code)
except requests.exceptions.ProxyError:
    print("Proxy connection failed; check endpoint and authentication.")
except requests.exceptions.SSLError:
    print("TLS verification failed; check the certificate configuration.")
except requests.exceptions.Timeout:
    print("Connection or read timed out; check reachability and load.")
except requests.exceptions.HTTPError as error:
    print("Target returned HTTP", error.response.status_code)
except requests.exceptions.RequestException:
    print("Request failed; inspect sanitized diagnostics.")

Why does an HTTPS request sometimes use an http:// proxy URL?

The dictionary key selects the destination scheme. The proxy URL scheme selects the connection to the proxy. An HTTP proxy can tunnel an HTTPS destination using CONNECT; the destination TLS connection then travels through that tunnel. Use https:// for the proxy endpoint only if the provider supports TLS to that endpoint. Do not change it just because the destination uses HTTPS.

Use SOCKS5 when the endpoint supports it

Install python -m pip install 'requests[socks]'. Set PROXY_URL to a supported socks5h://username:password@proxy-host:port endpoint and reuse the example. socks5h resolves the destination hostname through the proxy; socks5 resolves it locally. Confirm the protocol and port in your provider's connection details.

Check these errors first

407 or an authentication-related ProxyError: verify the username, password and any IP allowlist. Avoid printing the credential-bearing URL.
Timeout: check endpoint reachability and service capacity. The example sets separate connection and read limits; these are not an overall job deadline.
TLS failure: verify the endpoint scheme and certificate trust. Keep certificate verification enabled.
429 from the target: reduce the request rate and honor Retry-After where supplied.
Unexpected route: inspect HTTP_PROXY, HTTPS_PROXY, ALL_PROXY and NO_PROXY in the process environment. Test with an endpoint you control that reports the observed client IP.

Choose and test the proxy for your task

Use the IPHTML integration guide to obtain the correct connection format. Review residential proxies or datacenter proxies for your workload, and compare the current plans before scaling. If the route works but the response is unexpected, check the HTTP headers guide next.

References: Requests proxy configuration, Requests errors and timeouts, and urllib3 proxy protocol behavior.