My CI/CD pipeline on a self-hosted service could not reach external hosts because I had not configured HTTP_PROXY, HTTPS_PROXY, and NO_PROXY. This made me wonder why proxy configuration is usually exposed through environment variables that individual applications may or may not honor, rather than being enforced with a network namespace, firewall rules, or a transparent forward proxy.
My assumption is that these variables mainly tell a client where to send requests and do not otherwise change the tool's behavior. Is that accurate? Why is proxy handling considered an application-layer concern instead of a network-layer concern? Are there situations where an application needs to know that traffic is being proxied, such as authentication, HTTPS CONNECT handling, or deciding which hosts bypass the proxy?
3 Answers
These approaches solve different problems. Environment variables are cooperative: the application chooses to read them, select the appropriate HTTP or HTTPS behavior, authenticate to the proxy, and apply NO_PROXY rules. That is why proxy configuration belongs naturally in the application's settings, even though inconsistent support can be frustrating.
Network namespaces and transparent proxies are enforcement mechanisms. You can place a process in a namespace whose routing only permits traffic through a proxy, which is useful for untrusted CI jobs, sandboxes, or third-party plugins. The process cannot simply ignore an environment variable and open a direct socket. The tradeoff is that you must manage routing, DNS, firewall rules, and proxy behavior inside that environment. Namespaces are a kernel feature on modern Linux; the main cost is operational complexity, not installing a separate package.
The convention is much older than network namespaces, works without elevated privileges, and is portable across operating systems. A transparent redirect can also complicate HTTPS: the client needs to establish a CONNECT tunnel or otherwise trust a proxy certificate. With HTTP_PROXY and HTTPS_PROXY, the application can deliberately use the correct proxy protocol and handle proxy authentication.
That also explains why a proxy variable should not be treated as a security boundary. Any program that ignores it or creates a raw connection can bypass it unless the network itself prevents direct access.