Safeguard
Application Security

Your Edge Appends to X-Forwarded-For, So the First Hop Is Whatever the Caller Typed

Reading the first entry of X-Forwarded-For is reading user input. We captured what our load balancer actually sends, and it explains why a rate limiter can run for months without limiting anything.

Tomas Lindgren
Platform Engineer
7 min read

request.headers['x-forwarded-for'].split(',')[0] is user input. On a standard reverse proxy or load balancer setup it is not the client address, it is whatever the client typed, and anything you build on top of it is built on a value the caller controls.

That includes rate limits, IP allowlists, geo rules, abuse blocks, and the address you write into an audit record. This post is for whoever owns the code that reads the header. By the end you will know why the first hop lies, which value to trust instead, how to verify it in ten minutes, and the one network condition that invalidates the whole analysis.

X-Forwarded-For is a list, and every proxy appends to it

The header is a comma separated list of addresses, oldest first. A proxy that handles a request appends the peer address it saw and forwards the rest untouched. That is the specified behaviour and it is what implementations do.

The consequence is the part people miss. If the client sends the header itself, the proxy does not replace it. It appends to it. The forged value keeps its place at the front of the list, and the real address lands at the end.

What we measured

We captured traffic between our load balancer and the application with tcpdump. TLS terminates at the edge, so the internal hop is readable. The request was sent from a real address of 103.163.42.59, carrying two forged headers:

X-Forwarded-For: 198.51.100.77
X-Real-IP: 198.51.100.99

What arrived at the application:

X-Real-IP: 103.163.42.59
X-Forwarded-For: 198.51.100.77, 103.163.42.59

Two different behaviours in one hop. The edge appended to X-Forwarded-For, so the forged hop survived and kept the front position. The edge overwrote X-Real-IP, so the forged value is gone.

So on this topology:

  • X-Real-IP is trustworthy.
  • The last hop of X-Forwarded-For is trustworthy.
  • The first hop of X-Forwarded-For is whatever the caller typed.

The conventional idiom takes the first hop. It is the one wrong choice available.

What breaks, in order of how much it costs you

Rate limiting stops counting. If the bucket key comes from the first hop, rotating the header buys a fresh bucket on every request. The limiter still runs, still logs, still looks configured. It just never limits anything. We had exactly this on a public write endpoint, and the reason it went unnoticed is that the standard way to test a limiter is to hammer it from one machine, which passes.

IP allowlists invert. An allowlist keyed on the first hop admits anyone who can spell an allowed address. This is the failure that turns an internal-only admin route into a public one.

Audit records become fiction. We recorded the client address against a document signature. An attacker choosing what gets written into a legal record is a different class of problem from a rate limit, and it is the same one line bug.

Log poisoning and analytics. Anything downstream that groups by client IP, including your SIEM, can be filled with addresses of the caller's choosing. It is a cheap way to bury a real address in noise or to attribute activity to someone else.

The rule: parse from the right

Trust is a property of position. Every hop to the right of the entry your infrastructure added was added by your infrastructure. Everything to the left of it came from outside.

If you have exactly one trusted proxy, the last hop is the client. If you have two, the second from last is. Count your hops, hardcode the count, and read that many from the right. Do not infer it at runtime, because the number of trusted proxies is a deployment fact and inferring it from the header means reading the value you are trying to validate.

Here is the shape, in the form we shipped:

public static String of(HttpServletRequest req) {
    String real = req.getHeader("X-Real-IP");
    if (real != null && !real.isBlank()) return real.trim();

    String fwd = req.getHeader("X-Forwarded-For");
    if (fwd != null && !fwd.isBlank()) {
        String[] hops = fwd.split(",");
        for (int i = hops.length - 1; i >= 0; i--) {
            String hop = hops[i].trim();
            if (!hop.isEmpty()) return hop;
        }
    }
    return req.getRemoteAddr();
}

The empty hop trap

Note the loop rather than a straight hops[hops.length - 1]. A caller who ends the header with a trailing comma leaves an empty final segment:

X-Forwarded-For: 198.51.100.77, 103.163.42.59,

Split that and the last element is an empty string. Code that takes the last element and falls back to the socket address when it is blank does not get the client, it gets the load balancer. Every caller then collapses into a single bucket, which fails closed for a rate limiter and fails open for nothing, but it is still wrong and it is still silent. Walk back to the last non-empty hop.

Per stack

Most frameworks have a correct implementation already. Use it rather than parsing by hand.

  • nginx: set_real_ip_from <your edge CIDR>; plus real_ip_header X-Forwarded-For; and real_ip_recursive on;. This rewrites $remote_addr to the rightmost address not in a trusted range.
  • Express: app.set('trust proxy', 1) with the literal number of proxies in front. Then read req.ip. Do not use true, which trusts the whole chain and returns the leftmost value.
  • Spring: ForwardedHeaderFilter, and put it behind a check that the request actually came from your edge.
  • Go: no standard helper. Parse from the right, with a hardcoded trusted count.
  • Cloudflare: CF-Connecting-IP is set by Cloudflare and not appendable, so prefer it. Same for True-Client-IP on Akamai and Cloudflare Enterprise.
  • AWS ALB: appends, same as above. X-Forwarded-For rightmost is the client.

The concession that matters more than the parsing

All of this assumes the only route to your application is through the edge.

If the application port is reachable directly, from another tenant in the same VPC, from a misconfigured security group, from the public internet because someone published it for an internal tool, then no header means anything. The attacker sends X-Real-IP themselves and your trustworthy header is now the untrustworthy one. Check reachability before you tune the parser. A header is only as trustworthy as the guarantee that something you control wrote it, and that guarantee is a network property, not a code property.

The second concession: X-Real-IP is trustworthy on our edge because we verified our edge overwrites it. That is not universal. Some proxies pass it through untouched. Capture the traffic on your own path before trusting this post about your own setup.

How to verify in ten minutes

Do not test by sending many requests from one machine. That passes on broken code, because with a broken parser and no forged header the value still happens to be stable.

Test by rotating the header:

for i in $(seq 1 50); do
  curl -s -o /dev/null -w "%{http_code} " \
    -H "X-Forwarded-For: 198.51.100.$i" \
    https://your.app/some/rate/limited/endpoint
done

A correct limiter blocks partway through this loop, because every request is the same client. A broken one returns 200 fifty times. Run the same loop with a trailing comma on the header and with an empty header to cover the degenerate parses.

Then do it again with the allowlisted address of your choosing, and with an address you expect to see in your audit log, and check the log.

The implication

This is not an exotic bug. It is one line, the idiom appears in thousands of tutorials, and every security control keyed on client address inherits it. The reason it survives is that a broken parser and a correct one behave identically under every test that does not forge the header, which means it will not show up in your test suite, your load test, or your staging traffic. It shows up when somebody tries.

If you have a rate limiter, an allowlist, or an audit field, go look at the line that reads the header. It takes a minute, and the read is from the right.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.