Validate User-Supplied URLs Without SSRF

Accepting a URL is not the same as making it safe for your server to fetch. A URL can be syntactically valid yet point to a private service, resolve differently between checks and connection, or redirect to a prohibited destination. This guide covers a layered approach for backend URL fetch, preview, and import features.

Key Takeaways

Parse URLs with a maintained URL library; a regular expression or syntax check cannot establish that a network request is safe.
Prefer an exact hostname allowlist. If users may fetch arbitrary sites, validate the scheme, hostname, resolved IP addresses, and port under a clear policy.
Prevent time-of-check/time-of-use gaps: the connection must use the address that passed policy checks, or an egress proxy must enforce the policy at connection time.
Disable automatic redirects or ensure every redirect target is checked again before it is fetched.
Add network-level egress restrictions, timeouts, response limits, and monitoring. Application validation alone is not a complete SSRF defense.

Why URL Syntax Validation Is Not Enough

A URL parser answers questions such as whether an input can be interpreted as a URL and what its scheme, host, and path are. It does not establish that the destination is safe for your server to contact.

For example, a valid https URL might name a loopback address, a private-network host, or a public hostname that resolves to a private address. A permitted public site might then redirect to a prohibited destination. DNS answers can also change between validation and connection.

Treat URL handling as two separate tasks:

Input policy: Is this URL permitted by your product rules?
Connection policy: Is the actual network destination permitted at the moment the connection is made?

A parser helps with the first task. It cannot, by itself, guarantee the second.

Define a Narrow URL Policy

Start with the feature's requirements rather than accepting every URL a general-purpose parser understands.

Schemes: Allow only the schemes you need, usually https; allow http only if there is a justified product requirement. Reject schemes such as file, ftp, and application-specific schemes.
Hosts: Prefer an exact allowlist of hostnames when the feature can work with known services. If arbitrary public websites are necessary, apply hostname and resolved-address checks.
Ports: Restrict ports to the ones the feature requires, commonly 443 and, if http is supported, 80. Do not assume that a public IP on an unusual port is safe.
URL credentials: Reject URLs containing a username or password, such as https://user:pass@example.com/. These can confuse policy checks and expose secrets in logs.
Input size and form: Set a reasonable maximum input length and reject malformed or ambiguous inputs rather than trying to repair them silently.

Use a maintained URL parser for the language and runtime. Parse once into structured components, then apply policy to those components. Be careful that the parser, HTTP client, proxy, and any hostname-normalization code agree on how the destination is interpreted. If you support internationalized domain names, use a consistent, maintained IDNA implementation and compare normalized hostnames against the policy.

Validate Hostnames and IP Addresses

If a parsed hostname is an IP literal, parse it with a standard IP-address library and classify the resulting address. Do not rely on string prefixes or hand-written checks. IPv4 and IPv6 have different special-use ranges; IPv4-mapped IPv6 addresses and unusual textual forms also need consistent handling.

For hostnames, an exact allowlist is usually easier to reason about than a denylist. Compare complete normalized hostnames. A check for example.com must not accidentally accept example.com.attacker.invalid. If subdomains are intentionally allowed, enforce a label boundary and document which subdomains are trusted.

If users can request arbitrary hosts, resolve both A and AAAA records using the resolver behavior relevant to the actual fetch. Reject the hostname if any returned address is prohibited by your policy; checking only one answer can leave another usable route. The policy should exclude loopback, private, link-local, unspecified, multicast, and other special-purpose or non-public ranges, including IPv6 equivalents. Use a maintained IP classification library or a controlled egress service rather than a list that your team cannot keep current.

A globally routable address is not automatically safe: it may still identify a sensitive service reachable from your network, and network routing can differ by environment. Product-specific host restrictions and network egress controls remain important.

Account for DNS Rebinding

A vulnerable pattern is:

Resolve a hostname and confirm its address is allowed.
Pass the hostname to an HTTP client that resolves it again later.

The second lookup may return a different address. This is a time-of-check/time-of-use gap, sometimes exploited through DNS rebinding.

The fetch must connect to the address that passed the checks. A carefully designed transport can pin the validated address for the connection while still using the original hostname for the HTTP Host value and TLS certificate verification/SNI. This is easy to get wrong with custom transports, proxies, connection pools, and retries. Prefer a maintained client or controlled egress proxy that explicitly supports destination enforcement. Do not disable TLS certificate verification to make address pinning work.

If a proxy performs DNS resolution, enforce the address policy at that proxy; checking DNS only in the application may not protect the connection the proxy makes. Restrict outbound network access as a second layer so the fetch service cannot reach internal networks or sensitive infrastructure even if application checks fail.

Revalidate Every Redirect

Redirects are new destination requests, not proof that the original destination is safe. An allowed host can respond with a Location pointing to a private address, a disallowed hostname, or a different scheme.

A robust approach is to disable automatic redirect following. For each response:

Accept only redirect status codes your feature supports.
Resolve a relative Location against the current URL using a standards-compliant URL implementation.
Apply the complete URL, hostname, port, DNS, and IP policy to the new destination.
Connect only after that destination has passed the same connection-time checks.
Stop after a small, configured redirect limit and reject loops or invalid Location values.

Do not validate only the first URL and then hand it to a client that follows redirects automatically. If automatic redirects are necessary, use a client hook that can enforce the policy before every redirected request, and verify how that client handles DNS resolution and connection reuse. Avoid scheme downgrades, such as following an HTTPS URL to HTTP, unless explicitly required and safe for the feature.

Defensive Request Flow

Use this as an implementation-level design, not as a drop-in function. The address-pinning and transport behavior depends on your HTTP client and deployment, so verify those details for the stack you use.

fetch_user_url(input):
    reject input that exceeds the configured length
    url = parse with a maintained URL library
    reject if parsing fails or URL contains username/password
    reject unless scheme, hostname, and port match product policy

    for each request, including redirects:
        resolve hostname using the resolver used by the fetch path
        reject if resolution fails or any answer violates IP policy
        connect only to a validated address, or use an egress proxy
            that enforces the same policy at connection time
        preserve normal TLS hostname and certificate verification

        response = make request with automatic redirects disabled
        if response is not a redirect:
            enforce response size, time, and content-type limits
            return an appropriately handled response

        reject if redirect limit is reached
        url = resolve and validate the Location as a new URL

Keep the fetch service's permissions narrow. Use connection and total-request timeouts, cap response bytes while streaming (not only after buffering), limit redirects, and avoid forwarding inbound cookies or authorization headers to a user-selected destination. Restrict which response types your preview or import feature will process; parsing untrusted content can introduce risks beyond SSRF.

Defensive Design Checklist

Common Mistakes

Treating a successful parse as a security check. Parsing establishes structure, not destination trust.
Using a single regular expression. URL syntax, IP address forms, hostname normalization, and client behavior are too complex for one pattern to enforce the full policy.
Checking only the hostname string. A hostname may resolve to a prohibited address, and its DNS answer can change.
Checking one DNS answer. A hostname can have multiple A or AAAA records; the client may connect to an unchecked one.
Validating before a client performs its own lookup. A later lookup can invalidate the earlier decision.
Checking only the initial URL. Redirects can move the request to a destination that would have been rejected directly.
Relying only on IP deny rules. Address classifications and network reachability are not a substitute for product-specific host policy and egress controls.
Assuming a URL parser or online inspection tool makes fetching safe. The security decision must be enforced by the server and the network path that actually performs the request.

Best Practices

Prefer a fixed set of trusted destinations over arbitrary user-supplied hosts whenever possible. If arbitrary public URLs are essential, centralize fetching behind a small service with a documented policy, a constrained outbound network, and a transport that enforces destination checks at connection time.

Treat every redirect as a fresh request, keep the rules consistent across parsers, proxies, and HTTP clients, and test the deployed path—not just a standalone validation function. Review the policy when dependencies, resolvers, proxy settings, or network architecture change.

Conclusion

Safe URL fetching requires more than validating syntax. Combine strict input rules with hostname and IP checks, protection against DNS and redirect bypasses, connection-time enforcement, and network egress controls. If your design cannot ensure that the actual destination is checked, do not fetch arbitrary user-supplied URLs.

References

OWASP Server Side Request Forgery Prevention Cheat Sheet — guidance on SSRF defenses and destination validation.
RFC 1918: Address Allocation for Private Internets — IPv4 private address ranges.
RFC 6890: Special-Purpose IP Address Registries — special-purpose address registries.
RFC 9110, Section 15.4: Redirection 3xx — HTTP redirection semantics.

The live OWASP guidance could not be retrieved in this drafting environment. Check its current revision and the documentation for your HTTP client and egress proxy before implementation.

How to Validate User-Supplied URLs Without Creating an SSRF Risk