How to Safely Fetch URLs in a Node.js Backend

Fetching a user-supplied URL for a preview or inspection service can expose your backend to server-side request forgery (SSRF). An attacker may try to reach internal services, local ports, or cloud metadata endpoints through your server. URL parsing alone does not prevent this.

This tutorial shows how to restrict schemes and hosts, handle redirects, limit time and response size, and use network controls to protect the actual connection. The example targets Node.js 20 or newer and is a starting point—not a substitute for enforcing outbound network policy.

Key Takeaways

• Prefer an exact hostname allowlist and HTTPS-only URLs; reject credentials and unexpected ports.
• Validate every redirect destination. Do not let fetch follow redirects automatically.
• DNS checks performed before a request can be defeated by DNS changes between checking and connecting. Enforce destination restrictions at the connection or network layer.
• Block loopback, private, link-local, and other prohibited destinations for both IPv4 and IPv6.
• Apply a deadline, redirect limit, and streamed response-size limit. Use network egress controls as a separate layer of defense.

Start with a strict URL policy

A URL parser helps you inspect components, but it does not decide whether a destination is safe. For example, it will parse http://127.0.0.1/ successfully. You can inspect URL components, but the backend must enforce its own policy.

For a preview service, allow only the partner or customer domains the service genuinely needs. Exact hostnames are safer than suffix checks: a careless test such as hostname.endsWith("example.com") also accepts notexample.com. If subdomains are needed, check the boundary explicitly: hostname === "example.com" || hostname.endsWith(".example.com").

A basic policy should reject:

• Schemes other than https: (or another explicitly required scheme).
• Hostnames not on the allowlist, including IP-literal URLs unless there is a specific, controlled need for them.
• URL usernames or passwords, which can obscure the intended destination.
• Ports other than the ones the service explicitly supports.

You may also need to restrict paths and query parameters. A permitted public host can still expose sensitive actions or data if your service fetches arbitrary endpoints on it.

DNS resolution is part of the security boundary

A hostname is not an IP address. It may resolve to several IPv4 or IPv6 addresses, and the answers can change. A preflight dns.lookup() followed by an ordinary fetch() is not, by itself, a safe destination check: the HTTP client may resolve the name again when it connects. This time-of-check/time-of-use gap can let a changing DNS answer bypass the earlier check.

If your policy evaluates IP addresses, it must cover all returned addresses and the address actually used by the connection. Deny at least:

• IPv4 loopback (127.0.0.0/8), private ranges (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16), and link-local (169.254.0.0/16). The link-local range includes commonly targeted metadata addresses.
• IPv6 loopback (::1), unique-local (fc00::/7), and link-local (fe80::/10).
• Unspecified, multicast, IPv4-mapped IPv6, and any organization-specific internal or reserved ranges that your policy prohibits.

This list is not a complete IP-classification implementation. IPv4-mapped IPv6 and unusual address forms need careful normalization, and network layouts differ. Do not write a few string-prefix checks and treat them as a complete filter.

Use a controlled outbound proxy or connection layer that resolves the hostname, checks the resolved destination, and connects only to an approved address. Combine it with firewall or cloud network rules that deny access to internal, loopback, link-local, and metadata networks. If using a proxy, ensure the application cannot bypass it with direct outbound connections. DNS checks can be useful as an additional signal, but they do not replace connection-time enforcement.

Revalidate redirects and bound the request

A URL that passes validation can respond with a redirect to a prohibited destination. Disable automatic redirects, parse each Location relative to the current URL, and run the full URL policy again before following it. Keep a small redirect limit.

The example below enforces scheme, exact-host, credential, and port rules in the application. It also uses manual redirects, a shared deadline, and a streamed body-size cap. It does not pin DNS results or enforce destination IP restrictions. Use it only alongside the connection-layer and network controls described above. Replace the example host with a hostname you intentionally permit.

import { Buffer } from 'node:buffer';
import { isIP } from 'node:net';

const ALLOWED_HOSTS = new Set(['preview.example.com']);
const MAX_REDIRECTS = 3;
const MAX_RESPONSE_BYTES = 1_000_000;
const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]);

function validateTarget(url) {
  if (url.protocol !== 'https:') {
    throw new Error('Only HTTPS URLs are allowed');
  }
  if (url.username || url.password) {
    throw new Error('URLs with credentials are not allowed');
  }
  if (url.port && url.port !== '443') {
    throw new Error('Only port 443 is allowed');
  }

  const hostname = url.hostname.toLowerCase();
  if (isIP(hostname) !== 0 || !ALLOWED_HOSTS.has(hostname)) {
    throw new Error('Destination hostname is not allowed');
  }
}

async function readLimitedBody(body, maxBytes) {
  if (!body) return Buffer.alloc(0);

  const reader = body.getReader();
  const chunks = [];
  let total = 0;

  try {
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      total += value.byteLength;
      if (total > maxBytes) {
        await reader.cancel();
        throw new Error('Response exceeds the size limit');
      }
      chunks.push(Buffer.from(value));
    }
  } finally {
    reader.releaseLock();
  }

  return Buffer.concat(chunks, total);
}

export async function fetchPreview(input) {
  let current;
  try {
    current = new URL(input);
  } catch {
    throw new Error('Input must be an absolute, valid URL');
  }

  // One deadline applies to the complete redirect chain and body read.
  const signal = AbortSignal.timeout(5_000);

  for (let hop = 0; hop <= MAX_REDIRECTS; hop++) {
    validateTarget(current);

    const response = await fetch(current, {
      redirect: 'manual',
      signal,
      headers: { accept: 'text/html, text/plain;q=0.9' },
    });

    if (REDIRECT_STATUSES.has(response.status)) {
      const location = response.headers.get('location');
      await response.body?.cancel().catch(() => {});

      if (!location) throw new Error('Redirect has no Location header');
      if (hop === MAX_REDIRECTS) throw new Error('Too many redirects');

      current = new URL(location, current);
      continue;
    }

    if (!response.ok) {
      await response.body?.cancel().catch(() => {});
      throw new Error(`Upstream returned HTTP ${response.status}`);
    }

    const body = await readLimitedBody(response.body, MAX_RESPONSE_BYTES);
    return {
      finalUrl: current.href,
      contentType: response.headers.get('content-type'),
      body,
    };
  }

  throw new Error('Too many redirects');
}

The byte limit is enforced while reading rather than trusting Content-Length, which may be absent or inaccurate. The timeout covers the request and body read, and the redirect count prevents a long redirect chain. Apply content-type checks appropriate to your feature before parsing or returning the body. Avoid rendering fetched HTML as trusted content; sanitize it and use browser isolation controls if it will be displayed.

Put outbound network controls in place

Application checks and network controls address different failure modes. Configure outbound traffic so the service can reach only the destinations it needs. Depending on your deployment, that may mean an egress proxy, firewall rules, security groups, or a dedicated fetch worker in a restricted network.

The network policy should block internal and management networks over both IPv4 and IPv6, including loopback, private, link-local, and cloud metadata destinations. If arbitrary public websites must be fetched, allow public internet access only through a control that validates the final resolved and connected address. Monitor denied requests and keep the policy consistent across containers, workers, and staging environments.

Also consider abuse controls: authenticate callers where appropriate, rate-limit requests, cap concurrent fetches, and set limits on response size and processing time. These controls reduce resource-exhaustion risk; they do not replace destination validation.

Common Mistakes

• Treating new URL() as a security check. It parses and normalizes a URL; it does not authorize its destination.
• Checking only the original URL. Redirects can change the destination, so validate each hop.
• Relying on a DNS lookup before fetch(). A later connection may use a different DNS answer unless destination enforcement is tied to the actual connection.
• Blocking only IPv4 private addresses. IPv6, IPv4-mapped IPv6, link-local, and organization-specific ranges also matter.
• Using a hostname suffix without a boundary check. badexample.com can pass a naive endsWith("example.com") check.
• Trusting response headers for size limits. Enforce a byte limit as the body streams in.
• Assuming a proxy is always in use. Prevent direct egress paths that bypass the proxy's policy.

Best Practices

1. Prefer a small exact hostname allowlist over fetching arbitrary user-provided destinations.
2. Require HTTPS, reject credentials and unexpected ports, and constrain paths if the use case allows it.
3. Manually process redirects and reapply the entire destination policy at every hop.
4. Enforce IP restrictions at the connection or egress layer, not only with a preflight DNS check.
5. Apply a request deadline, redirect limit, streamed response cap, and concurrency and rate limits.
6. Test policy behavior in an isolated environment: permitted host, disallowed hostname, redirect to a disallowed host, timeout, oversized body, and blocked internal-address attempts. Do not test against systems without authorization.

Conclusion

A safer Node.js URL fetcher needs layered controls: strict URL policy, redirect revalidation, bounded requests, and network enforcement of the destination actually reached. Parsing and hostname checks are useful components, but neither is a complete SSRF defense.

References

• OWASP SSRF Prevention Cheat Sheet — guidance on destination allowlisting, DNS behavior, and network-layer defenses.
• Node.js: Global fetch — Node's Fetch API and request options.
• Node.js: URL — URL parsing and the WHATWG URL API.