Next.js Dynamic Sitemap from MongoDB Posts

A database-driven blog needs a sitemap that stays in step with its published posts. In the Next.js App Router, a sitemap.ts file can query MongoDB and return sitemap entries, but the details matter: use canonical URLs, keep duplicate slugs from producing duplicate entries, and emit lastModified only when you have a real, valid date.

This tutorial builds a sitemap from published MongoDB documents and explains what to do when the database is unavailable. The example assumes Next.js App Router, the official MongoDB Node.js driver, and documents with slug, status, updatedAt, and publishedAt fields.

Key Takeaways

Put the sitemap in app/sitemap.ts and return MetadataRoute.Sitemap entries.
Read only published posts with usable slugs; make the blog’s slug unique in MongoDB as well as deduplicating sitemap URLs.
Configure the site origin explicitly and normalize it to an origin, rather than building public URLs from request headers.
Add lastModified only when a stored date is valid and represents a meaningful update.
Let database failures surface as failures, and monitor them. Returning an empty sitemap can make a temporary outage look like a successful sitemap update.
A sitemap helps search engines discover URLs; submitting one does not guarantee that Google will crawl or index them.

1. Configure the site origin and MongoDB connection

Set server-side environment variables for the canonical site origin and MongoDB connection string:

SITE_URL=https://www.example.com
MONGODB_URI=mongodb+srv://<user>:<password>@<cluster>/<database>
MONGODB_DB=blog

Use the actual canonical host for your site, including its preferred www choice. Keep credentials out of source control. The code below expects the posts collection to contain documents such as:

{
  "slug": "url-design-guide",
  "status": "published",
  "publishedAt": "2025-01-10T12:00:00.000Z",
  "updatedAt": "2025-02-02T09:30:00.000Z"
}

Install the MongoDB driver if it is not already a dependency:

npm install mongodb

Create a connection module at lib/mongodb.ts. Reusing the promise in development avoids opening a new connection each time Next.js reloads a module:

import { MongoClient } from "mongodb";

const uri = process.env.MONGODB_URI;
if (!uri) {
  throw new Error("MONGODB_URI is not configured");
}

const globalForMongo = globalThis as typeof globalThis & {
  mongoClientPromise?: Promise<MongoClient>;
};

const clientPromise =
  globalForMongo.mongoClientPromise ?? new MongoClient(uri).connect();

if (process.env.NODE_ENV !== "production") {
  globalForMongo.mongoClientPromise = clientPromise;
}

export default clientPromise;

For production, also configure appropriate connection limits and network access for your deployment. Do not expose the connection string through a NEXT_PUBLIC_ variable.

2. Normalize the origin and select published posts

A sitemap URL should use the same canonical origin as your site’s public URLs. Parse the configured value with URL and use its normalized origin; reject credentials, query strings, fragments, and path prefixes so a misconfigured base URL cannot silently generate a different URL structure.

The query should filter out drafts and malformed slugs. It should fetch only the fields needed to construct sitemap entries. A unique index on slug for published posts is also a good data-integrity safeguard; sitemap-side deduplication is a second line of defense, not a replacement for that constraint.

3. Create `app/sitemap.ts`

This example includes the homepage, the blog index, and published post URLs. It normalizes the origin, encodes each slug as one path segment, deduplicates final URLs, and omits invalid or missing modification dates instead of inventing one.

import type { MetadataRoute } from "next";
import type { Document } from "mongodb";
import clientPromise from "@/lib/mongodb";

const databaseName = process.env.MONGODB_DB;
if (!databaseName) {
  throw new Error("MONGODB_DB is not configured");
}

interface SitemapPost extends Document {
  slug: string;
  status: string;
  updatedAt?: Date | string;
  publishedAt?: Date | string;
}

function getSiteOrigin(): string {
  const configured = process.env.SITE_URL;
  if (!configured) {
    throw new Error("SITE_URL is not configured");
  }

  const parsed = new URL(configured);
  if (parsed.protocol !== "https:" && parsed.protocol !== "http:") {
    throw new Error("SITE_URL must use HTTP or HTTPS");
  }
  if (parsed.username || parsed.password || parsed.search || parsed.hash) {
    throw new Error("SITE_URL must not contain credentials, a query, or a fragment");
  }
  if (parsed.pathname !== "/") {
    throw new Error("SITE_URL must be an origin, not a path prefix");
  }

  // URL.origin normalizes the scheme, hostname, and port and excludes a trailing slash.
  return parsed.origin;
}

function validLastModified(value: unknown): Date | undefined {
  let date: Date | undefined;

  if (value instanceof Date) {
    date = value;
  } else if (typeof value === "string") {
    date = new Date(value);
  }

  if (!date || !Number.isFinite(date.getTime())) {
    return undefined;
  }

  // A future timestamp is not a credible modification time for this sitemap.
  if (date.getTime() > Date.now()) {
    return undefined;
  }

  return date;
}

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  try {
    const origin = getSiteOrigin();
    const client = await clientPromise;
    const db = client.db(databaseName);

    const posts = await db.collection<SitemapPost>("posts").find(
      {
        status: "published",
        slug: { $type: "string", $ne: "" },
      },
      {
        projection: {
          _id: 0,
          slug: 1,
          updatedAt: 1,
          publishedAt: 1,
        },
      },
    ).toArray();

    const entries: MetadataRoute.Sitemap = [
      { url: `${origin}/` },
      { url: `${origin}/blog` },
    ];

    const postEntries = new Map<string, MetadataRoute.Sitemap[number]>();

    for (const post of posts) {
      const slug = post.slug.trim();
      if (!slug) continue;

      const url = new URL(`/blog/${encodeURIComponent(slug)}`, `${origin}/`).toString();
      const lastModified =
        validLastModified(post.updatedAt) ?? validLastModified(post.publishedAt);
      const existing = postEntries.get(url);

      // If duplicate documents exist, retain the newest valid date for that URL.
      const existingDate = existing?.lastModified;
      const existingTime =
        existingDate instanceof Date ? existingDate.getTime() : -Infinity;
      const nextTime = lastModified?.getTime() ?? -Infinity;

      if (!existing || nextTime > existingTime) {
        postEntries.set(url, {
          url,
          ...(lastModified ? { lastModified } : {}),
        });
      }
    }

    // Deduplicate all URLs, including any overlap between fixed and database routes.
    const uniqueEntries = new Map<string, MetadataRoute.Sitemap[number]>();
    for (const entry of [...entries, ...postEntries.values()]) {
      uniqueEntries.set(new URL(entry.url).toString(), entry);
    }

    return [...uniqueEntries.values()];
  } catch (error) {
    console.error("Unable to generate sitemap from MongoDB", error);
    throw error;
  }
}

The $type: "string" filter excludes documents whose slug is absent or not a string; the trim check also skips whitespace-only values. encodeURIComponent prevents a slug from being interpreted as multiple path segments. If your routing rules allow only lowercase letters, numbers, and hyphens, validate that stricter format in your publishing workflow too.

The duplicate handling keeps a single entry for each URL and, when duplicate documents have different valid dates, uses the newest one. Ideally, prevent the duplicate at write time with a suitable unique index that matches your publication model. If drafts and published posts can share a slug, a partial unique index may be more appropriate than a simple unique index; design that constraint around your application’s actual status transitions.

lastModified is based on stored post data, not the time the sitemap was generated. The example prefers a valid updatedAt and falls back to a valid publishedAt. If neither is valid, it leaves the field out. Update the stored timestamp when meaningful content changes; changing a counter or regenerating a document without changing its substantive content should not automatically imply that the page was meaningfully modified.

4. Decide how often the sitemap refreshes

A database query does not by itself define how often visitors receive fresh sitemap data. Next.js can cache route output depending on the route and configuration. For a sitemap that should be periodically regenerated, you can configure revalidation in app/sitemap.ts:

export const revalidate = 3600;

This requests revalidation on an hourly interval; the precise freshness behavior depends on your Next.js version and hosting platform. Choose an interval that balances how quickly new posts should appear against database load. If you need updates immediately after publishing, use a deliberate cache-invalidation strategy supported by your Next.js version and deployment platform, or serve the sitemap dynamically. Avoid assuming that a successful database write automatically invalidates a cached sitemap.

If the database is unavailable, the example logs the error and throws. That makes generation fail visibly rather than returning a successful-looking sitemap with every blog URL missing. Add monitoring for sitemap failures and test the behavior in your deployment environment. With caching or revalidation, a platform may continue serving previously generated output after a failed refresh; verify the behavior for your specific setup rather than treating it as a universal guarantee. A deliberate static fallback can be useful, but it should be clearly maintained and must not silently claim to include all current posts.

5. Check the generated sitemap

Run the app in the same environment where the database variables are configured, then request /sitemap.xml:

curl -i https://www.example.com/sitemap.xml

Check that:

The response is successful and contains XML.
Every listed URL uses the canonical scheme and hostname.
Drafts, blank slugs, and duplicate URLs are absent.
A post with a valid stored date has a correctly formed <lastmod> value.
A post without a valid date has no fabricated <lastmod> value.
Database or configuration failures are visible in logs and do not produce an empty sitemap that appears successful.

For a local test, use your local host in SITE_URL and request the local /sitemap.xml route. Do not infer that a sitemap is correct just because it returns HTTP 200; inspect its URLs and compare them with the routes that actually exist.

If the site grows beyond the sitemap limits accepted by search engines, split the URLs across sitemap files and publish a sitemap index. Keep each sitemap within Google’s documented limits of 50,000 URLs or 50 MB uncompressed. For larger collections, avoid loading every document into one application process; use pagination or a sitemap-index design appropriate to your data volume.

Common Mistakes

Using request headers as the canonical origin. Host headers can vary across proxies and requests. Use a trusted deployment setting for the public origin.
Including drafts or unpublishable records. Query for the exact published state and validate required route fields.
Treating URL parsing as deduplication. Normalize and deduplicate final URLs, and enforce slug uniqueness in the database where appropriate.
Setting lastModified to the current time for every entry. That reports sitemap generation time, not when the page content changed.
Converting an outage into an empty sitemap. Catching an error and returning [] hides a failure behind a successful response.
Assuming sitemap inclusion means indexing. A sitemap is a discovery signal, not a command to index every URL.
Ignoring the actual route structure. If posts are served from /articles/ rather than /blog/, generate the same canonical path your application uses.

Best Practices

Keep sitemap URL generation aligned with the application’s canonical URL rules, including hostname, protocol, and trailing-slash policy.
Validate slugs and timestamps when writing posts, then retain defensive checks when reading them for the sitemap.
Add database indexes for the fields used by the published-post query and enforce the slug uniqueness your publication rules require.
Set caching or revalidation intentionally and confirm it on your deployment platform.
Monitor generation failures and periodically compare sitemap entries with published, reachable pages.
Use robots directives and canonical tags consistently with the URLs you submit; a sitemap does not fix conflicting canonicalization or blocked pages.

Conclusion

A reliable Next.js dynamic sitemap is a small integration with important data-quality and operational details. Query only published posts, derive URLs from a normalized canonical origin, deduplicate entries, use genuine modification dates, and make database failures observable. Then verify the generated XML and remember that Google—not the sitemap—decides whether a URL is indexed.

References

Next.js: sitemap.xml file convention — documents the App Router sitemap convention and supported metadata fields.
Google Search Central: Build and submit a sitemap — explains sitemap formatting, size limits, submission, and the fact that submission does not guarantee crawling or indexing.