Duplicated whole page when accessing site by your-domain.com/index.html

[DmitryKorlas]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

Prerequisites

• I'm using the latest version of Docusaurus.
• I have tried the npm run clear or yarn clear command.
• I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
• I have tried creating a repro with https://new.docusaurus.io.
• I have read the console error message carefully (if applicable).

Description

Description

Accessing docusaurus build by direct URL domain.com/index.html causes a duplicated page issue.

It looks like shipped docusaurus serve CLI implements a redirect from index.html > /. And it is a specific logic which are not presented in http servers by default.

Background

I'm trying to migrate docusaurus 2.4.3>3.1.1 and faced an issues with merge-request integration in GitLab pipelines. GitLab pipeline produces a docusaurus build artifact which can be accessible by URL's like
some-domain.com/-/repoName/-/jobs/12345/artifacts/build/index.html.
Attempts to omit index.html part, i.e. some-domain.com/-/repoName/-/jobs/12345/artifacts/build/ caused of HTTP404 error. Probably it's possible to have a workaround fix by adjusting redirect rules for my case. Anyway, duplicated content behavior looks like a bug.

Reproducible demo

• https://tlzj5p-9095.csb.app/index.html (not sure how long it will be accessible)
• https://docusaurus.io/index.html

Steps to reproduce

1. install from scratch

npx create-docusaurus@latest my-website classic --typescript
cd my-website/
npm install
npm run build

2. run any http server to serve build over http (I'm using python http server to avoid side-effect redirects which shipped with docusaurus serve)

cd build
python3 -m http.server 9005 

3. visit http://localhost:9005/index.html

Expected behavior

No duplicated content appears

Actual behavior

Whole site is rendered twice. Duplicated top-level menu and page content.

You can also observe this on https://docusaurus.io/index.html

Your environment

• Public source code: https://codesandbox.io/p/devbox/beautiful-archimedes-qdeo7
• Public site URL:
  • https://tlzj5p-9095.csb.app/index.html (not sure how long it will be accessible)
  • https://docusaurus.io/index.html
• Docusaurus version used: 3.1.1
• Reproduced locally using OSX 14.2.1 + NodeJS 18.18.2

Self-service

• I'd be willing to fix this bug myself.

[slorber]

This has already been discussed here:

• Duplicated site content if navigating to */index.html documents directly #9552
• navigation to '/index.html' with a static web server duplicates the page's content. #9871

See my suggestions here: #9871 (comment)

You should serve pages by their canonical URLs. By default, this means serving them without the .html extension.

If this is a problem, there's often a "pretty URL" option on CDNs and static hosts you can turn on, otherwise you can use reverse proxy settings.

[DmitryKorlas]

Thanks for your response and valuable feedback! Maybe it would be helpful to mention about new behavior at the migration guides. I followed the instructions step by step, but this behavior was unexpected.

[DmitryKorlas]

Regarding my specific case with GitLab artifacts deployment for merge requests, I couldn't find the ability to implement a redirect on the GitLab side. GitLab integration requires an index.html file for viewing artifacts of MR.

I added a workaround trick to replace the URL on the client side. It looks messy, and I don't like it, but it resolved the issue. Perhaps it can be helpful to someone else:

// docusaurus.config.ts
import type {Config, HtmlTagObject} from '@docusaurus/types';

// see https://github.com/facebook/docusaurus/issues/9916
const redirect = `
(function() {
  var pathname = window.location.pathname;
  if (pathname.endsWith('index.html')) {
    history.replaceState(
      null,
      pathname,
      pathname.replace('index.html', '')
    );
  }
})();
`.replace(/\n/g, '');

const config: Config = {
  // ...
  headTags: [
    {
      tagName: 'script',
      attributes: {
        src: 'data:text/javascript,' + redirect,
      },
    },
  ] satisfies HtmlTagObject[],
};

[slorber]

We'll try to fix this for V4 in #10059