Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Sitemaps ping settings removed in https://github.com/DSpace/DSpace/pull/9165

...

  1. Keep your DSpace up to date. We are constantly adding new indexing improvements in new releases
  2. Ensure your DSpace is visible to search engines.
  3. Ensure your proxy is passing X-Forwarded headers to the User Interface

  4. Ensure the user interface is using server-side rendering (enabled by default)
  5. Ensure the sitemaps feature is enabled. (enabled by default)
  6. Ensure your robots.txt allows access to item "splash" pages and full text.
  7. Ensure item metadata appears in HTML headers correctly.
  8. Avoid redirecting file downloads to Item landing pages
  9. Turn OFF any generation of PDF cover pages
  10. As an aside, it's worth noting that OAI-PMH is generally not useful to search engines.  OAI-PMH has its own uses, but do not expect search engines to use it.

...

Ensure your proxy is passing X-Forwarded

...

headers to the User Interface

Some HTML tags important for SEO, such as the "citation_pdf_url" tag, require the full URL of your site.  The DSpace user interface will automatically attempt to "discover" that URL using HTTP Headers. 

...

In DSpace 7, server-side rendering is enabled by default (when running in production mode). However, it's important to ensure you do not disable it in production mode.  Per the frontend Installation instructions, you MUST also be running your user interface in production mode (via either yarn run serve:ssr or yarn start).

Because the DSpace user interface is based on Angular.io (which is a javascript framework), you MUST have server-side rendering enabled (which is the default) for search engines to fully index your side.  Server-side rendering allows your site to still function even when Javascript is turned off  in a user's browser.  Some web crawlers do not support Javascript (e.g. Google Scholar), so they will only interact with this server-side rendered content.DSpace use Angular Universal for

If you are unsure if server-side rendering , and it's enabled by default in Production mode via this configuration in src/environments/environment.production.ts:

Code Block
// Angular Universal Settings
universal: {
  preboot: true,
  ...
},

Per the frontend Installation instructions, you must also be running your production frontend/UI via either yarn run serve:ssr or yarn start.

For information, see "Universal (Server-side Rendering) settings" in User Interface Configuration

...

(SSR) is enabled, you can check to see if your site is accessible when Javascript is turned off.  For example, in Chrome, you should be able to do the following:

  1. Open your site in the Chrome browser
  2. Turn off (disable) Javascript using the Chrome instructions: https://developer.chrome.com/docs/devtools/javascript/disable/
  3. Click reload in your browser window to reload your site.
    1. If SSR is enabled, then you will still see your site's contents. You should be able to browse & search the site. (Keep in mind, pages may take longer to load because every request requires SSR.)  However, all dynamic menus or actions obviously will not work, as all pages will be static HTML.
    2. If SSR is disabled, then you will see a blank white page. You will not be able to see any content on your site.
  4. Don't forget to re-enable Javascript after you are done testing (see link above, or just close that window & reopen a new one)

DSpace use Angular Universal for server-side rendering, and it's enabled by default in Production mode via our production environment initialization in src/environments/environment.production.ts:

Code Block
// Angular Universal Settings
universal: {
  preboot: true,
  ...
},

For information, see "Universal (Server-side Rendering) settings" in User Interface Configuration HTML.)

Ensure the sitemaps feature is enabled

...

Notify the given URL that new sitemaps are available.  The URL of the new sitemap will be appended to the value of URL
Optionmeaning

-h

--help

Explain the arguments and options.

-s

--no_sitemaps

Do not generate a sitemap in sitemaps.org format.

-b

-no_htmlmap

Do not generate a sitemap in htmlmap format.

-a

--ping_all

Notify all configured search engines that new sitemaps are available.

-p URL

--ping URL

.

You can configure the list of "all search engines" by setting the value of sitemap.engineurls in dspace.cfg.

...