...

The Fedora 4.7.3 release is a backwards compatible refinement of the previous release, focused on:

• ...

Changes

Invalid Namespace Mappings

...

which fixes a namespace-corrupting bug that has the effect of Fedora being unable to successfully start after having been shut down.  

Detailed Bug Remediation Notes

Who is affected

This occurs when all the following conditions are true:

1. An earlier version of Fedora had been used to create objects, before upgrading to 4.7.1 and 4.7.2

2. One of the objects created in the repository prior to upgrade to 4.7.1 or 4.7.2 contains a triple using one or more of the following namespaces, and the prefix does NOT match the one shown:

1. skos: http://www.w3.org/2004/02/skos/core#

2. acl: http://www.w3.org/ns/auth/acl#

3. iana: http://www.iana.org/assignments/relation/

4. event: http://fedora.info/definitions/v4/event#

5. owl: http://www.w3.org/2002/07/owl#

6. prov: http://www.w3.org/ns/prov#

7. config: info:fedoraconfig/

For example, if an object prior to upgrade to Fedora 4.7.1 or 4.7.2 looked like the following, it would be OK since the prefix matches the above table:

@prefix skos:  <http://www.w3.org/2004/02/skos/core#> .
 
<> skos:prefLabel “foo” .

On the other hand, if an object prior to the upgrade to 4.7.1 or 4.7.2 looked like this, it will cause the namespace corrupting bug:

@prefix ns004:  <http://www.w3.org/2004/02/skos/core#> .
 
<> ns004:prefLabel “foo” .

Viewing the objects in a running 4.7.1 or 4.7.2 instance is insufficient for determining if this bug affects you, as the prefixes that you are shown will not necessarily match the prefixes originally present in the object. If you have any objects containing triples whose predicate or object contains one of the above namespaces, you are at risk.

What is the problem

Fedora 4.7.1 and 4.7.2 added some global, hard-coded namespaces and prefixes to the fedora-node-types.cnd file.  Any existing namespaces that conflict with these new hard-coded namespaces are deleted/overwritten when Fedora exits.  This results in an inconsistent Modeshape database.  The repository will function in this inconsistent state until a re-started is attempted, which will fail.

How to fix the issue

At a high-level you need to:

1. Take a backup

2. Fix the backup manually

3. Restore the backup into a new repository

Since it is somewhat difficult to determine if your repository is truly affected, a reasonable upgrade plan from 4.7.1 or 4.7.2 to 4.7.3 is:

1. Take a backup

2. Try simply upgrading 4.7.3 over the existing repository.  If it starts up successfully, you are not affected by the issue, and are done!

3. If Fedora 4.7.3 fails to start, then you need to fix the backup manually, and restore into an empty repository

Detailed instructions

1. Create backup

1. curl -X POST http://localhost:8080/fcrepo/rest/fcr:backup

2. You will be given a directory in message body.  For example: C:\Users\Dev\projects\dcs-package-ingest\package-ingest-integration\target\cargo\configurations\tomcat8x\temp\1496329063578-0

3. You will see one or more files named documents_N.bin.gz where N is a number like 000001 (e.g. documents_000001.bin.gz)

2. Unzip and inspect the documents_N.bin.gz files.  Look for modeshape node whose primaryType is ‘mode:namespaces’

1. `jq` is a great tool for inspecting very large json files.  To pretty-print so that tools like grep can be used, try , cat documents_00001.bin | jq .

1. Grep for ‘namespaces’.  You’ll see something like:

   Code Block

   { "metadata": { "id": "936198c317f1e7mode:namespaces" }, "content": { "key": "936198c317f1e7mode:namespaces", "parent": "936198c317f1e7jcr:system", "properties": { "http://www.jcp.org/jcr/1.0": { "primaryType": { "$name": "mode:namespaces" } } }, "children": [ { "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/1.0", "name": "jcr" }, { "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/nt/1.0", "name": "nt" }, { "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/mix/1.0", "name": "mix" }, { "key": "936198c317f1e7mode:namespaces-http://www.w3.org/2004/02/skos/core#", "name": "skos" } }

   
   

The dump contains a json object whose primary type is “mode:namespaces”.  Nested within it is a “children” array that enumerates the namespaces known to modeshape.  It appears that Modeshape chooses a single random prefix on all internal identifiers.  Acknowledging that prefix, the modeshape namespaces enumeration follows a simple and obvious pattern.

When Fedora encounters a novel rdf namespace and prefix, it adds it to the map.  In many cases, the prefix is auto-generated, e.g. “ns001”.  So if Fedora encountered a SKOS term, at any point, it may contain a namespace entry like:

{
    "key": "936198c317f1e7mode:namespaces-http://www.w3.org/2004/02/skos/core#",
    "name": "ns004"
}

In Fedora 4.7.1 and 4.7.2, a new set of hard-coded namespaces were published in the internal .cnd file used by modeshape.  When started, any pre-existing namespaces that match the hard-coded namespaces, yet differ by prefix, are overwritten.  So the namespaces table may now contain an entry:

{          
    "key": "936198c317f1e7mode:namespaces-http://www.w3.org/2004/02/skos/core#",
    "name": "skos"
}

This overwriting of the namespaces table is problematic.  Objects that existed before the repository had been upgraded to 4.7.1 or 4.7.2 will still contain their _old_ namespace references, such as:

{
  "metadata": {
    "id": "936198c317f1e7/jcr:system/jcr:nodeTypes/{httpwww.w3.org200402skoscore#}Concept"
  },
  "content": {
    "key": "936198c317f1e7/jcr:system/jcr:nodeTypes/{httpwww.w3.org200402skoscore#}Concept",
    "parent": "936198c317f1e7jcr:nodeTypes",
    "properties": {
      "http://www.jcp.org/jcr/1.0": {
        "isQueryable": true,
        "primaryType": {
          "$name": "nt:nodeType"
        },
        "supertypes": [],
        "hasOrderableChildNodes": false,
        "isMixin": true,
        "isAbstract": false,
        "nodeTypeName": "ns004:Concept" << NOTE: THIS IS THE PROBLEMATIC LINE <<
      }
    }
  }

This inconsistency is at the root of the problem.

3. For each dangling namespace, add a new entry to the mode:namespaces node, for example:

    "children": [
      {
        "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/1.0",
        "name": "jcr"
      },
      {
        "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/nt/1.0",
        "name": "nt"
      },
      {
        "key": "936198c317f1e7mode:namespaces-http://www.jcp.org/jcr/mix/1.0",
        "name": "mix"
      },
      {
        "key": "936198c317f1e7mode:namespaces-http://www.w3.org/2004/02/skos/core#",
        "name": "ns004"
      }
}

4. After the namespaces node has been fixed, gzip the file so that it retains its original name

5. Restore the backup via fcr:restore

Changes

Complete Listing of Resolved Tickets

...