Page History

...

Section

Column

width	50%

The Stanford Stephen J. Gould collection will be used to describe the initial implementation of this process. It is believed to represent a template for similar work with other born digital collections going forward.

The directory output for the Gould collection contains the EAD and the content and metadata files for both Media and file objects (irrelevant files not shown):

M1437 Gould
- Computer Media Photo
  - CM001.jpg
  - (etc)
- Disk Image
  - Wiki Markup
    CM001.001\[.dd\]
  - CM001.001.csv
  - CM001.001.txt
  - (etc)
- Display Derivatives
  - {filename}.htm
- EAD
- FTK xml
  - files
    - {filename}
  - Report_transformed.xml
  - Report.fo
  - Report.xml

Note that the first 2 directories map to objects describing the physical media and will be the source of creating the "unprocessed" collection, while Display Derivatives and FTK files map to individual file content & description and will be used to create the "processed" collection.

Column

width	50%

The Import/conversion process will produce this arrangement of media and file objects in DOR. These are shown in blue/bold; we are not concerned here how the Collection and Series nodes themselves get created, nor how much of the rest of the EAD is also represented as objects:

Collection object
- Series set -- Series 1 ..."
- :
- Series set -- "Series 6: Born Digital Materials"
  - Media object 1**** File object 1
    - File object 2
    - :
  - Media object 2
  - :

... while a simple descriptive elements like <type> allow other logical arrangements of the files based on the nature of their content. Note we are not concerned here how the Colleciton and Series nodes get created, nor how much of the rest of the EAD is also represented in DOR.

Any intellectual arrangement of this information -- categorization into genres (correspondence, novels, etc) for instance, or tracing iterations of a work across devices, etc -- will be a separate process of augmenting the descriptive metadata for these objects.

Each <file> segment will be a the basis of a separate DOR/Hypatia Digital Object. This is an example where the atomistic model adds overhead (separate metadata and content objects) and an integrated object combining commonMetadata and genericContent could be considered.

Collection and Series objects

The Collection and Born-Digital Series objects themselves are created first, ahead of FTK processing. All FTK processed materials for a collection are processed together and are members of the Born-Digial Series set. Media objects must be linked to the appropriate series via an isMemberOf relationship.

Media objects

The FTK processing must first create a set of media objects representing the physical media (hard drive, diskette, etc) on which the files were found. This has been described as a view of the "unprocessed" collection, meaning it has not been processed down to the individual units of content, the separate files. The following information is also contained in a CMnnn.txt file describing a specific disk image:

Sample of the starting lines of the .txt file describing the media object.

Panel

Created By AccessData® FTK® Imager 3.0.1.1467 110406

Case Information:
Acquired using: ADI3.0.1.1467
Case Number: M1437
Evidence Number: CM004
Unique Description:
Examiner: Peter Chan
Notes: 5.25 inch Floppy Disk

Note that some of this information is repeated as header information in the file-level FTK report below.

From: Disk Image // CMnnn.001.txt	maps to	notes
Evidence Number: CM004	descMetadata <mods:title>	Would correspond to EAD <c><unittitle>
Notes: 5.25 inch Floppy Disks	descmetadata <mods:physicalDescription> <mods:extent>	Would correspond to EAD <physdesc> in a node describing the media.
(implied)	RELS-EXT isMemberOfCollection	A link to the Collection object
(implied)	RELS-EXT isMemberOf	A link to the Series object

File objects

Sample of transformed FTK file available as input:

Panel

Wiki Markup

<ftk_report xmlns:fo="http://www.w3.org/1999/XSL/Format">
&nbsp;&nbsp;&nbsp; <series>Series 6: Born Digital Materials</series>
&nbsp;&nbsp;&nbsp; <collection_title>Stephen Jay Gould papers</collection_title>
&nbsp;&nbsp;&nbsp; <callnumber>M1437</callnumber>
&nbsp;&nbsp;&nbsp; <file>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <filename>BU3A5</filename>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <item_number>1004</item_number>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <filepath>CM006.001/NONAME \[FAT12\]/\[root\]/BU3A5</filepath>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <disk_image_no>CM006</disk_image_no>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <filesize>35654</filesize>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <filesize_unit>B</filesize_unit>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <file_creation_date>n/a</file_creation_date>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <file_accessed_date>n/a</file_accessed_date>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <file_modified_date>12/8/1988 6:48:48 AM (1988-12-08 14:48:48 UTC)</file_modified_date>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <md5_hash>976EDB782AE48FE0A84761BB608B1880</md5_hash>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <restricted>False</restricted>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <access_rights>Public</access_rights>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <title>The Burgess Shale and the Nature of History </title>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <filetype>WordPerfect 4.2</filetype>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <duplicate_File> </duplicate_File>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <export_path>files\BU3A5</export_path>
&nbsp;&nbsp;&nbsp; </file>

Each <file> segment will be a the basis of a separate Hypatia Digital Object. This is an example where the atomistic model adds overhead (separate metadata and content objects) and an integrated object combining commonMetadata and genericContent could be considered.

Collection and Series objects

The Collection and Born-Digital Series objects themselves are created first, ahead of FTK processing. All FTK processed materials for a collection are processed together and are members of the Born-Digial Series set. The following information is also contained in a CMnnn.txt file describing a specific disk image

Same as EAD <archdesc><title>
Note that both the Collection and Series object may originate with the EAD rather than from the FTK output described here.

Information from: FTK xml // Report_transformed.xml	maps to (within item objects)	notes
Information from: Disk Image // CMnnn.001.txt	maps to	notes	<collection_title>Stephen J. Gould Papers	Collection object descMetadata <mods:title> Series & item objects descMetadata <mods:location> (1)
<series>Series 6: Born Digital Materials	Series set object descMetadata <mods:title> Item objects descMetadata <mods:location> (1)	Same as EAD series <c><unittitle>
<note>5.25 inch Floppy Disks</note>	Media object descmetadata <mods:physicalDescription> <mods:extent>	Would correspond to EAD <physdesc> in a node describing the media.
<callnumber>M1437</callnumber>	Collection object descMetadata <mods:identifier type="unitid" displayLabel="Call Number">	Same as EAD <archdesc><did><unitid>
<filename>BU3A5</filename>	n/a	this is the original file name as it appeared on the original media.
<Item_Number>1004</Item_Number>	n/a	internal FTK reference only, to disambiguate references in the FTK report
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="57433a017cea736a-c3fe971b-40644f66-a3979b29-f84b5db6c3517d11a8454b8b"><ac:plain-text-body><![CDATA[	<filepath>CM006.001/NONAME [FAT12]/[root]/BU3A5</filepath>		location of file on original media ]]></ac:plain-text-body></ac:structured-macro> <ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9b9b9cd22305e25d-64978ab9-45aa4916-8f1783b1-d5a170855c78618aa5b97c4e"><ac:plain-text-body><![CDATA[everything after [root] can be taken as the fully qualified filename	]]></ac:plain-text-body></ac:structured-macro>
<disk_image_no>CM006</disk_image_no>	descMetadata <mods:location> (1)	This token, taken from the head of the <filepath>, is the only data link between the FTK output for a file object and the corresponding media object. We want a data link in descriptive metadata as well as an RDF link to the corresponding object.
<filesize>35654</filesize>		Could be used by conversion to compare against the file size as computed locally, a quick check prior to checksum validation?
<filesize_unit>B</filesize_unit>		Needed to correctly interpret <filesize>, if used
<file_creation_date>n/a</file_creation_date>	note?
<file_accessed_date>n/a</file_accessed_date>	note?
<file_modified_date>12/8/1988 6:48:48 AM (1988-12-08 14:48:48 UTC)</file_modified_date>	note?
<MD5_Hash>976EDB782AE48FE0A84761BB608B1880</MD5_Hash>		Used for checksum validation of a file during processing. This value will eventually be part of contentMetadata, but probably not as a value transferred from here.
<restricted>False</Restricted>		true=visible staff only, not discoverable .... Hypatia only
<type>Books</type>	descMetadata <mods:subject> <mods:topic>	<topic? or <genre>? authority?
<title>The Burgess Shale and the Nature of History</title>	descMetadata <mods:title>
<filetype>WordPerfect 4.2</filetype>	descMetadata <mods:note displayLabel="File type">
<Duplicate_File> </Duplicate_File>		* blank, null value or empty string - file is unique in collection, no duplicates * "M" - The main file in a duplicate relationship. Neither better nor worse than the duplicate file, but simply the file examined first. * "D" - indicates a duplicate file. Note that this is content duplication based on having the same checksum (name conflicts are different and handled another way). The two files may or may not have the same name. It is desirable to have a note and/or relationship in each record indicating the presence of a duplicate file in the collection. Details tbd.
<export_path>files\BU3A5.wp</export_path>		The file as saved by FTK for further processing.

...

Child pages

Versions Compared

Old Version 20

New Version 21

Key

Collection and Series objects

Media objects

File objects

Collection and Series objects