A command-line application that can be gracefully interrupted and restarted at any time, with little or no loss of progress.

Flow diagram

• Configuration
• Assessment
  
  • Scheduling
    
    • Determine what discovery is to be done
  • Discovery
    
    • Visit the client sites to build lists of URIs of Individuals for the index
    • Will the client site give us the last modified date for the individual?
  • Synchronizing
    
    • Record the results of discovery in the search index
    • Remove any URIs which are no longer viable
    • Add any new URIs.
    • Record that discovery was done.
• Retrieval
  
  • Prioritizing
    
    • Inspect the index to see which records should be updated.
    • Build a to-do list.
  • Processing - for each URI in the do-do list:
    
    • Modeling
      
      • Use LOD requests to build the model for the URI
    • Indexing
      • Build an index record from the model, and write to the search index.