Versions Compared

Old Version 4

changes.mady.by.user Jared Whiklo

Saved on

compared with

New Version 5

changes.mady.by.user Jared Whiklo

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Because Fedora publishes modification events on a JMS topic using a local ActiveMQ broker, one can write custom listener applications to handle these various workflows. By default, the repository's JMS broker supports both the OpenWire and STOMP protocols, which means that it is possible to write client listeners or consumers in a wide variety of languages, including PHP, Python, Ruby and Java, among others.

Message Headers

By default, Fedora publishes events to a topic on a local broker. This topic is named "fedora"

The contents of the message header and body are documented here.

Camel Messaging

For simple message-consuming applications, writing special-purpose applications may be an excellent choice. In contrast, once a repository begins making use of more complex message-based workflows or when there are multiple listener applications to manage, many repositories use systems such as Apache Camel to simplify the handling of these messages.

...

Please note that the hostnames used for Fedora and Solr in the snippets above are arbitrary. It is quite likely that these systems will be deployed on separate hosts and that the Camel routes will be deployed on yet another host. Camel makes it easy to distribute applications and replicate data asynchronously across an arbitrarily large number of independent systems.

Message Headers

By default, Fedora publishes events to a topic on a local broker. This topic is named "fedora"

The contents of the message header and body are documented here.

The fcrepo component for Camel is configured to recognize these headers and act appropriately.

Supporting Queues

The default configuration is fine for locally-deployed listeners, but it can be problematic in a distributed context. For instance, if the listener is restarted while a message is sent to the topic, that message may be missed. Furthermore, if there is a networking hiccup between Fedora's local broker and the remote listener, that too can result in lost messages. Instead, in this case, a queue may be better suited.

...


Code Block
languagexml
titleactivemq.xml customization: supporting a queue/fedora endpoint
linenumberstrue
<destinationInterceptors>
  <virtualDestinationInterceptor>
    <virtualDestinations>
      <compositeTopic name="fedora">
        <forwardTo>
          <queue physicalName="fedora"/>
        </forwardTo>
      </compositeTopic>
    </virtualDestinations>
  </virtualDestinationInterceptor>
</destinationInterceptors>

Now a consumer can pull messages from a queue without risk of losing messages.

This configuration, however, will not allow any other applications to read from the original topic. If it is necessary to have /topic/fedora available to consumers, this configuration will be useful:

Code Block
languagexml
titleactivemq.xml customization: supporting topic and queue endpoints
linenumberstrue
<destinationInterceptors>
  <virtualDestinationInterceptor>
    <virtualDestinations>
      <compositeTopic name="fedora" forwardOnly="false">
        <forwardTo>
          <queue physicalName="fedora"/>
        </forwardTo>
      </compositeTopic>
    </virtualDestinations>
  </virtualDestinationInterceptor>
</destinationInterceptors>

Now, both /topic/fedora and /queue/fedora will be available to consumers.

Distributed Brokers

The above example will allow you to distribute the message consumers across multiple machines without missing messages, but it can also be useful to distribute the message broker across multiple machines. This can be especially useful if you want to further decouple the message producers and consumers. It can also be useful for high-availability and failover support.

ActiveMQ supports a variety of distributed broker topologies. To push messages from both the message queue and topic to a remote broker, this configuration can be used:

Code Block
languagexml
titleactivemq.xml customization: distributed brokers
linenumberstrue
<networkConnectors>
  <networkConnector name="fedora_bridge" dynamicOnly="true" uri="static:(tcp://remote-host:61616)">
    <dynamicallyIncludedDestinations>
      <topic physicalName="fedora"/>
      <queue physicalName="fedora"/>
    </dynamicallyIncludedDestinations>
  </networkConnector>
</networkConnectors>

Protocol Support

ActiveMQ brokers support a wide variety of protocols. If Fedora's internal broker is bridged to an external broker, please remember to enable the proper protocols on the remote broker. This can be done like so:

...


Each transportConnector supports many additional options that can be added to this configuration.

Deployment

Camel routes can be deployed in any JVM container. In order to deploy to Jetty or Tomcat, the route must be built as a WAR file. This command will get you started:

...

Now, routes can be deployed (and re-deployed) by simply copying JAR files or XML documents to $KARAF_HOME/deploy.

Fedora Camel Toolbox

The Fedora project distributes camel routes for several common repository tasks as part of the fcrepo-camel-toolbox project, for use with Karaf version 4.x. Additional information is available on the Integration Services page. Detailed installation instructions are available as part of the project README and follow this pattern:

Code Block
languagebash
titleKaraf Shell
# install fcrepo-camel-toolbox (as of v4.1.0)
$> feature:repo-add mvn:org.fcrepo.camel/fcrepo-camel-toolbox/4.1.0/xml/features
 
# install fcrepo-camel-toolbox (as of v4.5.0)
$> feature:repo-add mvn:org.fcrepo.camel/toolbox-features/4.5.0/xml/features
 
# display available features
$> feature:list | grep fcrepo
 
# install feature
$> feature:install fcrepo-indexing-triplestore


Monitoring Your Camel Routes

It is often useful to keep runtime statistics for your camel routes. Hawtio is a web console for monitoring your messaging infrastructure, and it can be deployed in any JVM container, including Karaf, Tomcat or Jetty.

...