Page History

...

ImageMagick Media Filters

Overview

The ImageMagick Media Filters provide consistent, high quality thumbnails for image bitstreams, PDF bitstreams and PDF video (MP4) bitstreams.

These filters require a separate software installation of the conversion utilities (ImageMagick and Ghostscript: ImageMagick, Ghostscript (to support PDF thumbnails) and/or ffmpeg (to support MP4 thumbnails).  

The media filters use the library im4java to invoke the conversion utilities.  This library constructs a conversion command launches a sub-process to perform the generation of media files.

...

1. Install ImageMagick on your server.  The installation process differs based on your operating system.  For example, on Debian/Ubuntu, it's similar to this:

   Code Block
   apt-get install imagemagick

   
   

2. If you wish to generate PDF thumbnails, install Ghostscript on your server. The installation process differs based on your operating system.  For example, on Debian/Ubuntu, it's similar to this:

   Code Block
   apt-get install ghostscript

   
   

3. (New in 7.6) If you wish to generate MP4 (video) thumbnails, install FFmpeg on your server. The installation process differs based on your operating system.  For example, on Debian/Ubuntu, it's similar to this:

   Code Block
   apt-get install ffmpeg

   
   

4. The ImageMagick, Ghostscript, and Ghostscript FFmpeg executables should be accessible from the same directory (e.g. /usr/bin)
   1. This directory MUST be defined in the org.dspace.app.mediafilter.ImageMagickThumbnailFilter.ProcessStarter configuration as describe below.

DSpace Configuration

In the filter.plugins section of your dspace.cfg file, uncomment (or local.cfg) file, specify the ImageMagick media filter definitionfilters you wish to use.

# Make sure to always keep this plugin enabled if you want to support search within text documents
filter.plugin = Text Extractor

# NOTE: When "ImageMagick Image Thumbnail" is enabled, the default "JPEG Thumbnail" should NOT be enabled
filter.plugins = ImageMagick Image Thumbnail

# NOTE: When "ImageMagick PDF Thumbnail" is enabled, the default "PDFBox JPEG Thumbnail" should NOT be enabled
# Requires Ghostscript to also be installed
filter.plugins = ImageMagick PDF Thumbnail

# New in 7.6, \ this will generate thumbnails from video files.
# Requires ffmpeg to also be installed
filter.plugins = ImageMagick Video Thumbnail   

This will activate the following settings which are already present in dspace.cfg :(these do NOT need to be added, as they already exist)

   plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickImageThumbnailFilter = ImageMagick Image Thumbnail, \
   
plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilter = ImageMagick PDF Thumbnail
plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilterImageMagickVideoThumbnailFilter = ImageMagick PDFVideo Thumbnail

These media filters contain the several properties which can be configured.

...

The following property provides a path to the ImageMagick and GhostScript (convert), GhostScript (ghostscript), and ffmpeg utilities.

org.dspace.app.mediafilter.ImageMagickThumbnailFilter.ProcessStarter = /usr/bin

Supported file formats

Each of these ImageMagick filters has its own list of configurable file formats.  The defaults are usually best, but may be updated if you have custom bitstream formats.  These settings already exist in your dspace.cfg.

filter.org.dspace.app.mediafilter.ImageMagickImageThumbnailFilter.inputFormats = BMP, GIF, PNG, JPG, TIFF, JPEG, JPEG 2000
filter.org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilter.inputFormats = Adobe PDF
filter.org.dspace.app.mediafilter.ImageMagickVideoThumbnailFilter.inputFormats = Video MP4

Overwriting Existing Thumbnails

...

ERROR filtering, skipping bitstream:
        Item Handle: 1234/5678
        Bundle Name: ORIGINAL
        File Size: 30406135
        Checksum: c1df4b3a4755e9bed956383b61fc5042 (MD5)
        Asset Store: 0
org.im4java.core.CommandException: org.im4java.core.CommandException: convert.im6: not authorized `/tmp/impdfthumb6294641076817830415.pdf' @ error/constitute.c/ReadImage/454.

OR
 
org.im4java.core.CommandException: convert-im6.q16: attempt to perform an operation not allowed by the security policy 'PDF' @ error/constitute.c/IsCoderAuthorized/421

These may be caused by a change in your ImageMagick policy configuration on your server. 

...

• This exclusion was implemented to workaround a security vulnerability in Ghostscript reported here: https://www.kb.cert.org/vuls/id/332928This vulnerability looks like it may have been fixed
• According to that vulnerability report, this was patched in Ghostscript v9.25: https://www.ghostscript.com/doc/9.25/News.htm (Still pending verification – see launchpad ticket linked 24 (or above)

The newly added lines in the /etc/ImageMagick/policy.xml are these ones:

<!-- disable ghostscript format types -->
<policy domain="coder" rights="none" pattern="PS" />
<policy domain="coder" rights="none" pattern="EPS" />
<policy domain="coder" rights="none" pattern="PDF" />
<policy domain="coder" rights="none" pattern="XPS" />

If you wish To fix the error above requires you to re-enable ImageMagick to process Ghostscript format types, you can simply comment out those . That can be done by simply commenting out those new "policy" lines in the configuration file .  Be aware that, as the fix to this vulnerability is not yet verified, you should only do so at your own risk.(surround them with <!-- and --> to comment out)

Be aware that you MUST ensure you are running Ghostscript v9.24 or later to ensure that you are not at risk for the above security vulnerability in older versions of Ghostscript.

"convert-im6.q16: cache resources exhausted" errors

On Debian, and possibly other OSes, you may see errors like these when attempting to generate video thumbnails (especially if video files are big):

File: video.mp4.jpg
ERROR filtering, skipping bitstream:
        Item Handle: 1234/5678
        Bundle Name: ORIGINAL
        File Size: 146761357
        Checksum: 735ceb1b6b249afc84a5bb1b87ae0c02 (MD5)
        Asset Store: 0
org.im4java.core.CommandException: convert-im6.q16: cache resources exhausted `/tmp/magick-64dziU-1nQJjQHZYu4_R1fFP4l9en5iL.pam' @ error/cache.c/OpenPixelCache/4095.

These may be caused by too conservative resource policies in your policy.xml file. As an example, default values are located at /etc/ImageMagick-6/policiy.xml in Debian 11 (Bullseye), and are:

<policymap>
  <!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
  <policy domain="resource" name="memory" value="256MiB"/>
  <policy domain="resource" name="map" value="512MiB"/>
  <policy domain="resource" name="width" value="16KP"/>
  <policy domain="resource" name="height" value="16KP"/>
  <!-- <policy domain="resource" name="list-length" value="128"/> -->
  <policy domain="resource" name="area" value="128MP"/>
  <policy domain="resource" name="disk" value="1GiB"/>

To avoid the cache resources exhausted error, try increasing the resource limits policies. You may want to start by increasing the memory and disk policies (disk cache is used when the memory limit is reached). The actual values have to be adjusted depending on the size of your video bitstreams and the actual resources available in your installation. For example:

<policymap>
  <!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
  <policy domain="resource" name="memory" value="4GiB"/> <!-- memory limit increased from 256MiB to 4GiB -->
  <policy domain="resource" name="map" value="512MiB"/>
  <policy domain="resource" name="width" value="16KP"/>
  <policy domain="resource" name="height" value="16KP"/>
  <!-- <policy domain="resource" name="list-length" value="128"/> -->
  <policy domain="resource" name="area" value="128MP"/>
  <policy domain="resource" name="disk" value="4GiB"/> <!-- disk limit increased from 1GiB to 4GiB -->

For a detailed description of the ImageMagick limits, see https://imagemagick.org/script/command-line-options.php#limit.

Once the limits are properly set, a successful execution of the filter should show a message similar to:

File: video.mp4.jpg
FILTERED: bitstream 12345678-abcd-efgh-ijkl-1234567890ab (item: 1234/5678) and created 'video.mp4.jpg'