Page History
...
ImageMagick Media Filters
Table of Contents | ||||||
---|---|---|---|---|---|---|
|
Info |
---|
As of DSpace 7.6, the ImageMagick media filter also supports creating thumbnails of video (MP4) files, provided that "ffmpeg" is installed locally. See instructions below. |
Overview
The ImageMagick Media Filters provide consistent, high quality thumbnails for image bitstreams, PDF bitstreams and PDF video (MP4) bitstreams.
These filters require a separate software installation of the conversion utilities (ImageMagick and Ghostscript: ImageMagick, Ghostscript (to support PDF thumbnails) and/or ffmpeg (to support MP4 thumbnails).
The media filters use the library im4java to invoke the conversion utilities. This library constructs a conversion command launches a sub-process to perform the generation of media files.
Installation
Before ImageMagick Media Filters can be used, you must setup ImageMagick (and optionally Ghostscript) as follows:
Install ImageMagick
...
on your server. The installation process differs based on your operating system. For example, on Debian/Ubuntu, it's similar to this:
Code Block apt-get install imagemagick
If you wish to generate PDF thumbnails, install Ghostscript
...
on your server. The installation process differs based on your operating system. For example, on Debian/Ubuntu, it's similar to this:
Code Block apt-get install ghostscript
(New in 7.6) If you wish to generate MP4 (video) thumbnails, install FFmpeg on your server. The installation process differs based on your operating system. For example, on Debian/Ubuntu, it's similar to this:
Code Block apt-get install ffmpeg
- The ImageMagick, Ghostscript, and FFmpeg
...
- executables should be accessible from the same directory (e.g.
/usr/bin
)- This directory MUST be defined in the
org.dspace.app.mediafilter.ImageMagickThumbnailFilter.ProcessStarter
configuration as describe below.
- This directory MUST be defined in the
DSpace Configuration
In the filter.plugins section of your dspace.cfg (or local.cfg file) file, uncomment specify the ImageMagick media filter definitionfilters you wish to use.
Code Block | ||
---|---|---|
| ||
# Make sure to always keep this plugin enabled if you want to support search within text documents filter.plugin = Text Extractor # NOTE: When "ImageMagick Image Thumbnail" is enabled, the default "JPEG Thumbnail" should NOT be enabled filter.plugins = ImageMagick Image Thumbnail # NOTE: When "ImageMagick PDF Thumbnail" is enabled, the default "PDFBox JPEG Thumbnail" should NOT be enabled # Requires Ghostscript to also be installed filter.plugins = ImageMagick PDF Thumbnail # New in 7.6, \ this will generate thumbnails from video files. # Requires ffmpeg to also be installed filter.plugins = ImageMagick Video Thumbnail |
This will activate the following settings which are already present in dspace.cfg :(these do NOT need to be added, as they already exist)
Code Block |
---|
plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickImageThumbnailFilter = ImageMagick Image Thumbnail, \ plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilter = ImageMagick PDF Thumbnail plugin.named.org.dspace.app.mediafilter.FormatFilter = org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilterImageMagickVideoThumbnailFilter = ImageMagick PDFVideo Thumbnail |
These media filters contain the several properties which can be configured.
...
The following property provides a path to the ImageMagick and GhostScript (convert), GhostScript (ghostscript), and ffmpeg utilities.
Code Block |
---|
org.dspace.app.mediafilter.ImageMagickThumbnailFilter.ProcessStarter = /usr/bin |
Supported file formats
Each of these ImageMagick filters has its own list of configurable file formats. The defaults are usually best, but may be updated if you have custom bitstream formats. These settings already exist in your dspace.cfg.
Code Block |
---|
filter.org.dspace.app.mediafilter.ImageMagickImageThumbnailFilter.inputFormats = BMP, GIF, PNG, JPG, TIFF, JPEG, JPEG 2000
filter.org.dspace.app.mediafilter.ImageMagickPdfThumbnailFilter.inputFormats = Adobe PDF
filter.org.dspace.app.mediafilter.ImageMagickVideoThumbnailFilter.inputFormats = Video MP4 |
Overwriting Existing Thumbnails
...
PDFs optimized for physical printing often use the CMYK color space. On the web, however, the de facto color system is sRGB. By default, DSpace's ImageMagick-based thumbnailing system will create thumbnails that use the same color space as the source PDF. Most web browsers are not able to correctly display images that use the CMYK color space, which leads to images with visibly inaccurate colors. Furthermore, the JSPUI currently does not support the display of thumbnails that use the CMYK color space (see DS-2834).
If you are using Ghostscript version 9 or above, it is possible for DSpace to correctly convert images from CMYK to sRGB by providing it with appropriate ICC color profiles to use during thumbnail creation. Default ones are provided by most Ghostscript installations (version 9 or above). The following configuration options tell DSpace where those ICC profiles are located.
...
Providing ICC profiles to ImageMagick is optional. If these configuration properties are unset, no profiles will be supplied to ImageMagick, and thumbnails produced from PDFs using the CMYK color space will also use CMYK. The transformation from CMYK to RGB is optional, but keep in mind that thumbnails using CMYK will cause problems if you use JSPUI and display thumbnails on browse and search pages (see DS-2834).
Override ImageMagick Default Density
It is possible to override ImageMagick's default density of 72 DPI when creating PDF thumbnails. This increases the quality of resulting thumbnails at the expense of slightly longer execution times and higher memory usage. Any integer over 72 will help, but recommend 144 for a "2x" supersample. See the following configuration option in dspace.cfg
:
Code Block |
---|
# org.dspace.app.mediafilter.ImageMagickThumbnailFilter.density = 144 |
The effect is most notable on PDFs with a lot of text, gradients, or curved lines. See the pull request implementing this feature for more information and comparisons.
Additional Customization
...
Note: The PDF thumbnail generator is hard-coded to generate a thumbnail from the first page of the PDF.
Possible Errors / Issues
"convert.im6: not authorized" errors
On Ubuntu (possibly other OSes), you may see errors like these when attempting to generate PDF thumbnails:
Code Block |
---|
ERROR filtering, skipping bitstream:
Item Handle: 1234/5678
Bundle Name: ORIGINAL
File Size: 30406135
Checksum: c1df4b3a4755e9bed956383b61fc5042 (MD5)
Asset Store: 0
org.im4java.core.CommandException: org.im4java.core.CommandException: convert.im6: not authorized `/tmp/impdfthumb6294641076817830415.pdf' @ error/constitute.c/ReadImage/454.
OR
org.im4java.core.CommandException: convert-im6.q16: attempt to perform an operation not allowed by the security policy 'PDF' @ error/constitute.c/IsCoderAuthorized/421
|
These may be caused by a change in your ImageMagick policy configuration on your server.
In Ubuntu, the default "policy.xml" was recently updated to exclude all Ghostscript formats (including PDF, PS, etc). See this ticket: https://bugs.launchpad.net/ubuntu/+source/imagemagick/+bug/1796563
- This exclusion was implemented to workaround a security vulnerability in Ghostscript reported here: https://www.kb.cert.org/vuls/id/332928
- According to that vulnerability report, this was patched in Ghostscript v9.24 (or above)
The newly added lines in the /etc/ImageMagick/policy.xml
are these ones:
Code Block |
---|
<!-- disable ghostscript format types -->
<policy domain="coder" rights="none" pattern="PS" />
<policy domain="coder" rights="none" pattern="EPS" />
<policy domain="coder" rights="none" pattern="PDF" />
<policy domain="coder" rights="none" pattern="XPS" /> |
To fix the error above requires you to re-enable ImageMagick to process Ghostscript format types. That can be done by simply commenting out those new "policy" lines in the configuration file (surround them with <!-- and --> to comment out)
Be aware that you MUST ensure you are running Ghostscript v9.24 or later to ensure that you are not at risk for the above security vulnerability in older versions of Ghostscript.
"convert-im6.q16: cache resources exhausted" errors
On Debian, and possibly other OSes, you may see errors like these when attempting to generate video thumbnails (especially if video files are big):
Code Block |
---|
File: video.mp4.jpg
ERROR filtering, skipping bitstream:
Item Handle: 1234/5678
Bundle Name: ORIGINAL
File Size: 146761357
Checksum: 735ceb1b6b249afc84a5bb1b87ae0c02 (MD5)
Asset Store: 0
org.im4java.core.CommandException: convert-im6.q16: cache resources exhausted `/tmp/magick-64dziU-1nQJjQHZYu4_R1fFP4l9en5iL.pam' @ error/cache.c/OpenPixelCache/4095. |
These may be caused by too conservative resource policies in your policy.xml
file. As an example, default values are located at /etc/ImageMagick-6/policiy.xml
in Debian 11 (Bullseye), and are:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<policymap>
<!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
<policy domain="resource" name="memory" value="256MiB"/>
<policy domain="resource" name="map" value="512MiB"/>
<policy domain="resource" name="width" value="16KP"/>
<policy domain="resource" name="height" value="16KP"/>
<!-- <policy domain="resource" name="list-length" value="128"/> -->
<policy domain="resource" name="area" value="128MP"/>
<policy domain="resource" name="disk" value="1GiB"/> |
To avoid the cache resources exhausted
error, try increasing the resource limits policies. You may want to start by increasing the memory
and disk
policies (disk cache is used when the memory limit is reached). The actual values have to be adjusted depending on the size of your video bitstreams and the actual resources available in your installation. For example:
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
<policymap>
<!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
<policy domain="resource" name="memory" value="4GiB"/> <!-- memory limit increased from 256MiB to 4GiB -->
<policy domain="resource" name="map" value="512MiB"/>
<policy domain="resource" name="width" value="16KP"/>
<policy domain="resource" name="height" value="16KP"/>
<!-- <policy domain="resource" name="list-length" value="128"/> -->
<policy domain="resource" name="area" value="128MP"/>
<policy domain="resource" name="disk" value="4GiB"/> <!-- disk limit increased from 1GiB to 4GiB --> |
For a detailed description of the ImageMagick limits, see https://imagemagick.org/script/command-line-options.php#limit.
Once the limits are properly set, a successful execution of the filter should show a message similar to:
Code Block |
---|
File: video.mp4.jpg
FILTERED: bitstream 12345678-abcd-efgh-ijkl-1234567890ab (item: 1234/5678) and created 'video.mp4.jpg' |