Large Image Processing with ImageMagick convert – Need More Throughput

imageimagemagickthroughput

I am converting some largish images from a multi-image (pyramidal) tif to png format. The salient parts of the report from "identity -verbose" on the largest image are here:

Image: 
  Format: TIFF (Tagged Image File Format)
  Class: DirectClass
  Geometry: 72224x64080+0+0
  Resolution: 72x72
  Print size: 1003.11x890
  Units: PixelsPerInch
  Type: TrueColor
  Base type: TrueColor
  Endianess: MSB
  Colorspace: RGB
  Depth: 8-bit
  Channel depth:
    red: 8-bit
    green: 8-bit
    blue: 8-bit
    ...
  Page geometry: 72224x64080+0+0
    ...
  Scene: 2 of 12
  Compression: JPEG
  Orientation: TopLeft
  Properties:
    ...
  Filesize: 1.389GBB
  Number pixels: 4.6281GB
  Pixels per second: 5.516MB
  User time: 218.277u
  Elapsed time: 13:60.020
  Version: ImageMagick 6.7.1-0 2011-07-06 Q16 http://www.imagemagick.org

I am intending to use deepzoom composer to produce input for the Silverlight multiscaleimage control with this image. My question is how do I bring my system to its knees while processing these images with ImageMagick – it is taking too long to convert them. I have looked at a few articles, but I can't seem to get anywhere.

Some system and other related information:

 OS: Windows 7 64 bit.
 CPU: Intel Core2 Duo E7300 @ 2.66, 2.67
 RAM: 4.0 GB
 PAGEFILE: 8-12GB on non-OS disk
 "MAGICK_TMPDIR": Yet another empty, non-os disk with 140GB available.

Here is the result of "identify -list resource":

 File         Area       Memory          Map         Disk    Thread
 ------------------------------------------------------------------
 1536     4.1582GB    15.491GiB    30.981GiB    unlimited         2

I am running this command to extract the image referenced above:

convert "myFN.tif[2]" -limit file 8192GB -limit thread 32 "myFN%d.png"

Adding the two limit values did not seem to make a difference. When I run this, I average about 10% CPU utilization and have a pagefile commit size of 3BG. I can barely tell that it is running.

Q1) Is there anything else I can do to get ImageMagick to use more system resources? Most of the "large image" links I have found are asking the opposite question.

Q2) Changing "policy.xml" values (such as files) located here:
C:\Program Files\ImageMagick-6.7.1-Q16\www\source
did not seem to affect anything – the changes did not show up in the next "identify -list resource." Is there a trick to this?

Q3) Any other hints or ideas for this task?

Thanks,
David

Best Answer

libvips can convert pyramidal tiff directly into deepzoom pyramids. It's free, very fast and doesn't need much memory.

For example, I see:

$ vipsheader vips-pyr.tif 
vips-pyr.tif: 18008x22764 uchar, 3 bands, srgb, tiffload
$ time vips dzsave vips-pyr.tif x.zip
real    0m9.763s
user    0m19.700s
sys 0m4.644s
peak memory: 180mb

That's a 20,000 x 20,000 pyramidal tiff converted to deepzoom in under 10 seconds on a small laptop. It's writing a zip file containing the pyramid, so you can upload to a server immediately. Memory use scales with image width, so it'll do very large images --- I regularly process 250,000 x 250,000 pixel slides.

There's a chapter in the docs introducing dzsave.

Related Topic