Images in Java
Sections in this page
2012 Apr 17: this page now matches the radically restructured GRIP since version 12.1.1
Memory requirements
Let's get the size issue out of the way first. We have seen in another section that a typical photographic image these days occupies around 100 megabytes (Mb) or more of memory when it is loaded (uncompressed) for processing. That has an implication for the JVM because by default that will allocate only 64Mb to an application when it starts and increase that in steps of 16Mb up to 128Mb when necessary. If it needs any more it will throw an OutOfMemoryError, which is fatal. To prevent this it is necessary to specify initial and maximum memory sizes in the command line for running the application. Eg,
java -Xms256m -Xmx1024m classname
would start with 256 Mb and be able to expand to 1024 Mb (= 1 Gb).
The class representing 8- or 16-bit images
The standard Java class to instantiate for holding an image in memory is java.awt.image.BufferedImage. The Oracle API documentation shows that this can hold various kinds of image with different numbers of channels and 8 or 16 bits per channel. When loading an image from a disc file the type is set for us (see the image reading and writing page). It is important to ensure that when we come to display (a scaled down version of) the BufferedImage type it is compatible with the display hardware. Otherwise it can take a very long time for an image to be displayed or repainted. If we create an empty image it is necessary to use
java.awt.GraphicsEnvironment ge =
java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment ();
java.awt.GraphicsDevice gd = ge.getDefaultScreenDevice ();
java.awt.GraphicsConfiguration gc = gd.getDefaultConfiguration ();
java.awt.image.BufferedImage bim = gc.createCompatibleImage (width, height);
Similarly, when cloning an image or creating a destination image for an operation on an already loaded image:
BufferedImage dstBim = new BufferedImage (srcBim.getColorModel (),
srcBim.getRaster ().createCompatibleWritableRaster (dstWd, dstHt),
srcBim.isAlphaPremultiplied (), null);
In fact all of these details are taken care of in relevant classes in GRIP's API; see in particular net.grelf.image.Image and net.grelf.grip.ImPane.
Rasters
Contained within every BufferedImage object is a java.awt.image.WritableRaster object containing the multi-channel arrays of pixel values. The getWritableRaster () method of BufferedImage gets a reference to the raster. If you only want to read the pixel values and want to avoid inadvertently changing them, use instead the getRaster () method of BufferedImage, which gets a read-only reference to the raster, of the super-type java.awt.image.Raster.
We use a common pattern throughout our software for reading and writing pixel data in rasters. Here is one of the simplest instances of the pattern, similar to code in class Image8or16Base:
/** Invert the contrast in an image. */
public static void invert (java.awt.image.BufferedImage bim)
{
java.awt.image.WritableRaster wr = bim.getRaster ();
int maxLevel = Image8or16Base.getMaxLevel (bim);
int nBands = Image8or16Base.getNBands (bim);
int [] px = new int [nBands];
for (int y = 0; y < wr.getHeight (); y++)
{
for (int x = 0; x < wr.getWidth (); x++)
{
wr.getPixel (x, y, px);
for (int b = 0; b < nBands; b++)
{
px [b] = maxLevel - px [b];
}
wr.setPixel (x, y, px);
} }
} // invert
The classes in GRIP for images of up to 64-bits per channel
The interface net.grelf.image.Image currently has 4 implementations (in the same package): Image8, Image16, Image32 and Image64. Image64 holds pixels as 64-bit floating point numbers per channel but the others use integer pixel values.
Image 8 and Image16 use standard Java BufferedImages internally, as described in the previous section. This is because the standard javax.imageio class effectively handles the loading and saving of such images and there is no point in copying them to a different data structure after loading.
For images deeper than 16 bits per channel, GRIP has its own data structure. All objects in Java have a 12-byte overhead. In a multidimensional array each sub-array is an object. So if you naively create an image as, say, int [x][y][channel] then there is an overhead of 12 bytes per pixel plus 12 bytes per row, so the 3-dimensional array requires several times the amount of memory that it should. GRIP does much better by storing such an image as int [channel][x + width * y].
Such an image is scanned by using code such as this (within the Image32 class itself, or in a subclass):
/** Invert the contrast in this Image32. */
public void invert ()
{
int maxLevel = this.extremes.high;
for (int y = 0, i = 0; y < getHeight (); y++)
{
for (int x = 0; x < getWidth (); x++, i++)
{
for (int b = 0; b < getNBands (); b++)
{
data [b][i] = maxLevel - data [b][i];
} } }
int min = maxLevel - this.extremes.high;
int max = maxLevel - this.extremes.low;
this.extremes = new net.grelf.image.RangeInt (min, max);
} // invert
Notice that our Image classes hold a pair of numbers (a RangeInt or a RangeDouble) which are the minimum and maximum levels actually occurring, across all channels. They are updated whenever a processing operation is performed, as shown in the example above. Unlike BufferedImage, it is not assumed that the full theoretical range (0..Integer.MAX_VALUE, or Double.MIN_VALUE..Double.MAX_VALUE) is used. This makes it easier and quicker to extract meaningful histograms, for example.
Summary of image classes in GRIP
All in package net.grelf.image unless otherwise stated. The "Base" classes collect code that would otherwise have to be duplicated in their subclasses.
interface Image
abstract class ImageBase
abstract class Image8or16Base extends ImageBase implements Image
class Image8 extends Image8or16Base implements Image
class Image16 extends Image8or16Base implements Image
class Image32 extends ImageBase implements Image
class net.grelf.grip.Accumulator32 extends Image32
class Image64 extends ImageBase implements Image
class net.grelf.grip.Accumulator64 extends Image64
Processing any kind of image
For processing from another class (without direct access to the data array) you would write in the following style which avoids you having to know whether internally the image is a BufferedImage or in our own data structure.
/** Invert the contrast in an Image. */
public void invert (net.grelf.image.Image image)
{
int maxLevel = image.getRange ().high;
// getRange () is quick: does not scan the image
int nBands = image.getNBands ();
int [] px = new int [nBands];
for (int y = 0; y < image.getHeight (); y++)
{
for (int x = 0; x < image.getWidth (); x++)
{
px = image.getPixel (x, y);
for (int b = 0; b < nBands; b++)
{
px [b] = maxLevel - px [b];
}
image.setPixel (x, y, px);
} }
image.getOverallRange ();
// Slower: rescans the image and sets the extremes.
} // invert
In reality most of our image processing methods find minimum and maximum result values in the main pixel loop so they do not have to do a second scan at the end. The example above has been written more simply for clarity rather than for optimum performance.
Displaying an image
An instance of the class Image, introduced in the previous section, is wrapped in a net.grelf.grip.ImPane which handles all zooming, panning and scrolling. It in turn is wrapped in a net.grelf.grip.ImFrame which is a javax.swing.JFrame and displays itself as such. More details of the structure of an ImFrame can be seen on the next page.