Using AdMob for Windows Phone 7 XNA for Silverlight developers: Part 7 - Collision detection

Improving the image upload sample

Published on Thursday, March 17, 2011 12:38:00 AM UTC in Programming

In my extensive post about how you can upload images to an SQL database from your Silverlight client (read it here), I mentioned some potential problems with the amount of data that is being uploaded. The issue arises because when you load an (compressed) image from disk, it is converted to an uncompressed bitmap in memory. Storing that image in your database can result in data that in extreme cases can be up to 100x or even more bigger than the original image. In this post, I'll show you how you can improve the code to achieve a much better result with regards to that.

Prerequisites

I'm going to use the features of the ImageTools project on Codeplex to improve the image decoding and encoding process. The advantages of using such a library is that you increase the number of supported source formats (the user then can also select e.g. BMP or GIF images), and you have some control over how images are encoded. For example, you can set a compression/quality level for the JPEG encoder in that project.

You can either download the latest release (0.3) of the library from last October, or grab the current source code and build the assemblies yourself. Please note that compiling is not a straight forward process due to the used code contracts in the project. Unless you really need the bug fixes which apparently have been added since October using the pre-compiled binaries is much easier. In any case, you will have a set of assemblies needed for our project:

  • ImageTools: the core assembly
  • ImageTools.Utils: contains extension methods for the conversion process
  • ImageTools.IO.Bmp: BMP encoder/decoder
  • ImageTools.IO.Gif: GIF encoder/decoder
  • ImageTools.IO.Jpeg: JPEG encoder/decoder
  • ImageTools.IO.Png: PNG encoder/decoder
  • ICSharpCode.SharpZLib.Silverlight: a helper library required to encode PNGs

We are not using the other assemblies like ImageTools.Controls in our sample. It doesn't matter where the above assemblies are located on your hard disk, but you need to add a reference to them in the existing "ImageUploadSample" project from last time. I like to keep that kind of assemblies in a folder in my solution so it is more portable and can be compiled on other machines easily without fixing references or downloading separate packages first, but that is not a requirement.

Creating the image converter

If you recall the sample from the previous post you'll remember that the central place for the image conversion process were two methods we added to the automatically generated client-side proxy class for our database entity. Those two methods were responsible for both creating images from byte arrays as well as the other way round. Since this logic will become a little more complex now and have additional features added to optimize the conversion, I'd like to refactor that a bit. To this end, I create a new class ImageConverter:

/// <summary>
/// Decodes BMP, GIF, PNG and JPEG images from byte arrays
/// and is able encode bitmaps to byte arrays by choosing between
/// the most suitable format for minimizing file size (PNG or JPEG).
/// </summary>
public class ImageConverter
{
    private PngEncoder _pngEncoder;
    private JpegEncoder _jpegEncoder;

    /// <summary>
    /// Initializes a new instance of the <see cref="ImageConverter"/> class.
    /// </summary>
    public ImageConverter()
    {
        // add all available decoders
        Decoders.AddDecoder<BmpDecoder>();
        Decoders.AddDecoder<GifDecoder>();
        Decoders.AddDecoder<PngDecoder>();
        Decoders.AddDecoder<JpegDecoder>();

        // create a new PNG encoder and configure it
        // to write compressed images
        _pngEncoder = new PngEncoder();
        _pngEncoder.IsWritingUncompressed = false;

        // create a new JPEG encoder and configure it
        // to use a quality setting of 75
        _jpegEncoder = new JpegEncoder();
        _jpegEncoder.Quality = 75;
    }

When it is created, the image converter first adds all decoders the image tools project offers at the moment to the list of available decoders. This ensures we offer the user the maximum amount of comfort because they can now choose from a variety of source formats. (Note: in my tests, the PNG decoder choked on one of the PNG sample images I have used. I wasn't able to figure out what the reason was - opening and re-saving the image in a different image software solved the problem).

The rest of the code creates both a PNG encoder and a JPEG encoder and sets some of the available parameters, for example the quality setting of the JPEG encoder. These encoders will be used later to optimize the image size for storage.

Converting a byte array to an image

The "ExtendedImage" class of the image tools project uses asynchronous loading of images. This makes perfect sense when you're loading remote images because Silverlight's web access is asynchronous too; however, we are only decoding images from local data, and even Silverlight itself does that synchronously (as we've learned in the first post). I'm not sure what has driven this decision for the image tools library design, but since changing the already existing sample (which is completely built upon Silverlight's default behavior) to an asynchronous pattern would have required to rewrite the whole chain of loading local images I have turned that process into a synchronous method:

#region From byte array

private AutoResetEvent resetEvent = new AutoResetEvent(false);

/// <summary>
/// Synchronously converts a byte array to a writeable bitmap.
/// </summary>
/// <param name="buffer">The source buffer to use.</param>
/// <returns>A bitmap that contains the decoded image.</returns>
public WriteableBitmap FromByteArray(byte[] buffer)
{
    // create a stream from the source buffer
    MemoryStream ms = new MemoryStream(buffer);

    // this construct synchronizes the image creation as follows:
    // both the completed and failed events are handled and set
    // the reset event. After the code has set the source to the
    // memory stream, it waits for the reset event so it can
    // continue when the image creation has finished or failed.
    ExtendedImage image = new ExtendedImage();
    image.LoadingCompleted += (o, e) => resetEvent.Set();
    image.LoadingFailed += (o, e) => resetEvent.Set();
    image.DelayTime = 0;
    image.SetSource(ms);

    // wait for the loaded image
    resetEvent.WaitOne();

    // if loading has failed, throw an exception
    if (!image.IsFilled)
    {
        throw new Exception("Error while loading image.");
    }

    // convert the loaded image to a bitmap
    WriteableBitmap bitmap = image.ToBitmap();
    return bitmap;
}

#endregion

As you can see the actual conversion is very straight forward, because the library takes care of the heavy lifting in the conversion process: create the image and set the source to your memory stream, and once the image loading has completed, call the "ToBitmap" method - done.

Converting an image to a byte array

For the other way round, we are using the encoders that we have created in the constructor of the image converter class. Interestingly, unlike the decoding the encoders work synchronously out of the box. This is a mild surprise because one would think that encoding is more expensive.

Anyway, the encoding tries to do a bit of optimization here: it converts the given image to both PNG and JPEG formats and then decides what result to use, based on the final size. Since there is no way to predict the resulting file size reliably to choose between one of the two before actually encoding the image, and because the encoding is sufficiently fast, this is an acceptable way.

#region To byte array (optimized)

/// <summary>
/// Synchronously converts a bitmap to a byte array.
/// The used format can be JPEG or PNG, depending on which one
/// results in a smaller file size.
/// </summary>
/// <param name="bitmap">The bitmap to encode.</param>
/// <returns>The encoded image either in PNG or JPEG format.</returns>
public byte[] ToByteArrayOptimized(WriteableBitmap bitmap)
{
    ExtendedImage image = bitmap.ToImage();

    // encode to jpeg
    MemoryStream jpegStream = new MemoryStream();
    _jpegEncoder.Encode(image, jpegStream);

    // encode to png
    MemoryStream pngStream = new MemoryStream();
    _pngEncoder.Encode(image, pngStream);

    // decide which one we should use
    MemoryStream formatToUse = jpegStream.Length < pngStream.Length ? jpegStream : pngStream;
    byte[] result = formatToUse.ToArray();

    // done
    return result;
}

#endregion

Using the new image converter

To use the above new class, we only need to make a few changes to the methods we had already added to the client side proxy class of our database entity type last time. For example, it now obviously delegates the conversion to the image converter, which is declared as static field:

private static ImageConverter _converter = new ImageConverter();

The "RefreshBitmap" method of the entity type which is invoked when the byte array has been retrieved from the service to create the bitmap now simply uses this converter:

public void RefreshBitmap()
{
    // if we don't have image data, we cannot create a bitmap
    if (DatabaseImageData == null || DatabaseImageData.Data == null)
    {
        _bitmap = null;
    }
    else
    {
        // create a new bitmap and restore it from the binary data
        _bitmap = _converter.FromByteArray(DatabaseImageData.Data);
    }

    RaisePropertyChanged("Bitmap");
}

When the user selects a local image that should be stored in the database, we do the following:

  • Use the converter to create the actual bitmap from the byte array that is the selected image.
  • Let the converter optimize the image size using the above method.
  • Either take the optimized raw image data, or the original image data (whichever is smaller) to store it in the database entity.

It is absolutely possible that the attempt to optimize the image size results in a bigger image than before. For example, the user could have selected an image that is a heavily compressed JPEG, and the compression setting of our encoder then may actually increase the size.

On the other hand, there are several cases when we will be able to reduce the size of the image. Examples for this are: the user selects a BMP, a JPEG that has an unreasonable high quality setting, or a PNG that should better be encoded as JPEG due to its content.

public void SetBitmap(byte[] buffer)
{
    if (buffer == null)
    {
        throw new ArgumentNullException("buffer");
    }

    // create an actual image from the byte array
    _bitmap = _converter.FromByteArray(buffer);

    // update our own properties
    Width = _bitmap.PixelWidth;
    Height = _bitmap.PixelHeight;

    // get or create related image data entity
    DatabaseImageData imageData = DatabaseImageData;
    if (imageData == null)
    {
        imageData = new DatabaseImageData();
        DatabaseImageData = imageData;
    }

    // this is the optimizing step:
    // let the converter convert the image back to a byte array,
    // it tries to optimize the resulting size internally.
    // if however the original buffer is smaller than the optimized
    // converted image, we keep that one instead.
    var tempBuffer = _converter.ToByteArrayOptimized(_bitmap);

    if (tempBuffer.Length < buffer.Length)
    {
        // optimization succeeded (this will be the case for example
        // when the user selected a BMP, a poorly compressed JPEG
        // or a PNG that would better be compressed using JPEG.
        imageData.Data = tempBuffer;
    }
    else
    {
        // store the original image instead.
        // this happens if recompressing the original selection
        // results in a bigger file size, for example if the original
        // compression quality of a JPEG is worse that what the 
        // converter uses internally.
        imageData.Data = buffer;
    }

    // notify the outside world
    RaisePropertyChanged("Bitmap");
}

That's it! The byte array used as argument in this last method directly comes from the file the user has selected for upload. The rest of the project does not require any changes.

The result

The sample application from the previous post passed the uncompressed in-memory bitmaps through to the database. Even for a relatively small image with a resolution of 640x480 pixels, this always resulted in a size of 1,228,800 bytes (~1.17 MB, 640x480x4 bytes per pixel). The same image compressed as JPEG or PNG can, depending on the content of course, achieve a typical size of maybe 25 - 150 KBytes, thus saving a minimum amount of space of ~90% while still preserving a decent quality.

I hope this post and the complete source code you can download below gives you some ideas of how you can improve and tune the sample from the first post on the topic to get the best results for your project and to both increase performance and minimize the required storage space. Let me know if you have any suggestions or questions.

Download source code

Tags: Entity Framework · Image Tools · RIA Services · Silverlight