We use HTML5 Canvas elements for a number of features in our client apps, and we wanted to know once and for all – what would be the best format for us to export our results to?
The toDataURL method of Canvas can handle a number of formats. In Chrome it can export “image/png”, “image/jpeg” with a quality score of 0.0 to 1.0 in 0.1 increments, and “image/webm” in the same increments.
Rather than provide you with code or algorithms for determining “which is best” (something that is super subjective) I’m going to describe how we took decisions on what to benchmark and how. Hopefully we will give you the right questions to ask when you’re doing your own tests of this output.
1. What do we want to optimise?
There are three metrics you can measure with these compressions: how long it takes, how big the file is, and how good the quality is. The first two points aren’t too difficult, as you can do it in code and just get numbers out at the end. The third is significantly more subjective.
So we decided to do a comparison of file size and time, then look at the results from the most balanced solution and see what the quality was like.
2. How are we going to get the data?
Repeating tests is key in benchmarking, to ensure you can average out any anomalies. So we ran each compression 10 times and averaged out the time taken to do the whole lot. This wasn’t a huge number of tests, but we weren’t doing much else on the machine, and it was taking minutes to do each run – a suitable compromise.
Image compression is also nearly entirely subjective – it depends on what you’re compressing and how much you want to be able to make out afterwards. We had a very well defined class of images we wanted to compress, so we took a sample of 15 of those (50% of our initial test set) to run the entire set of tests on.
3. Enough questions, gimme the data
Here’s an overall plot of file size (x-axis) vs compression time (y-axis):
Bearing in mind that we were using 15 very different test images, and 21 different encoding types for each of those, that spread isn’t very far.
The outliers along the bottom are the PNGs – consistently quick, but the file sizes outstripped everything else. WebM and JPEG are reasonably well grouped together and display nonlinear characteristic in file size as the compression level is reduced.
That’s a lot of points to take in, and with labels it’s basically incomprehensible. This graph shows the averages of the time and size values, along with labels for the compression type used:
That makes a lot more sense. From this chart we can clearly see just how far out JPEG (at full quality) and PNG are in the size scale. We can also more clearly see the distribution of WebM vs JPEG – WebM is consistently faster at all quality levels (and the gap increases as the quality figure goes up).
4. Making the final decision
WebM is still a little too exotic to consider deploying at a large scale on the web for us. So we decided to go for JPEG 0.5 – the right compromise between speed (our least urgent measure) and size (our number 1 consideration), and the resulting image is of sufficiently high quality for us.
Bear in mind that these metrics were for one machine using only Chrome (the latest version) but hopefully the techniques employed will help you ensure that canvas image exports achieve the right results that you need.