Introduction

Testing is fundamental for the success of every software projects; however, if possible, this is even more true for jrawio, given that there is no documentation for camera raw formats and the huge variations among camera models even from the same manufacturer.

jrawio is developed in Test Driven mode (TDD):

  • When a new format/model must be supported, we first need a sample file, then we write a test on it, and when the test passes the new model is declared as “supported, tested”. The development is done in a separate branch, so it is visible to the official release only when the work has been completed.
  • When a bug is discovered, we also need a sample file that triggers the problem; the development proceeds in the same way as above described, with the difference than the camera is still declared as “supported, tested” during the development (as there are other passing tests related to that model).

Test files

Finding sample files has been traditionally a problem in the past; Tidalwave can't afford to buy a camera model of each brand, neither is conceivable to have a community made of a large number of committers, each maintaining the format for his/her camera. In the very first years of jrawio, we used samples randomly published in the internet, found thanks to googling or pointers provided in forums, etc. We faced with a serious problem (that also had an impact to jrawio development), as most of those samples disappeared after a few months (for random reasons, we suppose that most of the times the original publisher closed his website). Those files weren't released with any public license and could not redistributed in no way - a serious issue for any open source projec sensible to test files such as jrawio.

Fortunately, in the recent years the problem of camera raw formats has become relevant in photographers' communities (see OpenRAW, for instance) and some people have set up freely accessible repositories of samples collected from other people. The repositories we're aware of, and that we're using, are:

Still, we perceive a moderate risk in using this approach. While the situation has changed, since the above websites commit on making their files available, nothing guarantees about their future. For instance, the RAWPository has been dismissed and the corporate that managed it has been dissolved; it is still online, but who guarantees for the future?

For this reason, the preferred source of test files is the Imaging Test Set Repository, that we incepted and manage together with other people. It is hosted in a open source forge, which guarantees per se that published stuff won't be ever withdrawn, and the Creative Commons license that we choose guarantees that files can be redistributed, and eventually hosted by anyone elsewhere, if needed (in this respect, www.rawsamples.ch uses the Creative Commons license too, thus offering the same redistribution guarantee in case it goes away).

So, we strongly encourage test file contributions to the Imaging Test Set Repository. We can't use it exclusively, since only a small fraction of camera models and formats are provided, but we hope that the quantity will grow in future, thus reducing the dependency on other file repositories.

Downloading test files

The test file set for jrawio is quite large (800+ MB at the moment of writing) and will grow much more in future. For this reason, it doesn't make sense to commit test files to the source repository; instead, they are downloaded on demand by tests. They are stored in the directory defined by the property:

testset.cached.folder

that should be set in your local Maven settings (please see the document about Maven for more details). In this way you have the guarantee that files are downloaded only once (and you don't need a working internet connection after they have been downloaded). Without this setting, jrawio will save the test files under the temporary directory of the operating system, which in many cases is erased at each reboot.

Inspecting test results

Processed files are dumped as TIFF images in the directory $TMP/jrawio-test. As the number of test files grows, more and more free disk space (several gigabytes) is needed for storing all the processed files. For this reason, starting from jrawio 1.6, processed files are no more dumped by default: to re-activate the dumping, you must define the property:

jrawio.dumpImageAsTIFF=true

Another property can be used to dump a hexadecimal text dump of the raster (use it only when really needed since it takes several dozens of gigabytes and slows down the process a lot):

jrawio.dumpRasterAsText=true