• Application launcher and autostart feature contributed by Francesco Marella (fixes #590550, #612710).
• Improve GNOME location provider (patch by Gabriel de Perthuis) (fixes #598277).
• New ubuntu-mono-dark icons that fit the color guidelines better. Contributed by aleth (semi-fixes #596536).
• Improve error message when first adjustment method fails (fixes #596537).
• Translation updates: Basque (Ibai Oihanguren); Chinese (Jonathan Lumb);
  French (Hangman, XioNoX); German (Jan-Christoph Borchardt); Hebrew
  (dotancohen); Spanish (Fernando Ossandon)

Thank you contributors and donators! (btw, the donation will be spent on hosting for this website).

• Command line options for color adjustment methods changed. Procedure for setting specific screen (-s) or CRTC (-c) changed. See redshift -h for more information.
• Automatically obtain the location from the GNOME Clock applet if possible.
• Add application indicator GUI (by Francesco Marella) (fixes #588086).
• Add reset option (-x) that removes any color adjustment applied. Based on patch by Dan Helfman (fixes #590777).
• configure options for GUI changed; see configure --help for more information.
• New translations: German (Jan-Christoph Borchardt), Italian (Andrea Amoroso), Czech (clever_fox), Spanish (Fernando Ossandon), Finnish (Ilari Oras).

UPDATE: 1.4.1 released. It includes mono icons for Ubuntu (by Joern Konopka); the status icon toggles to show whether Redshift is on or off; and it has updated translations: Spanish (Fernando Ossandon), Russian (Чистый).

As you can see, it uses the graphics from the original game. It is beginning to approach a state where it’s actually playable. Here’s some more screens for your viewing pleasure:

The tiles were created with this python script: pngmaptiler-20100427. It will convert a PNG image to PNG tiles. The script depends on pycairo to do the cutting and scaling of the images. This example converts master.png to tiles of size 256×256 with 1 being the minimum zoom level and 4 being the maximum zoom:

$ pngmaptiler.py master.png 256 256 1 4

Flex flip book

The flip book is written in Adobe Flex and released under the CC BY license. It supports embedded images and flash movies, text labels, transparent pages and much more. I just needed a way to show a preview of some static PDF files so I adapted the code code for this purpose:

Flip book overlay on a web page

• Load book pages from XML description file (one image per page; only image files supported).
• Allow web page javascript to flip the pages.
• Random bug fixes…

The modified source code is available here: flipbook-20100308

WordPress integration

The wordpress plugin is still lacking a lot of features and probably has a load of bugs; but it is usable. The plugin uses the shortcode API to allow easy insertion of books into wordpress posts and pages. It is available here: book-20100308.

When you install and activate the plugin, it will create the necessary database tables to handle the books you upload. You need to manually create the folder on your web server where the uploaded pages will reside. The folder must be in the uploads folder and must be called book (usually wp-content/uploads/book/). It should have the same permissions as the uploads folder. There is currently a bug that will mess things up if you haven’t set the uploads folder in settings > miscellaneous, so you have to set that even if you use the default folder.

There is no admin interface for managing books but there is an HTTP API for creating books that is really easy to use. The database will have to be manipulated manually if you need to edit an existing book. A new book is created with the HTTP interface in wp-content/plugins/book/manage.php. You can for example use the utility cURL that is available on many linux systems (line breaks inserted for readability):

$ curl -F log=USERNAME -F pwd=PASSWORD -F name="Name of the book"
 -F "page[]=@page-1.jpg" -F "page[]=@page-2.jpg"
 example.com/wp-content/plugins/book/manage.php

This will create a new book in the database with the specified name and pages. You can upload as many page images (as either png or jpg) as you want. The example above won’t really be a book because it will only have two pages. USERNAME and PASSWORD should be the ones that you use to log in to your wordpress site. When the pages have been uploaded and the book created, the script will return the text “Success” followed by the id of the book that you will have to enter in the shortcode.

The shortcode is called book and has the following required parameters:

• book: the id of the book as returned by the manage.php script (it can also be found in the database manually).
• pdf: link to PDF version of the book.
• The content text will be the name of the book.

Next you should customize the CSS for the book display. Here’s an example of how it might look (text is in danish).

Creating image files from PDF documents

You need your book pages as image files for the above to work. Often times I find myself with PDF documents that I want to convert to images. I decided to use the poppler library for this conversion as it has the best rendering of fonts of the solutions I tested. I created a python script that does the conversion automatically: pdftopng-20100308. It is a command line utility that depends on the python-poppler bindings. Example:

$ pdftopng.py document.pdf output 420 595

This will create output-1.png, output-2.png etc. for each page in the PDF document. The images will be 420×595 in the example above. You can easily convert these to JPEG with imagemagick in linux if you want to save some bandwidth:

$ for f in output-*.png; do convert -quality 90 "$f" "${f%%.png}.jpg"; done

When you have a satisfactory JPEG file for each page you can upload them as a book with the cURL method above.

When I learned that there is a version for linux (xflux) I had to get that for my Ubuntu laptop. I was quite disappointed, however, when I discovered that not only does it not feature a sleek GUI like the windows version, it also simply does not work at all on my laptop. f.lux throws this message at me: “Sorry, we only support 24/32-bit displays right now” which must be a bug because I am running in 24-bit mode with the open source radeon driver.

Other features that are present in the windows version seem to be missing as well in xflux, like setting the daytime temperature. Ultimately I decided to code my own tool to adjust the color temperature. The result is an open source program called Redshift.

Please post comments here.

The Basics

The AccurateRip algorithm in itself is very simple. Since it operates on audio data extracted from CDs the basic unit is one audio CD sample, i.e. one 16-bit, little-endian, signed integer. As it is stereo, samples always appear in pairs, one sample for the left channel one for the right. The algorithm loops through these sample pairs while it keeps track of the 1-based index of the current pair, i.e. 1 is the index of the first pair in the data stream, 2 is the index of the next pair and so forth. For each pair the algorithm produces a 32-bit value by adding the first sample of the pair, to the second sample shifted left 16 places. This value is then multiplied by the index and added to the accumulated checksum. At the end of the data stream the result is simply the accumulated value:

1. Set the checksum value to zero, c = 0
2. Set the index value, i, to the start of the track
3. Loop while there is more samples in the track
   1. Read two samples (left and right), x_l, x_r
   2. Combine the samples, v = (x_r << 16) | x_l
   3. Update the checksum value, c += i⋅v
   4. Increment i
4. Return the value in c

This is not exactly a great checksum algorithm. It has been argued, that only about 97% of the bits in the data stream participate in the final checksum.  This disadvantage is of course offset by the large number of entries in the AccurateRip database. Fixing the checksum would require creating a new database from scratch.

AccurateRip Quirks

The checksum is not calculated the same way for all tracks on a disc; the first and last tracks are calculated differently. On the audio CD, samples are grouped into larger blocks called frames (sometimes known as sectors). Each frame contains 588 sample pairs (which is exactly 1/75 second worth of audio). In the first track, almost five frames are skipped at the start of the track. It is not exactly five frames because the last sample pair of frame five is actually used. Also note that the index value is incremented even for the samples that are not used in calculating the checksum (only step three in the loop of the algorithm is skipped). In the last track, exactly five frames are skipped at the end.

The effect of these cuts is that the checksum is calculated for a window on the data stream that is slightly shorter than the whole data stream. By sliding this window back and forth it is possible to obtain different checksums for different window offsets. This is important because several pressings of the same audio CDs often exist, the only difference between them being that the audio data on each pressing is slightly offset compared to the others. It would therefore be advantageous to use this sliding window to be able to determine whether a disc is in the AccurateRip database even if the disc registered in the database is of a different pressing.

Obviously, we could use the above algorithm, modified slightly, to calculate a checksum for each offset. That would require going through the loop 5880 to get all the possible checksums. It can be done much more efficiently!

The Math

Consider a sequence of values D = v₁, v₂, …, v_n calculated from sample pairs in a data stream as in step two in the loop of the above algorithm. Now, suppose that this is the data from the whole audio CD, and that each track is a subsequence of D. Let’s further assume that the sequence can be read only sequentially as, in practice, we would be working with some kind of data stream; either data from a CD or from audio files. To calculate the checksum of a track starting at index i of length k one would calculate the sum 1⋅v_i + 2⋅v_i+1 + … + k⋅v_i+k-1. This is the checksum at index i so lets call it c_i.

The goal here is the find checksums that are offset so lets look at that same track but this time offset by one. The checksum of this track is the sum 1⋅v_i+1 + 2⋅v_i+2 + … + k⋅v_i+k which we will call c_i+1. To see if there is an easy way of obtaining the offset checksum from the base checksum let’s subtract the last one from the first: c_i+1 – c_i = (1⋅v_i+1 + 2⋅v_i+2 + … + k⋅v_i+k) – (1⋅v_i + 2⋅v_i+1 + … + k⋅v_i+k-1) = k⋅v_i+k – (v_i + v_i+1 + … + v_i+k-1).

It can now be seen that c_i+1 = c_i + k⋅v_i+k – (v_i + v_i+1 + … + v_i+k-1), i.e. the checksum at offset one can be derived from the base checksum by adding a term and subtracting a different term. Now, let’s consider which parts of this formula we already know. Let’s also assume that we have just calculated the base checksum of the track, i.e. the next value in the sequence will be v_i+k since we have just read k values starting at index i. But v_i+k is exactly the value we need as part of the term that we’re going to add to the base checksum. We just need to read it and multiply it with k which is also a known value (the track length).

The term that we’re going to subtract is obviously just the sum of the values of the track. It would be trivial to compute this as we’re going through the values to compute the checksum anyway, so with a small modification of the algorithm this is no problem as well. It should be clear now that we don’t need to go through k values to obtain the checksum offset by one, we can just derive it from the base checksum by doing one simple addition and one subtraction. Let’s see if we can extend this to obtain more than just the first derived checksum. Let’s see if we can obtain c_i+2.

Reusing the formula obtained earlier, we see that c_i+2 = c_i+1 + k⋅v_(i+1)+k – (v_i+1 + v_i+2 + … + v_i+k). The right hand side has three terms. The first term is the first derived checksum which we have just calculated. The next term depends on the value v_(i+1)+k but this is the next value in the data stream since we’ve just read v_i+k. The last term is the tricky part. We need the sum, not of the base track this time, but of the track offset by one. We don’t have this since we didn’t loop through the track offset by one to calculate the checksum, we just derived it from the base checksum. We can, however, adjust the sum when we calculate a derived checksum. When we calculate the first derived checksum we need to adjust the sum so it is no longer the sum of the base track but is the sum of the track offset by one. We can do this by subtracting the first value in the track, then adding the last value in the track offset by one. This will in effect move the track window, to which the sum pertains, one step, and with just two operations we can get the correct sum. The only problem is that we need the first value of the track to do this. Either we need to seek back through the stream and retrieve this value again, or we could have saved it somewhere when we were calculating the checksum at the first index in the track.

The latter solution may seem like a waste of memory space, but if we use a clever trick, it actually isn’t. We will need to allocate some memory to save all the derived checksums anyway, so why not use this space temporarily to save a value from the beginning of the track? Each calculation of a derived checksum will need just one value to adjust the sum, so clearly, there’s exactly enough space. Furthermore, each value that is saved from the start of the track will need to be used once, after which it can be immediately overwritten by the derived checksum.

One last thing we need to consider is that the improved method sketched above can’t work directly on the first track in practice, because the first five frames (almost) are skipped but the index is still incremented. This makes it impossible to use the above formula to derive checksums from the base checksum of the first track because it was assumed that the track index would start at one. Now assume the more general case of a track whose index counter starts at, not one, but p. If we can derive a formula similar to the above from this more general case we can use it  to handle all tracks. The checksum of a track starting at index i but whose index counter starts at p+1, c_i,p = (p+1)⋅v_i + (p+2)⋅v_i+1 + … + (p+k)⋅v_k+i-1. Now let’s subtract c_i,p from c_i+1,p like above. Continuing this gets us c_i+1,p = c_i,p + (p+k)⋅v_k+i – (v_i + v_i+1 + … + v_k+i-1) – p⋅v_i. The first three terms on the right hand side are clearly equivalent to the formula above. The last term depends on the first value of the track, which we, conveniently, have already made available because we needed it to adjust the sum. Also notice that if p = 0 the last term disappears. This is what makes the formula work exactly as the simpler one derived above in the case where the index counter of the track starts at one.

Improved algorithm

First of all, it must be noted that the above math will be used to construct an algorithm that has to work on real computers, specifically computers that work with 32-bit fixed-width integers. Fortunately, the above formula can be used directly because it only make use of addition and multiplication operators which are compatible with modular arithmetic:

1. Allocate memory space for 5880 values
2. Set first memory field to zero; m[1] = 0
3. Set sum to zero; s = 0
4. Set the index value, i, to the start of the track minus (5880/2)+1
5. Loop k times, where k is the length of the track
   1. Read two samples (left and right); x_l, x_r
   2. Combine the samples; v = (x_r << 16) | x_l
   3. If i is less than 5880, save v in memory field i+1
   4. Update the sum; s += v
   5. Update the checksum value in the first memory field; m[1] += i⋅v
   6. Increment i
6. Loop 5880 minus one times, let i denote the number of the current loop
   1. Read two samples (left and right); x_l, x_r
   2. Combine the samples; v = (x_r << 16) | x_l
   3. Put aside the value in memory field i+1; f = m[i+1]
   4. Calculate the derived checksum by the above formula and store it in memory field i-1; m[i+1] = m[i] + (p+k)⋅v - s - p⋅f, where p is the offset added to the index counter, and k is the length of the track
   5. Adjust the sum; s = s + v - f
7. Return the values in memory space; the checksum at offset o will be in memory field (5880/2)+o

Notice how there’s only one loop that depends on the track length, followed by a loop of constant length that in practice will be very short compared to the first loop. There are no inner loops. Also notice, that seeking in the data stream is completely unecessary; the samples are always read sequentially.

A common use case of this algorithm would probably be to calculate checksums for entire discs. Interleaving the two loops for adjacent tracks can be done by calculating derived checksums of one track while calculating the base checksum for the next track. This way, the algorithm can calculate all checksums for all tracks with only one sequential read through the entire data stream.