I just implemented pretty much the same thing from scratch for my wedding gallery¹.
For each row, it tries 3-10 images, sums their aspect ratios, divides the total row width by the sum to get a candidate height, then picks the height that's closest to the average of the existing rows.
To make things look a bit nicer, it rejects candidate rows with the same number of images as the last row.
I might release the code on Github if I can make it modular enough. Currently it depends on jQuery but that's really only for element creation.