Our astronomical image-recognition software Astrometry.net (see Lang et al 2010) does a very good job on professional-grade astronomical images. It is less reliable on snapshots taken with normal, wide-angle lenses and fish-eye lenses. This is ironic, because from an information-theory point of view, they are much easier to recognize: They contain obvious, familiar constellations. The problem is that these wide-field shots have substantial geometric distortions in the field. These distortions foil Astrometry.net because they make the mapping from sky to focal plane non-conformal (squares on the sky don't go to squares on the image)
Another ironic thing about all this is that in fact these distortions are very common among cameras and very predictable. They should be extremely predictable using the EXIF meta-data in image headers, and even without that I bet the distortions live in a small family of possible choices. The project is: Find out what these standard distortions are, and fix Astrometry.net so it knows about them in advance, either by de-distorting the star list that it eats (this would be the easy option) or else by making the star-figure-matching step invariant to those kinds of distortions (that would be the hard option). Actually even easier would be to just have Astrometry.net automatically lower its tolerances as the asterisms get close to the field edges!