idiosyncracies.html 15 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272
  1. <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
  2. "http://www.w3.org/TR/html4/loose.dtd">
  3. <html>
  4. <head>
  5. <title>Maker Note Idio(t)syncracies</title>
  6. <link rel=stylesheet type='text/css' href='style.css' title='Style'>
  7. </head>
  8. <body>
  9. <h1 class='up'>Maker Note Idio(t)syncracies</h1>
  10. <p>It really is surprising how stupid some (...many, ...most?) manufacturers
  11. seem to be when it comes to writing what should be a fairly simple file
  12. format.</p>
  13. <p>One positive thing is that most manufacturers seem to have standardized on an
  14. EXIF-like IFD (Image File Directory) structure for their maker notes. But many
  15. problems arise because of a fundamental design flaw in the EXIF/TIFF format.
  16. Values longer than 4 bytes are stored at a location referenced by an offset from
  17. an absolute position in the file (where offset 0 is the start of the EXIF/TIFF
  18. information).</p>
  19. <p>The difficulty is that these offsets must be recalculated when a file is
  20. rewritten, but in general this is not possible (particularly for the maker
  21. notes) because the format of all information is not known. Some manufacturers
  22. have attempted to avoid this problem using offsets which are relative to the
  23. start of the maker note IFD instead of the usual start of EXIF. This is a good
  24. idea if implemented properly, but this is not done consistently. (And some
  25. manufacturers are not even consistent about how the offsets are calculated from
  26. one camera model to the next!)</p>
  27. <blockquote><font size='-1'><b>Technical aside:</b>
  28. <br>If EXIF were designed properly, all offsets would
  29. be relative to 4 bytes after the end of the IFD, which is the normal position
  30. for values to be stored, and all value data for the IFD would be stored in a
  31. block at this location. If this was done, an entire IFD could be relocated
  32. easily without causing problems.</font></blockquote>
  33. <p>Below is a list of idiosyncracies in files written by the digital cameras or
  34. software from various manufacturers. Many of these quirks relate to the offset
  35. problem mentioned above.</p>
  36. <hr>
  37. <p><a name="Canon"><b>Canon:</b></a> The 350D (firmware 1.0.1) gets the size of the thumbnail image
  38. wrong and reports it to be 10 bytes too long. This can cause the reported
  39. thumbnail image data to run off the end of the APP1 segment. A bug in version
  40. 1.0.4 of the 40D firmware causes it to write a maker note entry count that is
  41. one greater than it should be.</p>
  42. <p><a name="Casio"><b>Casio:</b></a> The preview image is referenced by two different offsets (the
  43. PreviewImage tag plus a PreviewImageStart/PreviewImageLength pair). Also, the
  44. offset for the PrintIM information is relative to the start of the IFD entry
  45. even though other offsets aren't.</p>
  46. <p><a name="Concord"><b>Concord:</b></a> Some models write PrintIM information with an entry-based
  47. offset like Casio.</p>
  48. <p><a name="GE"><b>General Electric:</b></a> A number of GE cameras store zero offsets for some
  49. maker note tags (possibly to indicate that the tags do not exist), and other
  50. offsets are 12 bytes too high for some models (like the A1230, E1035 and G2).
  51. </p>
  52. <p><a name="HP"><b>Hewlett-Packard:</b></a> The PhotoSmart 720 (one of the few HP models to use
  53. EXIF-format maker notes) uses a format code of 5 (rational64u) for tag 0x0204,
  54. but stores a rational32u value. Other models show about as much standardization
  55. as the Kodak point-and-shoot lineup. Also, some models (C945, M22, M23, R507,
  56. R607, R707, R717, R725, R727, R817, R818, R827, R927 and R960) write the EXIF
  57. ComponentsConfiguration incorrectly as ASCII characters (like the Leica M8 and
  58. M9).</p>
  59. <p><a name="Kodak"><b>Kodak:</b></a> Professional DCS Photo Desk software writes a cyclical EXIF
  60. directory such that the InteropIFD pointer points back to IFD0.
  61. Point-and-shoot models show little standardization in maker note format.
  62. Some models with IFD-format maker notes store incorrect count values for
  63. a number of tags (this is particularly nasty), and may contain blank IFD
  64. entries which are filled with 0xff's (not zeros like other makes).</p>
  65. <p><a name="Konica"><b>Konica:</b></a> The KD-300Z writes all maker notes offsets relative to the
  66. start of the individual IFD entry.</p>
  67. <p><a name="Kyocera"><b>Kyocera:</b></a> A number of models write all maker notes offsets relative
  68. to the start of the individual IFD entry.</p>
  69. <p><a name="Leica"><b>Leica:</b></a> The M8 and M9 write the EXIF ComponentsConfiguration value in
  70. ASCII instead of binary. The M8 writes EXIF ExposureCompensation and
  71. ShutterSpeedValue incorrectly as a unsigned rationals when they should be
  72. signed. This leads to crazy values like "+65536" for small negative exposure
  73. compensations, and "0 s" for long exposure times. (NOTE: These are all EXIF
  74. idiosyncracies since the values are in the standard EXIF, not the maker
  75. notes.) In DNG images, the M8 uses maker note offsets relative to the
  76. start of the maker notes in JPEG images (very reasonable), but relative to
  77. the end of the maker note header in DNG images. I think this was a mistake
  78. because this is changed in M9 DNG images to be the same as JPEG images.</p>
  79. <p>The Leica S2 maker note format is the MOST idiotic I have seen, and has the
  80. following peculiarities:</p>
  81. <ul>
  82. <li>It is stored as a trailer after the JPEG EOI (but referenced from a pointer
  83. inside the APP1 EXIF segment).</li>
  84. <li>Most of the offsets in this MakerNote IFD are relative to the start of the
  85. file instead of the EXIF TIFF header (which is particularly cruel because they
  86. are broken if other software simply adds a leading JFIF segment, but there is no
  87. simple way to detect that this has happened. Normally this could be detected by
  88. analyzing the pointers, but this doesn't work here because all of the unused
  89. data in the Leica maker notes make normal pointer assumptions impossible).</li>
  90. <li>The PreviewImage offset is relative to the start of the MakerNote data
  91. (which is MUCH more reasonable, but using two different offset bases in the same
  92. directory is yet another level of idiocy).</li>
  93. <li>In DNG images the maker notes use an absolute base offset. Yet again
  94. different from the M8 and M9. Consistency isn't Leica's strong suit.</li>
  95. </ul>
  96. <p>Strike that. The most idiotic award now goes to the Leica M (Typ 240), which
  97. adds these quirks (firmware 1.1.0.2):</p>
  98. <ul>
  99. <li>Tag 0x0301 has the same offset as the PreviewImage in the maker note IFD,
  100. although it looks like the data for this tag probably comes after the
  101. PreviewImage.</li>
  102. <li>Tag 0x0302 has an invalid offset (0xffffffff).</li>
  103. </ul>
  104. <p>OK, I give up on Leica. The new Leica S (Typ 007) attains a new level of
  105. stupidity by storing the preview image in the JPG file using a completely
  106. nonsensical technique (in IFD2 of the EXIF segment, with the data being stored
  107. outside the EXIF segment after the JPEG EOI), as well as various other brainless
  108. blunders. [@Leica: Try reading the MPF specification.]</p>
  109. <p><a name="Minolta"><b>Minolta:</b></a> An obvious bug in the firmware of the Z2 writes an incorrect
  110. offset for the 'MinoltaCameraSettings2' information -- it writes the offset of
  111. the offset itself instead of the offset of the value (hahaha!). Other
  112. offsets are correct.</p>
  113. <p><a name="Nikon"><b>Nikon:</b></a> D2H NEF files have huge blocks with all zero data (3.7 MB in
  114. my test file!).</p>
  115. <p><a name="Olympus"><b>Olympus:</b></a> The E-1 and E-300 have subdirectories in the maker notes, but
  116. not only does the data size of these subdirectories exclude the subdirectory
  117. value data, but also it is 2 bytes too small for the directory information
  118. itself (doh! -- they forgot to include the entry count). Similarly, the stored
  119. size of the maker note data block is too small for many models, which results
  120. in a loss of data if the maker notes are copied as a block when an image is
  121. rewritten.</p>
  122. <p><a name="Pentax"><b>Pentax:</b></a> The Optio 330 uses an offset for the PrintIM information which
  123. is relative to the start of the IFD entry (hmmm, like some Casio models...).
  124. Also, preview image offsets in the maker notes are given relative to the EXIF
  125. base rather than the maker note base (like all other maker notes offsets).</p>
  126. <p>The Optio 550, 555, 33WR and 43WR all specify a PrintIM directory at a the
  127. same offset of 0x29a with length 40 bytes, but the only PrintIM information in
  128. the file is nowhere near that offset and is 128 bytes long. Also for these
  129. models, tag 0x002e has a constant value of 0x6a6 even though its position
  130. changes. Finally, all of these models plus the Optio WP waste many kilobytes of
  131. space in each image with large unused data blocks in the EXIF information.</p>
  132. <p>The Optio 330RS and 430RS double reference the preview image information.</p>
  133. <p>Note that the worst problems are with the Optio 230, 330, and 430, which
  134. carry the Asahi brand name.</p>
  135. <p><a name="Photoshop"><b>Photoshop and Nikon Capture:</b></a> Both of these packages write TIFF IPTC
  136. information as 'int32u' (or 'LONG'). This is wrong
  137. (<a href="http://www.awaresystems.be/imaging/tiff/tifftags/iptc.html">see
  138. reference</a>). Nikon Capture goes one step further and simply ignores IPTC
  139. that is written correctly as 'undef' or 'int8u'. (So for compatibility,
  140. ExifTool also writes this incorrectly as 'int32u'.) Photoshop completely
  141. deletes the maker notes when an image is edited.</p>
  142. <p><a name="Ricoh"><b>Ricoh:</b></a> There is an IFD subdirectory in the Ricoh maker notes of both the
  143. Caplio RR30 and RR1. The RR30 uses standard EXIF offsets (relative to the start
  144. of the EXIF data), but for the RR1 the offsets are relative to the start of the
  145. subdirectory. The G700 uses MPF offsets relative to the start of the file,
  146. instead of the start of the MPF segment as per the MPF spec. The HX15 uses
  147. a standard EXIF maker note structure, but there are 2 extra padding bytes
  148. between the IFD entry count and the 1st IFD entry. The HZ15 and Pentax XG-1 (by
  149. Ricoh) both have an extra 2 bytes after the IFD entry counts. All value offsets
  150. are erroneously 0 for the HZ15, and there are other problems with the offsets
  151. stored by the XG-1.</p>
  152. <p><a name="Rollei"><b>Rollei:</b></a> The DK4010 writes all maker notes offsets relative to the
  153. start of the individual IFD entry.</p>
  154. <p><a name="Sanyo"><b>Sanyo:</b></a> The offsets written in the maker notes of the J1, J2, J3, S1,
  155. S3 and S4 have very little to do with reality. Apparently the Sanyo
  156. programmers have no understanding of the concept of an IFD offset.</p>
  157. <p><a name="Skanhex"><b>Skanhex:</b></a> With some Skanhex models (SX-210Z3, SX-330Z3, SX3300,
  158. SX410Z3), the 264-byte makernotes block contains no useful information, and
  159. overlaps values from the ExifIFD. For these models there is also a large block
  160. (typically 1195 bytes) of unreferenced information in the EXIF data immediately
  161. following the IteropIFD. This block begins with the character sequence
  162. "SKANH\0", and contains exactly the same information in all 20 of my sample
  163. images that contain this block (except for a variable amount of padding at the
  164. end with 0xff bytes). These quirks also affect some Gateway, Jenoptik, Medion,
  165. Samsung and Yakumo models built by Skanhex.</p>
  166. <p><a name="Toshiba"><b>Toshiba:</b></a> The PDR-3310 writes all maker notes offsets relative to the
  167. start of the individual IFD entry. (very similar to Konica KD-300Z)</p>
  168. <hr><a name="raw"></a>
  169. <h2>RAW file Idiosyncracies</h2>
  170. <p><a name="MinoltaMRW"><b>Minolta MRW:</b></a> The A200 stores the thumbnail image offset in IFD0
  171. relative to the start of file, while all other offsets are relative to the start
  172. of the TIFF header, which is 48 bytes into the file. Also, the A200 stores the
  173. StripOffsets and the StripByteCounts values in the wrong byte order.</p>
  174. <p><a name="SonyARW"><b>Sony ARW:</b></a> The maker notes of ARW images are not self-contained, so
  175. some information is lost when the images are rewritten by other software
  176. (including the Adobe DNG converter). The A100 with firmware 1.00 sets the high
  177. word of the thumbnail image offset to zero, but it should sometimes be 0x0001.
  178. (This problem is fixed for firmware 1.01.) Also with the A100, the
  179. JpgFromRawLength stored in IFD0 may be wrong (although this value is also stored
  180. in the MakerNotes and is correct here). As well, much information in these
  181. images is encrypted, which complicates things somewhat. Even the Sony IDC
  182. utility can't properly rewrite ARW files -- it corrupts the embedded MRW record
  183. when used to edit ARW images. Even funnier: IDC v3.0 will crash when loading
  184. some original A100 firmware 1.00 images, but no longer crashes if the images are
  185. first edited with ExifTool (probably because ExifTool fixes the above mentioned
  186. problems when it rewrites the image).</p>
  187. <p><a name="LeicaDNG"><b>Leica DNG:</b></a> The makernote offsets for the M8 are relative to the start
  188. of the makernote IFD in JPEG images, but relative to the start of the makernote
  189. header (8 bytes earlier) in DNG images. <i>[2009-09-09: This is fixed for the
  190. M9 which has offsets relative to the start of the makernote header for both JPEG
  191. and DNG.]</i></p>
  192. <p><a name="NikonNEF"><b>Nikon NEF:</b></a> Aside from the encryption that Nikon uses to try to hide
  193. some information in their maker notes, the NEF files in general seem fairly well
  194. behaved. Even so, the Nikon Transfer utility (version 1.3) still manages to
  195. corrupt some information in the 0th SubIFD when it is used to process NEF
  196. images. (Beware that other Nikon utilities may have this same problem if they
  197. use the same NEF writing routines.) But luckly the lost information isn't very
  198. important. (Only a few tags from the embedded full-sized preview image are
  199. lost: XResolution, YResolution and YCbCrPositioning.) Also, Nikon Transfer and
  200. Nikon Capture both write an incorrect size for the maker notes, which could
  201. cause loss of MakerNote information if the file is edited by other software (but
  202. this isn't a problem with ExifTool, which will fix this type of problem
  203. automatically when writing).</p>
  204. <p><a name="NikonNRW"><b>Nikon NRW:</b></a> Nikon should have just called this NEF with a different
  205. version number -- there should be no need to pollute the universe with zillions
  206. of unnecessary file extensions. Oh right, they weren't smart enough to include
  207. a file identifier containing a version number in their NEF images -- Doh! In
  208. these images, CFAPattern2 is written incorrectly with UNDEFINED instead of BYTE
  209. format.</p>
  210. <p><a name="PhaseOneIIQ"><b>Phase One IIQ:</b></a> Many values are referenced from more than one location
  211. in the TIFF structure of these images. For instance, the IFD0 strip data
  212. actually exists within the MakerNotes data block. This is a poor design, and
  213. leads to duplicated information when the image is rewritten.</p>
  214. <p><a name="RicohDNG"><b>Ricoh DNG:</b></a> The GR Digital IV (firmware 1.14) stores an incorrect
  215. length for the JPEG preview in SubIFD1.</p>
  216. <p><a name="SamsungSRW"><b>Samsung SRW:</b></a> Yet another TIFF-based RAW image with no proper file
  217. identifier. In these images the thumbnail is stored inside a SubIFD of IFD1
  218. instead of directly in IFD1 (dumb, dumb...). Also, the NX200 (firmware
  219. NX200_011181) uses a base offset for the X/YResolution values that is different
  220. from the PreviewImageStart pointer, both in the MakerNotes PreviewIFD. (Note
  221. that the NX100 uses the same base for both, so this is certainly a firmware bug
  222. for the NX200. <i>[2012-06-21: This problem now also affects the EX1, NX20 and
  223. WB2000] [2013-07-25: Add the NX2000 to this list]</i>)</p>
  224. <hr>
  225. <i>Created Mar. 25, 2005</i>
  226. <br><i>Last revised Sept. 4, 2015</i>
  227. <p class='lf'><a href="index.html">&lt;-- Back to ExifTool home page</a></p>
  228. </body>
  229. </html>