123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239 |
- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
- "http://www.w3.org/TR/html4/loose.dtd">
- <html>
- <head>
- <title>Writing Meta Information</title>
- <link rel=stylesheet type='text/css' href='style.css' title='Style'>
- <style type="text/css">
- <!--
- pre { color: #800; margin-left: 4em }
- -->
- </style>
- </head>
- <body>
- <div class='index'>
- <a href="#Abstract">Abstract</a>
- <br><a href="#Background">Background</a>
- <br><a href="#Implementation">Current Implementation</a>
- <br><a href="#Mandatory">Mandatory Tags</a>
- <br><a href="#JPEG">The JPEG Segment Size Limitation</a>
- <br><a href="#Preview">The Preview Image Problem</a> (<i>updated 2009-06-12</i>)
- <br><a href="#IFD0">IFD0/ExifIFD Ambiguity</a>
- </div>
- <h1 class='up'>Writing Meta Information</h1>
- <a name='Abstract'></a>
- <h3>Abstract</h3>
- <p>Writing meta information is more complicated than it may appear at first
- glance, which may be one reason why there are very few utilities around that do
- it. ExifTool uses tag names to identify the different pieces of meta information
- that can be extracted from a file. There are thousands of different tags that
- ExifTool recognizes, and many of these tag names are common between different
- metadata formats (the WhiteBalance tag is the worst offender, and can be found
- in 44 different places [ExifTool 8.33]), and sometimes the information can even
- be stored in different places within a single format. Couple this with the fact
- that many manufacturers store meta information in undocumented formats which
- must be reverse engineered (and each of which have their particular quirks), and
- you have a very complex situation.</p>
- <p>ExifTool attempts to simplify this situation as much as possible by making
- reasonable decisions about where to write the information you specify, yet it
- maintains flexibility by allowing you to configure its priorities if necessary,
- or even override the decision making process entirely.</p>
- <a name='Background'></a>
- <h3>Background</h3>
- <p>For a long time, I resisted adding write abilities to ExifTool even though it
- was an oft-requested feature. My concerns in adding this feature were:</p>
- <ol>
- <li>It would complicate the ExifTool interface and make it too confusing for
- typical users.</li>
- <li>It would complicate the code enough to slow down processing for normal use.</li>
- <li>It would take a LOT of work to implement.</li>
- </ol>
- <p>After thinking about this for a while, I was finally able to come up with some
- solutions:</p>
- <p>1. I designed an interface that I think is easy to use for people who
- don't want to know the details of the file structure, yet powerful enough for
- people who want to do very specific things to the information.</p>
- <p>2. I isolated all of the writing code as much as possible into separate files
- which autoload as required. This keeps the compilation fast for people who
- don't require the write feature. Also, I have left the reading routines
- unchanged, so they aren't slowed down by the extra code needed when writing
- information. Unfortunately, this meant I couldn't borrow a lot of code from the
- read routines (even more work for me!), but it had the advantage that I could
- perform additional optimizations in the write routines that I couldn't do
- otherwise. Although the startup costs of this implementation are fairly high
- (for writing only), it should be quite fast for batch writing of multiple files.</p>
- <p>3. I decided to bite the bullet and invest the time required (...guess what I
- did for my Christmas vacation!). Although I thought that a big project like
- this would be better suited to C++ (faster execution and a broader potential
- user base), after programming this so far in Perl I have grown to really
- appreciate the automatic memory handling and other great features of Perl such
- as hash lookups and incredible flexibility in text manipulations afforded by
- regular expressions.</p>
- <a name='Implementation'></a>
- <h3>Current Implementation</h3>
- <p>Currently, ExifTool can write most of the EXIF tags that anyone could reasonably
- want to change (but some tags are protected because they describe physical
- characteristics of the image that you can not change with ExifTool, eg.
- Compression). Also, all of the GPS, IPTC and XMP information and most of the
- MakerNotes information can be edited. This gives you great power, but with
- great power, comes great responsibility...</p>
- <p>It is possible for you to write nonsense into a file, which could cause other
- image readers to throw up their hands in despair and refuse to read the image.
- For this reason, it is best to always preserve the original copy of your image
- file. The "exiftool" script does this for you automatically by renaming the
- original file and always working on a copy.</p>
- <p>The writing logic for ExifTool is the reverse of the reading logic. You
- provide human-readable values and ExifTool will perform the conversions for you.
- For instance, you can set "WhiteBalance" to "Daylight" and ExifTool will change
- the value of WhiteBalance in the image wherever the tag is found provided that
- "Daylight" is a valid value for that location. ExifTool will even do some simple
- matching so that you could even just set it to "day", and ExifTool will search
- through the valid values and will choose the one that contains the string "day".
- If the value is ambiguous, the tag will not be set. If no tags can be set with
- the specified value, ExifTool returns an error message.</p>
- <p>The tag values can also be specified at a numerical level by disabling the
- print conversions that are normally applied. This can be done on a tag-by-tag
- basis or on a global basis through either the application or the API.</p>
- <p>As well as changing tag values wherever they are found in the image, exiftool
- will also create the tag in the preferred group if it didn't exist there before.
- By default, the preferred group is the first of the following where the tag is
- found: 1) EXIF, 2) IPTC, 3) XMP. Alternatively, the desired group (in family 0
- or 1) can be specified so ExifTool only writes the tag to a single location. For
- example, with the command line interface, this is done using an argument like
- "<code>-EXIF:WhiteBalance=Manual</code>".</p>
- <p>If a tag is added to a group that doesn't exist, the new group is created in
- the file, and required mandatory tags may be created. Conversely, if the last
- non-mandatory tag is deleted from a group, the group is removed from the file.</p>
- <a name='Mandatory'></a>
- <h3>Mandatory Tags</h3>
- <p>The EXIF and IPTC standards both specify some mandatory tags. ExifTool will
- automatically create many of these mandatory tags as required when writing new
- information (and remove them again when deleting information if only mandatory
- tags remain). However, some mandatory tags (particularly in the IPTC
- information) can not be easily added automatically, so it is left up to the user
- to add these tags if required.</p>
- <blockquote class='aside'>
- <b>Rant:</b> Let me say that the whole concept of mandatory tags is flawed.
- Instead of mandatory tags, the standard should specify default values to be
- assumed if the tags don't exist. A robust reader has to do this anyway, so it
- is redundant to require that this information must be written. In the case
- where there is no simple default value, the reader must be able to deal with the
- missing tag, otherwise it places the burden on the writer to magically pull a
- reasonable value out of thin air. Of course, you may say that the writer could
- get this information from the user, but conditions like this add an unnecessary
- level complexity to the user interface.
- </blockquote>
- <a name='JPEG'></a>
- <h3>The JPEG Segment Size Limitation</h3>
- <p>An unfortunate aspect of the JPEG format is that the size of a single segment
- is limited to less than 64 kB. With the 2-byte size word at the start of each
- segment, this leaves 65533 bytes for data. The EXIF specification states that
- the data must fit within a single <b>APP1</b> segment (which results in the
- preview image problem discussed below), however <b>APP13</b> Photoshop and
- <b>APP2</b> ICC Profile and FPXR information may span multiple segments. This
- multi-segment information is handled properly by ExifTool.</p>
- <p>JPEG comments may also exceed the size of a single <b>COM</b> segment. If
- necessary, comments are automatically split into separate segments when writing.
- However, when reading they are not joined together because some utilities store
- distinctly different comments in separate segments. To extract all JPEG
- comments into a single file, and combine any comments that may have been split
- into multiple segments, use "<code>exiftool -a -b -comment src.jpg >
- comment.out</code>".</p>
- <a name='Preview'></a>
- <h3>The Preview Image Problem</h3>
- <p>Writing the preview image in JPEG files poses many problems of its own. These
- problems stem from the fact that the JPEG standard is inadequate for storing
- large preview images due to the 64kB limit on segment size as mentioned above.
- (Note that TIFF images don't have this problem since they have a 4GB limit.)
- Some manufacturers get around this by appending the preview after the normal end
- of the JPEG file (JPEG EOI), but this causes complications because it means that
- the preview image pointers in the EXIF information now point outside the EXIF
- segment. This is truly unfortunate because it greatly complicates things for
- image writing software. Most other software can't deal with a preview image and
- will simply remove a preview like this when rewriting the file.</p>
- <p>However, as of ExifTool version 5, the preview images are handled properly
- when writing EXIF information in JPEG files. But for reasons of efficiency, the
- EXIF segment is not edited when writing information if no EXIF tags are being
- changed (eg. if only XMP or IPTC information is being edited). In this case,
- the preview image pointers could be invalidated because the length of the data
- between the EXIF segment (which comes near the start of the file) and the
- preview image (at the end of the file) is likely to change. ExifTool gets
- around this when reading JPEG images by looking for the preview at the end of
- the file and updating pointers if necessary, but the preview image may not be
- readable by other software (it should be noted though that very few image
- readers even know the preview image exists). However, the preview pointers in
- such a file can be fixed if necessary by simply using ExifTool to edit any EXIF
- information.</p>
- <p><b>2009-05-26</b>: New Samsung cameras recently started embedding preview
- images larger than 64 kB, and of course they created a new technique to do so.
- If they were smart, they would have developed a simple technique that could be
- used by others in the future, but of course they were stupid, and didn't think
- that far ahead. (Such is the normal path of dumb camera manufacturers when it
- comes to metadata.)</p>
- <p>The new Samsung models simply split the preview and write it to separate APP2
- segments with no header. If they had written a header (like "PREVIEW\0" for
- example), then the technique could be portable and useful. But they didn't.
- Without a header, the data can not easily be distinguised from other random APP2
- data, so this technique is not generally useful. Of course, there are
- disadvantages with splitting up a preview into separate JPEG segments, so this
- technique in general is not ideal.</p>
- <p><b>2009-06-12</b>:
- <a href="http://www.cipa.jp/english/hyoujunka/kikaku/cipa_e_kikaku_list.html">CIPA</a>
- has recently released a "<b>Multi-Picture Format</b>" standard for storing large
- images in JPEG files. Again, there is a big problem with this standard: It
- uses offsets that are relative to the start of the MPF header (in the new MPF
- APP2 segment) to reference images after the JPEG EOI. These offsets will
- quickly be broken if any data after the MPF segment changes length. This
- problem could have been avoided if offsets had been specified relative to the
- end of file, but it is too late for this now that the specification is
- public.</p>
- <p>The only workable alternative I can see is to enforce the rule that the MPF
- APP2 segment must come after all other APP segments. (It would have been smart
- if this was specified in the CIPA standard, but sadly this isn't the case.) If
- this is done, then metadata in the remaining APP segments (EXIF, IPTC, XMP, etc)
- can safely be edited without breaking the MPF offsets. I suggest that all
- metadata editors employ this strategy, regardless of the segment order specified
- in the standard (which says that the MPF APP2 segment must come immediately
- after the EXIF APP1 segment).</p>
- <a name='IFD0'></a>
- <h3>IFD0/ExifIFD Ambiguity</h3>
- <p>ExifTool has a preferred location (IFD) where it writes all EXIF tags.
- However, a number of tags are written to different locations by various digital
- cameras or image editors. Specifically, the following tags have been observed
- in both IFD0 and ExifIFD: Make, Model, Software, Artist, DateTimeOriginal,
- SensingMethod, CustomRendered, ExposureMode, WhiteBalance, DigitalZoomRatio and
- SceneCaptureType. To handle this ambiguity, ExifTool will delete the tag if it
- exists in IFD0 when it is written to ExifIFD, and vice versa.</p>
- <hr>
- <i>Created Dec. 30, 2004</i><br>
- <i>Last revised Oct. 4, 2010</i>
- <p class='lf'><a href="index.html"><-- Back to ExifTool home page</a></p>
- </body>
- </html>
|