Compare commits

...
391 Commits
Author SHA1 Message Date
James R. Barlow c14e10128a Bump version to -rc9 2015-08-29 16:43:22 -07:00
James R. Barlow 3270635192 ghostscript: quiet startup on rasterize 2015-08-28 04:51:36 -07:00
James R. Barlow 3d26257710 Add test cases for additional image formats 2015-08-28 04:51:11 -07:00
James R. Barlow c4f134d694 Prevent running validation on missing file after an exception is thrown 2015-08-28 04:48:29 -07:00
James R. Barlow 83f9dfbac4 Use png256 raster device when possible
Someone reported a bug where the .png input to unpaper ended up being
type 'P' (palette) for some reason, which was not supported in unpaper.

Not sure how it happened, but seemed easier to fix by explicitly
supporting. Here we use png256 if it would capture all colors in the
input file. It's up to tesseract/reportlab to make use of the palette
PNG when rendering.
2015-08-28 04:47:57 -07:00
James R. Barlow 3a445ad5f7 unpaper: support paletted files by conversion instead of bailing 2015-08-28 04:44:26 -07:00
James R. Barlow c6d106ec33 Throw exception if iccprofiles not found instead of returning None
So far iccprofiles were only missing for a user who had a custom and
possibly broken ghostscript installation.
2015-08-28 03:59:35 -07:00
James R. Barlow 2ce6834be4 Bump to -rc8 2015-08-24 01:25:01 -07:00
James R. Barlow b376672dbc Bug fix: exception thrown if input PDF was missing DocumentInfo block 2015-08-24 01:23:30 -07:00
James R. Barlow d07db8547f Merge branch 'master' of https://github.com/fritz-hh/OCRmyPDF 2015-08-23 12:30:46 -07:00
James R. Barlow aab08bfcc7 Fix requirements.txt problem 2015-08-23 12:30:40 -07:00
jbarlow83 e0a25494ee Explain the need for multi core, etc 2015-08-22 13:34:42 -07:00
James R. Barlow fd876d5e4e Merge branch 'develop' 2015-08-22 01:51:44 -07:00
James R. Barlow ee7f008ff5 Require unpaper 6.1; no messing around with broken versions 2015-08-22 01:51:08 -07:00
jbarlow83 d9161a6ddb Update README: docker run instructions 2015-08-22 01:50:13 -07:00
jbarlow83 f8d66768e3 Update README with docker install instructions 2015-08-22 01:33:12 -07:00
James R. Barlow 4f3673d14d Update notes for -rc6 2015-08-22 00:40:07 -07:00
James R. Barlow 1712fdb74a Merge branch 'feature/docker-debian' 2015-08-22 00:32:27 -07:00
James R. Barlow 3a5ffc79e0 Stock debian unpaper is no good; replace with 6.1 built from source
debian and ubuntu both install unpaper 0.4.2 or so. No .deb packages
available at higher version numbers although ArchLinux had something.
Considered making a separate image to handle building and install but
decided that was a premature optimization at this point, so just build
the unpaper that works. All tests pass.
2015-08-22 00:30:39 -07:00
James R. Barlow 859b063444 Fixup other docker test suite errors
Outstanding failures:
test_pageinfo::test_jpeg
tests involving unpaper due to version <6.1 failures
2015-08-20 02:37:03 -07:00
James R. Barlow bd61e7c644 dockerignore *.pyc
https://github.com/docker/docker/issues/13113
Docker kinda sucks. No recursive exclusion.
2015-08-20 02:27:07 -07:00
James R. Barlow c9abf282b5 Set docker locale to utf-8
Shocked, shocked, that there's a Linux distribution out that there isn't
doing the right thing and setting up utf-8 by default. (Many tests failed)
2015-08-20 01:44:30 -07:00
James R. Barlow 9dad40b5a3 Major overhaul of the Dockerfile
Switched from Ubuntu to debian:stretch because stretch has more recent
versions of our binary packages and starts smaller.  In particular,
stretch has both pillow==2.9.0 and reportlab==3.2.0 available as system
packages which saves the considerable hassle of install a toolchain.

Instead, a pyvenv is set up with access to system's site-packages (note:
needs two steps), making the binary-dependent packages available.  Then
the remaining packages are installed into the pyvenv with --no-cache-dir
to avoid saving files. And there we are.

Image is still very large (>500 MB), but programs like reportlab require
font rendering capabilities so they pull in large portions of the Linux
graphics stack. Not much will shrink that.
2015-08-20 01:25:31 -07:00
James R. Barlow 8e2d690cb0 Rework Dockerfile, setup.py to work with wheels for better cache use 2015-08-19 13:43:32 -07:00
James R. Barlow c132e091e1 Dockerfile: use local copy of application 2015-08-19 13:10:58 -07:00
James R. Barlow 630e6cbf1e pip chokes on Unicode filenames? 2015-08-18 23:56:30 -07:00
James R. Barlow 83ff5760a8 Dockerfile comment cleanup 2015-08-18 23:41:41 -07:00
James R. Barlow fed0ee638e Fix ruffus writing to RO directory in container 2015-08-18 23:30:06 -07:00
James R. Barlow cc161780df Replace fileinput with regular open-replace
fileinput is supposed to save time in these cases but it's not capable
of doing both in-place rewrites and working with a non-ascii encoding.
This was not noticed until characters outside of ASCII were picked up
by tesseract and saved in a HOCR file. Rework some surrounding code as
well and add multilingual test cases.
2015-08-18 23:27:50 -07:00
James R. Barlow 898b2b000a Works 2015-08-18 05:38:05 -07:00
James R. Barlow b3ee743ed7 WIP on docker 2015-08-18 04:46:25 -07:00
James R. Barlow ef17b669fe README needs ghostscript 2015-08-18 03:27:39 -07:00
James R. Barlow 2dff3e07ce Drop libxml2 dependency
It seems that Python's internal XML parser is good enough to do the job.
2015-08-17 15:26:07 -07:00
James R. Barlow 53c88093ad Bump to -rc5 2015-08-16 02:19:04 -07:00
James R. Barlow 0ec13d3a17 Fix test cases: minor issues
-os.environ directly modified when whole suite run, breaking subsequent
tests
-no longer trusting JHOVE for PDF/A validation
2015-08-16 01:57:35 -07:00
jbarlow83 0d5104049a Update README with better install instructions 2015-08-16 01:28:28 -07:00
James R. Barlow ce8fa69785 Update readme 2015-08-16 00:59:57 -07:00
James R. Barlow 30072e0c70 Pillow sucks
Far from being fluffy or friendly, Pillow silently allows installation
of itself without support for major image types.  Reportlab calls for
pillow 2.4.0.  On Ubuntu 14.04 LTS this will trigger an upgrade of
pillow that will be built without JPEG or ZLIB so it is effectively
neutered, and unfortunately Pillow will not detect this situation at
install time and guide users to a resolution.  Instead, you see nasty
stack traces.

So add a run-time check to ensure that Pillow is sane and capable of JPEG
and PNG support since both may be used internally.
2015-08-16 00:54:03 -07:00
James R. Barlow eb04a890b2 Relax Pillow requirement for Ubuntu 14.04 LTS 2015-08-15 15:55:56 -07:00
James R. Barlow 0c53adb04f setup: rollback lxml version to 3.3.3 - that's the latest in Ubuntu 14.04 2015-08-15 15:25:58 -07:00
James R. Barlow ee5a43fd47 setup: suppress jhove errors 2015-08-15 15:25:30 -07:00
James R. Barlow c43d6c2cbe Merge branch 'develop' of https://github.com/fritz-hh/OCRmyPDF into develop
Conflicts:
	setup.py
2015-08-15 15:18:41 -07:00
James R. Barlow 87aeeacb04 Fix erroneous instruction to "apt-get install tesseract"
Should be tesseract-ocr
2015-08-15 15:17:38 -07:00
James R. Barlow 6b26e9cad6 Fix erroneous instruction to "apt-get install tesseract"
Should be tesseract-ocr
2015-08-15 15:12:05 -07:00
James R. Barlow 85af0f0d03 Add test case for blank PDF page 2015-08-14 00:46:50 -07:00
James R. Barlow f6f4705ea3 Remove Java from setup.py 2015-08-14 00:44:56 -07:00
James R. Barlow a4702bff22 Possible fix for issue #111 2015-08-13 23:10:22 -07:00
James R. Barlow 73c5c48f79 Update notes 2015-08-13 23:08:29 -07:00
James R. Barlow adf495e8cc Remove JHOVE
JHOVE is not an effective PDF/A validator, as detailed in this article:
http://www.pdfa.org/2014/12/ensuring-long-term-access-pdf-validation-with-jhove/

In short, it's buggy. Out of 670 invalid PDF/A files in a test suite,
it only flagged 5.  It only looks for certain problems that Ghostscript
generated PDFs are unlikely to have.  So use qpdf as a final check for
general ill-formed PDF problems since it is quite reliable.

JHOVE 1 is no longer maintained. There's a JHOVE 2 but it has no PDF
support.  I also don't know if it's appropriate to bundle JHOVE, with an
LGPL, under this project and its current license.

Removing a dependency on Java is a huge win.  A world with less Java is
a world with less AbstractFactoryConstructorInterfaces.
2015-08-11 15:31:32 -07:00
James R. Barlow 9247ea00bf Improve ruffus exception handling
ruffus swallows the return code if the process of handling an exception
we hit an error in ruffus' own code, which can happen.  So pick through
its error stack and find out if there's an interesting return code in
there.  Had to use eval() of all things.

Also suppress the stack trace for normal error conditions that don't
need one.
2015-08-11 02:19:46 -07:00
James R. Barlow a1238d7bf9 Document override binary test 2015-08-11 00:44:43 -07:00
James R. Barlow 2d63268f0f Work around JHOVE bug for now, so that the test passes 2015-08-11 00:23:48 -07:00
James R. Barlow 1cb5f6a90d Refactor exit codes; test for missing tessdata
Some versions of tesseract installed by homebrew end up without a
functional tessdata folder, and tesseract is not helpful in this
situation, so add a new test to make sure our output is at least
indicative of the problem.

In the process of properly handling return codes I discovered
test_override_metadata triggers a NPE inside JHOVE probably due to the
Unicode character checking.  This could be specific to my JRE (1.6.0_65,
Oracle) but it's probably JHOVE's fault.  A valid PDF/A (per Acrobat)
is still generated.
2015-08-11 00:17:02 -07:00
James R. Barlow 8d848284df Fix code, test case: complain when GS fails to produce PDF/A
Modified pipeline to fix regression and return the proper error code if
we did not produce a PDF/A as expected.  The wrapper forces the output
to be PDF 1.3 which is not PDF/A compliant.

The funny thing is that in some cases JHOVE incorrectly states that a
file is PDF/A-1b compliant, well formed and valid, even when it is not
according to Acrobat XI and is missing the PDF/A metadata marker, as
far as I can tell.  JHOVE may not be as beneficial as hoped.
2015-08-10 16:05:00 -07:00
James R. Barlow 8fe54d1a5c Add new test case to check invalid PDF/A case
It revealed a regression - return code not the same as v2.x for invalid
PDF/A.  It's also not easy to get the return code out of ruffus.  Will
need to tweak the final step of the pipeline.
2015-08-10 13:57:28 -07:00
James R. Barlow 11dd9f14c3 setup.py: block unsafe 'upload', say to use twine instead 2015-08-09 14:16:30 -07:00
James R. Barlow 16d24f1166 Bump version to -rc4 2015-08-05 23:26:38 -07:00
James R. Barlow 97015ef775 Add a test case to check on the @argumentsfile syntax 2015-08-05 23:17:38 -07:00
James R. Barlow 2744dafb74 New test case: ensure metadata is preserved from input to output 2015-08-05 17:09:38 -07:00
James R. Barlow 7b268dbe1a Remove duplication in test case 2015-08-05 16:57:04 -07:00
James R. Barlow 8fcbbcef94 Improve usage text 2015-08-05 16:56:53 -07:00
James R. Barlow 8f93f0a06e Tidy docs 2015-08-05 16:56:30 -07:00
James R. Barlow 387142488c Kill duplicate file 2015-07-31 01:57:16 -07:00
James R. Barlow 6887e232fc Bug fix: exception from process timeout should be TimeoutExpired 2015-07-31 00:06:58 -07:00
James R. Barlow 6ac7ffd77b Merge branch 'feature/drop-mupdf-poppler' into develop 2015-07-30 23:38:27 -07:00
James R. Barlow b28faa582a Automatically use all available cores unless told not to 2015-07-30 23:20:21 -07:00
James R. Barlow 454ee029c8 Run final ghostscript in multithreaded mode
This step is serialized so all cores are not busy at this stage.
2015-07-30 23:20:04 -07:00
James R. Barlow a036de318e Replace mupdf and poppler with qpdf
Drop two dependencies and replace them with one that does the job of
both.  Smells like progress.

mupdf does PDF file repair and rendering
poppler does rendering and page splitting
qpdf does PDF file repair and page splitting
ghostscript does PDF file repair, rendering, and page splitting (sort of)

So we use qpdf.  Ghostscript's page splitting is supposed is less
efficient because it reprints the page (PDF -> Postscript -> PDF) and
possibly loses quality.  qpdf's library could be used to improve
performance.

This causes a slight performance regression:

py.test tests/test_main.py::test_maximum_options went from 187 seconds
up to 192.  This is likely due to O(n) serialized invocations of qpdf
compared to a single serialized call to pdfseparate.  Could improve on
this situation by using the example code in qpdf: pdf-split-pages.cc
or create marker files in split_pages() and then write a new @transform
function that would split pages on each CPU.  Probably not worth it,
overall, unless this causes problems on files with hundreds of pages.
2015-07-30 04:16:35 -07:00
James R. Barlow 9918c4020e Use img2pdf in test case because it does a better job 2015-07-30 03:35:56 -07:00
jbarlow83 3d6264e1b8 Fix formatting of 'motivation' 2015-07-28 17:58:26 -07:00
jbarlow83 1c25270503 Improve instructions for users that need sudo or venv 2015-07-28 17:55:56 -07:00
James R. Barlow 47e50f82c4 setup.py: allow mutool 1.7 2015-07-28 13:37:32 -07:00
James R. Barlow 27ecdfbba8 More fixes to error cases in setup.py 2015-07-28 13:05:23 -07:00
James R. Barlow 6901550065 Fix some installer issues 2015-07-28 12:41:24 -07:00
jbarlow83 6e6f918630 Actually link the release notes 2015-07-28 12:21:57 -07:00
jbarlow83 4633812246 Fix git clone command with one I tested ;) 2015-07-28 12:20:09 -07:00
jbarlow83 14bd1555aa Update README with more detailed instructions 2015-07-28 12:15:37 -07:00
James R. Barlow b9d7687fa0 Fixes: clarify install instructions and reactivate external program checks 2015-07-28 05:44:15 -07:00
James R. Barlow 93b36965e2 Merge branch 'develop'
# Conflicts:
#	RELEASE_NOTES.md
#	src/config.sh
#	src/hocrTransform.py
#	src/ocrPage.sh
2015-07-28 04:59:49 -07:00
James R. Barlow 9e0c443c2f -rc2: because pypi won't accept -rc1 2015-07-28 04:55:10 -07:00
James R. Barlow 60832152b1 Don't mess with options 2015-07-28 04:46:21 -07:00
James R. Barlow 6a160d22fe Update release notes, add copyrights 2015-07-28 04:36:58 -07:00
James R. Barlow e35526192c More test cases 2015-07-28 03:02:35 -07:00
James R. Barlow bea57bdded More test cases for other parameters 2015-07-28 02:31:18 -07:00
James R. Barlow 2a9da225e4 Minor tweaks to uncommon arguments 2015-07-28 02:25:50 -07:00
James R. Barlow a3f37de9b5 Test cases for --tesseract-timeout 2015-07-28 01:47:30 -07:00
James R. Barlow 6064160953 Get rid of subprocess call on import of tesseract, unpaper -- bit nasty 2015-07-28 01:00:29 -07:00
James R. Barlow 8508141314 Drop nose, all tests working reasonably again
Although the real issue was that the ruffus pipeline cannot be executed
twice in the same process due to its reliance on global variables.

The new OO pipeline in ruffus 2.6 would be one resolution that would
allow for more comprehensive testing as opposed to farming out the
execution to subprocess and inspecting the results, as is currently
done.
2015-07-28 00:43:22 -07:00
James R. Barlow 1c95597882 nose can't really handle external tests so looking into py.test instead
Specifically it trips over the need to reimport ocrmypdf.main.  That in
turn raises questions about whether to make that function into an
external script that imports ocrmypdf... or something else.  Would be
possible with a loop that manipulates sys_argv and then reloads
ocrmypdf.main; might need that anyway.
2015-07-27 22:07:04 -07:00
James R. Barlow 587fa63c8e --oversample: Default to 0 2015-07-27 20:42:16 -07:00
James R. Barlow b40eec4cb0 Add --oversample test for hocr rendering 2015-07-27 17:18:02 -07:00
James R. Barlow 7bcd48c269 Add test to confirm that metadata is transferred to final PDF/A 2015-07-27 16:11:51 -07:00
James R. Barlow 2e7cd52c0f Improve argument handling, test cases 2015-07-27 15:39:54 -07:00
James R. Barlow 77d4cb367e Put ghostscript in a module 2015-07-27 15:22:00 -07:00
James R. Barlow 2c45c5abc6 Implement tesseract timeout 2015-07-27 04:23:37 -07:00
James R. Barlow a89afabd79 Implement tesseract PDF rendering as an alternative
It's much better a rendering text baselines than hocr and seems to
produce small file sizes, so it's progress.  Not available for
Tesseract 3.02 obviously, so both modes need to remove available.
2015-07-27 04:20:49 -07:00
James R. Barlow 03f7c9bf07 setup.py: Only do program checks when installing 2015-07-27 02:14:51 -07:00
James R. Barlow d5f4862749 setup.py: check for third party program requirements 2015-07-27 01:45:17 -07:00
James R. Barlow 8aced0b6d3 More testing: JPEG 2015-07-27 00:25:43 -07:00
James R. Barlow 6b9adef684 Don't create inline images in output PDFs
...except that Ghostscript will sometimes turn out of line images into
inline images on its own, possibly if file size is small.
2015-07-26 21:43:49 -07:00
James R. Barlow 5440d988fc Make this PDF a whole image page
Originally it had a smaller image centred in a page, which is not quite
supported.
2015-07-26 18:32:50 -07:00
James R. Barlow 30da4fc569 pageinfo: drop pdftotext and use PyPDF instead 2015-07-26 18:23:37 -07:00
James R. Barlow 2c1b5e100b Test cases for pageinfo; complain about inline images 2015-07-26 18:18:41 -07:00
James R. Barlow 3684f278ed Add some pageinfo test cases; found problem with inline images 2015-07-26 15:24:42 -07:00
James R. Barlow 6c3cb6acba Remove redundant *res_render 2015-07-26 12:56:10 -07:00
James R. Barlow b98ba8d174 Replace .md with .rst
Github supports both, and PyPI expects .rst files, so use .rst and make
everyone happy.

Auto-converted using pandoc
find . -name '*.md' | parallel pandoc --from=markdown --to=rst --output='{.}.rst' '{}'
http://bfroehle.com/2013/04/26/converting-md-to-rst/
2015-07-26 03:01:18 -07:00
James R. Barlow d3088829af More packaging changes: move jhove, fix console script 2015-07-26 01:52:08 -07:00
James R. Barlow 9aaaba1714 Packaging stuff 2015-07-25 23:45:13 -07:00
Jim Barlow 9adb0d696f Prepare for Python packaging - move to ocrmypdf folder 2015-07-25 18:22:04 -07:00
Jim Barlow c270f1ba5f Update release notes so far 2015-07-25 18:18:37 -07:00
Jim Barlow 7b255b575a Metadata override from command lien 2015-07-25 18:12:25 -07:00
Jim Barlow d7a9f3a2ab Transfer Unicode document information from input PDF to output PDF
What a pain getting Unicode right, but there it is.

I cannot find anything to confirm that it is acceptable to put the PDF/A
definition file at the end of the Ghostscript inputs.  I did this because
Ghostscript seems to copy document info from the last document on the
list so reportlab's information "wins" in normal order, so it fixes that
issue, and reportlab 'helpfully' fills in all of those fields even if it
does not have information.

It could also work to pass document information along to reportlab, and
set it in each output PDF: .debug.pdf, .rendered.pdf, and .page.pdf to
ensure that whatever page is last in the pipeline has the right
information. Or perhaps it's possible to write a Postscript trailer that
overwrites any previous docinfo with no side effects, but I can't find
any information on how to do that.  I don't think it's worth pursuing
unless this arrangement causes some problem with PDF/A generation.

On a minor note, Jhove misreads the way I have encoded the strings in
producing its validation log.  It reads them as UTF-16 little endian, so
will tend to produce a string of Asian characters in place of the real
data.
2015-07-25 18:05:25 -07:00
Jim Barlow abf2e7e9bb Copy document metadata from source document into output (untested)
This works for ASCII only; will do Unicode version.
2015-07-25 15:31:02 -07:00
Jim Barlow 72e5fa9ba0 Reimplement debug pages 2015-07-25 14:14:02 -07:00
Jim Barlow 32c1078d2c Reimplement skip text pages 2015-07-25 14:13:32 -07:00
Jim Barlow 133f901a69 Change @subdivide to @split
@split is for "1 to many" operations, so it's the right tool for this
case.
2015-07-25 02:58:34 -07:00
Jim Barlow 42cd683ec0 Try to make pdfinfo less obnoxious by printing too many decimals 2015-07-25 02:47:59 -07:00
Jim Barlow 151eb05377 For now, unpaper is the only deskew provider 2015-07-25 01:46:16 -07:00
Jim Barlow 16177d0a52 Remove ability to override temporary (working) folder
Little point to this feature - on most platforms the environment
variable can be overridden if desired to set a new root location.

At the same time, this change removes the ability to resume a partially
executed pipeline by deleting all of the results on failure.  If -k is
provided then the temporary files will survive but there's no way to
resume from them.  Because resuming doesn't really work away and would
only be useful to users experiencing very specific problems, this is
probably not worth it, so no major loss.  The intent of -k is to assist
debugging.
2015-07-25 01:45:26 -07:00
Jim Barlow 5ce544289f Automatically try to use all available CPUs 2015-07-25 01:10:14 -07:00
Jim Barlow 77bd35c3c7 Remove duplicate test folder 2015-07-25 01:00:40 -07:00
Jim Barlow 0c5c208db0 Goodbye, so long, farewell, shell... 2015-07-25 00:57:07 -07:00
Jim Barlow 60eb745331 Split selecting final image and render PDF result into separate tasks
Simplifies the logic - one deals with all images, the other details
with an image and .hocr. Also add JPEG reconversion.
2015-07-25 00:54:00 -07:00
Jim Barlow 9f90b5cb0a Modularize unpaper; get -d and -c working again 2015-07-25 00:22:56 -07:00
Jim Barlow 5adff94545 Remove more dead/old code 2015-07-24 15:41:24 -07:00
Jim Barlow aa2baabfa9 Implement deskew and clean using unpaper 2015-07-24 15:19:37 -07:00
Jim Barlow 75c2b23efc Cleanup externals 2015-07-24 02:01:19 -07:00
Jim Barlow 6451017962 Implement oversample 2015-07-24 01:56:44 -07:00
Jim Barlow 0f857a6a34 Put .rendered.pdf files into temp folder 2015-07-24 01:56:19 -07:00
Jim Barlow 7638a88a6a Change 'clean' to 'repair' for clarity since 'clean' is what unpaper does 2015-07-24 01:55:54 -07:00
Jim Barlow bed12d2021 Remove 'pdftoppm' renderer
Ghostscript is more reliable than Poppler's pdftoppm renderer. gs is
also a hard dependency, as the only open source tool that can produce
a PDF/A file, while Poppler could be removed.  pdftoppm has awkward
syntax with some special handling needed for different versions.  I have
found isolated rendering bugs with pdftoppm as well.

With that, I'm removing supporting for multiple rasterizers.

A minor advantage of pdftoppm is that its code produced JPEGs where
possible, but this can be achieved with gs.
2015-07-24 01:35:33 -07:00
Jim Barlow 587569fcb6 Tidy up 2015-07-24 01:27:01 -07:00
Jim Barlow 8c0dc9a06d Platform independent search for iccprofiles for PDF/A 2015-07-24 01:18:46 -07:00
Jim Barlow 289e4025ad First successful PDF/A produced by new pipeline 2015-07-23 23:28:32 -07:00
Jim Barlow 5476eafe4c Rasterize PDF pages and generate .hocr files 2015-07-23 23:09:29 -07:00
Jim Barlow df32f283cd Langauge checking 2015-07-23 18:38:59 -07:00
Jim Barlow 68ecaac9cc Add tesseract version check 2015-07-23 17:06:00 -07:00
Jim Barlow cffd4623ca Add PDF/A validation 2015-07-23 17:05:34 -07:00
Jim Barlow 6dc2782e80 Can now generate PDF/A files, multipage and single page 2015-07-23 04:57:31 -07:00
Jim Barlow 5df187c086 Wrap a proxy around pdfinfo block so it can be passed around processes 2015-07-23 03:49:30 -07:00
Jim Barlow 7fd172e41e Get rid of chdir, replace deprecated @split with @subdivide 2015-07-23 03:09:03 -07:00
Jim Barlow 619528a1b5 Try a method for passing along the pdfinfo struct 2015-07-23 02:39:42 -07:00
Jim Barlow 596d468c14 Reinstate WrapperLogger with more multiprocessing fixes 2015-07-23 02:26:09 -07:00
Jim Barlow eddbf1060a diff --git a/src/ocrmypdf.py b/src/ocrmypdf.py
index 68d1591..95afa8f 100755
--- a/src/ocrmypdf.py
+++ b/src/ocrmypdf.py
@@ -24,6 +24,7 @@ import ruffus.cmdline as cmdline
 from .hocrtransform import HocrTransform

 import warnings
+import multiprocessing

 warnings.simplefilter('ignore', pypdf.utils.PdfReadWarning)

@@ -96,7 +97,7 @@ debugging.add_argument(
     '-k', '--keep-temporary-files', action='store_true',
     help="keep temporary files (helpful for debugging)")
 debugging.add_argument(
-    '-g' ,'--debug-rendering', action='store_true',
+    '-g', '--debug-rendering', action='store_true',
     help="render each page twice with debug information on second page")

@@ -106,51 +107,19 @@ if not options.temp_folder:
     options.temp_folder = 'tmp'

-_logger, _logger_mutex = cmdline.setup_logging(__name__, options.log_file,
-                                               options.verbose)
+log, log_mutex = cmdline.setup_logging(__name__, options.log_file,
+                                       options.verbose)

-class WrappedLogger:
-
-    def __init__(self, my_logger, my_mutex):
-        self.logger = my_logger
-        self.mutex = my_mutex
-
-    def log(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.log(*args, **kwargs)
-
-    def debug(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.debug(*args, **kwargs)
-
-    def info(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.info(*args, **kwargs)
-
-    def warning(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.warning(*args, **kwargs)
-
-    def error(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.error(*args, **kwargs)
-
-    def critical(self, *args, **kwargs):
-        with self.mutex:
-            self.logger.critical(*args, **kwargs)
-
-log = WrappedLogger(_logger, _logger_mutex)
-
-
-def re_symlink(input_file, soft_link_name, log=log):
+def re_symlink(input_file, soft_link_name, log, mutex):
     """
     Helper function: relinks soft symbolic link if necessary
     """
     if input_file == soft_link_name:
-        log.debug("Warning: No symbolic link made. You are using " +
-                     "the original data directory as the working directory.")
+        with mutex:
+            log.debug("Warning: No symbolic link made. You are using " +
+                      "the original data directory as the working directory.")
         return

@@ -161,12 +130,14 @@ def re_symlink(input_file, soft_link_name, log=log):
         try:
             os.unlink(soft_link_name)
         except:
-            log.debug("Can't unlink %s" % (soft_link_name))
+            with mutex:
+                log.debug("Can't unlink %s" % (soft_link_name))

     if not os.path.exists(input_file):
         raise Exception("trying to create a broken symlink to %s" % input_file)

-    log.debug("os.symlink(%s, %s)" % (input_file, soft_link_name))
+    with mutex:
+        log.debug("os.symlink(%s, %s)" % (input_file, soft_link_name))

     os.symlink(
2015-07-23 02:22:12 -07:00
Jim Barlow 33731a6864 Move pageinfo code out of the pipeline 2015-07-23 02:17:13 -07:00
Jim Barlow 0c36cd2e24 Fix errors related to use working directory
Mainly workaround lack of @split(...output_dir) in ruffus
2015-07-23 01:16:05 -07:00
Jim Barlow 5cef1be26d New pipeline runs, splits pages 2015-07-22 22:58:13 -07:00
Jim Barlow e89f482c3d Fixes from early testing of new pipeline 2015-07-22 22:51:38 -07:00
Jim Barlow fe3e40305d Learn to split PDF into pages 2015-07-22 22:46:00 -07:00
Jim Barlow a92b5ceb6b Begin unifying main script and page script 2015-07-22 22:30:00 -07:00
Jim Barlow 0e7e7d8437 Suppress the xref warning for now 2015-07-22 11:24:14 -07:00
Jim Barlow f47fa98f33 Fixes to colorspace and other inquiries 2015-07-22 11:24:06 -07:00
Jim Barlow d3d5879911 Replace pdfimages -list call to poppler with PyPDF test for image
The immediate reason for doing this is that (newer?) versions of parse()
seem to choke on the parse string. It appears to trigger exponential
behavior in the underlying regex. In any case, replacing subprocesses
with native Python is usually better.
2015-07-22 11:22:12 -07:00
Jim Barlow b2168e11db Require Py3 for tests 2015-07-22 11:21:33 -07:00
Jim Barlow 6d5d8be708 New test: check skew 2015-07-22 04:00:59 -07:00
Jim Barlow ce2dbdf372 Add another test 2015-07-22 03:16:19 -07:00
Jim Barlow ec8a35a7a6 Basic test cases 2015-07-22 02:59:25 -07:00
Jim Barlow f6577c22c3 Complete wrapping of logger/logger_mutex 2015-07-22 02:57:13 -07:00
Jim Barlow 43d6c03093 Implement oversampling in ocrpage.py 2015-03-27 18:32:55 -07:00
Jim Barlow 1870f116bb More consistent spacing 2015-03-24 23:05:42 -07:00
Jim Barlow 8b87def013 Don't presume two jobs 2015-03-24 23:04:49 -07:00
Jim Barlow de599d97b5 Tidy up readme 2015-03-24 23:04:33 -07:00
Jim Barlow 5d7e6b45c4 Cleanup logger 2015-03-24 22:46:33 -07:00
Jim Barlow c6091bcfe1 Change python2 -> python3 for readlink() 2015-03-24 22:36:13 -07:00
Jim Barlow 466a8a1318 It's now py3 that uses lxml, reportlab 2015-03-19 17:12:32 -07:00
Jim Barlow a99ba3b696 Add rudimentary support for combining OCR layer with existing content
It appears to be very fragile due to weaknesses in PyPDF. Better
option is probably to use pdftk's watermark feature.
2015-03-10 14:28:38 -07:00
Jim Barlow 9229f7c6cc Add option to render text as invisible OCR text
Prior to this change, hocrtransform would render printable text (black
on white) and then a fully opaque image on top of the text. According to
the PDF spec, text that is the output of OCR should be marked invisible,
so that PDF viewers /know/ it's OCR output in a document that might mix
OCR and text overlays. Another benefit is that PDF viewers would know
to skip rendering text if they are not smart enough to figure out the
image will completely overwrite it.

However, for debug, visible text is nice, so retain it as an option.
2015-02-22 12:43:27 -08:00
Jim Barlow bf114bb188 Clean up pixel transform logic with namedtuple 2015-02-21 14:14:34 -08:00
Jim Barlow b8eed2f861 More PEP8/lint 2015-02-21 13:00:46 -08:00
Jim Barlow ccb1e347be Call HocrTransform directly instead of through a subprocess 2015-02-20 17:20:48 -08:00
Jim Barlow 8698974f11 Rename hocrTransform -> hocrtransform 2015-02-20 16:47:36 -08:00
Jim Barlow f2c79c4341 Convert hocrtransform to py3 2015-02-20 16:38:24 -08:00
Jim Barlow 4966d1346b Module marker for src folder 2015-02-20 15:43:05 -08:00
Jim Barlow 4a9337f757 PEP8 2015-02-20 15:42:06 -08:00
Jim Barlow db311fb6a2 Add support for -b (skip big pages) 2015-02-20 15:26:33 -08:00
Jim Barlow 02c1dcec8e Remove filenames from .hocr files
As documented, Tesseract does not escape the filename when inserting it
into .hocr, potentially creating an invalid XML file as a result. Since
there is no use for the title, regex it and nuke it.
2015-02-13 13:41:14 -08:00
Jim Barlow 52dc74d3ce Support Tesseract 3.03 quirk: .html vs .hocr extension 2015-02-11 10:24:10 -08:00
Jim Barlow cc2af2bc15 Convert the final image to a JPEG if the original image was a JPEG
Of course, this introduces recompression artifacts, and is unnecessary
if no options are given that modify the final image (no -d, -c, -i).
But rather than worry about that, it would be better to ultimately find
a way to combine the original PDF page with the output PDF text in the
case where we want no changes to the original. This is good enough for
now.

The better option can apparently be achieved using pdftk background, or
probably better, PyPDF2's merge. If Tesseract PDF generation is used
then we need a way to remove the image. Tesseract PDF generation at 3.03
does layout better (I think) and also properly encodes the hidden layer,
which is less likely to give display issues (I think).
2015-02-11 10:23:45 -08:00
Jim Barlow 638c6db05d Use the appropriate PNG rendered given the types of image present 2015-02-11 03:32:00 -08:00
Jim Barlow f7db8d9aff Use Ghostscript -> PNG instead of pdftoppm for rendering
Ghostscript has the clunkiest imaginable syntax, obtuse documentation,
quirky behavior, and poor diagnostics... but it *actually works* unlike
pdftoppm/poppler which gets things wrong.

In this case I observed poppler incorrectly decompresses certain CCITT
encoded monochrome PDFs. So set up Ghostscript to do the job instead.

For the moment this performs monochrome -> RGB conversion via reportlab.
2015-02-11 03:13:07 -08:00
Jim Barlow 564fb7a87e Support Ghostscript 9.14's new color conversion engine (not portable)
The flag -dUseCIEColor is now deprecated, as it invokes the old engine
which introduces color errors. The new engine requires a PDF/A file
header with hardcoded location of a ICC profile to use, now included in
the project. Portable iterations should generate a PDFA_def.ps based on
the target system; for now OS X with homebrew is presumed.

I have selected sRGB since scanners tend to capture RGB and printing
is not a major consideration for PDF/A.

Also note all file paths given to gs must be absolute. May its creators
be forever haunted for their failure to document this unexpected quirk.
2015-02-09 15:33:49 -08:00
Jim Barlow 4d88e64774 Standardize tmpfile prefix 2015-02-09 15:02:49 -08:00
Jim Barlow 26f1163b46 Handle case where a page contains no images - don't OCR
It doesn't make much sense to do anything with an all vector page
except extract the page unmodified.
2015-02-08 20:05:54 -08:00
Jim Barlow 40058e99e0 Implement debug text only page option 2015-02-08 19:51:41 -08:00
Jim Barlow bece4c3e02 Describe what decision was made based on -f and -s and presence of text 2015-02-08 19:51:18 -08:00
Jim Barlow f0f6b57c87 When deciding on OCR, check for presence of text rather than a font
It appears to be possible to have a PDF with an embedded font that is
either unused or used only for whitespace. So check for some amount of
actual text instead.
2015-02-08 17:38:27 -08:00
Jim Barlow dc2a4ab044 Logic error 2015-02-08 17:33:35 -08:00
Jim Barlow b16d6f5b81 Implement skipping OCR when -s is specified
Appears to be necessary to disable each state of the pipeline that is
inactive, not just initial and terminal stages of an inactive segment.
If nothing else this makes what is going on more explicit.
2015-02-08 17:26:16 -08:00
Jim Barlow 69ce6ff7b5 Not a named param 2014-11-22 15:35:05 -08:00
Jim Barlow 32ba50b8dc Add Tesseract timeout to keep things reasonable 2014-11-14 02:06:23 -08:00
Jim Barlow 36aca45f35 The -dci options now work (and valid combinations thereof) 2014-11-14 00:23:22 -08:00
Jim Barlow 925290342d Leptonica deskew can handle .pnm input, unlike imagemagick 2014-11-13 23:20:25 -08:00
Jim Barlow 4dc0370c57 Add leptonica deskew 2014-11-13 16:53:26 -08:00
Jim Barlow b92f8e43f2 Run as a module instead 2014-11-13 16:52:53 -08:00
Jim Barlow 22b0733a1d Merge branch 'feature/findskew' into develop 2014-11-13 16:00:27 -08:00
Jim Barlow 6021684ab6 Attempt to fix multiprocessing pickling error 2014-11-13 15:58:57 -08:00
Jim Barlow f4b1d0cdfe Fix symlink error that occurs in multipage processing 2014-11-13 15:58:36 -08:00
Jim Barlow d0d8048621 Comments 2014-10-17 17:28:31 -07:00
Jim Barlow cfd119325d Use abspath instead of relpath for temporary directory symlink 2014-10-11 17:48:56 -07:00
Jim Barlow ad30833ffc Support missing tess_cfg_files parameter when omitted by OCRmyPDF.sh 2014-10-11 17:48:33 -07:00
Jim Barlow e5c79a6666 Use TIFFs as intermediates
pdftoppm in recent versions (0.26.4,5) seems to be incapable of
producing valid TIFFs, so have it dump a .pnm file and let ImageMagick
figure out how to convert it to TIFF. This is not ideal, but at least
it works.
2014-10-10 01:54:16 -07:00
Jim Barlow 63dc753c1b Standardize intermediate filenames better
convert .pnm -deskew <...> .pnm seems to have a bug that produces an
invalid .pnm file which later causes tesseract (specifically,
leptonica) to choke (using 3.02/1.71 as versions, respectively). Will
change pipeline to use tiffs internally since they are less stupid.
2014-10-10 01:30:43 -07:00
Jim Barlow 017bc1f252 Basic error handling 2014-10-10 01:07:46 -07:00
Jim Barlow bcd67c009d Sort of working, but fragile; uses tmp folder properly now 2014-10-10 00:35:49 -07:00
fritz-hh 635358884e start rewrite ocrmypdf in python 2014-10-09 22:53:08 +02:00
Jim Barlow 2f6cfafdfc Now produces a finished OCR-PDF page 2014-10-08 03:54:06 -07:00
Jim Barlow 25234fa30b First crack at Ruffus, working well 2014-10-08 03:21:28 -07:00
fritz-hh 5b17341804 Merge remote-tracking branch 'origin/v2.x' into v3.x 2014-10-07 22:06:05 +02:00
fritz-hh 9bedfa9a72 fixes #95
Exit if the output path points to a folder
Exit if the output path point to an existing file
2014-10-07 16:42:10 +02:00
fritz-hh e1f1220970 make clear it is a draft from v3.x branch 2014-10-03 16:23:02 +02:00
fritz-hh 5855bcd1fe Merge remote-tracking branch 'origin/v2.x' into v3.x 2014-10-03 16:21:49 +02:00
fritz-hh a14af5b9ee make clear it is a draft from v2.x branch 2014-10-03 16:18:20 +02:00
fritz-hh f11c03750e typo 2014-10-03 16:16:17 +02:00
fritz-hh ea5cfa40c1 Update ROADMAP.md 2014-10-03 16:13:56 +02:00
fritz-hh c562754d81 typo 2014-10-03 16:11:26 +02:00
fritz-hh 90d892512a roadmap usage updated 2014-10-03 16:09:59 +02:00
fritz-hh 9c6fedb15b usage corrected [-f|-s] 2014-10-03 16:07:06 +02:00
fritz-hh 3a7175115f roadmap arguments specified 2014-10-03 16:03:02 +02:00
fritz-hh 98c41f3223 typo in usage 2014-10-03 15:44:14 +02:00
fritz-hh d101e96e16 roadmap: better layout 2014-10-03 15:30:29 +02:00
fritz-hh a446b6c440 roadmap rename steps 2014-10-03 15:19:20 +02:00
fritz-hh b1fec0f1b1 roadmap detailed 2014-10-03 15:17:53 +02:00
fritz-hh 1dfdc93745 draft roadmap for v3.x 2014-10-03 15:02:17 +02:00
fritz-hh 6c5ee4095c default language now set in the config.sh file 2014-09-30 23:28:22 +02:00
fritz-hh 986fbf63a4 Introduce -s option + fix bug when -C no set
- Introduce -s option to no ocr pages containing fonts
- Solve issue with -f and -s if -C is not set
2014-09-30 23:16:31 +02:00
fritz-hh 2612105d32 correct download path 2014-09-29 22:29:25 +02:00
fritz-hh 954fe13f54 update release notes for v2.2-stable 2014-09-29 22:25:02 +02:00
fritz-hh bb5a00685e Make clear this is a draft 2014-09-28 21:10:04 +02:00
Jim Barlow dabbddb04e deskew and clean 2014-09-27 15:03:07 -07:00
fritz-hh 5f173e5acb return right return code
Python does not map the expression to its return code automatically, so
this line returns success regardless of the reportlab version installed.
(I also realized that hasattr is superfluous).
2014-09-27 00:53:10 +02:00
fritz-hh b28ff40aea remove reportlab patch. fixes #91
remove patch that was required for versions of reportlab <3.0 (fixed in
3.0 now)
patch was necessary in order to reduce size of graysclage / b&w images
in pdf
2014-09-26 23:58:19 +02:00
Jim Barlow fccfb4589e Moving quickly - we can now output .ppm files at correct resolution 2014-09-26 04:43:15 -07:00
Jim Barlow 5384c98013 Initial ocrpage.py rewrite into python3 2014-09-26 04:19:41 -07:00
fritz-hh 2ed2307573 Merge pull request #89 from jbarlow83/feature/readlink-osx
More portable solution (works also on OS X) to get OCRmyPDF.sh path (following simlinks)
2014-09-25 23:09:26 +02:00
Jim Barlow 3f8a2d8d3e Eliminate readlink entirely and do the same thing on all platforms 2014-09-25 13:47:35 -07:00
fritz-hh 1a13b7c85f Check if the input file exist
Previously I checked only if the folder in which the input file should
be exists
2014-09-25 22:03:45 +02:00
Jim Barlow d7130a1e56 Merge branch 'feature/keep-text-pages' into develop 2014-09-25 03:50:21 -07:00
Jim Barlow f69054cb17 Fix parameter order problems
Put TESS_CFG_FILES last because it is optional and can be blank. If
omitted it breaks the sequence of subsequent parameters. Also cleanup
text output in this new mode.
2014-09-25 03:50:01 -07:00
Jim Barlow 80dc6eca2c Merge branches 'feature/readlink-osx' and 'feature/keep-text-pages' into develop
Conflicts:
	OCRmyPDF.sh
2014-09-25 03:14:10 -07:00
Jim Barlow d250fbb3d6 Fix call to readlink on OS X
readlink -f is a GNU coreutils extension, so not available on OS X and
other platforms.
2014-09-25 03:11:27 -07:00
Jim Barlow 09bbe92611 Add command line option to skip pages that contain font data
If a page contains font data, the script would abort, unless -f was given,
in which case it would use pdftoppm to rasterize the font into a bitmap
and then attempt to OCR it. -f is almost certainly not what users want
unless they want to debug OCR or something.

If a PDF already has fonts it either was OCR'd already, or it is
a composite file containing, for example, some scanned documents appended
to a text report.  In the latter case, this -s option provides OCR on
pages that don't have it without changing those that do, and if a PDF
was completely OCRed it will be converted to PDF/A.  In batch jobs with
a mix of OCR and non-OCR the implicit conversion to PDF/A is also useful.
2014-09-25 02:43:40 -07:00
Jim Barlow 69d922e096 Check for missing pdftoppm when poppler installed with --disable-splash-output
When I upgraded to poppler 0.24.5, pdftoppm was not compiled because the
script had --disable-splash-output set for some reason.

For OS X Homebrew the solution is:
brew uninstall poppler
brew install poppler --with-splash-output
2014-09-25 02:30:29 -07:00
fritz-hh d510e7e4ae prevent new spurious jhove message to be displayed 2014-09-24 23:43:37 +02:00
fritz-hh 5893290dd9 update to jhove v1.11 2014-09-24 23:17:39 +02:00
fritz-hh 5c3bbc4031 typo in OCRmyPDF.sh 2014-09-22 21:22:38 +02:00
fritz-hh 27cd8cf0db add link to heise open source 2014-09-20 20:47:02 +02:00
fritz-hh b403016d5b Release notes updated for v2.1-stable 2014-09-20 19:50:32 +02:00
fritz-hh 5a81823969 Merge pull request #82 from orbitcowboy/v2.x
Fixed typo
2014-09-20 19:02:33 +02:00
fritz-hh 17801401cd Merge pull request #83 from DorianScholz/v2.x
- small changes to make this work on Ubuntu 12.04 called via symlink
- lowered minimum parallel version
2014-09-20 18:59:57 +02:00
Dorian Scholz 5c7b2a2a36 lowered minimum version for parallel to 20121122 2014-09-10 13:27:59 +02:00
Dorian Scholz 1db06de287 added BASEPATH to allow for execution via symlink 2014-09-10 13:26:14 +02:00
Martin Ettl 3904178d44 Fixed typo 2014-09-09 07:01:04 +02:00
fritz-hh 8bb9c3610c Merge pull request #81 from MoritzFago/v2.x
fixed tipo ghostcript to ghostscript
2014-09-08 18:31:00 +02:00
MoritzFago 7dcc382ccc fixed tipo ghostcript to ghostscript 2014-09-08 16:52:49 +02:00
fritz-hh b71fc807d2 Merge pull request #77 from andysigner/v2.x
Fixed typo in help text
2014-05-23 19:51:20 +02:00
Andy Signer 15d28d970a Fixed typo in help text 2014-05-23 12:41:31 +02:00
fritz-hh e083a860e9 Merge pull request #73 from andreas-christ/v2.x
Fixed typo in import of reportlab.
2014-04-27 23:20:48 +02:00
Andreas Christ 6463b9dd84 Fixed typo in import of reportlab. 2014-04-27 19:15:46 +02:00
fritz-hh c873de6ca4 Consider that the hocr file has not always the same name
Closes #72
2014-04-27 16:01:11 +02:00
fritz-hh b70863b47e support both older and newer versions of reportlab
closes #71
2014-04-27 15:53:20 +02:00
fritz-hh 3546f84c6d ignore *.pyc files 2014-04-27 15:46:55 +02:00
Jim Barlow 1d98917db9 Add command line option to skip pages that contain font data
If a page contains font data, the script would abort, unless -f was given,
in which case it would use pdftoppm to rasterize the font into a bitmap
and then attempt to OCR it. -f is almost certainly not what users want
unless they want to debug OCR or something.

If a PDF already has fonts it either was OCR'd already, or it is
a composite file containing, for example, some scanned documents appended
to a text report.  In the latter case, this -s option provides OCR on
pages that don't have it without changing those that do, and if a PDF
was completely OCRed it will be converted to PDF/A.  In batch jobs with
a mix of OCR and non-OCR the implicit conversion to PDF/A is also useful.
2014-02-06 23:11:54 -08:00
fritz-hh 1c34fd69cf RELEASE_NOTES update prior delivery of v2.0-stable 2014-01-25 22:14:05 +01:00
fritz-hh 4cf38404cc fixes #51
Allow tesseract 3.02.01 to be used.
Even 3.02.01 fails in few cases (see issue #28). I decided to allow this
version anyway because 3.02.02 is not yet available for some widespread
linux distributions
2014-01-25 21:58:50 +01:00
Jim Barlow 112fb5098b Expose pixFindSkew API 2014-01-21 21:36:41 -08:00
Jim Barlow 5ace6906c7 Bug fix: leptonica generates .png when asked to produce .pbm/pgm/ppm
Leptonica does not interpret those extensions correctly.  However, when
asked to produce a .pnm file, it will produce the expected .pbm/pgm/ppm
file depending on the input.  So ask it to produce a .pnm and then
adjust the extension.

And add a test case.
2014-01-21 21:35:58 -08:00
Jim Barlow 8cfbdaf0d0 Fix a silly typo, and other minor cleanup 2014-01-19 19:06:19 -08:00
Jim Barlow 6703434976 Replace ImageMagick-convert with Leptonica 2014-01-19 14:47:51 -08:00
Jim Barlow 62edc15cd7 Implement ctypes wrapper around Leptonica to access its deskew function
A few design notes:
Leptonica's deskew is far superior to ImageMagick's convert -deskew command --
around 30-40x faster.  Subjectively the output appears to this contributor to
be of higher quality as well.  The difference is the algorithm: ImageMagick
uses the complex Hough transform to find the skew angle, while Leptonica uses
the simpler method, Postl's variance of differential line sums -- conceptually, shear the image and check for straight horizontal.  In this case
simplicity wins.  Finding the skew angle is the bulk of the work.

Leptonica's author explains the advantages of his approach here:
http://www.leptonica.com/skew-measurement.html

Leptonica is the low-level library that Tesseract depends on.  Hence, this
project already depends on Leptonica.  Leptonica can read and write most
common image file types on its own.

Unfortunately its error handling is poor: it seldom returns any meaningful
error codes.  The best it manages is writing messages to stderr, which in
the context of a verbose script is just confusing since the error's source
is not indicated.  The problem is compounded by Tesseract's use of Leptonica,
which will produce exactly the same errors in some cases.  So we trap stderr
between calls to Leptonica and parse it for a few different types of error
message.

leptonica.py is Python 2/3 compatible and set up to provide access to other
Leptonica functions as needed.  Of particular interest are its orientation
detection (including flip and rotation errors) which it does by comparing
text ascenders to descenders.

There is a PyPI "pylepthonica" package, however it is out of date by a few
years, and it implements all of Leptonica with Python wrappers -- so it is
massive, with one .py file at 2.5 MB.  This module is loosely inspired by
pyleptonica but more modern, up to date, and contains only limited
functionality.
2014-01-19 14:28:52 -08:00
fritz-hh be830ddc31 List supported languages
In case lan is not supported, list the supported languages in the error
message
2014-01-18 22:22:19 +01:00
fritz-hh 18322b424f fixes #60
Check if the languages option provided to tesseract (-l) are supported
2014-01-18 21:38:22 +01:00
fritz-hh 6901c60db4 more robust way to check tesseract version
better way of checking if the tesseract version is compatible with the
script.
If the required tess version is 3.02.02, and the actual version is 3.03,
the script would have told before that the version is too old, because
303<30202, now it compares 3.03>3.0202
2014-01-18 21:02:15 +01:00
fritz-hh e369ce6766 config file: version updated to v2.0-rc2 2014-01-16 21:22:24 +01:00
fritz-hh 64e4e5d91e release notes updated for v2.0-rc2 2014-01-16 21:19:15 +01:00
fritz-hh efce7de9ae wording corrected 2014-01-15 23:08:26 +01:00
fritz-hh 38c64ac689 dependency to pdftk removed
concatenation is now done also with ghostscript
2014-01-15 21:23:42 +01:00
fritz-hh 6d203e3eee portability improvements + minor changes 2014-01-15 21:23:41 +01:00
fritz-hh 81f461e557 disclaimer added 2014-01-14 23:46:33 +01:00
fritz-hh 988bde1387 tmpfiles to $TMPDIR + better portability (mktemp)
mktemp: consider both FreeBSD/OSX and Linux OS having incompatible
syntax
From now on temporary files are saved in the folder specified by the
environment variable $TMPDIR
2014-01-14 22:57:10 +01:00
fritz-hh aedbabdbe8 merged pull request from oxplot 2014-01-14 22:29:41 +01:00
fritz-hh 6ed53e53c7 Readme improved 2014-01-14 19:47:28 +01:00
Mansour Behabadi a78630ce99 Make src scripts executable
Signed-off-by: Mansour Behabadi <mansour@oxplot.com>
2014-01-14 17:50:46 +11:00
Mansour Behabadi 6653066784 Use --gnu in parralell and XX for mktemp
Signed-off-by: Mansour Behabadi <mansour@oxplot.com>
2014-01-14 17:49:24 +11:00
fritz-hh e40f1fa081 better handling of ligatures: fixes #58 2014-01-13 23:13:15 +01:00
fritz-hh a872ce751d config file restructured
to be make which parameters are allowed to be changed by the user
2014-01-13 22:11:28 +01:00
fritz-hh 317846fbdc Check of tmp folder creation was successful 2014-01-13 22:05:26 +01:00
fritz-hh f581a55544 Merge pull request #57 from jbarlow83/for-upstream/tmpfolder
Fix temporary folder name generation collisions
2014-01-13 12:31:02 -08:00
fritz-hh 447b291e70 minor changes 2014-01-13 18:03:44 +01:00
fritz-hh 01d07253e8 indicate python2 to be used in header 2014-01-13 18:03:43 +01:00
fritz-hh 034a466094 Merge pull request #56 from jbarlow83/for-upstream/hocr-selfwidth
Fix AttributeError on self.width if Tesseract finds no OCR text
2014-01-13 08:44:16 -08:00
fritz-hh c6211e2335 Merge pull request #55 from jbarlow83/for-upstream/check-poppler
Verify that pdftoppm is the Poppler version, not xpdf version
2014-01-13 08:42:33 -08:00
Jim Barlow 1d03a6417d Verify that pdftoppm is the Poppler version, not xpdf version 2014-01-12 22:12:09 -08:00
Jim Barlow 1d62ef27a2 Fix AttributeError on self.width if Tesseract finds no OCR text
self.width remains undefined unless hOCR finds text.  It might not, if
a page contains only an image for example.

Full error message is:
AttributeError: ‘hocrTransform’ object has no attribute ‘width’
2014-01-12 22:10:15 -08:00
Jim Barlow 996048dc08 Fix temporary folder name generation collisions
First, the regular expression matches everything after the first period
in a filename.  Adding the $ make it match the last, so that filenames
such as “Report.1.pdf” get trimmed to “Report.1”.

Next use mktemp to get the OS to create a temporary folder.  It will
guarantee a unique directory name beginning with prefix, even if parallel
processes are at work.
2014-01-12 22:05:11 -08:00
fritz-hh bf02ee3bdc Resolved conflits with jbarlow83 pull request 2014-01-12 15:37:14 +01:00
fritz-hh a3c7fba02d minor changes (comments) 2014-01-11 22:26:29 +01:00
fritz-hh a8cd7febf6 remove spurious space in img number
Tell the script that "nbImg" is a number, so that leading/trailing
spaces are removed
2014-01-11 22:15:53 +01:00
fritz-hh 20c008b84f avoid spurious error msg if no image in pdf 2014-01-11 22:05:19 +01:00
fritz-hh 7cd73566be check if python libs are installed
Check if reportlab and lxml are installed, otherwise exist with an error
2014-01-11 17:08:26 +01:00
fritz-hh e56fd53d06 poppler syntax (rather than xpdf syntax) 2014-01-11 16:19:52 +01:00
fritz-hh 810b1b3b3e Merge pull request #48 from jbarlow83/for-upstream/osx-errors
Fix pdffonts error when filename contains a space
2014-01-11 07:10:12 -08:00
fritz-hh cb0b033fe7 Merge branch 'v2.x' of https://github.com/fritz-hh/OCRmyPDF into v2.x 2014-01-11 15:52:01 +01:00
fritz-hh 46f673a3b7 exit if bad parallel/tesseract version installed 2014-01-10 22:59:33 +01:00
fritz-hh 455303b3d4 parallel version added in RELEASE_NOTES 2014-01-10 22:12:58 +01:00
Jim Barlow 24a84d6380 Fix pdffonts error when filename contains a space 2014-01-09 16:44:24 -08:00
Jim Barlow 9aa2171052 Monkeypatch reportlab to output grayscale and monochrome colorspaces 2014-01-09 16:36:26 -08:00
Jim Barlow 3a46ea1f36 Merge branch 'for-upstream/pdftoppm-error' into for-upstream/mono 2014-01-09 16:20:05 -08:00
Jim Barlow d33779f301 Detect monochrome images and extract them as PBM (1 bpp) 2014-01-09 16:15:24 -08:00
Jim Barlow d6ea0793b8 Fix ocrPage.sh pdftoppm error on OS X 10.9 2014-01-09 16:04:37 -08:00
fritz-hh 4e5e5bb925 version changed to v2.x 2014-01-08 20:57:55 +01:00
fritz-hh 3232ed8e38 link to releases updated 2014-01-08 20:56:34 +01:00
fritz-hh 29d6748af8 release_notes and readme updated for v2.0-rc1 2014-01-07 23:13:42 +01:00
fritz-hh 828f195071 erroneous exit code corrected 2014-01-07 21:57:18 +01:00
fritz-hh b0b7e32783 fixes #40 and code cleanup 2014-01-07 21:51:15 +01:00
fritz-hh c1103c0248 check tesseract version
fixes #41
versions older than 3.02.02 are known to produce invalid hocr output (in
some cases)
2014-01-07 21:04:28 +01:00
fritz-hh 940a016e95 link to issue tracking system added 2014-01-06 23:12:15 +01:00
fritz-hh c6cc098e47 create symbolic links and not copy
If deskew and/or cleanup is not requested, do not copy the files, but
just create symbolic link.
This saves disk place and makes the script slightly quicker
2014-01-06 23:08:35 +01:00
fritz-hh 54f47ab89b Minor change 2014-01-06 22:41:43 +01:00
fritz-hh fc3de64dce Changed debug page name
In order to have the debug page after the normal panel in the final PDF
file
2014-01-06 22:41:29 +01:00
fritz-hh 414c4e3f3c round dpi value correctly 2014-01-06 22:32:11 +01:00
fritz-hh 6a9f38d31e removed unused variables 2014-01-06 22:23:41 +01:00
fritz-hh aa4256d35c fixes #44
The x/y resolutions are not computed separately anymore.
We do not check anymore if x and y resolutions are different (not
measure could anyway be taken if they were not equal...)
2014-01-06 22:23:00 +01:00
fritz-hh 8a1241ba44 minor changes (indentation and fct name) 2014-01-06 22:05:49 +01:00
fritz-hh 7eab052e0f Improved consistency of tmp file names 2014-01-06 22:00:58 +01:00
fritz-hh 552d19e36b v1.1-stable added in release notes 2014-01-06 20:09:46 +01:00
fritz-hh 463b04e795 typo 2014-01-06 20:09:12 +01:00
fritz-hh c0d8508264 minor change in log msg 2014-01-06 19:30:19 +01:00
fritz-hh 6ef4ba31e2 help and documentation improved 2014-01-05 22:02:12 +01:00
fritz-hh 10a3d26291 default PDI definition moved to cfg file 2014-01-05 22:01:45 +01:00
fritz-hh ab994b32ee explanations added for no_ligature cfg file 2014-01-05 22:01:10 +01:00
fritz-hh 9352b71d78 Copyright added 2014-01-05 22:00:12 +01:00
fritz-hh 71593421ed minor change 2014-01-05 21:22:31 +01:00
fritz-hh 2754970f37 Echo arguments of script in debug mode 2014-01-04 21:43:41 +01:00
fritz-hh 5945454597 Support for -f option
Fixes #16
2014-01-04 21:24:33 +01:00
fritz-hh 884dbce712 copyright years updated 2014-01-04 21:20:29 +01:00
fritz-hh 8ee1bc6598 Minor change 2014-01-04 21:19:51 +01:00
fritz-hh 7d76c46731 Check if page already contains a font 2014-01-04 18:05:21 +01:00
fritz-hh f8ccf42c06 path to tmp folder now defined in config.sh 2014-01-04 17:24:35 +01:00
fritz-hh 0abe0f1f10 minor change 2014-01-03 17:00:35 +01:00
fritz-hh ee8a5d80ff echo also java version in debug mode 2014-01-03 16:27:11 +01:00
fritz-hh f08893b5c8 Support section added 2014-01-03 16:13:17 +01:00
fritz-hh 41cd88506e Echo version of the used tools
Fixes #35
2014-01-03 15:59:51 +01:00
fritz-hh 081223b138 Delete 2013_09_LED_und_Energiesparlampen.pdf
file committed by mistake... So deleting it now
2013-12-31 23:38:38 +01:00
fritz-hh 4e60c9ba09 Warn user in case of low resolution 2013-12-30 23:55:26 +01:00
fritz-hh 95fe7cd3bc Oversampling + more than 1 img
- Oversampling resolution can now be set from the cmd line (-o option)
- If a page contains more than one image, warn the user but process the
page anyway with a default resolution
2013-12-30 23:44:38 +01:00
fritz-hh 79ec1d994e Automatic oversampling
- If resolution is too low (<250dpi) perform automatic oversampling of
the image
- comments improved
- log messages improved
2013-12-30 22:27:10 +01:00
fritz-hh 045362425f minor change 2013-12-30 19:16:29 +01:00
fritz-hh 2b2637fbc3 minor change 2013-12-30 18:21:03 +01:00
fritz-hh bfc4f7a28d better resolution handling (fixes #38)
- dpi computation moved to in dedicated function
- do not exit in case of resolution mismatch (fixes #38)
- comments improved
2013-11-29 10:37:09 +01:00
fritz-hh 407670e1f3 Minor change 2013-11-29 10:34:05 +01:00
fritz-hh d0671d81b5 New log level added (LOG_WARN) 2013-11-29 10:33:46 +01:00
fritz-hh 7a74ebbcc3 comments and log messages improved 2013-11-29 00:39:44 +01:00
fritz-hh 9e69800332 typo 2013-11-28 00:37:57 +01:00
fritz-hh 7542188592 Removed bashism
== does not exist in bourne shell
2013-11-27 23:44:45 +01:00
fritz-hh b4a23c005d fixes #34
tell GNU parallel to protect against evaluation by the sub shell (-q
flag).
This is required in case the file name passed as argument contains
special characters like "#"
2013-11-27 23:15:54 +01:00
fritz-hh 5e0f8be4b1 Various improvements
-Constants moved to config.sh
- Use "python2" cmd instead of "python"
- few other minor changes
2013-11-27 22:34:21 +01:00
fritz-hh 50dee55606 File Test_Issue_#28 renamed 2013-11-27 22:30:43 +01:00
fritz-hh da5cd01fe4 copyright line added 2013-05-06 23:13:29 +03:00
fritz-hh d3fb317d41 readme updated
new feature: Process several pages in parallel if more than one CPU core
is available
2013-05-06 21:54:41 +02:00
fritz-hh 88ddeb1fb6 OCRmyPDF.sh: added dependency to GNU parallel 2013-05-06 21:54:05 +02:00
fritz-hh f9e2e74bf3 Merge remote-tracking branch 'origin/v1.x' into v2.x 2013-05-06 21:35:34 +02:00
fritz-hh 87e01aff60 readme updated for v1.0-stable 2013-05-06 21:29:15 +02:00
fritz-hh 7e8481186a OCRmyPDF.sh: metadata not added anymore
Removed feature to add metadata in final pdf file (because it lead to to
final PDF file that does not comply to the PDF/A-1 format)
2013-05-06 21:26:33 +02:00
fritz-hh 2b0103a4e6 basic implementation of parallel page processing
- basic implementation of parallel page processing using GNU parallel
- processing around 40% faster on dual core processor
2013-05-05 22:33:54 +02:00
fritz-hh 064d4be83c Merge remote-tracking branch 'origin/v1.x' into v2.x
Conflicts:
	OCRmyPDF.sh

Fixes #31
2013-05-05 21:01:17 +02:00
fritz-hh ab536d5678 OCRmyPDF.sh: fixes issue for files having spaces
fixes #31
2013-05-05 20:56:45 +02:00
fritz-hh 9db805c4ad new file to OCR one page
Required to perform OCR of several pages in parallal (using GNU
parallel)
2013-05-05 20:45:27 +02:00
fritz-hh f7923a9761 OCRmyPDF.sh: few variables renamed for clarity 2013-05-05 20:44:03 +02:00
fritz-hh fd52650255 .gitattribute: handle *.jar and *.pdf as binary 2013-05-05 16:54:41 +02:00
fritz-hh f0fe295175 jhove config: fixes #29 2013-05-05 16:36:54 +02:00
fritz-hh 2f89aa3935 .gitignore corrected + jhove jar files added
.gitignore file corrected, because it prevented some required jhove
binary files from being checked in (jar files)
2013-05-04 22:01:03 +02:00
fritz-hh 5aa27343e0 delete test file 2013-05-04 21:58:01 +02:00
fritz-hh 5ce2841389 JHove: deleted doc + source
Deleted number of jhove files that are not required
(documentation and java source code mainly)
Goal: reduce size of the package
2013-05-04 21:55:39 +02:00
fritz-hh e4ffb58269 OCRmyPDF.sh: provision for parallel pages processing 2013-05-02 22:06:16 +02:00
fritz-hh 2ce3d9e19d added file to reproduce #28 2013-05-02 17:21:17 +02:00
fritz-hh 9271fe73a8 OCRmyPDF.sh: fixes #27
The fix should now be compatible to most implementation of grep
2013-05-02 16:51:46 +02:00
fritz-hh edaa70b97f OCRmyPDF.sh: fixes #25 and fixes #26
- In debug mode: compute and echo time required for processing
- Resolutions (x/y) that are nearly equal are not supported (because the
test did not take into account imprecision due to trauncation)
2013-05-01 15:58:55 +02:00
fritz-hh ab07f4deea OCRmyPDF.sh: handling of path with spaces
- corrected fct absolutePath() to handle path with spaces correctly
- pdf title metadata: split on case change file name
- change of owner/group/permission removed from code
- improved logging
2013-05-01 13:44:20 +02:00
fritz-hh beb1d7ab54 release notes: updated for v1.0-rc2 2013-04-29 12:27:43 +03:00
fritz-hh 5ce3e9bfec OCRmyPDF.sh: Version number updated 2013-04-29 12:19:19 +03:00
fritz-hh 2bed210a30 OCRmyPDF.sh: added metadata in final pdf file
- added metadata in final pdf file: fixes #4
- improved logging of PDF/A validation results
2013-04-28 22:18:34 +02:00
fritz-hh 2441551156 OCRmyPDF.sh: final pdf same owner & permissions
fixes #9
2013-04-28 15:54:31 +02:00
fritz-hh 15baca5e08 HocrTransform.py: exist if page size if not found
fixes #21
2013-04-28 14:56:14 +02:00
fritz-hh 062ef0ca3a OCRmyPDF.sh: keep tmp files in debug mode
fixes #22
2013-04-28 14:43:21 +02:00
fritz-hh 24b4686944 release notes: unpaper version added 2013-04-27 14:03:17 +03:00
fritz-hh d3d1c20ca2 Correct version number
fixes #19
2013-04-27 14:00:59 +03:00
fritz-hh 7f7b81154f Merge branch 'master' of https://github.com/fritz-hh/OCRmyPDF 2013-04-26 19:37:26 +02:00
fritz-hh 6372cec6b8 OCRmyPDF.sh: Fixed major problem with deskew
After deskew the images was cropped to the wrong size
2013-04-26 19:37:02 +02:00
fritz-hh 5ec875325e Update README.md 2013-04-26 18:46:18 +03:00
fritz-hh 4d80709cfd Update README.md 2013-04-26 18:43:15 +03:00
fritz-hh 2642c1b3d3 Update README.md 2013-04-26 18:00:58 +03:00
950 changed files with 50573 additions and 312642 deletions
+21
View File
@@ -0,0 +1,21 @@
bin/
build/
dist/
include/
lib/
ocrmypdf.egg-info/
staging/
.git/
.ruffus_history.sqlite
MANIFEST.in
*.sublime*
*.pdf
*.rst
*.pyc
*/*.pyc
*/*/*.pyc
*/*/*/*.pyc
*/*/*/*/*.pyc
*/*/*/*/*/*.pyc
*/*/*/*/*/*/*.pyc
*/*/*/*/*/*/*/*.pyc
+5 -19
View File
@@ -1,22 +1,8 @@
# Always use Unix convention for new lines
* text eol=lf
# Custom for Visual Studio
*.cs diff=csharp
*.sln merge=union
*.csproj merge=union
*.vbproj merge=union
*.fsproj merge=union
*.dbproj merge=union
# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
# These files are binary and should be left untouched
# (binary is a macro for -text -diff)
*.jar binary
*.pdf binary
*.PDF binary
+14 -164
View File
@@ -1,167 +1,17 @@
releasePackage.sh
tmp/
log/
#################
## Eclipse
#################
*.pydevproject
.project
.metadata
*.pyc
tests/output/
.ruffus_history.sqlite
*.sublime-*
/*.pdf
build/
dist/
*.egg-info/
venv/
*/test/output
bin/
tmp/
*.tmp
*.bak
*.swp
*~.nib
local.properties
.classpath
.settings/
.loadpath
# External tool builders
.externalToolBuilders/
# Locally stored "Eclipse launch configurations"
*.launch
# CDT-specific
.cproject
# PDT-specific
.buildpath
#################
## Visual Studio
#################
## Ignore Visual Studio temporary files, build results, and
## files generated by popular Visual Studio add-ons.
# User-specific files
*.suo
*.user
*.sln.docstates
# Build results
[Dd]ebug/
[Rr]elease/
*_i.c
*_p.c
*.ilk
*.meta
*.obj
*.pch
*.pdb
*.pgc
*.pgd
*.rsp
*.sbr
*.tlb
*.tli
*.tlh
*.tmp
*.vspscc
.builds
*.dotCover
## TODO: If you have NuGet Package Restore enabled, uncomment this
#packages/
# Visual C++ cache files
ipch/
*.aps
*.ncb
*.opensdf
*.sdf
# Visual Studio profiler
*.psess
*.vsp
# ReSharper is a .NET coding add-in
_ReSharper*
# Installshield output folder
[Ee]xpress
# DocProject is a documentation generator add-in
DocProject/buildhelp/
DocProject/Help/*.HxT
DocProject/Help/*.HxC
DocProject/Help/*.hhc
DocProject/Help/*.hhk
DocProject/Help/*.hhp
DocProject/Help/Html2
DocProject/Help/html
# Click-Once directory
publish
# Others
[Bb]in
[Oo]bj
sql
TestResults
*.Cache
ClientBin
stylecop.*
~$*
*.dbmdl
Generated_Code #added for RIA/Silverlight projects
# Backup & report files from converting an old project file to a newer
# Visual Studio version. Backup files are not needed, because we have git ;-)
_UpgradeReport_Files/
Backup*/
UpgradeLog*.XML
############
## Windows
############
# Windows image file caches
Thumbs.db
# Folder config file
Desktop.ini
#############
## Python
#############
*.py[co]
# Packages
*.egg
*.egg-info
dist
build
eggs
parts
bin
var
sdist
develop-eggs
.installed.cfg
# Installer logs
pip-log.txt
# Unit test / coverage reports
.coverage
.tox
#Translations
*.mo
#Mr Developer
.mr.developer.cfg
# Mac crap
.DS_Store
include/
lib/
pip-selfcheck.json
pyvenv.cfg
-17
View File
@@ -1,17 +0,0 @@
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
+87
View File
@@ -0,0 +1,87 @@
# OCRmyPDF
#
# VERSION 3.0.0
FROM debian:stretch
MAINTAINER James R. Barlow <jim@purplerock.ca>
# Add unprivileged user
RUN useradd docker \
&& mkdir /home/docker \
&& chown docker:docker /home/docker
# Update system and install our dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
locales \
ghostscript \
tesseract-ocr \
tesseract-ocr-deu tesseract-ocr-spa tesseract-ocr-eng tesseract-ocr-fra \
qpdf \
poppler-utils \
python3 \
python3-pip \
python3-venv \
python3-reportlab \
python3-pil
# Enforce UTF-8
# Borrowed from https://index.docker.io/u/crosbymichael/python/
RUN dpkg-reconfigure locales && \
locale-gen C.UTF-8 && \
/usr/sbin/update-locale LANG=C.UTF-8
ENV LC_ALL C.UTF-8
# Build unpaper 6.1
RUN apt-get install -y \
wget \
gcc \
libavformat-dev \
libavcodec-dev \
libavutil-dev \
autoconf \
automake \
make \
pkg-config \
xsltproc
WORKDIR /root
RUN wget https://github.com/Flameeyes/unpaper/archive/unpaper-6.1.tar.gz
RUN tar xf unpaper-6.1.tar.gz
WORKDIR /root/unpaper-unpaper-6.1
RUN autoreconf -i
RUN ./configure CFLAGS="-O2 -march=native -pipe -flto"
RUN make -j install
RUN apt-get remove -y \
gcc \
autoconf \
automake \
pkg-config \
xsltproc \
make
RUN apt-get autoremove -y && apt-get clean -y
RUN rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Set up a Python virtualenv and take all of the system packages, so we can
# rely on the platform packages rather than importing GCC and compiling them
RUN pyvenv /appenv \
&& pyvenv --system-site-packages /appenv
COPY . /application/
# Install application and dependencies
# In this arrangement Pillow and reportlab will be provided by the system
RUN . /appenv/bin/activate; \
pip install --upgrade pip \
&& pip install --no-cache-dir /application \
&& pip install --no-cache-dir -r /application/test_requirements.txt
USER docker
WORKDIR /home/docker
ENV DEFAULT_RUFFUS_HISTORY_FILE=/tmp/.{basename}.ruffus_history.sqlite
ENV OCRMYPDF_TEST_OUTPUT=/tmp/test-output
ENV OCRMYPDF_IN_DOCKER=1
# Must use array form of ENTRYPOINT
# Non-array form does not append other arguments, because that is "intuitive"
ENTRYPOINT ["/application/docker-wrapper.sh"]
+20
View File
@@ -0,0 +1,20 @@
Copyright (c) 2013-2015, The OCRmyPDF Authors
Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+3
View File
@@ -0,0 +1,3 @@
recursive-exclude tests/output *
include requirements.txt
include test_requirements.txt
+2 -332
View File
@@ -1,336 +1,6 @@
#!/bin/sh
##############################################################################
# Copyright (c) 2013: fritz-hh from Github (https://github.com/fritz-hh)
# Copyright (c) 2013-14: fritz-hh from Github (https://github.com/fritz-hh)
##############################################################################
VERSION="alpha0"
usage() {
cat << EOF
--------------------------------------------------------------------------------------
Script aimed at generating a searchable PDF file from a PDF file containing only images.
(The script performs optical character recognition of each respective page using the
tesseract engine)
Copyright: fritz from NAS4Free forum
Version: $VERSION
Usage: OCRmyPDF.sh [-h] [-v] [-g] [-k] [-d] [-c] [-i] [-l language] [-C filename] inputfile outputfile
-h : Display this help message
-v : Increase the verbosity (this option can be used more than once)
-g : Activate debug mode:
- Generates a PDF file containing each page twice (once with the image, once without the image
but with the OCRed text as well as the detected bounding boxes)
- Set the verbosity to the highest possible
-k : Do not delete the temporary files
-d : Deskew each page before performing OCR
-c : Clean each page before performing OCR
-i : Incorporate the cleaned image in the final PDF file (by default the original image
image, or the deskewed image if the -d option is set, is incorporated)
-l : Set the language of the PDF file in order to improve OCR results (default "eng")
Any language supported by tesseract is supported.
-C : Pass an additional configuration file to the tesseract OCR engine.
(this option can be used more than once)
Note: The configuration file must be available in the "tessdata/configs" folder
of your tesseract installation
inputfile : PDF file to be OCRed
outputfile : The PDF/A file to be generated
--------------------------------------------------------------------------------------
EOF
}
#################################################
# Get an absolute path from a relative path to a file
#
# Param1 : Relative path
# Returns: 1 if the folder in which the file is located does not exist
# 0 otherwise
#################################################
absolutePath() {
local wdsave absolutepath
wdsave="`pwd`"
! cd "`dirname "$1"`" 1> /dev/null 2> /dev/null && return 1
absolutepath="`pwd`/`basename $1`"
cd "$wdsave"
echo "$absolutepath"
return 0
}
# Initialization of constants
EXIT_BAD_ARGS="1" # possible exit codes
EXIT_BAD_INPUT_FILE="2"
EXIT_MISSING_DEPENDENCY="3"
EXIT_INVALID_OUPUT_PDFA="4"
EXIT_OTHER_ERROR="5"
LOG_ERR="0" # 0=only error messages
LOG_INFO="1" # 1=error messages and some infos
LOG_DEBUG="2" # 2=debug level logging
SRC="./src" # location of the source folder (except source of external tools like jhove)
JHOVE="./jhove/bin/JhoveApp.jar" # java SW for validating the final PDF/A
JHOVE_CFG="./jhove/conf/jhove.conf" # location of the jhove config file
# Initialization the configuration parameters with default values
VERBOSITY="$LOG_ERR" # default verbosity level
LAN="eng" # default language of the PDF file (required to get good OCR results)
KEEP_TMP="0" # do not delete the temporary files (default)
PREPROCESS_DESKEW="0" # 0=no, 1=yes (deskew image)
PREPROCESS_CLEAN="0" # 0=no, 1=yes (clean image to improve OCR)
PREPROCESS_CLEANTOPDF="0" # 0=no, 1=yes (put cleaned image in final PDF)
DEBUG_MODE="0" # 0=no, 1=yes (generates each PDF page twice, with and without image)
TESS_CFG_FILES="" # list of additional configuration files to be used by tesseract
# Parse optional command line arguments
while getopts ":hvgkdcil:C:" opt; do
case $opt in
h) usage ; exit 0 ;;
v) VERBOSITY=$(($VERBOSITY+1)) ;;
g) VERBOSITY="10"; DEBUG_MODE="1" ;;
k) KEEP_TMP="1" ;;
d) PREPROCESS_DESKEW="1" ;;
c) PREPROCESS_CLEAN="1" ;;
i) PREPROCESS_CLEANTOPDF="1" ;;
l) LAN="$OPTARG" ;;
C) TESS_CFG_FILES="$OPTARG $TESS_CFG_FILES" ;;
\?)
echo "Invalid option: -$OPTARG" >&2
usage
exit $EXIT_BAD_ARGS ;;
:)
echo "Option -$OPTARG requires an argument" >&2
usage
exit $EXIT_BAD_ARGS ;;
esac
done
# Remove the optional arguments parsed above.
shift $((OPTIND-1))
# Check if the number of mandatory parameters
# provided is as expected
if [ "$#" -ne "2" ]; then
echo "Exactly one mandatory argument shall be provided" >&2
usage
exit $EXIT_BAD_ARGS
fi
! absolutePath "$1" > /dev/null && echo "The folder in which the input file should be located does not exist. Exiting..." >&2 && exit $EXIT_BAD_ARGS
FILE_INPUT_PDF="`absolutePath "$1"`"
! absolutePath "$2" > /dev/null && echo "The folder in which the output file should be generated does not exist. Exiting..." >&2 && exit $EXIT_BAD_ARGS
FILE_OUTPUT_PDFA="`absolutePath "$2"`"
# set script path as working directory
cd "`dirname $0`"
[ $VERBOSITY -ge $LOG_INFO ] && echo "OCRmyPDF version: $VERSION"
# check if the required utilities are installed
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Checking if all dependencies are installed"
! command -v identify > /dev/null && echo "Please install ImageMagick. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v pdfimages > /dev/null && echo "Please install poppler-utils. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v pdftoppm > /dev/null && echo "Please install poppler-utils. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v pdftk > /dev/null && echo "Please install pdftk. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
[ $PREPROCESS_CLEAN -eq 1 ] && ! command -v unpaper > /dev/null && echo "Please install unpaper. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v tesseract > /dev/null && echo "Please install tesseract and tesseract-data. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v python > /dev/null && echo "Please install python, and the python libraries: reportlab, lxml. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v gs > /dev/null && echo "Please install ghostcript. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
! command -v java > /dev/null && echo "Please install java. Exiting..." >&2 && exit $EXIT_MISSING_DEPENDENCY
# Initialize path to temporary files
TMP_FLD="./tmp/`date +"%Y%m%d_%H%M"`.filename.`basename "$FILE_INPUT_PDF" | sed s/[.][^.]*//`"
FILE_SIZE_PAGES="$TMP_FLD/page-sizes.txt" # size in pt of the respective page of the input PDF file
FILES_OCRed_PDFS="${TMP_FLD}/*-ocred.pdf" # string matching all 1 page PDF files that need to be merged
FILE_OUTPUT_PDF="${TMP_FLD}/ocred.pdf" # name of the OCRed PDF file before conversion to PDF/A
FILE_VALIDATION_LOG="${TMP_FLD}/pdf_validation.log" # log file containing the results of the validation of the PDF/A file
# Create tmp folder
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Creating temporary folder"
rm -r -f "${TMP_FLD}"
mkdir -p "${TMP_FLD}"
# get the size of each pdf page (width / height) in pt (inch*72)
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Input file: Extracting size of each page (in pt)"
! identify -format "%w %h\n" "$FILE_INPUT_PDF" > "$FILE_SIZE_PAGES" \
&& echo "Could not get size of PDF pages. Exiting..." >&2 && exit $EXIT_BAD_INPUT_FILE
sed -I "" '/^$/d' "$FILE_SIZE_PAGES" # removing empty lines (last one should be)
numpages=`cat "$FILE_SIZE_PAGES" | wc -l | sed 's/^ *//g'`
[ $VERBOSITY -ge $LOG_INFO ] && echo "Input file: The file has $numpages pages"
# Itterate the pages of the input pdf file
cpt="1"
while read pageSize ; do
# add leading zeros to the page number
page=`printf "%04d" $cpt`
[ $VERBOSITY -ge $LOG_INFO ] && echo "Processing page $page"
# create the name of the required file
curOrigImg="$TMP_FLD/${page}_Image" # original image available in the current PDF page
# (the image file may have a different orientation than in the pdf file)
curHocr="$TMP_FLD/$page.hocr" # hocr file to be generated by the OCR SW for the current page
curOCRedPDF="$TMP_FLD/${page}-ocred.pdf" # PDF file containing the image + the OCRed text for the current page
curOCRedPDFDebug="$TMP_FLD/${page}-debug-ocred.pdf" # PDF file containing data required to find out if OCR worked correctly
# get width / height of PDF page (in pt)
widthPDF=`echo $pageSize | cut -f1 -d" "`
heightPDF=`echo $pageSize | cut -f2 -d" "`
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: size ${heightPDF}x${widthPDF} (h*w in pt)"
# extract raw image from pdf file to compute resolution
# unfortunatelly this image can have another orientation than in the pdf...
# so we will have to extract it again later
pdfimages -f $page -l $page -j "$FILE_INPUT_PDF" "$curOrigImg" 1>&2
# count number of extracted images
nbImg=`ls -1 "$curOrigImg"* | wc -l`
[ $nbImg -ne "1" ] && echo "Expecting exactly 1 image on page $page (found $nbImg). Exiting..." >&2 && exit $EXIT_BAD_INPUT_FILE
# Get characteristics of the extracted image
curImg=`ls -1 "$curOrigImg"*`
propCurImg=`identify -format "%w %h %[colorspace]" "$curImg"`
widthCurImg=`echo "$propCurImg" | cut -f1 -d" "`
heightCurImg=`echo "$propCurImg" | cut -f2 -d" "`
colorspaceCurImg=`echo "$propCurImg" | cut -f3 -d" "`
# switch height/width values if the image has not the right orientation
# we make here the assumption that vertical/horizontal dpi are equal
# we will check that later
if [ $((($heightPDF-$widthPDF)*($heightCurImg-$widthCurImg))) -lt 0 ]; then
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Extracted image has wrong orientation. Inverting image height/width values"
tmpval=$heightCurImg
heightCurImg=$widthCurImg
widthCurImg=$tmpval
fi
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: size ${heightCurImg}x${widthCurImg} (h*w pixel)"
# compute the resolution of the image
dpi_x=$(($widthCurImg*72/$widthPDF))
dpi_y=$(($heightCurImg*72/$heightPDF))
[ "$dpi_x" -ne "$dpi_y" ] && echo "X/Y resolutions not equal ($dpi_x/$dpi_y). Exiting..." >&2 && exit $EXIT_BAD_INPUT_FILE
dpi="$dpi_x"
# Identify if page image should be saved as ppm (color) or pgm (gray)
ext="ppm"
opt=""
if [ $colorspaceCurImg == "Gray" ]; then
ext="pgm"
opt="-gray"
fi
curImgPixmap="$TMP_FLD/$page.$ext"
curImgPixmapDeskewed="$TMP_FLD/$page.deskewed.$ext"
curImgPixmapClean="$TMP_FLD/$page.cleaned.$ext"
# extract current page as image with right orientation and resoltution
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Extracting image as $ext file (${dpi} dpi)"
! pdftoppm -f $page -l $page -r $dpi $opt "$FILE_INPUT_PDF" > "$curImgPixmap" \
&& echo "Could not extract page $page as $ext from $FILE_INPUT_PDF. Exiting..." >&2 && exit $EXIT_OTHER_ERROR
# if requested deskew image (without changing its size in pixel)
if [ "$PREPROCESS_DESKEW" -eq "1" ]; then
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Deskewing image"
! convert "$curImgPixmap" -deskew 40% -gravity center -extent ${heightCurImg}x${widthCurImg} "$curImgPixmapDeskewed" \
&& echo "Could not deskew \"$curImgPixmap\". Exiting..." >&2 && exit $EXIT_OTHER_ERROR
else
cp "$curImgPixmap" "$curImgPixmapDeskewed"
fi
# if requested clean image with unpaper to get better OCR results
if [ "$PREPROCESS_CLEAN" -eq "1" ]; then
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Cleaning image with unpaper"
! unpaper --dpi $dpi --mask-scan-size 100 \
--no-deskew --no-grayfilter --no-blackfilter --no-mask-center --no-border-align \
"$curImgPixmapDeskewed" "$curImgPixmapClean" 1> /dev/null \
&& echo "Could not clean \"$curImgPixmapDeskewed\". Exiting..." >&2 && exit $EXIT_OTHER_ERROR
else
cp "$curImgPixmapDeskewed" "$curImgPixmapClean"
fi
# perform OCR
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Performing OCR"
! tesseract -l "$LAN" "$curImgPixmapClean" "$curHocr" hocr $TESS_CFG_FILES 1> /dev/null 2> /dev/null \
&& echo "Could not OCR file \"$curImgPixmapClean\". Exiting..." >&2 && exit $EXIT_OTHER_ERROR
mv "$curHocr.html" "$curHocr"
# embed text and image to new pdf file
if [ "$PREPROCESS_CLEANTOPDF" -eq "1" ]; then
image4finalPDF="$curImgPixmapClean"
else
image4finalPDF="$curImgPixmapDeskewed"
fi
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Embedding text in PDF"
! python $SRC/hocrTransform.py -r $dpi -i "$image4finalPDF" "$curHocr" "$curOCRedPDF" \
&& echo "Could not create PDF file from \"$curHocr\". Exiting..." >&2 && exit $EXIT_OTHER_ERROR
# if requested generate special debug PDF page with visible OCR text
if [ $DEBUG_MODE -eq "1" ] ; then
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Page $page: Embedding text in PDF (debug page)"
! python $SRC/hocrTransform.py -b -r $dpi "$curHocr" "$curOCRedPDFDebug" \
&& echo "Could not create PDF file from \"$curHocr\". Exiting..." >&2 && exit $EXIT_OTHER_ERROR
fi
# delete temporary files created for the current page
# to avoid using to much disk space in case of PDF files having many pages
if [ $KEEP_TMP -eq 0 ]; then
rm "$curOrigImg"*.*
rm "$curHocr"
rm "$curImgPixmap"
rm "$curImgPixmapDeskewed"
rm "$curImgPixmapClean"
fi
# go to next page of the pdf
cpt=$(($cpt+1))
done < "$FILE_SIZE_PAGES"
# concatenate all pages
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Output file: Concatenating all pages"
! pdftk $FILES_OCRed_PDFS cat output "$FILE_OUTPUT_PDF" \
&& echo "Could not concatenate individual PDF pages (\"$FILES_OCRed_PDFS\") to one file. Exiting..." >&2 && exit $EXIT_OTHER_ERROR
# insert metadata (copy metadata from input file)
#echo "Output file: Inserting metadata"
# TODO (may work with pdftk update_info)
# the name of the file may be used as title
# convert the pdf file to match PDF/A format
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Output file: Converting to PDF/A"
! gs -dQUIET -dPDFA -dBATCH -dNOPAUSE -dUseCIEColor \
-sProcessColorModel=DeviceCMYK -sDEVICE=pdfwrite -sPDFACompatibilityPolicy=2 \
-sOutputFile=$FILE_OUTPUT_PDFA "$FILE_OUTPUT_PDF" 1> /dev/null 2> /dev/null \
&& echo "Could not convert PDF file \"$FILE_OUTPUT_PDF\" to PDF/A. Exiting..." >&2 && exit $EXIT_OTHER_ERROR
# validate generated pdf file (compliance to PDF/A)
[ $VERBOSITY -ge $LOG_DEBUG ] && echo "Output file: Checking compliance to PDF/A standard"
java -jar "$JHOVE" -c "$JHOVE_CFG" -m PDF-hul "$FILE_OUTPUT_PDFA" > "$FILE_VALIDATION_LOG"
grep -i "Status|Message" "$FILE_VALIDATION_LOG" # summary of the validation
[ $VERBOSITY -ge $LOG_DEBUG ] && cat "$FILE_VALIDATION_LOG"
# check the validation results
pdf_valid=1
grep -i 'ErrorMessage' "$FILE_VALIDATION_LOG" && pdf_valid=0
grep -i 'Status.*not valid' "$FILE_VALIDATION_LOG" && pdf_valid=0
grep -i 'Status.*Not well-formed' "$FILE_VALIDATION_LOG" && pdf_valid=0
! grep -i 'Profile:.*PDF/A-1' "$FILE_VALIDATION_LOG" && pdf_valid=0
# delete temporary files
if [ $KEEP_TMP -eq 0 ]; then
rm -r -f "${TMP_FLD}"
fi
[ $pdf_valid -ne 1 ] && echo "Output file: The generated PDF/A file is INVALID" >&2 && exit $EXIT_INVALID_OUPUT_PDFA
[ $VERBOSITY -ge $LOG_INFO ] && echo "Output file: The generated PDF/A file is VALID"
exit 0
python3 -m ocrmypdf.main "$@"
-40
View File
@@ -1,40 +0,0 @@
OCRmyPDF
========
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched
To get the script usage, call: ./OCmyPDF.sh -h
Features
--------
- Generate a searchable PDF/A file from a PDF file containing only images
- Keep the exact resolution of the original embedded images
- If requested deskew and / or clean the image before performing OCR
- Validate the generated file against the PDF/A specification using jhove
- Provides debug mode to enable easy verification of the OCR results
Motivation
----------
I searched the web for a free command line tool to OCR PDF files on linux/unix:
I found many, but none of them were really satisfying.
- Either they produced PDF files with misplaced text under the image (making copy/paste impossible)
- Or they changed the resolution of the embedded images
- Or they generated PDF file having a ridiculous big size
- Or they crashed when trying to OCR some of my PDF files
- Or they did not produce valid PDF files (even though they were readable with my current PDF reader)
- On top of that none of them produced PDF/A files (format dedicated for long time storage / archiving)
... so I decided to develop my own tool (using various existing scripts as an inspiration)
Install
--------
Download OCRmyPDF here: https://github.com/fritz-hh/OCRmyPDF/tags
Copy the file in onto your linux/unix machine and extract it.
Run: "sh ./OCRmyPDF.sh" to get the script usage
If not yet installed, the script will notify you about dependencies that need to be installed
+245
View File
@@ -0,0 +1,245 @@
OCRmyPDF
========
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to
be searched.
Main features
-------------
- Generates a searchable
`PDF/A <https://en.wikipedia.org/?title=PDF/A>`__ file from a regular PDF
only containing images
- Places OCRed text accurately below the image to ease copy / paste
- Keeps the exact resolution of the original embedded images
- or if requested oversamples the images before OCRing so as to get
better results
- When possible, copies input images directly to output without transcoding them,
to preserve image quality
- Keeps file size about the same
- If requested deskews and/or cleans the image before performing OCR
- Validates input and output files
- Provides debug mode to enable easy verification of the OCR results
- Processes several pages in parallel when more than one CPU core is
available
- Uses Tesseract OCR engine
For details: please consult the `release notes <RELEASE_NOTES.rst>`__
Motivation
----------
I searched the web for a free command line tool to OCR PDF files on
Linux/UNIX: I found many, but none of them were really satisfying.
- Either they produced PDF files with misplaced text under the image (making copy/paste impossible)
- Or they did not display correctly some escaped HTML characters located in the hOCR file produced by the OCR engine
- Or they changed the resolution of the embedded images
- Or they generated PDF files having a ridiculous big size
- Or they crashed when trying to OCR some of my PDF files
- Or they did not produce valid PDF files (even though they were readable with my current PDF reader)
- On top of that none of them produced PDF/A files (format dedicated for long time storage)
... so I decided to develop my own tool (using various existing scripts
as an inspiration)
Installation
------------
Download OCRmyPDF here: https://github.com/fritz-hh/OCRmyPDF/releases
You can install it to a Python virtual environment or system-wide.
Installing the Docker container
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For many users, installing the Docker container will be easier than installing all of OCRmyPDF's dependencies. For Windows, it is the only option.
If you have `Docker <https://docs.docker.com/>`__ installed on your system, you can install
a Docker container of the latest release.
Follow the Docker installation instructions for your platform. If you can run this command
successfully, your system is ready to download and execute the image::
docker run hello-world
OCRmyPDF will use all available CPU cores. By default, the VirtualBox machine instance on Windows and OS X has only a single CPU core enabled. Use the VirtualBox Manager to determine the name of your Docker container host, and then follow these optional steps to enable multiple CPUs::
# Optional
docker-machine stop "yourVM"
VBoxManage modifyvm "yourVM" --cpus 2 # or whatever number of core is desired
docker-machine start "yourVM"
eval $(docker-machine env "yourVM")
Assuming you have a Docker engine running somewhere, you can run these commands to download
the image::
docker pull jbarlow83/ocrmypdf
Then tag it to give a more convenient name, just ocrmypdf::
docker tag jbarlow83/ocrmypdf ocrmypdf
You can then run using the command::
docker run ocrmypdf --help
To execute the OCRmyPDF on a local file, you must `provide a writable volume to the Docker image <https://docs.docker.com/userguide/dockervolumes/>`__, such as this in this template::
docker run -v "$(pwd):/home/docker" <other docker arguments> ocrmypdf <your arguments to ocrmypdf>
In this worked example, the current working directory contains an input file called `test.pdf` and the output will go to `output.pdf`::
docker run -v "$(pwd):/home/docker" ocrmypdf --skip-text test.pdf output.pdf
Note that `ocrmypdf` has its own separate -v argument to control debug verbosity. All Docker arguments should before the `ocrmypdf` container name and all arguments to `ocrmypdf` should be listed after.
Installing on Mac OS X Yosemite
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If it's not already present, `install Homebrew <http://brew.sh/>`__
Update Homebrew::
brew update
Install or upgrade the required Homebrew packages, if any are missing::
brew install libpng openjpeg jbig2dec # image libraries
brew install qpdf
brew install ghostscript
brew install python3
brew install libxml2
brew install leptonica
brew install tesseract
It is also recommended that install Pillow and confirm it can read and write JPEG and PNG files::
pip3 install --upgrade pip
pip3 install --upgrade pillow
To test that your Python imaging library (Pillow) can access JPEG and PNG files, try this command::
python3 -c "from PIL import Image; im = Image.new('1', (1, 1)); im.save('test.png'); im.save('test.jpg')"
If you have trouble getting Pillow to access JPEG and PNG files, `review the installation instructions <https://pillow.readthedocs.org/installation.html>`__.
You can then install OCRmyPDF from PyPI::
pip3 install ocrmypdf
The command line program should now be available::
ocrmypdf --help
Installing on Ubuntu 14.04 LTS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Installing on Ubuntu 14.04 LTS (trusty) is more difficult than other options, because of certain bugs in package installation.
Update apt-get::
sudo apt-get update
sudo apt-get upgrade
Install system dependencies::
sudo apt-get install \
zlib1g-dev \
libjpeg-dev \
ghostscript \
tesseract-ocr \
qpdf \
unpaper \
python3-pip \
python3-pil \
python3-pytest \
python3-reportlab
If you wish install OCRmyPDF to the system Python, then install as follows (note this installs new packages
into your system Python, which could interfere with other programs)::
sudo pip3 install ocrmypdf
If you wish to install OCRmyPDF to a virtual environment to isolate system Python from modified, you can
follow these steps. This includes a workaround `for a known, unresolved issue in Ubuntu 14.04's ensurepip
package <http://www.thefourtheye.in/2014/12/Python-venv-problem-with-ensurepip-in-Ubuntu.html>`__::
sudo apt-get install python3-venv
python3 -m venv venv-ocrmypdf --without-pip
source venv-ocrmypdf/bin/activate
wget -O - -o /dev/null https://bootstrap.pypa.io/get-pip.py | python
deactivate
pyvenv --system-site-packages venv-ocrmypdf
source venv-ocrmypdf/bin/activate
pip install ocrmypdf
Ubuntu 14.04 only installs `unpaper` version 0.4.2, which is not supported by OCRmyPDF because it is produces invalid output. This program is an optional dependency, and provides page deskewing and cleaning. See `Dockerfile <Dockerfile>`__ for an example of how to building unpaper 6.1 from source. If you choose to install unpaper later, OCRmyPDF will use the foremost version on the system PATH.
Installing HEAD revision from sources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To install the HEAD revision from sources in development mode::
git clone -b master https://github.com/fritz-hh/OCRmyPDF.git
cd OCRmyPDF
pip3 install -e .
On certain Linux/UNIX platforms such as Ubuntu, you may need to use
run the install command as superuser::
sudo pip3 install -e .
Note that this will alter your system's Python distribution. If you prefer
to not install as superuser, you can install the package in a Python virtual environment::
git clone -b master https://github.com/fritz-hh/OCRmyPDF.git
pyvenv venv
source venv/bin/activate
cd OCRmyPDF
pip3 install -e .
If your platform does not have ``pip3``, make sure that Python 3.4+ and the `pip`
package are installed.
To run the program::
ocrmypdf --help
If not yet installed, the script will notify you about dependencies that
need to be installed. The script requires specific versions of the
dependencies. Older version than the ones mentioned in the release notes
are likely not to be compatible to OCRmyPDF.
Support
-------
In case you detect an issue, please:
- Check if your issue is already known
- If no problem report exists on github, please create one here:
https://github.com/fritz-hh/OCRmyPDF/issues
- Describe your problem thoroughly
- Append the console output of the script when running the debug mode
(-v 1 option)
- If possible provide your input PDF file as well as the content of the
temporary folder (using a file sharing service like
www.file-upload.net)
Press & Media
-------------
- `c't 1-2014, page 59 <http://www.heise.de/ct/inhalt/2014/1/58/>`__:
Detailed presentation of OCRmyPDF v1.0 in the leading German IT
magazine c't
- `heise Open Source, 09/2014: Texterkennung mit
OCRmyPDF <http://www.heise.de/-2356670>`__
Disclaimer
----------
The software is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied.
-37
View File
@@ -1,37 +0,0 @@
RELEASE NOTES
=============
Please always read this file before installing the package
Download software here: https://github.com/fritz-hh/OCRmyPDF/tags
v1.0-rc1 (2013-04-26):
====
New features
------------
- First release candidate
Changes
-------
- N/A
Fixes
-----
- N/A
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0_17"
+518
View File
@@ -0,0 +1,518 @@
RELEASE NOTES
=============
Please always read this file before installing the package
Download software here: https://github.com/fritz-hh/OCRmyPDF/tags
v3.0:
=====
New features
------------
- Easier installation with a Docker container or Python's ``pip`` package manager
- Eliminated many external dependencies, so it's easier to setup
- Now installs ``ocrmypdf`` to ``/usr/local/bin`` or equivalent for system-wide
access and easier typing
- Improved command line syntax and usage help (``--help``)
- Tesseract 3.03+ PDF page rendering can be used instead for better positioning
of recognized text (``--pdf-renderer tesseract``)
- PDF metadata (title, author, keywords) are now transferred to the
output PDF
- PDF metadata can also be set from the command line (``--title``, etc.)
- Automatic repairs malformed input PDFs if possible
- Added test cases to confirm everything is working
- Added option to skip extremely large pages that take too long to OCR and are
often not OCRable (e.g. large scanned maps or diagrams); other pages are still
processed (``--skip-big``)
- Added option to kill Tesseract OCR process if it seems to be taking too long on
a page, while still processing other pages (``--tesseract-timeout``)
- Less common colorspaces (CMYK, palette) are now supported by conversion to RGB
- Multiple images on the same PDF page are now supported
Changes
-------
- New, robust rewrite in Python 3.4+ with ruffus_ pipelines
- Now uses Ghostscript 9.14's improved color conversion model to preserve PDF colors
- All "tasks" in the pipeline can be executed in parallel on any
available CPUs, increasing performance
- The ``-o DPI`` argument has been phased out, in favor of ``--oversample DPI``, in
case we need ``-o OUTPUTFILE`` in the future
- Removed several dependencies, so it's easier to install. We no
longer use:
- GNU parallel_
- ImageMagick_
- Python 2.7
- Poppler
- MuPDF_ tools
- shell scripts
- Java and JHOVE_
- libxml2
- Some new external dependencies are required or optional, compared to v2.x:
- Ghostscript 9.14+
- qpdf_ 5.0.0+
- Unpaper_ 6.1 (optional)
- some automatically managed Python packages
.. _ruffus: http://www.ruffus.org.uk/index.html
.. _parallel: https://www.gnu.org/software/parallel/
.. _ImageMagick: http://www.imagemagick.org/script/index.php
.. _MuPDF: http://mupdf.com/docs/
.. _qpdf: http://qpdf.sourceforge.net/
.. _Unpaper: https://github.com/Flameeyes/unpaper
.. _JHOVE: http://jhove.sourceforge.net/
Release candidates
------------------
- rc9:
- fix issue #118: report error if ghostscript iccprofiles are missing
- fixed another issue related to #111: PDF rasterized to palette file
- add support image files with a palette
- don't try to validate PDF file after an exception occurs
- rc8:
- fix issue #111: exception thrown if PDF is missing DocumentInfo dictionary
- rc7:
- fix error when installing direct from pip, "no such file 'requirements.txt'"
- rc6:
- dropped libxml2 (Python lxml) since Python 3's internal XML parser is sufficient
- set up Docker container
- fix Unicode errors if recognized text contains Unicode characters and system locale is not UTF-8
- rc5:
- dropped Java and JHOVE in favour of qpdf
- improved command line error output
- additional tests and bug fixes
- tested on Ubuntu 14.04 LTS
- rc4:
- dropped MuPDF in favour of qpdf
- fixed some installer issues and errors in installation instructions
- improve performance: run Ghostscript with multithreaded rendering
- improve performance: use multiple cores by default
- bug fix: checking for wrong exception on process timeout
- rc3: skipping version number intentionally to avoid confusion with Tesseract
- rc2: first release for public testing to test-PyPI, Github
- rc1: testing release process
Compatibility notes
-------------------
- ``./OCRmyPDF.sh`` script is still available for now
- Stacking the verbosity option like ``-vvv`` is no longer supported
- The configuration file ``config.sh`` has been removed. Instead, you can
feed a file to the arguments for common settings:
::
ocrmypdf input.pdf output.pdf @settings.txt
where ``settings.txt`` contains *one argument per line*, for example:
::
-l
deu
--author
A. Merkel
--pdf-renderer
tesseract
Fixes
-----
- Handling of filenames containing spaces: fixed
Notes and known issues
----------------------
- Some dependencies may work with lower versions than tested, so try
overriding dependencies if they are "in the way" to see if they work.
- ``--pdf-renderer tesseract`` will output files with an incorrect page size in Tesseract 3.03,
due to a bug in Tesseract.
- PDF files containing "inline images" are not supported and won't be for the 3.0 release. Scanned
images almost never contain inline images.
v2.2-stable (2014-09-29):
=========================
New features
------------
- None
Changes
-------
- Update to jhove v1.11
- Request the python library reportlab v3.0 or newer (So that we could remove a patch to the previous version of reportlab leading to issues for some users)
Fixes
-----
- Fix bug on Mac OS X (resolution of simlink to OCRmyPDF.sh script) (thanks to jbarlow83)
- Check if the input pdf file exists before to continue
Tested with
-----------
- Operating system: FreeBSD 9.2
- Dependencies:
- parallel 20140822
- poppler-utils 0.24.5
- ImageMagick 6.8.9-4 2014-09-17
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.8
- ghostcript (gs): 9.06
- java: openjdk version "1.7.0_65"
v2.1-stable (2014-09-20):
=========================
New features
------------
- None
Changes
-------
- None
Fixes
-----
- Allow execution via simlink
- Add support for tesseract 3.03
- Add support for newer version of reportlab
- Lowered minimum version of gnu parallel
- Various typo
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- parallel 20130222
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v2.0-stable (2014-01-25):
=========================
New features
------------
- Check if the language(s) passed using the -l option is supported by
tesseract (fixes #60)
Changes
-------
- Allow OCRmyPDF to be used with tesseract 3.02.01, even though OCR
might fail for few PDF file (see issue #28). Rationale: For some
linux distribution, no newer version than tesseract 3.02.01 is
available
Fixes
-----
- More robust algorithm for checking the version of the installed
tesseract package
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- parallel 20130222
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v2.0-rc2 (2014-01-16):
======================
New features
------------
- None
Changes
-------
- Size reduction of final PDF file: (fixes #50)
- Support for monochrome (Black&White) images (massive size reduction
in final PDF: >80%)
- Reduced size of grayscale images (by 13% on test PDF file)
- Preventing fi, fl ligatures does not require anymore to pass an
additional config file to tesseract using the -C option (fixes #58)
- Location of temporary folder according to content of environment
variable TMPDIR.
- Dependency to pdftk removed
- Check for compatible versions of dependencies: (fixes #51)
- parallel and tesseract
- python libraries reportlab and lxml
Fixes
-----
- Improved portability with various shells (dash, bash, tcsh) and OS
(FreeBSD, MAC OSX, Linux) (fixes #59)
- Corrected bug in case the input PDF file contains a space character
(fixes #48)
- Prevent spurious error message in case there is no image in a PDF
page
- Prevent collision of temporary folder names (fixes #57)
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- parallel 20130222
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v2.0-rc1 (2014-01-07):
======================
New features
------------
- Huge performance improvement on machines having multiple CPU/cores
(processing of several pages concurrently) (fixes #18)
- By default prevent from processing a PDF file already containing
fonts (i.e. text)(it can be overridden with the -f flag) (fixes #16)
- Warn if the resolution is too low to get reasonable OCR results
(fixes #37)
- New option (-o) to perform automatic oversampling if the image
resolution is too low. This can improve OCR results.
- Warn if using a tesseract version older than v3.02.02 (as older
versions are known to produce invalid output) (fixes #41)
- Echo version of the installed dependencies (e.g. tesseract) in debug
mode in order to ease support (fixes #35)
- Echo the arguments passed to the script in debug mode to ease support
Changes
-------
- In debug mode: The debug page is now placed after the respective
"normal" page
- Reduced disk space usage in temporary folder if -d (deskew) or -c
(cleanup) options are not selected
- New file src/config.sh containing various configuration parameters
- Documentation of the tesseract config file "tess-cfg/no\_ligature"
improved
- Improved consistency of the temporary file names
Fixes
-----
- Improved robustness:
- in case vertical resolution differs from horizontal resolution (fixes
#38)
- in case a PDF page contains more than one image (fixes #36)
- Fix a problem occurring if python 3 is the standard interpreter
(fixes #33)
- Fix a problem occurring if the input PDF file contains special
characters like "#" (fixes #34)
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- parallel 20130222
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v1.1-stable (2014-01-06):
=========================
New features
------------
- N/A
Changes
-------
- N/A
Fixes
-----
- Fixed syntax error (bashism) leading to an error message on certain
systems (fixes #42)
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v1.0-stable (2013-05-06):
=========================
New features
------------
- In debug mode: compute and echo time required for processing (fixes
#26)
Changes
-------
- Removed feature to add metadata in final pdf file (because it lead to
to final PDF file that does not comply to the PDF/A-1 format)
- Removed feature to set same owner & permissions in final PDF file
than in input file
- Removed many unused jhove files (e.g. documentation, \*.java and
\*.class files)
Fixes
-----
- Correction to handle correctly path and input PDF files having spaces
(fixes #31)
- Resolutions (x/y) that are nearly equal are now supported (fixes #25)
- Fix compatibility issue with Ubuntu server 12.04 / Ubuntu server
10.04 / Linux Mint 13 Maya and probably other Linux distributions
(fixes #27)
- Commit missing jhove files (\*.jar mainly) due to wrong .gitignore
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v1.0-rc2 (2013-04-29):
======================
New features
------------
- Keep temporary files if debug mode is set (fixes #22)
- Set same owner & permissions in final PDF file than in input file
(fixes #9)
- Added metadata in final pdf file (fixes #4)
Changes
-------
- N/A
Fixes
-----
- Fixed wrong image cropping when deskew option is activated
- Exit with error message if page size is not found in hocr file (fixes
#21)
- Various minor fixes in log messages
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
v1.0-rc1 (2013-04-26):
======================
New features
------------
- First release candidate
Changes
-------
- N/A
Fixes
-----
- N/A
Tested with
-----------
- Operating system: FreeBSD 9.1
- Dependencies:
- poppler-utils 0.22.2
- ImageMagick 6.8.0-7 2013-03-30
- Unpaper 0.3
- tesseract 3.02.02
- Python 2.7.3
- pdftk 1.45
- ghoscript (gs): 9.06
- java: openjdk version "1.7.0\_17"
+5
View File
@@ -0,0 +1,5 @@
#!/bin/bash
. /appenv/bin/activate
cd /home/docker
exec ocrmypdf "$@"
-502
View File
@@ -1,502 +0,0 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
[This is the first released version of the Lesser GPL. It also counts
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
Licenses are intended to guarantee your freedom to share and change
free software--to make sure the software is free for all its users.
This license, the Lesser General Public License, applies to some
specially designated software packages--typically libraries--of the
Free Software Foundation and other authors who decide to use it. You
can use it too, but we suggest you first think carefully about whether
this license or the ordinary General Public License is the better
strategy to use in any particular case, based on the explanations below.
When we speak of free software, we are referring to freedom of use,
not price. Our General Public Licenses are designed to make sure that
you have the freedom to distribute copies of free software (and charge
for this service if you wish); that you receive source code or can get
it if you want it; that you can change the software and use pieces of
it in new free programs; and that you are informed that you can do
these things.
To protect your rights, we need to make restrictions that forbid
distributors to deny you these rights or to ask you to surrender these
rights. These restrictions translate to certain responsibilities for
you if you distribute copies of the library or if you modify it.
For example, if you distribute copies of the library, whether gratis
or for a fee, you must give the recipients all the rights that we gave
you. You must make sure that they, too, receive or can get the source
code. If you link other code with the library, you must provide
complete object files to the recipients, so that they can relink them
with the library after making changes to the library and recompiling
it. And you must show them these terms so they know their rights.
We protect your rights with a two-step method: (1) we copyright the
library, and (2) we offer you this license, which gives you legal
permission to copy, distribute and/or modify the library.
To protect each distributor, we want to make it very clear that
there is no warranty for the free library. Also, if the library is
modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
restrictive license from a patent holder. Therefore, we insist that
any patent license obtained for a version of the library must be
consistent with the full freedom of use specified in this license.
Most GNU software, including some libraries, is covered by the
ordinary GNU General Public License. This license, the GNU Lesser
General Public License, applies to certain designated libraries, and
is quite different from the ordinary General Public License. We use
this license for certain libraries in order to permit linking those
libraries into non-free programs.
When a program is linked with a library, whether statically or using
a shared library, the combination of the two is legally speaking a
combined work, a derivative of the original library. The ordinary
General Public License therefore permits such linking only if the
entire combination fits its criteria of freedom. The Lesser General
Public License permits more lax criteria for linking other code with
the library.
We call this license the "Lesser" General Public License because it
does Less to protect the user's freedom than the ordinary General
Public License. It also provides other free software developers Less
of an advantage over competing non-free programs. These disadvantages
are the reason we use the ordinary General Public License for many
libraries. However, the Lesser license provides advantages in certain
special circumstances.
For example, on rare occasions, there may be a special need to
encourage the widest possible use of a certain library, so that it becomes
a de-facto standard. To achieve this, non-free programs must be
allowed to use the library. A more frequent case is that a free
library does the same job as widely used non-free libraries. In this
case, there is little to gain by limiting the free library to free
software only, so we use the Lesser General Public License.
In other cases, permission to use a particular library in non-free
programs enables a greater number of people to use a large body of
free software. For example, permission to use the GNU C Library in
non-free programs enables many more people to use the whole GNU
operating system, as well as its variant, the GNU/Linux operating
system.
Although the Lesser General Public License is Less protective of the
users' freedom, it does ensure that the user of a program that is
linked with the Library has the freedom and the wherewithal to run
that program using a modified version of the Library.
The precise terms and conditions for copying, distribution and
modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
program which contains a notice placed by the copyright holder or
other authorized party saying it may be distributed under the terms of
this Lesser General Public License (also called "this License").
Each licensee is addressed as "you".
A "library" means a collection of software functions and/or data
prepared so as to be conveniently linked with application programs
(which use some of those functions and data) to form executables.
The "Library", below, refers to any such software library or work
which has been distributed under these terms. A "work based on the
Library" means either the Library or any derivative work under
copyright law: that is to say, a work containing the Library or a
portion of it, either verbatim or with modifications and/or translated
straightforwardly into another language. (Hereinafter, translation is
included without limitation in the term "modification".)
"Source code" for a work means the preferred form of the work for
making modifications to it. For a library, complete source code means
all the source code for all modules it contains, plus any associated
interface definition files, plus the scripts used to control compilation
and installation of the library.
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running a program using the Library is not restricted, and output from
such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
appropriate copyright notice and disclaimer of warranty; keep intact
all the notices that refer to this License and to the absence of any
warranty; and distribute a copy of this License along with the
Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) The modified work must itself be a software library.
b) You must cause the files modified to carry prominent notices
stating that you changed the files and the date of any change.
c) You must cause the whole of the work to be licensed at no
charge to all third parties under the terms of this License.
d) If a facility in the modified Library refers to a function or a
table of data to be supplied by an application program that uses
the facility, other than as an argument passed when the facility
is invoked, then you must make a good faith effort to ensure that,
in the event an application does not supply such function or
table, the facility still operates, and performs whatever part of
its purpose remains meaningful.
(For example, a function in a library to compute square roots has
a purpose that is entirely well-defined independent of the
application. Therefore, Subsection 2d requires that any
application-supplied function or table used by this function must
be optional: if the application does not supply it, the square
root function must still compute square roots.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Library,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Library, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote
it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Library.
In addition, mere aggregation of another work not based on the Library
with the Library (or with a work based on the Library) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may opt to apply the terms of the ordinary GNU General Public
License instead of this License to a given copy of the Library. To do
this, you must alter all the notices that refer to this License, so
that they refer to the ordinary GNU General Public License, version 2,
instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
This option is useful when you wish to copy part of the code of
the Library into a program that is not a library.
4. You may copy and distribute the Library (or a portion or
derivative of it, under Section 2) in object code or executable form
under the terms of Sections 1 and 2 above provided that you accompany
it with the complete corresponding machine-readable source code, which
must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange.
If distribution of object code is made by offering access to copy
from a designated place, then offering equivalent access to copy the
source code from the same place satisfies the requirement to
distribute the source code, even though third parties are not
compelled to copy the source along with the object code.
5. A program that contains no derivative of any portion of the
Library, but is designed to work with the Library by being compiled or
linked with it, is called a "work that uses the Library". Such a
work, in isolation, is not a derivative work of the Library, and
therefore falls outside the scope of this License.
However, linking a "work that uses the Library" with the Library
creates an executable that is a derivative of the Library (because it
contains portions of the Library), rather than a "work that uses the
library". The executable is therefore covered by this License.
Section 6 states terms for distribution of such executables.
When a "work that uses the Library" uses material from a header file
that is part of the Library, the object code for the work may be a
derivative work of the Library even though the source code is not.
Whether this is true is especially significant if the work can be
linked without the Library, or if the work is itself a library. The
threshold for this to be true is not precisely defined by law.
If such an object file uses only numerical parameters, data
structure layouts and accessors, and small macros and small inline
functions (ten lines or less in length), then the use of the object
file is unrestricted, regardless of whether it is legally a derivative
work. (Executables containing this object code plus portions of the
Library will still fall under Section 6.)
Otherwise, if the work is a derivative of the Library, you may
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications.
You must give prominent notice with each copy of the work that the
Library is used in it and that the Library and its use are covered by
this License. You must supply a copy of this License. If the work
during execution displays copyright notices, you must include the
copyright notice for the Library among them, as well as a reference
directing the user to the copy of this License. Also, you must do one
of these things:
a) Accompany the work with the complete corresponding
machine-readable source code for the Library including whatever
changes were used in the work (which must be distributed under
Sections 1 and 2 above); and, if the work is an executable linked
with the Library, with the complete machine-readable "work that
uses the Library", as object code and/or source code, so that the
user can modify the Library and then relink to produce a modified
executable containing the modified Library. (It is understood
that the user who changes the contents of definitions files in the
Library will not necessarily be able to recompile the application
to use the modified definitions.)
b) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (1) uses at run time a
copy of the library already present on the user's computer system,
rather than copying library functions into the executable, and (2)
will operate properly with a modified version of the library, if
the user installs one, as long as the modified version is
interface-compatible with the version that the work was made with.
c) Accompany the work with a written offer, valid for at
least three years, to give the same user the materials
specified in Subsection 6a, above, for a charge no more
than the cost of performing this distribution.
d) If distribution of the work is made by offering access to copy
from a designated place, offer equivalent access to copy the above
specified materials from the same place.
e) Verify that the user has already received a copy of these
materials or that you have already sent this user a copy.
For an executable, the required form of the "work that uses the
Library" must include any data and utility programs needed for
reproducing the executable from it. However, as a special exception,
the materials to be distributed need not include anything that is
normally distributed (in either source or binary form) with the major
components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies
the executable.
It may happen that this requirement contradicts the license
restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
library, provided that the separate distribution of the work based on
the Library and of the other library facilities is otherwise
permitted, and provided that you do these two things:
a) Accompany the combined library with a copy of the same work
based on the Library, uncombined with any other library
facilities. This must be distributed under the terms of the
Sections above.
b) Give prominent notice with the combined library of the fact
that part of it is a work based on the Library, and explaining
where to find the accompanying uncombined form of the same work.
8. You may not copy, modify, sublicense, link with, or distribute
the Library except as expressly provided under this License. Any
attempt otherwise to copy, modify, sublicense, link with, or
distribute the Library is void, and will automatically terminate your
rights under this License. However, parties who have received copies,
or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
9. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Library or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Library (or any work based on the
Library), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Library or works based on it.
10. Each time you redistribute the Library (or any work based on the
Library), the recipient automatically receives a license from the
original licensor to copy, distribute, link with or modify the Library
subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Library at all. For example, if a patent
license would not permit royalty-free redistribution of the Library by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Library.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply,
and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
12. If the distribution and/or use of the Library is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Library under this License may add
an explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
13. The Free Software Foundation may publish revised and/or new
versions of the Lesser General Public License from time to time.
Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library
specifies a version number of this License which applies to it and
"any later version", you have the option of following the terms and
conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
copyrighted by the Free Software Foundation, write to the Free
Software Foundation; we sometimes make exceptions for this. Our
decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
If you develop a new library, and you want it to be of the greatest
possible use to the public, we recommend making it free software that
everyone can redistribute and change. You can do so by permitting
redistribution under these terms (or, alternatively, under the terms of the
ordinary General Public License).
To apply these terms, attach the following notices to the library. It is
safest to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least the
"copyright" line and a pointer to where the full notice is found.
<one line to give the library's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
Also add information on how to contact you by electronic and paper mail.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the library, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
<signature of Ty Coon>, 1 April 1990
Ty Coon, President of Vice
That's all there is to it!
-17
View File
@@ -1,17 +0,0 @@
JHOVE - JSTOR/Harvard Object Validation Environment
Copyright 2003-2008 by JSTOR and the President and Fellows of Harvard College
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU Lessor General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
USA
-227
View File
@@ -1,227 +0,0 @@
JHOVE - JSTOR/Harvard Object Validation Environment
Copyright 2003-2012 by JSTOR and the President and Fellows of Harvard College
JHOVE is made available under the GNU Lesser General Public License (LGPL;
see the file LICENSE for details)
Rev. 1.9, 2012-12-17
JHOVE (the JSTOR/Harvard Object Validation Environment, pronounced "jhove")
is an extensible software framework for performing format identification,
validation, and characterization of digital objects.
o Format identification is the process of determining the format to which a
digital object conforms: "I have a digital object; what format is it?"
o Format validation is the process of determining the level of compliance of a
digital object to the specification for its purported format: "I have an
object purportedly of format F; is it?"
o Format characterization is the process of determing the format-specific
significant properties of an object of a given format: "I have an object of
format F; what are its salient properties?"
These actions are frequently necessary during routine operation of digital
repositories and for digital preservation activities.
The output from JHOVE is controlled by output handlers. JHOVE uses an
extensible plug-in architecture; it can be configured at the time of its
invocation to include whatever specific format modules and output handlers
that are desired. The initial release of JHOVE includes modules for
arbitrary byte streams, ASCII and UTF-8 encoded text, AIFF and WAVE audio,
GIF, JPEG, JPEG 2000, TIFF, and PDF; and text and XML output handlers.
The JHOVE project is a collaboration of JSTOR and the Harvard University
Library. Development of JHOVE was funded in part by the Andrew W. Mellon
Foundation. JHOVE is made available under the GNU Lesser General Public
License (LGPL; see the file LICENSE for details).
JHOVE is currently being maintained by indpendent developers.
REQUIREMENTS
1. Java J2SE 1.5
(JHOVE was originally implemented using the Sun J2SE SDK 1.4.1 and has
been tested to work with 1.5)
2. If you would like to compile the JHOVE source code, then
Apache Ant, a Java-based build tool <http://ant.apache.org/> is necessary.
Note that the JAVA_HOME environment variable must be set appropriately for
Ant to work properly.
(JHOVE was implemented and tested using Ant 1.5.1.)
DISTRIBUTION
The JHOVE distribution package includes:
jhove/ # JHOVE home directory
COPYING # GNU Lesser General Public License
LICENSE # JHOVE license information
README
RELEASENOTES # JHOVE release notes
bin/
jhove.jar # JHOVE API package
jhove-handler.jar # Standard output handler package
jhove-module.jar # Standard module package
JhoveApp.jar # JHOVE command line application
JhoveView.jar # JHOVE with Swing GUI front-end
build.xml # Ant configuration file
classes/
build.xml # Ant configuration file
edu/ ... # JHOVE API packages
ADump.* # AIFF dump utility class
GDump.* # GIF dump utility class
Jhove.* # JHOVE main class
JDump.* # JPEG dump utility class
J2Dump.* # JPEG 2000 dump utility class
PDump.* # PDF dump utility class
TDump.* # TIFF dump utility class
UserHome.* # user.home property utility class
WDump.* # WAVE dump utility class
conf/
jhove.conf # JHOVE configuration file
jhove.xsd # JHOVE output schema
jhoveConfig.xsd # JHOVE configuration file schema
doc/
*.html # API documentation
...
examples/ # Sample files
ascii/ ...
gif/ ...
jpeg/ ...
jpeg2000/ ...
pdf/ ...
tiff/ ...
utf-8/ ...
adump* # AIFF dump Bourne shell driver
adump.bat* # AIFF dump DOS shell driver script
gdump* # GIF dump Bourne shell driver
gdump.bat* # GIF dump DOS shell driver script
jdump* # JPEG dump Bourne shell driver
jdump.bat* # JPEG dump DOS shell driver script
j2dump* # JPEG 2000 dump Bourne shell driver
j2dump.bat* # JPEG 2000 dump DOS shell driver
jhove.tmpl* # Template for JHOVE Bourne shell driver script
jhove_bat.tmpl* # Template for JHOVE DOS shell driver script
pdump* # PDF dump Bourne shell driver
pdump.bat* # PDF dump DOS shell driver script
tdump* # TIFF dump Bourne shell driver
tdump.bat* # TIFF dump DOS shell driver script
userhome* # user.home Bourne shell driver
userhome.bat* # user.home DOS shell driver script
wdump* # WAVE dump Bourne shell driver
wdump.bat* # WAVE dump DOS shell driver script
INSTALLATION
Edit the configuration file, jhove/conf/jhove.conf, and set the absolute
pathname of the JHOVE home directory and the temporary directory (in which
temporary files are created):
<jhoveHome>jhove-home-directory</jhoveHome>
<tempDirectory>temporary-directory</tempDirectory>
The JHOVE home directory is the top-most directory in the distribution TAR
or ZIP file. On Unix systems, "/var/tmp" is an appropriate temporary
directory; on Windows, "C:\Temp". For example, if the distribution TAR
file is disaggregated on a Unix system in the directory "/users/stephen/
projects", then the configuration file should read:
<jhoveHome>/users/stephen/projects/jhove</jhoveHome>
<tempDirectory>/var/tmp</jhoveHome>
In the JHOVE home directory, copy the JHOVE Bourne shell driver script
template, "jhove.tmpl", to "jhove" (or the equivalent Windows shell
script, "jhove_bat.tmpl" to "jhove.bat"), and set the
JHOVE home directory, Java home directory, and Java interpreter:
JHOVE_HOME=jhove-home-directory
JAVA_HOME=java-home-directory
JAVA=java-interpreter
The JAVA_HOME property should provide the absolute pathname of the Java
runtime or SDK installation; JAVA should provide the absolute pathname of the
Java interpreter. For example:
JHOVE_HOME=/users/stephen/projects/jhove
JAVA_HOME=/usr/local/j2re1.4.1_02
JAVA=$JAVA_HOME/bin/java
In the DOS shell driver script, jhove.bat, the equivalent three
variables are:
SET JHOVE_HOME=jhove-home-directory
SET JAVA_HOME=java-home-directory
SET JAVA=%JAVA_HOME%\bin\java
For example:
SET JHOVE_HOME="C:\Program Files\jhove"
SET JAVA_HOME="C:\Program Files\java\j2re1.4.1_02"
SET JAVA=%JAVA_HOME%\bin\java
The quotation marks are necessary because of the embedded space characters.
On Windows platforms it may also be necessary to add the Java bin subdirectory
to the System PATH environment variable:
PATH=C:\Program Files\java\j2re1.4.1_02\bin;...
(For information on setting a Windows environment variable, consult your local
documentation or system administrator.)
USAGE
java Jhove [-c config] [-m module] [-h handler] [-e encoding] [-H handler]
[-o output] [-x saxclass] [-t tempdir] [-b bufsize]
[-l loglevel] [[-krs] dir-file-or-uri [...]]
where -c config Configuration file pathname
-m module Module name
-h handler Output handler name (defaults to TEXT)
-e encoding Character encoding used by output handler (defaults to UTF-8)
-H handler About handler name
-o output Output file pathname (defaults to standard output)
-x saxclass SAX parser class (defaults to J2SE default)
-t tempdir Temporary directory in which to create temporary files
-b bufsize Buffer size for buffered I/O (defaults to J2SE 1.4 default)
-l loglevel Logging level
-k Calculate CRC32, MD5, and SHA-1 checksums
-r Display raw data flags, not textual equivalents
-s Format identification based on internal signatures only
dir-file-or-uri Directory or file pathname or URI of formated content
stream
All named modules and output handlers must be found on the Java CLASSPATH at
the time of invocation. The JHOVE driver script, jhove/jhove, automatically
sets the CLASSPATH and invokes the Jhove main class:
jhove [-c config] [-m module] [-h handler] [-e encoding] [-H handler]
[-o output] [-x saxclass] [-t tempdir] [-b bufsize] [-l loglevel]
[[-krs] dir-file-or-uri [...]]
The following additional programs are available, primarily for testing
and debugging purposes. They display a minimally processed, human-readable
version of the contents of AIFF, GIF, JPEG, JPEG 2000, PDF, TIFF, and WAVE
files:
java ADump aiff-file
java GDump gif-file
java JDump jpeg-file
java J2Dump jpeg2000-file
java PDump pdf-file
java TDump tiff-file
java WDump wave-file
For convenience, the following driver scripts are also available:
adump aiff-file
gdump gif-file
jdump jpeg-file
j2dump jpeg2000-file
pdump pdf-file
tdump tiff-file
wdump wave-file
The JHOVE Swing-based GUI interface can be invoked from a command shell from
the jhove/bin sub-directory:
java -jar JhoveView.jar -c <configFile>
where <configFile> is the pathname of the JHOVE configuration file.
-1676
View File
File diff suppressed because it is too large Load Diff
-78
View File
@@ -1,78 +0,0 @@
<project name="Jhove" default="dist" basedir=".">
<description>Project build file
Jhove - JSTOR/Harvard Object Validation Environment
Version 1.0 2004-09-10
Copyright 2004 by JSTOR and the President and Fellows of Harvard College
</description>
<!-- ant (or ant dist) Build everything
ant debug Build everything with debug enabled
ant clean Delete backup files
ant cleanclass Delete backup and class files
ant cleandist Delete backup, class, and jar files
ant javadoc Build javadocs
-->
<!-- set global properties for this build -->
<property name="bin" location="bin"/>
<property name="classes" location="classes"/>
<property name="doc" location="doc"/>
<target name="dist" description="Create distribution">
<ant dir="${classes}" inheritAll="false">
<property name="dbg" value="off"/>
</ant>
<chmod file="jhove" perm="ugo+x"/>
<chmod file="${bin}/JhoveView.jar" perm="ugo+x"/>
</target>
<target name="debug" description="Create distribution with debug enabled">
<ant dir="${classes}" inheritAll="false">
<property name="dbg" value="on"/>
</ant>
</target>
<target name="view" description="Create JhoveView application">
<ant dir="${classes}" target="view" inheritAll="false">
<property name="dbs" value="on"/>
</ant>
</target>
<target name="clean" description="Delete backup files">
<ant dir="${classes}" target="main-clean" inheritAll="false"/>
</target>
<target name="cleanclass" depends="clean">
<ant dir="${classes}" target="main-cleanclass" inheritAll="false"/>
</target>
<target name="cleandist" depends="cleanclass">
<delete file="${bin}/jhove.jar"/>
<delete file="${bin}/jhove-handler.jar"/>
<delete file="${bin}/jhove-module.jar"/>
<delete file="${bin}/JhoveApp.jar"/>
<delete file="${bin}/JhoveView.jar"/>
</target>
<target name="javadoc">
<javadoc sourcepath="${classes}" destdir="${doc}"
windowtitle="JHOVE Documentation"
Overview="${classes}/overview.html">
<package name="edu.harvard.hul.ois.jhove"/>
<package name="edu.harvard.hul.ois.jhove.handler"/>
<package name="edu.harvard.hul.ois.jhove.handler.audit"/>
<package name="edu.harvard.hul.ois.jhove.module"/>
<package name="edu.harvard.hul.ois.jhove.module.aiff"/>
<package name="edu.harvard.hul.ois.jhove.module.gif"/>
<package name="edu.harvard.hul.ois.jhove.module.html"/>
<package name="edu.harvard.hul.ois.jhove.module.iff"/>
<package name="edu.harvard.hul.ois.jhove.module.jpeg"/>
<package name="edu.harvard.hul.ois.jhove.module.jpeg2000"/>
<package name="edu.harvard.hul.ois.jhove.module.pdf"/>
<package name="edu.harvard.hul.ois.jhove.module.tiff"/>
<package name="edu.harvard.hul.ois.jhove.module.wave"/>
<package name="edu.harvard.hul.ois.jhove.module.xml"/>
<package name="edu.harvard.hul.ois.jhove.viewer"/>
</javadoc>
</target>
</project>
-235
View File
@@ -1,235 +0,0 @@
/**********************************************************************
* JDump - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2005 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.aiff.*;
import java.io.*;
/**
* Dump contents of AIFF file in human-readable format.
* @author Gary McGath
*/
public class ADump extends Dump {
/* Fixed value for first 4 bytes */
private static final int[] sigByte =
{ 0X46, 0X4F, 0X52, 0X4D };
private static final boolean ENDIAN = true; /* bigEndian */
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java ADump file");
System.exit (-1);
}
try {
FileInputStream file = new FileInputStream (args[0]);
BufferedInputStream buffer = new BufferedInputStream (file);
DataInputStream stream = new DataInputStream (buffer);
ADump dump = new ADump (); // Just to access contained classes
long os = 0;
for (int i=0; i<4; i++) {
int ch;
ch = stream.readUnsignedByte();
if (ch != sigByte[i]) {
System.out.println ("No AIFF FORM header");
System.exit (-2);
}
}
os += 4;
long ckSize = ModuleBase.readUnsignedInt (stream, ENDIAN);
// Read the file type
StringBuffer formType = new StringBuffer (4);
for (int i=0; i<4; i++) {
int ch = ModuleBase.readUnsignedByte(stream);
formType.append((char) ch);
}
System.out.println ("00000000: FORM " + ckSize + ": " + formType);
boolean aiff_c = false;
if (formType.toString ().equals ("AIFC")) {
aiff_c = true;
}
StringBuffer sbuf = new StringBuffer ();
boolean endOfFile = false;
while (!endOfFile) {
// Read chunks
try {
sbuf.setLength(0);
// Read chunk name.
for (int i=0; i<4; i++) {
int ch = ModuleBase.readUnsignedByte(stream);
sbuf.append((char) ch);
}
String ckID = sbuf.toString ();
// Read size (excluding chunk name and size fields)
ckSize = ModuleBase.readUnsignedInt (stream, ENDIAN);
System.out.print (leading (os, 8) + os + ": " + ckID +
" " + ckSize);
long alreadyRead = 0;
if (ckID.equals ("AESD")) {
int [] aes = new int[24];
for (int i=0; i<24; i++) {
aes[i] = ModuleBase.readUnsignedByte (stream);
}
System.out.print (": " + aes[0]);
for (int i=1; i<24; i++) {
System.out.print ("," + aes[i]);
}
alreadyRead = 24;
}
else if (ckID.equals ("ANNO") ||
ckID.equals ("AUTH") ||
ckID.equals ("(c) ") ||
ckID.equals ("NAME")) {
sbuf.setLength (0);
for (int i=0; i<ckSize; i++) {
int ch = ModuleBase.readUnsignedByte (stream);
sbuf.append ((char) ch);
}
System.out.print (": \"" + sbuf.toString () + "\"");
alreadyRead = ckSize;
}
else if (ckID.equals ("APPL")) {
sbuf.setLength (0);
for (int i=0; i<4; i++) {
int ch = ModuleBase.readUnsignedByte (stream);
sbuf.append ((char) ch);
}
System.out.print (": " + sbuf.toString ());
alreadyRead = 4;
}
else if (ckID.equals ("COMM")) {
int numChannels = ModuleBase.readSignedShort (stream,
ENDIAN);
long numSampleFrames =
ModuleBase.readUnsignedInt (stream, ENDIAN);
int sampleSize = ModuleBase.readSignedShort (stream,
ENDIAN);
byte [] buf = new byte[10];
for (int i=0; i<10; i++) {
buf[i] = (byte) ModuleBase.readSignedByte (stream);
}
ExtDouble xd = new ExtDouble (buf);
double sampleRate = xd.toDouble ();
System.out.print (": " + numChannels + " " +
numSampleFrames + " " + sampleSize +
" " + sampleRate);
alreadyRead = 18;
if (aiff_c) {
sbuf.setLength (0);
for (int i=0; i<4; i++) {
int ch = ModuleBase.readUnsignedByte (stream,
null);
sbuf.append ((char) ch);
}
System.out.print (" " + sbuf.toString ());
int count = ModuleBase.readUnsignedByte (stream,
null);
alreadyRead = 23;
sbuf.setLength (0);
for (int i=0; i<count; i++) {
int ch = ModuleBase.readUnsignedByte (stream,
null);
sbuf.append ((char) ch);
alreadyRead++;
}
System.out.print (" \"" + sbuf.toString () + "\"");
}
}
else if (ckID.equals ("COMT")) {
int numComments = ModuleBase.readUnsignedShort (stream,
ENDIAN,
null);
System.out.print (": " + numComments);
alreadyRead = 2;
}
else if (ckID.equals ("FVER")) {
long timestamp = ModuleBase.readUnsignedInt (stream,
ENDIAN,
null);
System.out.print (": " + timestamp);
alreadyRead = 4;
}
else if (ckID.equals ("INST")) {
int baseNote = ModuleBase.readSignedByte (stream);
int detune = ModuleBase.readSignedByte (stream);
int lowNote = ModuleBase.readSignedByte (stream);
int highNote = ModuleBase.readSignedByte (stream);
int lowVelocity = ModuleBase.readSignedByte (stream);
int highVelocity = ModuleBase.readSignedByte (stream);
int gain = ModuleBase.readSignedShort (stream, ENDIAN);
System.out.print (": " + baseNote + " " + detune +
" " + lowNote + "," + highNote +
" " + lowVelocity + "," +
highVelocity + " " + gain);
alreadyRead = 8;
}
else if (ckID.equals ("MARK")) {
int numMarkers = ModuleBase.readUnsignedShort (stream,
ENDIAN);
System.out.print (": " + numMarkers);
alreadyRead = 2;
}
else if (ckID.equals ("SSND")) {
long offset = ModuleBase.readUnsignedInt (stream,
ENDIAN);
long blockSize = ModuleBase.readUnsignedInt (stream,
ENDIAN);
System.out.print (": " + offset + " " + blockSize);
alreadyRead = 8;
}
System.out.println ();
// Actual number of bytes to skip must be even.
long actSize = ckSize;
if ((actSize & 1) != 0) {
actSize++;
}
actSize -= alreadyRead;
stream.skipBytes ((int) actSize);
os += ckSize + 8;
}
catch (EOFException e) {
endOfFile = true;
}
}
}
catch (Exception e) {
}
}
}
-264
View File
@@ -1,264 +0,0 @@
/**********************************************************************
* GDump - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
//import java.util.*;
/**
* Dump contents of GIF file in human-readable format.
*/
public class GDump
extends Dump
{
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java GDump file");
System.exit (-1);
}
try {
FileInputStream file = new FileInputStream (args[0]);
BufferedInputStream buffer = new BufferedInputStream (file);
DataInputStream stream = new DataInputStream (buffer);
boolean bigEndian = false;
String signature = readChars (stream, 3);
String version = readChars (stream, 3);
System.out.println ("00000000: \"" + signature + version + "\"");
int width = ModuleBase.readUnsignedShort(stream, bigEndian, null);
int height = ModuleBase.readUnsignedShort(stream, bigEndian, null);
int packedFields = stream.readUnsignedByte ();
int backgroundColor = stream.readUnsignedByte ();
int aspectRatio = stream.readUnsignedByte ();
System.out.println ("00000006: LogicalScreenDescriptor: " +
width + "x" + height + " 0x" +
leading (packedFields, 2) +
Integer.toHexString (packedFields) + " " +
backgroundColor + " " + aspectRatio);
long os = 13;
boolean globalColorTable = (packedFields & 0x80) != 0;
int globalColorTableSize = packedFields & 0x07;
if (globalColorTable) {
System.out.println (leading (os, 8) + os +
": GlobalColorTable:");
os = colorTable (stream, os, globalColorTableSize);
}
for (int op = 0; (op = stream.readUnsignedByte ()) != -1;) {
if (op == 0x00) {
System.out.println (leading (os, 8) + os +
": BlockTerminator");
os++;
}
else if (op == 0x2c) {
int imageLeft = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int imageTop = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int imageWidth = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int imageHeight = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
packedFields = stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os +
": ImageDescriptor: " + imageLeft +
"," + imageTop + " " + imageWidth +
"x" + imageHeight + " 0x" +
leading (packedFields, 2) +
Integer.toHexString (packedFields));
os += 10;
boolean localColorTable = (packedFields & 0x80) != 0;
int localColorTableSize = packedFields & 0x07;
if (localColorTable) {
System.out.println (leading (os, 8) + os +
": LocalColorTable:");
os = colorTable (stream, os, localColorTableSize);
}
int size = stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os +
": ImageData: " + size);
os = subBlocks (stream, ++os);
}
else if (op == 0x21) {
int label = stream.readUnsignedByte ();
if (label == 0x01) {
int blockSize = stream.readUnsignedByte ();
int gridLeft = ModuleBase.readUnsignedShort(stream,
bigEndian,
null);
int gridTop = ModuleBase.readUnsignedShort(stream,
bigEndian,
null);
int gridWidth = ModuleBase.readUnsignedShort(stream,
bigEndian,
null);
int gridHeight= ModuleBase.readUnsignedShort(stream,
bigEndian,
null);
int cellWidth = stream.readUnsignedByte ();
int cellHeight = stream.readUnsignedByte ();
int foreground = stream.readUnsignedByte ();
int background = stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os +
": PlainTextExtension: " +
blockSize + " " + gridLeft + "," +
gridTop + " " + gridWidth + "x" +
gridHeight + " " + cellWidth +
"x" + cellHeight + " " +
foreground + "," + background);
os += 15;
os = subBlocks (stream, os);
}
else if (label == 0xf9) {
int blockSize = stream.readUnsignedByte ();
packedFields = stream.readUnsignedByte ();
int delayTime = ModuleBase.readUnsignedShort(stream,
bigEndian,
null);
int colorIndex= stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os +
": GraphicControlExtension: " +
blockSize + " 0x" +
leading (packedFields, 2) +
packedFields + " " + delayTime +
" " + colorIndex);
os += 7;
}
else if (label == 0xfe) {
System.out.println (leading (os, 8) + os +
": CommentExtension: \"" +
"\"");
os += 2;
os = subBlocks (stream, os);
}
else if (label == 0xff) {
int blockSize = stream.readUnsignedByte ();
String appId = readChars (stream, 8);
int [] authCode = new int [3];
authCode[0] = stream.readUnsignedByte ();
authCode[1] = stream.readUnsignedByte ();
authCode[2] = stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os +
": ApplicationExtension: " +
blockSize + " \"" + appId +
"\" " + authCode[0] + "," +
authCode[1] + "," + authCode[2]);
os += 14;
os = subBlocks (stream, os);
}
else {
String hex = Integer.toHexString (label);
System.out.println (leading (os-1, 8) + (os-1) +
": Unknown extension block: 0x" +
leading (hex, 2) + hex);
os += 2;
}
}
else if (op == 0x3b) {
System.out.println (leading (os, 8) + os +
": Trailer: 0x3b");
os++;
break;
}
else {
String hex = Integer.toHexString (op);
System.out.println (leading (os, 8) + os +
": Unknown block: 0x" +
leading (hex, 2) + hex);
os++;
}
}
stream.close ();
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (-2);
}
}
/**
* Read and display a color table.
* @param stream Data input stream
* @param os Current byte offset
* @param size Color table size
* @return Updated byte offset
*/
private static long colorTable (DataInputStream stream, long os, int size)
throws IOException
{
int n = 2<<size;
for (int i=0; i<n; i++) {
long r = stream.readUnsignedByte ();
long g = stream.readUnsignedByte ();
long b = stream.readUnsignedByte ();
System.out.println (leading (os, 8) + os + ": " +
leading (i, 3, ' ') + i + ": " +
leading (r, 3, ' ') + r + "," +
leading (g, 3, ' ') + g + "," +
leading (b, 3, ' ') + b);
os += 3;
}
return os;
}
/**
* Read and display a sequence of sub-blocks.
* @param stream Data input stream
* @param os Current byte offset
* @return Updated byte offset
*/
private static long subBlocks (DataInputStream stream, long os)
throws IOException
{
int size;
while ((size = stream.readUnsignedByte ()) != 0) {
System.out.println (leading (os, 8) + os + ": SubBlock: " + size);
os += 1;
for (int i=0; i<size; i++) {
stream.readUnsignedByte ();
os += 1;
}
}
System.out.println (leading (os, 8) + os + ": BlockTerminator");
return os + 1;
}
}
-216
View File
@@ -1,216 +0,0 @@
/**********************************************************************
* JDump - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
import java.util.*;
/**
* Dump contents of JPEG2000 file in human-readable format.
*/
public class J2Dump extends Dump {
/* Fixed value for first 12 bytes */
private static final int[] sigByte =
{
0X00,
0X00,
0X00,
0X0C,
0X6A,
0X50,
0X20,
0X20,
0X0D,
0X0A,
0X87,
0X0A };
private static final boolean ENDIAN = true; /* bigEndian */
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java J2Dump file");
System.exit (-1);
}
try {
FileInputStream file = new FileInputStream (args[0]);
BufferedInputStream buffer = new BufferedInputStream (file);
DataInputStream stream = new DataInputStream (buffer);
J2Dump dump = new J2Dump (); // Just to access contained classes
long os = 0;
boolean bigEndian = true;
int i;
for (i = 0; i < 12; i++) {
int ch;
ch = stream.readUnsignedByte();
if (ch != sigByte[i]) {
System.out.println ("No JPEG 2000 header");
System.exit (-2);
}
}
os += 12;
boolean endOfFile = false;
Stack boxStack = new Stack ();
while (!endOfFile) {
//If there are boxes on the stack, see if there's space
//left in the top one.
Box boxtop;
for (;;) {
boxtop = null;
if (boxStack.isEmpty ()) {
break;
}
boxtop = (Box) boxStack.peek ();
if (boxtop.bytesLeft > 0) {
break;
}
else {
boxStack.pop ();
}
}
// Read the header of a JP2 box
Box box = dump.new Box (stream);
try {
box.read ();
}
catch (EOFException e) {
endOfFile = true;
break;
}
os += box.length - box.bytesLeft;
// If it's contained in a superbox, subtract
// this box from its remaining length
if (boxtop != null) {
boxtop.bytesLeft -= box.length;
}
System.out.println (leading (os, 8) + os + ": " +
stackPrefix (boxStack) + box.type + " " + box.length);
if (box.isSuperbox ()) {
boxStack.push (box);
}
else {
os += box.bytesLeft;
stream.skipBytes((int) box.bytesLeft);
}
// A "length" of 0 means the box occupies the rest of the file.
if (box.length == 0) {
endOfFile = true;
}
}
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (-2);
}
}
/* Constructs a qualifying prefix to indicate nested boxes. */
private static String stackPrefix (Stack boxStack)
{
StringBuffer retval = new StringBuffer ();
// In defiance of gravity, we rummage through the stack
// of boxes starting at the bottom.
for (int i = 0; i < boxStack.size(); i++) {
Box box = (Box) boxStack.elementAt (i);
// Remove trailing spaces from types for better readability
retval.append (box.type.trim() + "/");
}
return retval.toString ();
}
/** Local class for defining JPEG2000 boxes. */
class Box {
public String type;
public long length;
public long bytesLeft;
public boolean hasBoxes;
DataInputStream dstream;
public Box (DataInputStream stream)
{
this.dstream = stream;
}
/** Reads a box header and sets up for reading contents. */
public void read () throws IOException
{
length = ModuleBase.readUnsignedInt (dstream, ENDIAN, null);
long headerLength = 8;
type = read4Chars ();
// If the length field is 1, there is an 8-byte extended
// length field.
if (length == 1) {
length = ModuleBase.readSignedLong(dstream, true, null);
headerLength = 16;
}
bytesLeft = length - headerLength;
}
/* Reads a 4-character name */
private String read4Chars() throws IOException {
StringBuffer sbuf = new StringBuffer(4);
for (int i = 0; i < 4; i++) {
int ch = ModuleBase.readUnsignedByte(dstream, null);
sbuf.append((char) ch);
}
return sbuf.toString();
}
/** Returns true if this box contains other boxes.
* At present, we don't deal with the insides of boxes
* that contain both data and boxes (e.g., cref). */
public boolean isSuperbox ()
{
// If it's a known superbox type, we return true.
// If we've left any out, that will merely make us
// lose the subboxes of that type.
String [] supertypes = { "asoc", "cgrp", "comp", "drep",
"ftbl", "jp2h", "jpch",
"jplh", "res ", "uuid" };
for (int i = 0; i < supertypes.length; i++) {
if (supertypes[i].equals (type)) {
return true;
}
}
return false;
}
}
}
-404
View File
@@ -1,404 +0,0 @@
/**********************************************************************
* JDump - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
//import java.util.*;
/**
* Dump contents of JPEG file in human-readable format.
*/
public class JDump
extends Dump
{
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java JDump file");
System.exit (-1);
}
try {
FileInputStream file = new FileInputStream (args[0]);
BufferedInputStream buffer = new BufferedInputStream (file);
DataInputStream stream = new DataInputStream (buffer);
boolean bigEndian = true;
long os = 0;
boolean endOfImage = false;
boolean readingECS = false;
boolean haveCode = false;;
int nECS = 0;
int code = 0;
while (!endOfImage) {
if (!readingECS) {
if (!haveCode) {
code = stream.readUnsignedByte ();
for (int i=0;
(code = stream.readUnsignedByte ()) == 0xff;
i++) {
System.out.println (leading (os, 8) + os +
": fill 0xff");
os++;
}
}
}
else {
boolean ff = false;
int length = 0;
while (true) {
code = stream.readUnsignedByte ();
length++;
if (code == 0xff) {
ff = true;
}
else if (ff) {
if (code != 0x00) {
length -= 2;
break;
}
else {
ff = false;
}
}
}
System.out.println (leading (os, 8) + os + ": ECS" +
nECS + " " + length + " ...");
os += length;
nECS++;
readingECS = false;
haveCode = true;
continue;
}
if (code == 0x01) {
System.out.println (leading (os, 8) + os + ": TEM");
}
else if ((code >= 0xc0 && code <= 0xc3) ||
(code >= 0xc5 && code <= 0xc7) ||
(code >= 0xc9 && code <= 0xcb) ||
(code >= 0xcd && code <= 0xcf)) {
int n = code - 0xc0;
/****************** int length = markerSegment (stream, bigEndian); */
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int P = stream.readUnsignedByte ();
int Y = ModuleBase.readUnsignedShort (stream, bigEndian,
null);
int X = ModuleBase.readUnsignedShort (stream, bigEndian,
null);
int Nf = stream.readUnsignedByte ();
int [] Ci = new int [Nf];
int [] Hi = new int [Nf];
int [] Vi = new int [Nf];
int [] Tqi = new int [Nf];
for (int i=0; i<Nf; i++) {
Ci[i] = stream.readUnsignedByte ();
Hi[i] = stream.readUnsignedByte ();
Tqi[i] = stream.readUnsignedByte ();
}
System.out.print (leading (os, 8) + os + ": SOF" + n +
" " + length + ": " + P + " " + X +
"x" + Y + " " + Nf + ":");
for (int i=0; i<Nf; i++) {
System.out.print (" " + Ci[i] + "," + Hi[i] + "," +
Tqi[i]);
}
System.out.println ();
os += length;
}
else if (code == 0xc4) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DHT" + " " +
length + " ...");
os += length;
}
else if (code == 0xc8) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": JPG" + " " +
length + " ...");
os += length;
}
else if (code == 0xcc) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DAC" + " " +
length + " ...");
os += length;
}
else if (code >= 0xd0 && code <= 0xd7) {
int m = code - 0xd0;
System.out.println (leading (os, 8) + os + ": RST" + m);
readingECS = true;
}
else if (code == 0xd8) {
System.out.println (leading (os, 8) + os + ": SOI");
}
else if (code == 0xd9) {
System.out.println (leading (os, 8) + os + ": EOI");
endOfImage = true;
break;
}
else if (code == 0xda) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": SOS" + " " +
length + " ...");
os += length;
readingECS = true;
}
else if (code == 0xdb) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DQT" + " " +
length + " ...");
os += length;
}
else if (code == 0xdc) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DNL" + " " +
length + " ...");
os += length;
}
else if (code == 0xdd) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DRI" + " " +
length + " ...");
os += length;
}
else if (code == 0xde) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DHP" + " " +
length + " ...");
os += length;
}
else if (code == 0xdf) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": EXP" + " " +
length + " ...");
os += length;
}
else if (code == 0xe0) {
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
String id = readChars (stream, 5);
if (id.equals ("JFIF\0")) {
int major = stream.readUnsignedByte ();
int minor = stream.readUnsignedByte ();
int units = stream.readUnsignedByte ();
int xDensity = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int yDensity = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int xThumbnail = stream.readUnsignedByte ();
int yThumbnail = stream.readUnsignedByte ();
System.out.print (leading (os, 8) + os + ": APP0 " +
"\"" + id + "\" " + major + "." +
minor + " " + units + " " +
xDensity + "x" + yDensity + " "
+ xThumbnail + "x" + yThumbnail);
int n = length - 16;
if (n > 0) {
for (int i=0; i<n; i++) {
stream.readUnsignedByte ();
}
System.out.print (" " + n + " ...");
}
System.out.println ();
}
else if (id.equals ("JFXX\0")) {
int ext = stream.readUnsignedByte ();
String hex = Integer.toHexString (ext);
System.out.print (leading (os, 8) + os + ": APP0 " +
"0x" + leading (hex, 2) + hex);
int n = length-3;
if (n > 0) {
for (int i=0; i<n; i++) {
stream.readUnsignedByte ();
}
System.out.print (" " + n + " ...");
}
System.out.println ();
}
else {
System.out.println (leading (os, 8) + os + ": APP0 " +
length + " ...");
for (int i=2; i<length; i++) {
stream.readUnsignedByte ();
}
}
os += length;
}
else if (code >= 0xe1 && code <= 0xef) {
int n = code - 0xe0;
/*
int length = markerSegment (stream, bigEndian);
*/
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
if ((n == 1 || n == 2) && length >= 8) {
String id = readChars (stream, 4);
int NULL = stream.readUnsignedByte ();
int padding = stream.readUnsignedByte ();
if (id.equals ("Exif") || id.equals ("FPXR")) {
System.out.println (leading (os, 8) + os +
": APP" + n + " \"" + id +
"\" " + NULL + " " + padding +
" " + (length-8) + ": ...");
}
else {
System.out.println (leading (os, 8) + os +
": APP" + n + " " + length +
" ...");
}
for (int i=8; i<length; i++) {
stream.readUnsignedByte ();
}
}
else {
System.out.println (leading (os, 8) + os + ": APP" +
n + " " + length + " ...");
for (int i=2; i<length; i++) {
stream.readUnsignedByte ();
}
}
os += length;
}
else if (code == 0xf0) {
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
int v = stream.readUnsignedByte ();
int rev = stream.readUnsignedByte ();
System.out.print (leading (os, 8) + os + ": VER " +
v + "." + rev);
int n = length - 4;
if (n > 0) {
int [] cap = new int [n];
for (int i=0; i<n; i++) {
cap[i] = stream.readUnsignedByte ();
System.out.print (((i==0) ? " " : ",") + cap[i]);
}
}
System.out.println ();
os += length;
}
else if (code == 0xf1) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DTI " +
length + " ...");
os += length;
}
else if (code == 0xf2) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DTT " +
length + " ...");
os += length;
}
else if (code == 0xf3) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": SRF " +
length + " ...");
os += length;
}
else if (code == 0xf4) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": SRS " +
length + " ...");
os += length;
}
else if (code == 0xf5) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DCR " +
length + " ...");
os += length;
}
else if (code == 0xf6) {
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": DQS " +
length + " ...");
os += length;
}
else if (code >= 0xf7 && code <= 0xfd) {
int n = code - 0xf0;
int length = markerSegment (stream, bigEndian);
System.out.println (leading (os, 8) + os + ": JPG" + n +
" " + length + " ...");
os += length;
}
else if (code == 0xfe) {
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
String comment = readChars (stream, length-2);
System.out.println (leading (os, 8) + os + ": COM \"" +
comment + "\"");
os += length;
}
else {
int length = markerSegment (stream, bigEndian);
String hex = Integer.toHexString (code);
System.out.println (leading (os, 8) + os + ": RES (0x" +
leading (hex, 2) + hex + ") " +
length + " ...");
}
os += 2;
}
stream.close ();
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (-2);
}
}
/**
* Read marker segment data
* @param stream Data input stream
* @param bigEndian True if big-endian
* @return Length of marker segment
*/
private static int markerSegment (DataInputStream stream,
boolean bigEndian)
throws IOException
{
int length = ModuleBase.readUnsignedShort (stream,
bigEndian,
null);
for (int i=2; i<length; i++) {
stream.readUnsignedByte ();
}
return length;
}
}
-313
View File
@@ -1,313 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2007 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
//import java.io.*;
import java.util.*;
public class Jhove
{
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
/** Application name. */
private static final String NAME = "Jhove";
/** Application build date, YYYY, MM, DD. */
private static int [] DATE;
/** Application release number. */
private static String RELEASE;
/** Application invocation syntax. */
private static final String USAGE = "java " + NAME + " [-c config] " +
"[-m module] [-h handler] [-e encoding] [-H handler] [-o output] " +
"[-x saxclass] [-t tempdir] [-b bufsize] [-l loglevel] [[-krs] " +
"dir-file-or-uri [...]]";
/** Copyright information. */
private static final String RIGHTS =
"Derived from software Copyright 2004-2011 " +
"by the President and Fellows of Harvard College. " +
"Version 1.7 and higher independently released. " +
"Released under the GNU Lesser General Public License.";
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
public static void main (String [] args)
{
RELEASE = JhoveBase._release; // possibly safer than final static init
DATE = JhoveBase.DATE;
/* Make sure we have a satisfactory version of Java. */
String version = System.getProperty ("java.vm.version");
if (version.compareTo ("1.5.0") < 0) {
//System.err.println (NAME + ": Java 1.4 or higher is required");
System.out.println ("Java 1.5 or higher is required");
System.exit (-1);
}
try {
/**********************************************************
* Initialize the application state object.
**********************************************************/
App app = new App (NAME, RELEASE, DATE, USAGE, RIGHTS);
/**********************************************************
* Retrieve the configuration file.
**********************************************************/
String configFile = JhoveBase.getConfigFileFromProperties ();
String saxClass = JhoveBase.getSaxClassFromProperties ();
/* Pre-parse the command line for -c and -x config options.
* With Windows, we have to deal with quote marks on our own. With Unix,
* the shell takes care of quotes for us. */
boolean quoted = false;
for (int i=0; i<args.length; i++) {
if (quoted) {
int len = args[i].length ();
if (args[i].charAt (len-1) == '"') {
quoted = false;
}
}
else {
if (args[i].equals ("-c")) {
if (i < args.length-1) {
configFile = args[++i];
}
}
else if (args[i].equals ("-x")) {
if (i <args.length-1) {
saxClass = args[++i];
}
}
else if (args[i].charAt (0) == '"') {
quoted = true;
}
}
}
/**********************************************************
* Initialize the JHOVE engine.
**********************************************************/
String encoding = null;
String tempDir = null;
int bufferSize = -1;
String moduleName = null;
String handlerName = null;
String aboutHandler = null;
String logLevel = "SEVERE";
String outputFile = null;
boolean checksum = false;
boolean showRaw = false;
boolean signature = false;
List<String> list = new ArrayList<String> ();
/**********************************************************
* Parse command line arguments:
* -m module Module name
* -h handler Output handler
* -e encoding Output encoding
* -H handler About handler
* -o output Output file pathname
* -t tempdir Directory for temp files
* -b bufsize Buffer size for buffered I/O
* -k Calculate checksums
* -r Display raw numeric flags
* -s Check internal signatures only
* dirFileOrUri Directories, file pathnames, or URIs
*
* The following arguments were defined in previous
* versions, but are now obsolete
* -p param OBSOLETE
* -P param OBSOLETE
**********************************************************/
quoted = false;
StringBuffer filename = null;
for (int i=0; i<args.length; i++) {
if (quoted) {
int len = args[i].length ();
if (args[i].charAt (len-1) == '"') {
filename.append (" " + args[i].substring (0, len-1));
list.add (filename.toString ());
quoted = false;
}
else {
filename.append (" " + args[i]);
}
}
else {
if (args[i].equals ("-c")) {
i++;
}
else if (args[i].equals ("-m")) {
if (i < args.length-1) {
moduleName = args[++i];
}
}
else if (args[i].equals ("-p")) {
// Obsolete -- but eat the next arg for compatibility
if (i <args.length-1) {
@SuppressWarnings("unused")
String moduleParam = args[++i];
}
}
else if (args[i].equals ("-h")) {
if (i < args.length-1) {
handlerName = args[++i];
}
}
else if (args[i].equals ("-P")) {
// Obsolete -- but eat the next arg for compatibility
if (i <args.length-1) {
@SuppressWarnings("unused")
String handlerParam = args[++i];
}
}
else if (args[i].equals ("-e")) {
if (i < args.length-1) {
encoding = args[++i];
}
}
else if (args[i].equals ("-H")) {
if (i < args.length-1) {
aboutHandler = args[++i];
}
}
else if (args[i].equals ("-l")) {
if (i < args.length-1) {
logLevel = args[++i];
}
}
else if (args[i].equals ("-o")) {
if (i < args.length-1) {
outputFile = args[++i];
}
}
else if (args[i].equals ("-x")) {
i++;
}
else if (args[i].equals ("-t")) {
if (i <args.length-1) {
tempDir = args[++i];
}
}
else if (args[i].equals ("-b")) {
if (i <args.length-1) {
try {
bufferSize = Integer.parseInt (args[++i]);
}
catch (NumberFormatException e) {
System.err.println ("Invalid buffer size, using default.");
}
}
}
else if (args[i].equals ("-k")) {
checksum = true;
}
else if (args[i].equals ("-r")) {
showRaw = true;
}
else if (args[i].equals ("-s")) {
signature = true;
}
else if (args[i].charAt (0) != '-') {
if (args[i].charAt (0) == '"') {
filename = new StringBuffer ();
filename.append (args[i].substring (1));
quoted = true;
}
else {
list.add (args[i]);
}
}
}
}
if (quoted) {
list.add (filename.toString ());
}
JhoveBase je = new JhoveBase ();
je.setLogLevel (logLevel);
je.init (configFile, saxClass);
if (encoding == null) {
encoding = je.getEncoding ();
}
if (tempDir == null) {
tempDir = je.getTempDirectory ();
}
if (bufferSize < 0) {
bufferSize = je.getBufferSize ();
}
Module module = je.getModule (moduleName);
if (module == null && moduleName != null) {
System.out.println ("Module '" + moduleName + "' not found");
System.exit (-1);
}
OutputHandler about = je.getHandler (aboutHandler);
if (about == null && aboutHandler != null) {
System.out.println ("Handler '" + aboutHandler + "' not found");
System.exit (-1);
}
OutputHandler handler = je.getHandler (handlerName);
if (handler == null && handlerName != null) {
System.out.println ("Handler '" + handlerName + "' not found");
System.exit (-1);
}
String [] dirFileOrUri = null;
int len = list.size ();
if (len > 0) {
dirFileOrUri = new String [len];
for (int i=0; i<len; i++) {
dirFileOrUri[i] = (String) list.get (i);
}
}
/**********************************************************
* Invoke the JHOVE engine.
**********************************************************/
je.setEncoding (encoding);
je.setTempDirectory (tempDir);
je.setBufferSize (bufferSize);
je.setChecksumFlag (checksum);
je.setShowRawFlag (showRaw);
je.setSignatureFlag (signature);
je.dispatch (app, module, about, handler, outputFile,
dirFileOrUri);
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (-1);
}
}
}
-229
View File
@@ -1,229 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2008 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.viewer.*;
import javax.swing.*;
/**
* JhoveView - JSTOR/Harvard Object Validation Environment.
*/
public class JhoveView
{
/******************************************************************
* PRIVATE CLASS FIELDS.
*
* Application constants.
******************************************************************/
/** Application name. */
private static final String NAME = "JhoveView";
/** Application build date, YYYY, MM, DD. */
private static int [] _date;
/** Usage string is meaningless here. */
private static final String USAGE = null;
/** Application release number. */
private static String _release;
/** Default character encoding */
//private static final String DEFAULT_ENCODING = "UTF-8";
/** Copyright information. */
private static final String RIGHTS =
"Derived from software Copyright 2004-2011 " +
"by the President and Fellows of Harvard College. " +
"Version 1.7 and higher independently released. " +
"Released under the GNU Lesser General Public License.";
/******************************************************************
* Configuration constants.
******************************************************************/
/** Configuration file property. */
// private static final String CONFIG_PROPERTY = "edu.harvard.hul.ois." +
// "jhove.config";
/** Jhove directory */
// private static final String JHOVE_DIR = "jhove";
/** Config directory */
// private static final String CONFIG_DIR = "conf";
/** SAX parser class property. */
// private static final String SAX_PROPERTY = "edu.harvard.hul.ois.jhove." +
// "saxClass";
/******************************************************************
* Action constants.
******************************************************************/
/******************************************************************
* Exit code constants.
******************************************************************/
/** General error. */
private static final int ERROR = -1;
/** Incompatible Java VM. */
private static final int INCOMPATIBLE_VM = -2;
/** File not writable. */
//private static final int FILE_NOT_WRITABLE = -11;
/** No module specified. */
//private static final int NO_MODULE = -21;
/** Module not found. */
//private static final int MODULE_NOT_FOUND = -22;
/** Output handler not found. */
//private static final int HANDLER_NOT_FOUND = -31;
/** No object specified. */
//private static final int NO_OBJECT = -41;
/** URL not accessible. */
//private static final int URL_NOT_ACCESSIBLE = -42;
/** File not found. */
//private static final int FILE_NOT_FOUND = -43;
/** File not readable. */
//private static final int FILE_NOT_READABLE = -44;
/** SAX parser not found. */
//private static final int PARSER_NOT_FOUND = -51;
/******************************************************************
* PUBLIC CLASS METHODS.
******************************************************************/
/**
* Stub constructor.
*/
private JhoveView ()
{
}
/**
* Application main entry point.
* @parm args Command line arguments
*/
public static void main (String [] args)
{
_release = JhoveBase._release; // Seems safer than final init
_date = JhoveBase.DATE;
/* Make sure we have a satisfactory version of Java. */
String version = System.getProperty ("java.vm.version");
if (version.compareTo ("1.4.0") < 0) {
//System.err.println (NAME + ": Java 1.4 or higher is required");
errorAlert ("Java 1.4 or higher is required");
System.exit (INCOMPATIBLE_VM);
}
// If we're running on a Macintosh, put the menubar at the top
// of the screen where it belongs.
System.setProperty ("apple.laf.useScreenMenuBar", "true");
App app = new App (NAME, _release, _date, USAGE, RIGHTS);
try {
/**********************************************************
* Retrieve the configuration file.
**********************************************************/
String configFile = JhoveBase.getConfigFileFromProperties ();
String saxClass = JhoveBase.getSaxClassFromProperties ();
/**********************************************************
* Initialization:
* configFile Configuration file pathname
* saxClass SAX parser class
**********************************************************/
/* Pre-parse the command line for -c and -x config options. */
boolean quoted = false;
for (int i=0; i<args.length; i++) {
if (quoted) {
int len = args[i].length ();
if (args[i].charAt (len-1) == '"') {
quoted = false;
}
}
else {
if (args[i].equals ("-c")) {
if (i < args.length-1) {
configFile = args[++i];
}
}
else if (args[i].equals ("-x")) {
if (i <args.length-1) {
saxClass = args[++i];
}
}
else if (args[i].charAt (0) == '"') {
quoted = true;
}
}
}
/**********************************************************
* Initialize the JHOVE engine.
**********************************************************/
JhoveBase je = new JhoveBase ();
try {
je.init (configFile, saxClass);
}
catch (JhoveException e) {
errorAlert (e.getMessage ());
// Keep going, so user can correct in editor
}
// Create the main window to select a file.
JhoveWindow jwin = new JhoveWindow (app, je);
jwin.setVisible (true);
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (ERROR);
}
}
/* Displays an error alert. */
private static void errorAlert (String msg)
{
JFrame hiddenFrame = new JFrame ();
// Truncate long messages so the alert isn't wider
// than the screen
if (msg.length() > 80) {
msg = msg.substring (0, 79) + "...";
}
JOptionPane.showMessageDialog (hiddenFrame,
msg, "Jhove Error", JOptionPane.ERROR_MESSAGE);
}
}
-115
View File
@@ -1,115 +0,0 @@
/**********************************************************************
* PDump - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.pdf.*;
import java.io.*;
/**
* Dump contents of PDF file in human-readable format.
*/
public class PDump
extends Dump
{
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java PDump file");
System.exit (-1);
}
try {
RandomAccessFile file = new RandomAccessFile (args[0], "r");
Tokenizer tokenizer = new FileTokenizer (file);
Token token = null;
long offset = 0;
while ((token = tokenizer.getNext ()) != null) {
System.out.print (leading (offset, 8) + offset + ": ");
if (token instanceof ArrayEnd) {
System.out.println ("ArrayEnd");
}
else if (token instanceof ArrayStart) {
System.out.println ("ArrayStart");
}
else if (token instanceof Comment) {
System.out.println ("Comment \"" +
((Comment) token).getValue () + "\"");
}
else if (token instanceof DictionaryEnd) {
System.out.println ("DictionaryEnd");
}
else if (token instanceof DictionaryStart) {
System.out.println ("DictionaryStart");
}
// else if (token instanceof Hexadecimal) {
// System.out.println ("Hexadecimal[" +
// (((Hexadecimal) token).isPDFDocEncoding () ?
// "PDF" : "UTF-16") + "] \"" +
// ((Hexadecimal) token).getValue () +
// "\"");
// }
else if (token instanceof Keyword) {
System.out.println ("Keyword \"" +
((Keyword) token).getValue () + "\"");
}
else if (token instanceof Literal) {
System.out.println ("Literal[" +
(((Literal)token).isPDFDocEncoding () ?
"PDF" : "UTF-16") + "] \"" +
((Literal) token).getValue () + "\"");
}
else if (token instanceof Name) {
System.out.println ("Name \"" +
((Name) token).getValue () + "\"");
}
else if (token instanceof Numeric) {
Numeric numeric = (Numeric) token;
if (numeric.isReal ()) {
System.out.println ("Numeric " + numeric.getValue ());
}
else {
System.out.println ("Numeric " +
numeric.getIntegerValue ());
}
}
else if (token instanceof Stream) {
System.out.println ("Stream " +
((Stream) token).getLength ());
}
else {
System.out.println (token);
}
offset = tokenizer.getOffset ();
}
}
catch (Exception e) {
e.printStackTrace (System.err);
System.exit (-2);
}
}
}
-361
View File
@@ -1,361 +0,0 @@
/**********************************************************************
* TDump - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2004 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.tiff.*;
import java.io.*;
import java.util.*;
/**
* Dump contents of TIFF file in human-readable format.
*/
public class TDump
extends Dump
{
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
/** Count of IFDs. */
private static int _nIFDs;
/** Sorted associative map of tags. */
private static Map _tags;
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java TDump [-bs] file");
System.exit (-1);
}
String tiff = null;
boolean nobyte = false;
boolean nosub = false;
for (int i=0; i<args.length; i++) {
if (args[i].equals ("-b")) {
nobyte = true;
}
else if (args[i].equals ("-s")) {
nosub = true;
}
else if (tiff == null) {
tiff = args[i];
}
}
/* Accumlate all tag definitions in a sorted map. */
_tags = new TreeMap ();
int err = 0;
try {
/* Read TIFF header. */
RandomAccessFile file = new RandomAccessFile (tiff, "r");
boolean bigEndian = true;
int b1 = ModuleBase.readUnsignedByte (file);
int b2 = ModuleBase.readUnsignedByte (file);
if (b1 == 0x49 && b2 == b1) {
bigEndian = false;
}
int magic = ModuleBase.readUnsignedShort (file, bigEndian);
long offset = ModuleBase.readUnsignedInt (file, bigEndian);
_tags.put ("00000000", "\"" + ((char) b1) + ((char) b2) + "\" (" +
(bigEndian ? "big" : "little") + " endian) " + magic +
" LONG @" + offset);
/* Read IFDs. */
_nIFDs = 0;
while ((offset = readIFD (file, bigEndian, offset, nobyte,
nosub)) > 0) {
}
file.close ();
}
catch (Exception e) {
e.printStackTrace (System.out);
err = -2;
}
finally {
/* Display all tags in offset-sorted order. */
Iterator iter = _tags.keySet ().iterator ();
while (iter.hasNext ()) {
String os = (String) iter.next ();
System.out.println (os + ": " + (String) _tags.get (os));
}
if (err != 0) {
System.exit (err);
}
}
}
/**
* Read IFDs.
* @param file Open TIFF file
* @param bigEndian True if big-endian
* @param offset Byte offset of IFD
* @param nobyte If true, only display the first 8 bytes of data of
* type BYTE
* @param nosub If true, do not parse subIFDs
*/
private static long readIFD (RandomAccessFile file, boolean bigEndian,
long offset, boolean nobyte, boolean nosub)
throws Exception
{
int nIFD = ++_nIFDs;
List subIFDs = new ArrayList ();
List stripByteCounts = new ArrayList ();
List stripOffsets = new ArrayList ();
file.seek (offset);
int nEntries = ModuleBase.readUnsignedShort (file, bigEndian);
_tags.put (leading (offset, 8) + offset, "IFD " + nIFD + " with " +
nEntries + " entries");
String name = null;
for (int i=0; i<nEntries; i++) {
long os = offset + 2 + i*12;
file.seek (os);
int tag = ModuleBase.readUnsignedShort (file, bigEndian);
int type = ModuleBase.readUnsignedShort (file, bigEndian);
long count = ModuleBase.readUnsignedInt (file, bigEndian);
StringBuffer buffer = new StringBuffer (tag + " (" +
TiffTags.tagName(tag) +
") " + IFD.TYPE[type] +
" " + count);
if (type == IFD.ASCII) {
if (count > 4) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
StringBuffer ascii = new StringBuffer ();
for (int j=0; j<count-1; j++) {
ascii.append ((char) ModuleBase.readUnsignedByte (file));
}
buffer.append (" = \"" + ascii.toString () + "\"");
}
else if (type == IFD.BYTE ||
type == IFD.UNDEFINED) {
if (count > 4) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
long ct = count;
if (nobyte && count > 8) {
ct = 8;
}
for (int j=0; j<ct; j++) {
long by = ModuleBase.readUnsignedByte (file);
buffer.append (" " + by);
}
if (nobyte && count > 8) {
buffer.append (" ...");
}
}
else if (type == IFD.DOUBLE) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo + " =");
for (int j=0; j<count; j++) {
double db = ModuleBase.readDouble (file, bigEndian);
buffer.append (" " + db);
}
}
else if (type == IFD.FLOAT) {
if (count > 1) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
for (int j=0; j<count; j++) {
float fl = ModuleBase.readFloat (file, bigEndian);
buffer.append (" " + fl);
}
}
else if (type == IFD.LONG || type == IFD.IFD) {
if (count > 1) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo + " = ");
}
else if (tag == 330 || /* Sub IFD */
tag == 34665 || /* EXIF IFD */
tag == 34853 || /* EXIF GPS IFD */
tag == 40965) { /* EXIF Interoperability IFD */
buffer.append (" @");
}
else {
buffer.append (" = ");
}
for (int j=0; j<count; j++) {
long in = ModuleBase.readUnsignedInt (file, bigEndian);
if (j > 0) {
buffer.append (" ");
}
buffer.append (in);
if (tag == 330 || /* Sub IFD */
tag == 34665 || /* EXIF IFD */
tag == 34853 || /* EXIF GPS IFD */
tag == 40965) { /* EXIF Interoperability IFD */
subIFDs.add (new Long (in));
}
else if (tag == 273) {
stripOffsets.add (new Long (in));
}
else if (tag == 279) {
stripByteCounts.add (new Long (in));
}
}
}
else if (type == IFD.RATIONAL) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo + " =");
for (int j=0; j<count; j++) {
long numer = ModuleBase.readUnsignedInt (file, bigEndian);
long denom = ModuleBase.readUnsignedInt (file, bigEndian);
buffer.append (" " + numer + "/" + denom);
}
}
else if (type == IFD.SBYTE) {
if (count > 4) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
for (int j=0; j<count; j++) {
long by = ModuleBase.readSignedByte (file);
buffer.append (" " + by);
}
}
else if (type == IFD.SHORT) {
if (count > 2) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
for (int j=0; j<count; j++) {
int sh = ModuleBase.readUnsignedShort (file, bigEndian);
buffer.append (" " + sh);
if (tag == 273) {
stripOffsets.add (new Long ((long) sh));
}
else if (tag == 279) {
stripByteCounts.add (new Long ((long) sh));
}
}
}
else if (type == IFD.SLONG) {
if (count > 1) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
for (int j=0; j<count; j++) {
int in = ModuleBase.readSignedInt (file, bigEndian);
buffer.append (" " + in);
}
}
else if (type == IFD.SRATIONAL) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo + " =");
for (int j=0; j<count; j++) {
long numer = ModuleBase.readSignedInt (file, bigEndian);
long denom = ModuleBase.readSignedInt (file, bigEndian);
buffer.append (" " + numer + "/" + denom);
}
}
else if (type == IFD.SSHORT) {
if (count > 2) {
long vo = ModuleBase.readUnsignedInt (file, bigEndian);
file.seek (vo);
buffer.append (" @" + vo);
}
buffer.append (" =");
for (int j=0; j<count; j++) {
int sh = ModuleBase.readSignedShort (file, bigEndian);
buffer.append (" " + sh);
}
}
_tags.put (leading (os, 8) + os, buffer.toString ());
}
if (!nosub) {
for (int i=0; i<subIFDs.size (); i++) {
long os = ((Long) subIFDs.get (i)).longValue ();
while ((os = readIFD (file, bigEndian, os, nobyte, nosub)) > 0) {
}
}
}
long os = offset + 2 + nEntries*12;
file.seek (os);
long next = ModuleBase.readUnsignedInt (file, bigEndian);
_tags.put (leading (os, 8) + os, "NextIFDOffset LONG @" + next);
int len = stripOffsets.size ();
if (len > 0) {
for (int j=0; j<len; j++) {
long start = ((Long) stripOffsets.get (j)).longValue ();
long count = ((Long) stripByteCounts.get (j)).longValue ();
long finish= start + count - 1;
_tags.put (leading (start, 8) + start, "(Image " + nIFD +
",strip " + (j+1) + ") IMAGEDATA " + count +
": ... " + leading (finish, 8) + finish);
}
}
return next;
}
}
-33
View File
@@ -1,33 +0,0 @@
/**********************************************************************
* UserHome - JSTOR/Harvard Object Validation Environment
* Copyright 2006 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import java.io.*;
/**
* Determine the default value of the Java user.home property.
*/
public class UserHome
{
public static void main (String [] args)
{
System.out.println ("Default user.home is \"" +
System.getProperty ("user.home") + "\"");
}
}
-196
View File
@@ -1,196 +0,0 @@
/**********************************************************************
* JDump - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2005 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
/**
* Dump contents of WAVE file in human-readable format.
*
* @author Gary McGath
*
*/
public class WDump extends Dump {
/* Fixed value for first 4 bytes */
private static final int[] sigByte =
{ 0X52, 0X49, 0X46, 0X46 };
private static final boolean ENDIAN = false; /* little endian */
/******************************************************************
* MAIN ENTRY POINT.
******************************************************************/
/**
* Main entry point.
* @param args Command line arguments
*/
public static void main (String [] args)
{
if (args.length < 1) {
System.err.println ("usage: java WDump file");
System.exit (-1);
}
try {
FileInputStream file = new FileInputStream (args[0]);
BufferedInputStream buffer = new BufferedInputStream (file);
DataInputStream stream = new DataInputStream (buffer);
ADump dump = new ADump (); // Just to access contained classes
long os = 0;
for (int i=0; i<4; i++) {
int ch;
ch = stream.readUnsignedByte();
if (ch != sigByte[i]) {
System.out.println ("No AIFF FORM header");
System.exit (-2);
}
os++;
}
long ckSize = ModuleBase.readUnsignedInt (stream, ENDIAN);
os += 4;
// Read the file type
String formType = read4Chars (stream);
os += 4;
System.out.println ("00000000: RIFF " + ckSize + ": " + formType);
boolean endOfFile = false;
while (!endOfFile) {
// Read chunks
try {
// Read chunk name.
String ckID = read4Chars (stream);
// Read size (excluding chunk name and size fields)
ckSize = ModuleBase.readUnsignedInt (stream, ENDIAN);
System.out.print (leading (os, 8) + os + ": " + ckID +
" " + ckSize);
long alreadyRead = 0;
if (ckID.equals ("fact")) {
long sampleLength =
ModuleBase.readUnsignedInt (stream, ENDIAN);
System.out.print (": " + sampleLength);
alreadyRead = 4;
}
else if (ckID.equals ("fmt ")) {
int formatTag = ModuleBase.readUnsignedShort (stream,
ENDIAN);
int channels = ModuleBase.readUnsignedShort (stream,
ENDIAN);
long samplesPerSec =
ModuleBase.readUnsignedInt (stream, ENDIAN);
long avgBytesPerSec =
ModuleBase.readUnsignedInt (stream, ENDIAN);
int blockAlign = ModuleBase.readUnsignedShort (stream,
ENDIAN);
String hex = Integer.toHexString (formatTag);
System.out.print (": 0x" + leading (hex, 4) + hex +
" " + channels + " " +
samplesPerSec + " " +
avgBytesPerSec + " " + blockAlign);
alreadyRead = 14;
if (ckSize > 14) {
int bitsPerSample =
ModuleBase.readUnsignedShort (stream, ENDIAN);
System.out.print (" " + bitsPerSample);
alreadyRead = 16;
if (ckSize > 16) {
int size =
ModuleBase.readUnsignedShort (stream,
ENDIAN);
System.out.print (" " + size);
alreadyRead = 18;
if (size == 22) {
int validBitsPerSample =
ModuleBase.readUnsignedShort (stream,
ENDIAN);
long channelMask =
ModuleBase.readUnsignedInt (stream,
ENDIAN);
hex = Long.toHexString (channelMask);
System.out.print (" " +
validBitsPerSample +
" 0x" +
leading (hex, 8) + hex +
" 0x");
for (int i=0; i<4; i++) {
long guid =
ModuleBase.readUnsignedInt (
stream,
ENDIAN);
hex = Long.toHexString (guid);
System.out.print (leading (hex, 8) +
hex);
}
alreadyRead = 40;
}
}
}
}
System.out.println ();
if (ckID.equals ("list") || ckID.equals ("LIST")) {
readNestedChunks (ckID, stream, ckSize, os + 8);
}
else {
stream.skipBytes ((int) (ckSize - alreadyRead));
}
os += ckSize + 8;
}
catch (EOFException e) {
endOfFile = true;
}
}
}
catch (Exception e) {
}
}
/* The "list" and "LIST" chunks (which are two distinct chunk types)
hold nested chunks. */
private static void readNestedChunks
(String ckID, DataInputStream stream, long ckSize, long os)
throws IOException
{
String listType = read4Chars (stream);
System.out.println ("List type = " + listType);
while (ckSize > 0) {
String subCkID = read4Chars (stream);
long subCkSize = ModuleBase.readUnsignedInt (stream, ENDIAN, null);
System.out.println (leading (os, 8) + os + ": " +
ckID + "/" + subCkID + " " + subCkSize);
stream.skipBytes ((int) subCkSize);
os += subCkSize + 8;
ckSize -= subCkSize + 8;
}
}
private static String read4Chars (DataInputStream stream)
throws IOException
{
StringBuffer sbuf = new StringBuffer(4);
for (int i = 0; i < 4; i++) {
int ch = ModuleBase.readUnsignedByte(stream, null);
sbuf.append((char) ch);
}
return sbuf.toString ();
}
}
-128
View File
@@ -1,128 +0,0 @@
<project name="Jhove" default="main-dist" basedir=".">
<description>Main build file
Jhove - JSTOR/Harvard Object Validation Environment
Version 1.1 2006-02-13
Copyright 2004-2006 by JSTOR and the President and Fellows of Harvard College
</description>
<!-- set global properties for this build -->
<property name="classpath" value=".:../bin/jhove.jar"/>
<property name="api" location="edu/harvard/hul/ois/jhove"/>
<property name="viewer" location="edu/harvard/hul/ois/jhove/viewer"/>
<property name="root" location=".."/>
<property name="manifest" location="${viewer}/META-INF"/>
<target name="main-init" description="Create the time stamp">
<tstamp>
<format property="timestamp.isoformat" pattern="yyyy-MM-dd'T'HH:mm:ss"/>
</tstamp>
</target>
<target name="main-dist" depends="api,main,view"/>
<target name="api" description="Compile the API">
<ant dir="${api}" inheritAll="false">
<property name="dbg" value="${dbg}"/>
</ant>
</target>
<target name="main-compile" depends="main-init"
description="Compile the main file">
<javac compiler="javac1.5" target="1.5" srcdir="." debug="${dbg}" classpath="${classpath}"/>
</target>
<target name="main" depends="main-compile, app-manifest"
description="Build the app jar">
<jar destfile="${root}/bin/JhoveApp.jar" basedir="." index="true"
manifest="./META-INF/MANIFEST.MF"
includes="Jhove.class,
GDump.class, JDump.class, PDump.class, TDump.class,
J2Dump.class,J2Dump$Box.class,
ADump.class,
WDump.class, UserHome.class,
edu/harvard/hul/ois/jhove/*.class,
edu/harvard/hul/ois/jhove/*.xsd,
edu/harvard/hul/ois/jhove/handler/*.class,
edu/harvard/hul/ois/jhove/handler/audit/*.class,
edu/harvard/hul/ois/jhove/module/*.class,
edu/harvard/hul/ois/jhove/module/aiff/*.class,
edu/harvard/hul/ois/jhove/module/gif/*.class,
edu/harvard/hul/ois/jhove/module/html/*.class,
edu/harvard/hul/ois/jhove/module/html/*.dtd,
edu/harvard/hul/ois/jhove/module/html/*.ent,
edu/harvard/hul/ois/jhove/module/iff/*.class,
edu/harvard/hul/ois/jhove/module/jpeg/*.class,
edu/harvard/hul/ois/jhove/module/jpeg2000/*.class,
edu/harvard/hul/ois/jhove/module/pdf/*.class,
edu/harvard/hul/ois/jhove/module/tiff/*.class,
edu/harvard/hul/ois/jhove/module/wave/*.class,
edu/harvard/hul/ois/jhove/module/xml/*.class"/>
</target>
<target name="app-manifest" depends="main-init">
<mkdir dir="${manifest}"/>
<manifest file="./META-INF/MANIFEST.MF">
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Built-On" value="${timestamp.isoformat}"/>
<attribute name="Main-Class" value="Jhove"/>
<section name="JHOVE">
<attribute name="Version" value="1.0"/>
<attribute name="Package" value="Jhove app"/>
<attribute name="Rights" value="Copyright 2004-2006 by JSTOR and the President and Fellows of Harvard College"/>
</section>
</manifest>
</target>
<target name="view-manifest" depends="main-init">
<mkdir dir="${manifest}"/>
<manifest file="${manifest}/MANIFEST.MF">
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Built-On" value="${timestamp.isoformat}"/>
<attribute name="Main-Class" value="JhoveView"/>
<section name="JHOVE">
<attribute name="Version" value="1.0"/>
<attribute name="Package" value="Swing Viewer"/>
<attribute name="Rights" value="Copyright 2004-2006 by JSTOR and the President and Fellows of Harvard College"/>
</section>
</manifest>
</target>
<target name="view" description="Compile the viewer application"
depends="api,main,view-manifest">
<jar destfile="${root}/bin/JhoveView.jar" basedir="." index="true"
manifest="${manifest}/MANIFEST.MF"
includes="JhoveView.class,
edu/harvard/hul/ois/jhove/*.class,
edu/harvard/hul/ois/jhove/handler/*.class,
edu/harvard/hul/ois/jhove/handler/audit/*.class,
edu/harvard/hul/ois/jhove/module/*.class,
edu/harvard/hul/ois/jhove/module/aiff/*.class,
edu/harvard/hul/ois/jhove/module/gif/*.class,
edu/harvard/hul/ois/jhove/module/html/*.class,
edu/harvard/hul/ois/jhove/module/html/*.dtd,
edu/harvard/hul/ois/jhove/module/html/*.ent,
edu/harvard/hul/ois/jhove/module/iff/*.class,
edu/harvard/hul/ois/jhove/module/jpeg/*.class,
edu/harvard/hul/ois/jhove/module/jpeg2000/*.class,
edu/harvard/hul/ois/jhove/module/pdf/*.class,
edu/harvard/hul/ois/jhove/module/tiff/*.class,
edu/harvard/hul/ois/jhove/module/wave/*.class,
edu/harvard/hul/ois/jhove/module/xml/*.class,
edu/harvard/hul/ois/jhove/viewer/*.gif,
edu/harvard/hul/ois/jhove/viewer/*.class"/>
</target>
<target name="main-clean">
<delete>
<fileset dir="." includes="*~ \#*"/>
</delete>
<ant dir="${api}" target="api-clean" inheritAll="false"/>
</target>
<target name="main-cleanclass">
<delete>
<fileset dir="." includes="*.class*"/>
</delete>
<ant dir="${api}" target="api-cleanclass" inheritAll="false"/>
</target>
</project>
@@ -1,940 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2005 by JSTOR and the President and Fellows of Harvard College
*
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.util.*;
/**
* Encapsulation of the AES Metadata for Audio documents
*
* @author Gary McGath
*
*/
public class AESAudioMetadata
{
/******************************************************************
* PUBLIC CLASS FIELDS.
******************************************************************/
/** Big-endian constant. */
public static final int BIG_ENDIAN = 0;
/** Little-endian constant. */
public static final int LITTLE_ENDIAN = 1;
/** Analog / digital labels. */
public static final String [] A_D = {
"ANALOG", "PHYS_DIGITAL", "FILE_DIGITAL"
};
/** Values for primary identifier type */
public static final String
FILE_NAME = "FILE_NAME",
OTHER = "OTHER";
/** Constant for an undefined integer value. */
public static final int NULL = -1;
/** Constant for an undefined floating-point value. */
public static final double NILL = -1.0;
/******************************************************************
* PRIVATE INSTANCE FIELDS.
*
******************************************************************/
/** Constant value for the SchemaVersion field */
public static final String SCHEMA_VERSION = "1.02b";
/** Constant value for the disposition field */
private static final String DEFAULT_DISPOSITION = "validation";
private String _analogDigitalFlag;
private String _appSpecificData;
private String _audioDataEncoding;
private int _byteOrder;
private String _disposition;
private List _faceList;
private long _firstSampleOffset;
private String _format;
private List _formatList;
private int _numChannels;
private String _primaryIdentifier;
private String _primaryIdentifierType;
private String _primaryIdentifierOtherType;
private String _schemaVersion;
private String _specificationVersion;
private String[] _use;
/* Most recently added FormatRegion */
private FormatRegion _curFormatRegion;
/* Most recently added Face */
private Face _curFace;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/** Instantiate a <code>NisoImageMetadata</code> object.
*/
public AESAudioMetadata ()
{
_schemaVersion = SCHEMA_VERSION;
_disposition = DEFAULT_DISPOSITION;
_analogDigitalFlag = null;
_format = null;
_specificationVersion = null;
_audioDataEncoding = null;
_primaryIdentifier = null;
_primaryIdentifierType = null;
_use = null;
// We add one format region to get started. In practice,
// that one is all we're likely to need. but more can be
// added if necessary.
_formatList = new LinkedList ();
_faceList = new LinkedList ();
addFormatRegion ();
addFace ();
_numChannels = NULL;
_byteOrder = NULL;
_firstSampleOffset = NULL;
}
/******************************************************************
* PUBLIC STATIC INTERFACES.
*
******************************************************************/
/**
* Public interface to the nested FormatRegion object. Instances
* of this should be created only by addFormatRegion, but can be
* accessed through the public methods of this interface.
*/
public static interface FormatRegion {
/** Returns the bit depth. */
public int getBitDepth ();
/** Returns the bitrate reduction (compression information).
* This will be an array of seven strings (which may be
* empty, but should never be null) interpreted as follows:
* <ul>
* <li>0: codecName
* <li>1: codecNameVersion
* <li>2: codecCreatorApplication
* <li>3: codecCreatorApplicationVersion
* <li>4: codecQuality
* <li>5: dataRate
* <li>6: dataRateMode
* </ul>
*/
public String[] getBitrateReduction ();
/** Returns the sample rate. */
public double getSampleRate ();
/** Returns the word size. */
int getWordSize ();
/** Returns <code>true</code> if the region is empty. */
public boolean isEmpty ();
/** Sets the bit depth value. */
public void setBitDepth (int bitDepth);
/** Sets the bitrate reduction information to null (no compression). */
public void clearBitrateReduction ();
/** Sets the bitrate reduction (aka compression type). */
public void setBitrateReduction (String codecName,
String codecNameVersion,
String codecCreatorApplication,
String codecCreatorApplicationVersion,
String codecQuality,
String dataRate,
String dataRateMode);
/** Sets the sample rate. */
public void setSampleRate (double sampleRate);
/** Sets the word size. */
public void setWordSize (int wordSize);
}
/**
* Public interface to the nested TimeDesc object. Instances
* of this should be created only by appropriate methods, but can be
* accessed through the public methods of this interface.
*/
public static interface TimeDesc {
/** Returns the hours component. */
public int getHours ();
/** Returns the minutes component. */
public int getMinutes ();
/** Returns the seconds component. */
public int getSeconds ();
/** Returns the frames component of the fraction of a second.
* We always consider frames to be thirtieths of a second. */
public int getFrames ();
/** Returns the samples remaining after the frames part of
* the fractional second. */
public int getSamples ();
/** Returns the sample rate on which the samples remainder
* is based. */
public double getSampleRate ();
}
/** Public interface to the nested Face object. Instances
* of this should be created only by appropriate methods, but can be
* accessed through the public methods of this interface. */
public static interface Face {
/** Returns an indexed FaceRegion. */
public FaceRegion getFaceRegion (int i);
/** Adds a FaceRegion. This may be called repeatedly to
* add multiple FaceRegions. */
public void addFaceRegion ();
/** Returns the starting time. */
public TimeDesc getStartTime ();
/** Returns the duration. */
public TimeDesc getDuration ();
/** Returns the direction. */
public String getDirection ();
/** Sets the starting time. This will be converted
* into a TimeDesc. */
public void setStartTime (long samples);
/** Sets the duration. This will be converted
* into a TimeDesc. */
public void setDuration (long samples);
/** Sets the direction. This must be one of the
* directionTypes. FORWARD is recommended for most
* or all cases.
*/
public void setDirection (String direction);
/* End of interface Face */
}
/** Public interface to the nested FaceRegion object. Instances
* of this should be created only by appropriate methods, but can be
* accessed through the public methods of this interface. */
public static interface FaceRegion {
/** Returns the starting time. */
public TimeDesc getStartTime ();
/** Returns the duration. */
public TimeDesc getDuration ();
/** Returns the channel map locations. The array length must
* equal the number of channels. */
public String[] getMapLocations ();
/** Sets the starting time. */
public void setStartTime (long samples);
/** Sets the duration. */
public void setDuration (long samples);
/** Sets the channel map locations. The array length must
* equal the number of channels. */
public void setMapLocations (String[] locations);
/* End of interface FaceRegion */
}
/******************************************************************
* STATIC MEMBER CLASSES.
*
******************************************************************/
/** The implementation of the FormatRegion interface. The combination
* of a public interface and a private implementation is suggested
* in _Java in a Nutshell_.
*/
class FormatRegionImpl implements FormatRegion {
private int _bitDepth;
private double _sampleRate;
private int _wordSize;
private String[] _bitrateReduction;
public FormatRegionImpl () {
_bitDepth = NULL;
_sampleRate = NILL;
_wordSize = NULL;
_bitrateReduction = null;
}
/** Returns bit depth. */
public int getBitDepth ()
{
return _bitDepth;
}
/** Returns the bitrate reduction (compression information).
* This will be an array of seven strings (which may be
* empty but not null) interpreted respectively as follows:
* <ul>
* <li>0: codecName
* <li>1: codecNameVersion
* <li>2: codecCreatorApplication
* <li>3: codecCreatorApplicationVersion
* <li>4: codecQuality
* <li>5: dataRate
* <li>6: dataRateMode
* </ul>
*/
public String[] getBitrateReduction ()
{
return _bitrateReduction;
}
/** Returns sample rate. */
public double getSampleRate ()
{
return _sampleRate;
}
/** Returns word size. */
public int getWordSize ()
{
return _wordSize;
}
/** Returns true if the FormatRegion contains only
* default values. */
public boolean isEmpty ()
{
return _bitDepth == NULL &&
_sampleRate == NILL &&
_wordSize == NULL;
}
/** Sets bit depth. */
public void setBitDepth (int bitDepth)
{
_bitDepth = bitDepth;
}
/** Sets the bitrate reduction information to null (no compression). */
public void clearBitrateReduction ()
{
_bitrateReduction = null;
}
/** Sets the bitrate reduction (compression type). */
public void setBitrateReduction (String codecName,
String codecNameVersion,
String codecCreatorApplication,
String codecCreatorApplicationVersion,
String codecQuality,
String dataRate,
String dataRateMode)
{
_bitrateReduction = new String[7];
_bitrateReduction[0] = codecName;
_bitrateReduction[1] = codecNameVersion;
_bitrateReduction[2] = codecCreatorApplication;
_bitrateReduction[3] = codecCreatorApplicationVersion;
_bitrateReduction[4] = codecQuality;
_bitrateReduction[5] = dataRate;
_bitrateReduction[6] = dataRateMode;
}
/** Sets sample rate. */
public void setSampleRate (double sampleRate)
{
_sampleRate = sampleRate;
}
/** Sets word size. */
public void setWordSize (int wordSize)
{
_wordSize = wordSize;
}
/* End of FormatRegionImpl */
}
/** The implementation of the TimeDesc interface. The combination
* of a public interface and a private implementation is suggested
* in _Java in a Nutshell_.
*/
class TimeDescImpl implements TimeDesc
{
private int _hours;
private int _minutes;
private int _seconds;
private int _frames;
private int _samples;
private double _sampleRate;
private int _frameCount;
/* Constructor rewritten to avoid rounding errors when converting to
* TCF. Now uses integer remainder math instead of floating point.
* Changed the base unit from a double representing seconds to a long
* representing samples. Changed all existing calls (that I could find)
* to this method to accomodate this change.
*
* @author David Ackerman
*/
public TimeDescImpl (long samples)
{
long _sample_count = samples;
_frameCount = 30;
_sampleRate = _curFormatRegion.getSampleRate ();
/* It seems that this method is initially called before a valid
* sample rate has been established, causing a divide by zero
* error.
*/
if (_sampleRate < 0) {
_sampleRate = 44100.0; //reasonable default value
}
long sample_in_1_frame = (long)(_sampleRate/_frameCount);
long sample_in_1_second = sample_in_1_frame * _frameCount;
long sample_in_1_minute = sample_in_1_frame * _frameCount * 60;
long sample_in_1_hour = sample_in_1_frame * _frameCount * 60 * 60;
long sample_in_1_day = sample_in_1_frame * _frameCount * 60 * 60 * 24;
// BWF allows for a negative timestamp but tcf does not, so adjust
// time accordingly
// this might be a good place to report a warning during validation
if (_sample_count < 0) {
_sample_count += sample_in_1_day;
_sample_count = (_sample_count % sample_in_1_day);
}
_hours = (int)(_sample_count / sample_in_1_hour);
_sample_count -= (_hours * sample_in_1_hour);
_minutes = (int)(_sample_count / sample_in_1_minute);
_sample_count -= (_minutes * sample_in_1_minute);
_seconds = (int)(_sample_count / sample_in_1_second);
_sample_count -= (_seconds * sample_in_1_second);
_frames = (int)(_sample_count / sample_in_1_frame);
_sample_count -= (_frames * sample_in_1_frame);
_samples = (int)_sample_count;
/* At present TCF does not have the ability to handle time stamps
* > midnight. Industry practice is to roll the clock forward to
* zero or back to 23:59:59:29... when crossing this boundary
* condition.
*/
_hours = _hours % 24;
}
/** Returns the hours component. */
public int getHours () {
return _hours;
}
/** Returns the minutes component. */
public int getMinutes () {
return _minutes;
}
/** Returns the seconds component. */
public int getSeconds () {
return _seconds;
}
/** Returns the frames component of the fraction of a second.
* We always consider frames to be thirtieths of a second. */
public int getFrames () {
return _frames;
}
/** Returns the samples remaining after the frames part of
* the fractional second. */
public int getSamples () {
return _samples;
}
/** Returns the sample rate on which the samples remainder
* is based. */
public double getSampleRate () {
return _sampleRate;
}
} /* End of TimeDescImpl */
/** The implementation of the Face interface. The combination
* of a public interface and a private implementation is suggested
* in _Java in a Nutshell_.
*/
class FaceImpl implements Face {
List _regionList;
TimeDesc _startTime;
TimeDesc _duration;
String _direction;
/** Constructor. Initially the duration is set
* to null, indicating unknown value. */
public FaceImpl ()
{
_regionList = new ArrayList ();
_startTime = new TimeDescImpl (0);
_duration = null;
}
/** Returns an indexed FaceRegion. */
public FaceRegion getFaceRegion (int i) {
return (FaceRegion) _regionList.get (i);
}
/** Adds a FaceRegion. This may be called repeatedly to
* add multiple FaceRegions. */
public void addFaceRegion () {
_regionList.add (new FaceRegionImpl ());
}
/** Returns the starting time. Will be zero if not
* explicitly specified. */
public TimeDesc getStartTime () {
return _startTime;
}
/** Returns the duration. May be null if the duration
* is unspecified. */
public TimeDesc getDuration () {
return _duration;
}
/** Returns the direction. */
public String getDirection ()
{
return _direction;
}
/** Sets the starting time. This will be converted
* into a TimeDesc. */
public void setStartTime (long samples)
{
_startTime = new TimeDescImpl (samples);
}
/** Sets the duration. This will be converted
* into a TimeDesc. */
public void setDuration (long samples)
{
_duration = new TimeDescImpl (samples);
}
/** Sets the direction. This must be one of the
* directionTypes. FORWARD is recommended for most
* or all cases.
*/
public void setDirection (String direction)
{
_direction = direction;
}
/* End of FaceImpl */
}
/** The implementation of the Face interface. The combination
* of a public interface and a private implementation is suggested
* in _Java in a Nutshell_.
*/
class FaceRegionImpl implements FaceRegion {
private TimeDesc _startTime;
private TimeDesc _duration;
private String[] _mapLocations;
public FaceRegionImpl ()
{
_startTime = new TimeDescImpl (0);
_duration = null;
}
/** Returns the starting time. */
public TimeDesc getStartTime () {
return _startTime;
}
/** Returns the duration. */
public TimeDesc getDuration () {
return _duration;
}
/** Returns the channel map locations. The array length
* will equal the number of channels. */
public String[] getMapLocations ()
{
return _mapLocations;
}
/** Sets the duration. This will be converted
* into a TimeDesc. */
public void setStartTime (long samples) {
_startTime = new TimeDescImpl (samples);
}
/** Sets the duration. This will be converted
* into a TimeDesc. */
public void setDuration (long samples)
{
_duration = new TimeDescImpl (samples);
}
/** Sets the channel map locations. The array length must
* equal the number of channels. */
public void setMapLocations (String[] locations)
{
_mapLocations = locations;
}
/* End of FaceRegionImpl */
}
/* End of inner classes */
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/** Returns analog/digital flag. Value will always be
* "FILE_DIGITAL" in practice. */
public String getAnalogDigitalFlag ()
{
return _analogDigitalFlag;
}
/** Returns application-specific data. We assume this is
* representable in String format.
*/
public String getAppSpecificData ()
{
return _appSpecificData;
}
/** Returns audio data encoding. */
public String getAudioDataEncoding ()
{
return _audioDataEncoding;
}
/** Returns the bitrate reduction (compression information).
* This will be an array of seven strings (which may be
* empty, but should never be null) interpreted as follows:
* <ul>
* <li>0: codecName
* <li>1: codecNameVersion
* <li>2: codecCreatorApplication
* <li>3: codecCreatorApplicationVersion
* <li>4: codecQuality
* <li>5: dataRate
* <li>6: dataRateMode
* </ul>
*/
public String[] getBitrateReduction ()
{
return _curFormatRegion.getBitrateReduction();
}
/* Returns the sample rate. */
public double getSampleRate ()
{
return _curFormatRegion.getSampleRate ();
}
/** Return the byte order: 0 = big-endian; 1 = little-endian. */
public int getByteOrder ()
{
return _byteOrder;
}
/** Returns disposition. */
public String getDisposition ()
{
return _disposition;
}
/** Gets the list of Faces. Normally there will be only one face
* in a digital file. */
public List getFaceList ()
{
return _faceList;
}
/** Return the offset of the first byte of sample data. */
public long getFirstSampleOffset ()
{
return _firstSampleOffset;
}
/** Returns format name. */
public String getFormat ()
{
return _format;
}
/** Gets the list of Format Regions. Since one is created
* automatically on initialization, it's possible that the
* list will contain a Format Region with only default values.
* This should be checked with isEmpty ().
*/
public List getFormatList ()
{
return _formatList;
}
/** Returns the names of the map locations.
* The returned
* value is an array whose length equals the number of
* channels and whose elements correspond to channels 0, 1,
* etc.
*/
public String[] getMapLocations() {
return _curFace.getFaceRegion(0).getMapLocations();
}
/** Returns number of channels. */
public int getNumChannels ()
{
return _numChannels;
}
/** Returns primary identifier. */
public String getPrimaryIdentifier ()
{
return _primaryIdentifier;
}
/** Returns primary identifier type. */
public String getPrimaryIdentifierType ()
{
return _primaryIdentifierType;
}
/** Returns schema version. */
public String getSchemaVersion ()
{
return _schemaVersion;
}
/** Returns specification version of the document format. */
public String getSpecificationVersion ()
{
return _specificationVersion;
}
/** Returns the use (role of the document).
* The value returned is an array of two strings,
* the useType and the otherType. */
public String[] getUse ()
{
return _use;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/** Sets the analog/digital flag. The value set should always
* be "FILE_DIGITAL". */
public void setAnalogDigitalFlag (String flagType)
{
_analogDigitalFlag = flagType;
}
/** Sets the bitrate reduction (compression type). */
public void setBitrateReduction (String codecName,
String codecNameVersion,
String codecCreatorApplication,
String codecCreatorApplicationVersion,
String codecQuality,
String dataRate,
String dataRateMode)
{
_curFormatRegion.setBitrateReduction (codecName,
codecNameVersion, codecCreatorApplication,
codecCreatorApplicationVersion,
codecQuality, dataRate, dataRateMode);
}
/** Set the bitrate reduction information to null (no compression). */
public void clearBitrateReduction ()
{
_curFormatRegion.clearBitrateReduction ();
}
/** Sets the byte order.
* @param order Byte order: 0 = big-endian, 1 = little-endian
*/
public void setByteOrder (int order)
{
_byteOrder = order;
}
/** Sets the byte order.
*/
public void setByteOrder (String order)
{
if (order.substring (0, 3).toLowerCase ().equals ("big")) {
_byteOrder = BIG_ENDIAN;
}
else if (order.substring (0, 6).toLowerCase ().equals ("little")) {
_byteOrder = LITTLE_ENDIAN;
}
}
/** Sets the audio data encoding. */
public void setAudioDataEncoding (String audioDataEncoding)
{
_audioDataEncoding = audioDataEncoding;
}
/** Set the application-specific data. For present purposes,
* we assume this is representable as a text string. */
public void setAppSpecificData (String data)
{
_appSpecificData = data;
}
/** Sets the bit depth. */
public void setBitDepth (int bitDepth)
{
_curFormatRegion.setBitDepth (bitDepth);
}
/** Sets the disposition. */
public void setDisposition (String disposition)
{
_disposition = disposition;
}
/** Sets the direction.
* This must be one of the values
* FORWARD, REVERSE, A_WIND, B_WIND, C_WIND, D_WIND,
* FRONT, BACK. FORWARD may be the only one that
* makes sense for digital formats.
*/
public void setDirection (String direction)
{
_curFace.setDirection (direction);
}
/** Sets the duration in samples.
* This affects the current face and its first FaceRegion.
*/
public void setDuration (long duration)
{
_curFace.setDuration (duration);
_curFace.getFaceRegion(0).setDuration (duration);
}
/** Sets the offset of the first byte of sample data. */
public void setFirstSampleOffset (long offset)
{
_firstSampleOffset = offset;
}
/** Sets the format name. */
public void setFormat (String format)
{
_format = format;
}
/** Sets the array of channel map locations. The length
* of the array must equal the number of channels. */
public void setMapLocations (String[] locations) {
_curFace.getFaceRegion(0).setMapLocations (locations);
}
/** Sets the number of channels. */
public void setNumChannels (int numChannels)
{
_numChannels = numChannels;
}
/** Sets the primary identifier. */
public void setPrimaryIdentifier (String primaryIdentifier)
{
_primaryIdentifier = primaryIdentifier;
}
/** Sets the primary identifier type. If the primary identifier
* type is OTHER, use setOtherPrimaryIdentifierType instead.
*/
public void setPrimaryIdentifierType (String primaryIdentifierType)
{
_primaryIdentifierType = primaryIdentifierType;
}
/** Sets the primary identifier type as "OTHER", and
* set the otherType.
*/
public void setOtherPrimaryIdentifierType (String otherType)
{
_primaryIdentifierType = "OTHER";
_primaryIdentifierOtherType = otherType;
}
/** Sets the sample rate. */
public void setSampleRate (double sampleRate)
{
_curFormatRegion.setSampleRate (sampleRate);
}
/** Sets the specification version of the document format.*/
public void setSpecificationVersion (String specificationVersion)
{
_specificationVersion = specificationVersion;
}
/** Sets the start time in samples.
* This affects the current face and its first FaceRegion.
*/
public void setStartTime (long samples)
{
_curFace.setStartTime (samples);
_curFace.getFaceRegion(0).setStartTime (samples);
}
/** Sets the role of the document. Permitted values are
* ORIGINAL_MASTER, PRESERVATION_MASTER, PRODUCTION_MASTER,
* SERVICE, PREVIEW, or OTHER.
* If useType is "OTHER", then otherType
* is significant. Since OTHER is the only meaningful
* value for a digital document, the code assumes this will always
* be the case and uses otherType. */
public void setUse (String useType, String otherType)
{
_use = new String[] {useType, otherType};
}
/** Sets the word size. */
public void setWordSize (int wordSize)
{
_curFormatRegion.setWordSize (wordSize);
}
/** Adds a FormatRegion object to a FormatSize list.
* The most recently added FormatRegion object will
* be filled in by setBitDepth, setSampleRate, and
* setWordSize.
*/
public void addFormatRegion ()
{
_curFormatRegion = new FormatRegionImpl ();
_formatList.add (_curFormatRegion);
}
/** Adds a Face.
*/
public void addFace ()
{
_curFace = new FaceImpl ();
_faceList.add (_curFace);
_curFace.addFaceRegion();
}
}
@@ -1,167 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* Encapsulates information about agents, either individual
* persons or corporate bodies.
*/
public class Agent
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
private String _name;
private AgentType _type;
private String _address;
private String _telephone;
private String _fax;
private String _email;
private String _web;
private String _note;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates an Agent given a name and an AgentType.
*/
public Agent (String name, AgentType type)
{
_name = name;
_type = type;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Returns the value of the address property.
*/
public String getAddress ()
{
return _address;
}
/**
* Returns the value of the email property.
*/
public String getEmail ()
{
return _email;
}
/**
* Returns the value of the fax property.
*/
public String getFax ()
{
return _fax;
}
/**
* Returns the value of the name property.
*/
public String getName ()
{
return _name;
}
/**
* Returns the value of the note property.
*/
public String getNote ()
{
return _note;
}
/**
* Returns the value of the telephone property.
*/
public String getTelephone ()
{
return _telephone;
}
/**
* Returns the value of the type property.
*/
public AgentType getType ()
{
return _type;
}
/**
* Returns the value of the web property.
*/
public String getWeb ()
{
return _web;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/**
* Sets the value of the address property.
*/
public void setAddress (String address)
{
_address = address;
}
/**
* Sets the value of the email property.
*/
public void setEmail (String email)
{
_email = email;
}
/**
* Sets the value of the fax property.
*/
public void setFax (String fax)
{
_fax = fax;
}
/**
* Sets the value of the name property.
*/
public void setName (String name)
{
_name = name;
}
/**
* Sets the value of the note property.
*/
public void setNote (String note)
{
_note = note;
}
/**
* Sets the value of the telephone property.
*/
public void setTelephone (String telephone)
{
_telephone = telephone;
}
/**
* Sets the value of the web property.
*/
public void setWeb (String web)
{
_web = web;
}
}
@@ -1,60 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for an Agent.
* Applications will not create or modify AgentTypes, but will
* use one of the predefined AgentType instances COMMERCIAL, GOVERNMENT,
* EDUCATIONAL, NONPROFIT, STANDARD, or OTHER.
*
* @see Agent
*/
public final class AgentType
extends EnumerationType
{
/******************************************************************
* PUBLIC STATIC INSTANCES.
******************************************************************/
/**
* Agent type for a commercial entity.
*/
public static final AgentType COMMERCIAL = new AgentType ("Commercial");
/**
* Agent type for a governmental body.
*/
public static final AgentType GOVERNMENT = new AgentType ("Government");
/**
* Agent type for an educational institution.
*/
public static final AgentType EDUCATIONAL = new AgentType ("Educational");
/**
* Agent type for a non-profit organization.
*/
public static final AgentType NONPROFIT = new AgentType ("Non-profit");
/**
* Agent type for a standards body.
*/
public static final AgentType STANDARD = new AgentType ("Standards body");
/**
* Agent type that doesn't fit the other categories.
*/
public static final AgentType OTHER = new AgentType ("Other");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create AgentTypes directly.
**/
private AgentType (String value)
{
super (value);
}
}
@@ -1,40 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
*
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for the analog/digital
* flag of AESAudioMetadata.
* Applications will not create or modify instances of this class, but will
* use one of the predefined AnalogDigitalFlagType instances.
*
* @author Gary McGath
*
*/
public class AnalogDigitalFlagType extends EnumerationType {
/** Enumeration instance for analog data */
public static final AnalogDigitalFlagType ANALOG =
new AnalogDigitalFlagType ("ANALOG");
/** Enumeration instance for physical digital data */
public static final AnalogDigitalFlagType PHYS_DIGITAL =
new AnalogDigitalFlagType ("PHYS_DIGITAL");
/** Enumeration instance for FILE digital data */
public static final AnalogDigitalFlagType FILE_DIGITAL =
new AnalogDigitalFlagType ("FILE_DIGITAL");
/**
* Applications will never create PropertyTypes directly.
**/
private AnalogDigitalFlagType (String value)
{
super (value);
}
}
@@ -1,128 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
//import java.io.*;
import java.util.*;
/**
* The application class for JHOVE. One instance of this class is
* generated by the application. Various utility and informational
* methods are contained here.
*/
public class App
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Application build date. */
private Date _date;
/** Application name. */
private String _name;
/** Application release identifier. */
private String _release;
/** Application rights statement. */
private String _rights;
/** Application invocation syntax. */
private String _usage;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates an App object.
* @param name Name of the application.
* @param release Release identifier.
* @param date Last modification date of the application code,
* in the form of an array of three numbers.
* <code>date[0]</code> is the year,
* <code>date[1]</code> the month, and
* <code>date[2]</code> the day.
* @param usage String summarizing command line usage, to
* output when app name is typed with no
* arguments or flags.
* @param rights Copyright or other rights statement.
*/
public App (String name, String release, int [] date, String usage,
String rights)
{
Calendar calendar = new GregorianCalendar ();
calendar.set (date[0], date[1]-1, date[2]);
_name = name;
_release = release;
_date = calendar.getTime ();
_usage = usage;
_rights = rights;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/**
* Returns application code creation date
*/
public Date getDate ()
{
return _date;
}
/**
* Returns the name of this application
*/
public String getName ()
{
return _name;
}
/**
* Returns the release identifier for this application.
*/
public String getRelease ()
{
return _release;
}
/**
* Returns the rights string.
*/
public String getRights ()
{
return _rights;
}
/**
* Returns the usage string, which should be output when the user
* enters a command line with only the application name.
*/
public String getUsage ()
{
return _usage;
}
/******************************************************************
* Serialization methods.
******************************************************************/
/**
* Outputs detailed information about the application,
* including configuration, available modules and handlers,
* etc.
*/
public void show (OutputHandler handler)
{
handler.show (this);
}
}
@@ -1,66 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
*
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
/**
* Class for providing an InputSource to XMPHandler,
* with a ByteArrayInputStream as the basis of the
* InputSource. This is suitable for a number of modules.
*
* @author Gary McGath
*
*/
public class ByteArrayXMPSource extends XMPSource {
/* The underlying ByteArrayInputStream. */
ByteArrayInputStream _instrm;
/**
* Constructor based on ByteArrayInputStream.
*
* @param instrm ByteArrayInputStream containing the XMP
*/
public ByteArrayXMPSource (ByteArrayInputStream instrm)
throws IOException
{
super (new InputStreamReader
(new XMLWrapperStream (instrm)));
_instrm = instrm;
// Prepare for resetting.
instrm.mark (instrm.available ());
}
/**
* Constructor based on ByteArrayInputStream with encoding.
*
* @param instrm ByteArrayInputStream containing the XMP
*/
public ByteArrayXMPSource (ByteArrayInputStream instrm,
String encoding)
throws IOException
{
super (new InputStreamReader
(new XMLWrapperStream (instrm), encoding));
_instrm = instrm;
// Prepare for resetting.
instrm.mark (instrm.available ());
}
/* (non-Javadoc)
* @see edu.harvard.hul.ois.jhove.XMPSource#resetReader()
*/
protected void resetReader() {
_instrm.reset ();
_reader = new InputStreamReader (_instrm);
}
}
@@ -1,25 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* An interface for supporting a general, simple callback function.
*/
public interface Callback
{
/**
* A generic callback function. Any class which needs to support
* callback can implement callback and pass a reference to itself
* to the function that does the callback.
*
* @param selector An indicator of the function to be performed.
* Interpretation is determined by the implementing class.
* @param parm Whatever data may be appropriate to the callback.
*
* @return As specified by the implementing class.
*/
public int callback (int selector, Object parm);
}
@@ -1,93 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates the result of calculations which provide a greater
* or lesser degree of confirmation of the integrity of a digital
* object's content, including checksums, CRC's, message digests,
* etc.
*
* @see ChecksumType
* @see Checksummer
*/
public class Checksum
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
private ChecksumType _type;
private String _value;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates a Checksum with a given value and type
*/
public Checksum (String value, ChecksumType type)
{
_value = value;
_type = type;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/**
* Returns this Checksum's type
*/
public ChecksumType getType ()
{
return _type;
}
/**
* Returns this Checksum's value
*/
public String getValue ()
{
return _value;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/**
* Sets the type of this Checksum
*/
public void setType (ChecksumType type)
{
_type = type;
}
/**
* Sets the value of this Checksum
*/
public void setValue (String value)
{
_value = value;
}
/******************************************************************
* Put here as a convenience for checksum calculators
*****************************************************************/
/**
* Maps unsigned byte value (0 to 256) to signed byte value (-128 to 127).
*/
public static byte unsignedByteToByte (int value)
{
return (byte) ((value < 127) ? value : value - 256);
}
}
@@ -1,145 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2006 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.IOException;
import java.io.*;
/**
* A ChecksumInputStream is a FilterInputStream with the added
* functionality of calculating checksums as it goes.
*
* The idea of replacing this with java.util.zip.CheckedInputStream
* looks very tempting, but we need the byte count, which
* CheckedInputStream doesn't provide.
*
* @author Gary McGath
*
*/
public class ChecksumInputStream extends FilterInputStream {
private InputStream subsumedStream;
private Checksummer _cksummer;
private long _nBytes;
/**
* Constructor.
*
* @param stream Stream to be filtered
* @param cksummer Object to calculate checksum on the bytes
* as they are read
*/
public ChecksumInputStream(InputStream stream, Checksummer cksummer) {
super (stream);
subsumedStream = stream;
_cksummer = cksummer;
_nBytes = 0;
}
/**
* Reads a byte from the subsumed stream, updating
* the byte count and the checksums.
*/
public int read() throws IOException {
int ch = subsumedStream.read ();
if (ch >= 0) {
_nBytes++;
if (_cksummer != null) {
_cksummer.update (ch);
}
}
return ch;
}
/**
* Reads some number of bytes from the input stream and
* stores them into the buffer array b. The number of
* bytes actually read is returned as an integer.
*
* All bytes read are fed through the checksummer.
*/
public int read(byte[] b) throws IOException
{
int len = subsumedStream.read (b);
// Careful here -- don't want to add -1 bytes at EOF
if (len > 0) {
if (_cksummer != null) {
_cksummer.update (b);
}
_nBytes += len;
}
return len;
}
/**
* Reads up to len bytes of data from the input stream
* into an array of bytes. An attempt is made to read as
* many as len bytes, but a smaller number may be read,
* possibly zero. The number of bytes actually read is
* returned as an integer.
*
* All bytes read are fed through the checksummer.
*/
public int read(byte[] b, int off, int len) throws IOException
{
len = subsumedStream.read (b, off, len);
// Careful here -- don't want to add -1 bytes at EOF
if (len > 0) {
if (_cksummer != null) {
_cksummer.update (b, off, len);
}
_nBytes += len;
}
return len;
}
/**
* Skips n bytes.
* Reads them and feeds them through the checksummer.
*/
public long skip (long n) throws IOException {
long nret = 0;
while (n > 0) {
// grab the data in reasonable buffer-sized chunks.
int bufsize = (int) (n > 8192 ? 8192 : n);
byte[] buf = new byte[bufsize];
int nread = read (buf);
if (nread <= 0) {
break;
}
nret += nread;
n -= nread;
}
return nret;
}
/**
* Closes the subsumed stream.
*/
public void close () throws IOException
{
subsumedStream.close();
}
/**
* Returns the byte count to date on the stream.
* This returns the number of bytes read.
* Because of buffering, this is not a reliable
* indicator of how many bytes have actually been processed.
*/
public long getNBytes ()
{
return _nBytes;
}
/**
* Returns the Checksummer object.
*/
public Checksummer getChecksummer ()
{
return _cksummer;
}
}
@@ -1,37 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for a Checksum on a content
* stream or file.
* Applications will not create or modify ChecksumTypes, but will
* use one of the predefined ChecksumType instances
* CRC32, MD5, or SHA1.
*
* @see Checksum
*/
public final class ChecksumType
extends EnumerationType
{
/** 32-bit Cyclical Redundancy Checksum. */
public static final ChecksumType CRC32 = new ChecksumType ("CRC32");
/** 128-bit Message Digest 5. */
public static final ChecksumType MD5 = new ChecksumType ("MD5");
/** 160-bit Secure Hash Algorithm. */
public static final ChecksumType SHA1 = new ChecksumType ("SHA-1");
/**
* Applications will never create ChecksumTypes directly.
**/
private ChecksumType (String value)
{
super (value);
}
}
@@ -1,193 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2006 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.security.*;
import java.util.zip.*;
/**
* The Checksummer class encapsulates the calculation of the
* CRC32, MD5, and SHA-1 checksums.
*/
public class Checksummer implements java.util.zip.Checksum
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Byte count. */
protected long _nByte;
/** CRC32 object. */
private CRC32 _crc32;
/** MD5 message digest. */
private MessageDigest _md5;
/** SHA-1 message digest. */
private MessageDigest _sha1;
/**
* Creates a Checksummer, with instances of each of
* CRC32, MD5 MessageDigest, and SHA-1 MessageDigest.
* If one or both of the MessageDigests aren't supported
* on the current platform, they are left as null.
*
* @see CRC32
* @see MessageDigest
*/
public Checksummer ()
{
reset ();
}
/** Resets all checksums and the byte count to their
* initial values.
*/
public void reset ()
{
_nByte = 0;
_crc32 = new CRC32 ();
try {
_md5 = MessageDigest.getInstance ("MD5");
_sha1 = MessageDigest.getInstance ("SHA-1");
}
catch (NoSuchAlgorithmException e) {
}
}
/** getValue is required by the Checksum interface, but
* we can return only one of the three values. We
* return the CRC32 value, since that's the one which
* is guaranteed to be available.
*/
public long getValue ()
{
return _crc32.getValue ();
}
/**
* Updates the checksum with the argument.
* Called when a signed byte is available.
*/
public void update (byte b)
{
_crc32.update (b);
if (_md5 != null) {
_md5.update (b);
}
if (_sha1 != null) {
_sha1.update (b);
}
}
/**
* Updates the checksum with the argument.
* Called when an unsigned byte is available.
*/
public void update (int b)
{
byte sb;
if (b > 127) {
sb = (byte) (b - 256);
}
else {
sb = (byte) b;
}
update (sb);
}
/**
* Updates the checksum with the argument.
* Called when a byte array is available.
*/
public void update (byte[] b)
{
_crc32.update (b);
if (_md5 != null) {
_md5.update (b);
}
if (_sha1 != null) {
_sha1.update (b);
}
}
/**
* Updates the checksum with the argument.
* Called when a byte array is available.
*/
public void update (byte[] b, int off, int len)
{
_crc32.update (b, off, len);
if (_md5 != null) {
_md5.update (b, off, len);
}
if (_sha1 != null) {
_sha1.update (b, off, len);
}
}
/**
* Returns the value of the CRC32 as a hex string.
*/
public String getCRC32 ()
{
return padLeadingZeroes
(Long.toHexString (_crc32.getValue ()), 8);
}
/**
* Returns the value of the MD5 digest as a hex string.
* Returns null if the digest is not available.
*/
public String getMD5 ()
{
String value = null;
if (_md5 != null) {
StringBuffer buffer = new StringBuffer ();
byte [] digest = _md5.digest ();
for (int i=0; i<digest.length; i++) {
int un = (digest[i] >= 0) ? digest[i] : 256+digest[i];
buffer.append (padLeadingZeroes
(Integer.toHexString (un), 2));
}
value = buffer.toString ();
}
return value;
}
/**
* Returns the value of the SHA-1 digest as a hex string.
* Returns null if the digest is not available.
*/
public String getSHA1 ()
{
String value = null;
if (_sha1 != null) {
StringBuffer buffer = new StringBuffer ();
byte [] digest = _sha1.digest ();
for (int i=0; i<digest.length; i++) {
int un = (digest[i] >= 0) ? digest[i] : 256+digest[i];
buffer.append (padLeadingZeroes
(Integer.toHexString (un), 2));
}
value = buffer.toString ();
}
return value;
}
/** Pad a hexadecimal (or other numeric) string out to
* the specified length with leading zeroes. */
private String padLeadingZeroes (String str, int len)
{
// This is optimized for adding just one leading zero
// or none, which will be the usual case.
while (str.length () < len) {
str = "0" + str;
}
return str;
}
}
@@ -1,395 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.util.*;
import org.xml.sax.*;
/**
* SAX Parser for the configuration file.
*/
public class ConfigHandler
extends org.xml.sax.helpers.DefaultHandler
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
private String _class;
protected StringBuffer _content;
private Map<String, String> _extension;
/** The schema name */
private final static String configSchemaName =
"jhoveConfig.xsd";
/** The list of handlers. Each element in the List is an
* array of two Strings representing the class and the initialization
* string. */
private List<String[]> _handler;
/** The list of handler parameters. Each element in the List is
* a List of Strings (which may be empty but not null) representing
* parameters to be passed to the module. List elements are in
* one-to-one correspondence with _handler.
*/
protected List<List<String>> _handlerParams;
private String _init;
private List<String> _param;
private String _tempDir;
private String _mixVsn;
private String _encoding;
private String _logLevel;
private int _bufferSize;
private String _jhoveHome;
private int _sigBytes;
protected boolean _isHandler;
/* _isModule is protected rather than private so that subclasses
* can add elements to the module element. */
protected boolean _isModule;
private boolean _isTempDir;
private boolean _isMixVsn;
private boolean _isEncoding;
private boolean _isBufferSize;
private boolean _isJhoveHome;
private boolean _isLogLevel;
private boolean _isSigBytes;
/** The list of modules. Each element in the List is an
* array of two Strings representing the class and the initialization
* string. */
protected List<ModuleInfo> _module;
/** The list of module parameters. Each element in the List is
* a List of Strings (which may be empty but not null) representing
* parameters to be passed to the module. List elements are in
* one-to-one correspondence with _module.
*/
protected List<List<String>> _modParams;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates a ConfigHandler.
*/
public ConfigHandler ()
{
_module = new ArrayList<ModuleInfo> ();
_handler = new ArrayList<String[]> ();
_modParams = new ArrayList<List<String>> ();
_handlerParams = new ArrayList<List<String>> ();
_isModule = false;
_isHandler = false;
_isTempDir = false;
_isEncoding = false;
_isBufferSize = false;
_isJhoveHome = false;
_isLogLevel = false;
_bufferSize = -1;
_encoding = null;
_extension = new Hashtable<String, String> ();
_tempDir = null;
_mixVsn = null;
_sigBytes = 1024;
_logLevel = null;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/**
* Returns the List of Modules specified by the config file.
* Each element of the List is a String[2] whose elements are
* the module class name and initialization value.
*
* @see Module
*/
public List<ModuleInfo> getModule ()
{
return _module;
}
/**
* Returns the List of module parameters specified by the config file.
* Each element of the List is a List (possibly empty) of Strings
* whose elements are parameters to pass to the module. The
* values returned by <code>getModuleParams()</code> are in
* one-to-one correspondence with those return by <code>getModule()</code>.
*/
public List<List<String>> getModuleParams ()
{
return _modParams;
}
/**
* Returns the List of handler parameters specified by the config file.
* Each element of the List is a List (possibly empty) of Strings
* whose elements are parameters to pass to the output handler. The
* values returned by <code>getHandlerParams()</code> are in
* one-to-one correspondence with those return by <code>getHandler()</code>.
*/
public List<List<String>> getHandlerParams ()
{
return _handlerParams;
}
/**
* Returns the List of OutputHandlers specified by the config file.
*
* @see OutputHandler
*/
public List<String[]> getHandler ()
{
return _handler;
}
/**
* Returns the temporary directory path specified by the config file,
* with final path separator.
*/
public String getTempDir ()
{
return _tempDir;
}
/** Returns the MIX schema version specified by the config file.
* Acceptable values are "0.2" and "1.0" and "2.0".
*/
public String getMixVsn ()
{
return _mixVsn;
}
/** Returns the number of bytes to examine when looking for an
* indefinitely positioned signature, or checking the first
* sigBytes bytes of a file in lieu of a signature.
*/
public int getSigBytes ()
{
return _sigBytes;
}
/**
* Returns the character encoding specified by the config file.
*/
public String getEncoding ()
{
return _encoding;
}
/**
* Returns the buffer size specified in the config file.
*
* @return the buffer size, or -1 if none specified
*/
public int getBufferSize ()
{
return _bufferSize;
}
/**
* Return an associative map of configuration extensions.
*/
public Map<String, String> getExtensions ()
{
return _extension;
}
/**
* Returns the path to the application's home directory,
* with final path separator.
*/
public String getJhoveHome ()
{
return _jhoveHome;
}
/**
* Returns the name of the desired log level. This should be the
* name of one of the predefined values of java.util.logging.Level,
* e.g., "WARNING", "INFO", "ALL". The default level is SEVERE.
*/
public String getLogLevel ()
{
return _logLevel;
}
/******************************************************************
* SAX parser methods.
******************************************************************/
/**
* SAX parser callback method.
*/
public void startElement (String namespaceURI, String localName,
String rawName, Attributes atts)
throws SAXException
{
_content = new StringBuffer ();
if (rawName.equals ("module")) {
_isModule = true;
_init = null;
_param = new ArrayList<String> (1);
_class = null;
}
else if (rawName.equals ("outputHandler")) {
_isHandler = true;
_init = null;
_param = new ArrayList<String> (1);
_class = null;
}
else if (rawName.equals ("tempDirectory")) {
_isTempDir = true;
}
else if (rawName.equals ("mixVersion")) {
_isMixVsn = true;
}
else if (rawName.equals ("defaultEncoding")) {
_isEncoding = true;
}
else if (rawName.equals ("bufferSize")) {
_isBufferSize = true;
}
else if (rawName.equals ("jhoveHome")) {
_isJhoveHome = true;
}
else if (rawName.equals ("logLevel")) {
_isLogLevel = true;
}
else if (rawName.equals ("sigBytes")) {
_isSigBytes = true;
}
}
/**
* SAX parser callback method.
*/
public void characters (char [] ch, int start, int length)
throws SAXException
{
_content.append (ch, start, length);
}
/**
* SAX parser callback method.
*/
public void endElement (String namespaceURI, String localName,
String rawName)
throws SAXException
{
if (_isModule) {
if (rawName.equals ("class")) {
_class = _content.toString ();
}
else if (rawName.equals ("init")) {
_init = _content.toString ();
}
else if (rawName.equals ("param")) {
_param.add (_content.toString ());
}
else if (rawName.equals ("module")) {
ModuleInfo modInfo = new ModuleInfo( _class, _init);
_module.add (modInfo);
_modParams.add (_param);
_isModule = false;
}
}
else if (_isHandler) {
if (rawName.equals ("class")) {
_class = _content.toString ();
}
else if (rawName.equals ("init")) {
_init = _content.toString ();
}
else if (rawName.equals ("param")) {
_param.add (_content.toString ());
}
else if (rawName.equals ("outputHandler")) {
String [] tuple = { _class, _init };
_handler.add (tuple);
_handlerParams.add (_param);
_isHandler = false;
}
}
else if (_isTempDir) {
_tempDir = _content.toString ().trim ();
_isTempDir = false;
}
else if (_isMixVsn) {
_mixVsn = _content.toString ().trim ();
_isMixVsn = false;
}
else if (_isSigBytes) {
try {
_sigBytes = Integer.parseInt (_content.toString ().trim ());
}
catch (NumberFormatException e) {}
_isSigBytes = false;
}
else if (_isEncoding) {
_encoding = _content.toString ().trim ();
_isEncoding = false;
}
else if (_isJhoveHome) {
_jhoveHome = _content.toString ().trim ();
_isJhoveHome = false;
}
else if (_isLogLevel) {
_logLevel = _content.toString ().trim ();
_isLogLevel = false;
}
else if (_isBufferSize) {
try {
_bufferSize = Integer.parseInt (_content.toString ().trim ());
}
catch (NumberFormatException e) {
/* Just ignore a malformed number */
}
_isBufferSize = false;
}
else if (!rawName.equals ("jhoveConfig")) {
_extension.put (rawName, _content.toString ().trim ());
}
}
/** EntityResolver designed to locate the config schema. It tries to find it
* as a local resource.
*
* It appears that not all SAX implementations will actually call this
* function for schema resolution, so this isn't a guarantee that the
* schema in the config file won't be called directly. But hopefully
* it will cut down on the burden on the server with the official
* schema copy.
*/
public InputSource resolveEntity (String publicId, String systemId)
throws SAXException, IOException {
if (systemId.endsWith (configSchemaName)) {
try {
URL resURL = this.getClass().getResource("jhoveConfig.xsd");
InputStream strm = resURL.openStream ();
return new InputSource (strm);
}
catch (Exception e) {}
}
// If we couldn't get the local resource, use default location methods
return super.resolveEntity (publicId, systemId);
}
}
@@ -1,212 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
*
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.util.*;
import javax.swing.*;
import edu.harvard.hul.ois.jhove.viewer.ConfigWindow;
/**
*
* Class to write out configuration information to the configuration file.
* To minimize the chance of getting into a bad state, it writes to a
* temporary file, then replaces the old config file with that file,
* rather than directly overwriting the existing file.
*
* @author Gary McGath
*
*/
public class ConfigWriter {
private PrintWriter _confOut;
private File _tempFile;
private File _confFile;
ConfigWindow _parent;
/**
* Constructor.
* Creates a temporary file for writing and creates an OutputStreamWriter
* to write to it. If there is already a file located by
* <code>file</code>, it will not be replaced or overwritten
* until <code>writeFile</code> has successfully written out
* the temporary file.
*
* @param file Location of the configuration file
*
* @param parent The ConfigWindow which invoked this instance.
* May be null if invoked to write a default config file.
*/
public ConfigWriter (File file, ConfigWindow parent) throws IOException
{
_confFile = file;
_parent = parent;
// Set up a temporary file to write to.
String path = file.getParent();
file.getParentFile().mkdirs(); // Make sure the directory exists
_tempFile = File.createTempFile ("jho", ".conf", new File (path));
//_tempFile.createNewFile();
FileOutputStream ostrm = new FileOutputStream (_tempFile);
OutputStreamWriter osw = new OutputStreamWriter (ostrm, "UTF-8");
_confOut = new PrintWriter (osw);
}
/**
* Writes out the content of the file to the temporary file,
* then deletes the existing configuration file (as specified
* by the constructor parameter) and renames the temporary file
* to the configuration file.
*
* If the temporary file can't be written, or the configuration
* file can't be replaced, a warning dialog is put up and the
* configuration file remains unchanged.
*/
public void writeFile (List<ModuleInfo> modules,
List<String[]> handlers,
File homeDir,
File tempDir,
String encoding,
int bufferSize) throws IOException
{
writeHead ();
// Write the home and temp directories. Home must always be valid.
_confOut.println (" <jhoveHome>" +
encodeContent (homeDir.getPath ()) +
"</jhoveHome>");
// Write out the encoding
if (encoding != null && encoding.length() > 0) {
_confOut.println (" <defaultEncoding>" +
encodeContent (encoding) + "</defaultEncoding>");
}
if (tempDir != null) {
_confOut.println (" <tempDirectory>" +
encodeContent (tempDir.getPath ()) +
"</tempDirectory>");
}
// Write the buffer size if not default
if (bufferSize > 0) {
_confOut.println (" <bufferSize>" + bufferSize +
"</bufferSize>");
}
// Write out the modules
ListIterator<ModuleInfo> iter = modules.listIterator ();
while (iter.hasNext ()) {
ModuleInfo minfo = iter.next ();
// The class must be non-null, but init may be null.
// If the class is empty, it's a user error (probably
// clicked "Add" and then lost track of it). Don't
// write it out.
if (!"".equals (minfo.clas)) {
_confOut.println (" <module>");
_confOut.println (" <class>" + encodeContent (minfo.clas) +
"</class>");
if (minfo.init != null && minfo.init.length () > 0) {
_confOut.println (" <init>" + encodeContent (minfo.init) +
"</init>");
}
/** tuple[2] and beyond are parameters */
if (minfo.params != null) {
for (int i = 0; i < minfo.params.length; i++) {
_confOut.println (" <param>" + encodeContent(minfo.params[i]) +
"</param>");
}
}
_confOut.println (" </module>");
}
}
// Write out the handlers
ListIterator<String[]> hiter = handlers.listIterator ();
while (hiter.hasNext ()) {
String handler = hiter.next ()[0];
if (handler.length() > 0) { // Don't write out blank handler names
_confOut.println (" <outputHandler>");
_confOut.println (" <class>" + encodeContent (handler) +
"</class>");
_confOut.println (" </outputHandler>");
}
}
writeTail ();
_confOut.close ();
// Replace the old file with the new.
if (_confFile.exists () && !_confFile.delete ()) {
if (_parent != null) {
JOptionPane.showMessageDialog(_parent,
"Can't replace old config file",
"Error",
JOptionPane.ERROR_MESSAGE);
}
_tempFile.delete ();
}
else {
_tempFile.renameTo (_confFile);
}
}
/* Write the fixed lines which begin the config file */
private void writeHead () throws IOException
{
_confOut.println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
_confOut.println("<jhoveConfig version=\"1.0\"");
_confOut.println(" xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"");
_confOut.println(" xmlns=\"http://hul.harvard.edu/ois/xml/ns/jhove/jhoveConfig\"");
// Note that the open quote of this line isn't closed till the next line; that's intentional.
_confOut.println("xsi:schemaLocation=\"http://hul.harvard.edu/ois/xml/ns/jhove/jhoveConfig");
_confOut.println(" http://hul.harvard.edu/ois/xml/xsd/jhove/jhoveConfig.xsd\">");
}
/* Write out the fixed end of the config file */
private void writeTail () throws IOException
{
_confOut.println("</jhoveConfig>");
}
/**
* Encodes a content String in XML-clean form, converting characters
* to entities as necessary. The null string will be
* converted to an empty string.
*/
private static String encodeContent (String content)
{
if (content == null) {
content = "";
}
StringBuffer buffer = new StringBuffer (content);
int n = 0;
while ((n = buffer.indexOf ("&", n)) > -1) {
buffer.insert (n+1, "amp;");
n +=5;
}
n = 0;
while ((n = buffer.indexOf ("<", n)) > -1) {
buffer.replace (n, n+1, "&lt;");
n += 4;
}
n = 0;
while ((n = buffer.indexOf (">", n)) > -1) {
buffer.replace (n, n+1, "&gt;");
n += 4;
}
return buffer.toString ();
}
}
@@ -1,114 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
/**
* A FilterInputStream which passes only a specified
* number of bytes, then returns an EOF condition.
*
* @author Gary McGath
*
*/
public class CountedInputStream extends FilterInputStream {
private int bytesLeft;
/**
* @param instrm The InputStream being counted
* @param count The number of bytes to be allowed
*/
public CountedInputStream(InputStream instrm, int count) {
super(instrm);
bytesLeft = count;
}
/** Reads a single byte from the stream and decrements
* the count of remaining bytes. If the count
* is exhausted, returns -1 to signify end of file. */
public int read() throws IOException {
if (bytesLeft <= 0) {
return -1;
}
else {
int ch = super.read ();
if (ch != -1) {
--bytesLeft;
}
return ch;
}
}
/**
* Reads some number of bytes from the input stream and
* stores them into the buffer array b. The number of
* bytes actually read is returned as an integer.
*
* The number of bytes read will not exceed the number
* of bytes remaining in the count. The count is
* decremented by the number of bytes actually read.
*/
public int read(byte[] b) throws IOException
{
int len = b.length;
int bytesRead;
if (len <= bytesLeft) {
// Limit doesn't affect us, do normal read
bytesRead = super.read (b);
}
else {
// Limit the read to bytesLeft
bytesRead = super.read (b, 0, bytesLeft);
}
bytesLeft -= bytesRead;
return bytesRead;
}
/**
* Reads up to len bytes of data from the input stream
* into an array of bytes. An attempt is made to read as
* many as len bytes, but a smaller number may be read,
* possibly zero. The number of bytes actually read is
* returned as an integer.
*
* The number of bytes read will not exceed the number
* of bytes remaining in the count. The count is
* decremented by the number of bytes actually read.
*/
public int read(byte[] b, int off, int len) throws IOException
{
int bytesRead;
if (len <= bytesLeft) {
// Limit doesn't affect us, do normal read
bytesRead = super.read (b, off, len);
}
else {
bytesRead = super.read (b, off, bytesLeft);
}
bytesLeft -= bytesRead;
return bytesRead;
}
/**
* Skips n bytes.
* Decrements the count by the number of bytes
* actually skipped.
*/
public long skip (long n) throws IOException {
long bytesRead = super.skip (n);
if (bytesLeft < bytesRead) {
bytesLeft = 0;
}
else {
bytesLeft -= bytesRead;
}
return bytesRead;
}
}
@@ -1,128 +0,0 @@
package edu.harvard.hul.ois.jhove;
import java.io.File;
import java.io.IOException;
import java.lang.reflect.Field;
import java.util.ArrayList;
import java.util.List;
import edu.harvard.hul.ois.jhove.module.*;
/** This class creates a default configuration if no valid configuration file
* is found. */
public class DefaultConfigurationBuilder {
private final static String FILE_SEP = System.getProperty ("file.separator");
private final static String HOME_DIR = System.getProperty ("user.home");
private final static String JHOVE_DIR = HOME_DIR + FILE_SEP + "jhove";
private final static String TEMP_DIR = System.getProperty("java.io.tmpdir");
private final static String DEFAULT_ENCODING = "utf-8";
private final static int DEFAULT_BUFFER_SIZE = 131072;
private File configFile;
/** Constructor. A location for the file may be specified or
* left null, */
public DefaultConfigurationBuilder (File location) {
if (location != null) {
configFile = location;
}
else {
configFile = new File (JHOVE_DIR +
FILE_SEP + "conf" +
FILE_SEP + "jhove.conf");
}
}
public void writeDefaultConfigFile () throws IOException {
// if (TEMP_DEBUG) {
// String configFileName = "null";
// if (configFile != null)
// configFileName = configFile.getAbsolutePath();
// System.out.println ("writeDefaultConfigFile: path is " + configFileName);
// }
ConfigWriter cw = new ConfigWriter (configFile, null);
List<ModuleInfo> modules = getModules();
// TextHandler, XmlHandler, and AuditHandler are loaded by
// default, so there are no handlers to put in the config file.
List<String[]> handlers = new ArrayList<String[]> ();
File homeDir = new File (JHOVE_DIR);
File tempDir = new File (TEMP_DIR);
try {
cw.writeFile(modules, handlers, homeDir, tempDir,
DEFAULT_ENCODING, DEFAULT_BUFFER_SIZE);
}
catch (IOException e) {
// if (TEMP_DEBUG)
// e.printStackTrace();
throw e;
}
}
// public void writeDefaultConfigFile () throws IOException {
// ConfigWriter cw = new ConfigWriter (configFile, null);
// List<ConfigWriter.ModuleInfo> modules = getModules();
// // TextHandler, XmlHandler, and AuditHandler are loaded by
// // default, so there are no handlers to put in the config file.
// List<String> handlers = new ArrayList<String> ();
// File homeDir = new File (JHOVE_DIR);
// File tempDir = new File (TEMP_DIR);
// cw.writeFile(modules, handlers, homeDir, tempDir,
// DEFAULT_ENCODING, DEFAULT_BUFFER_SIZE);
// }
public File getConfigFile () {
return configFile;
}
protected List<ModuleInfo> getModules () {
int nModules = builtInModules.length;
ArrayList<ModuleInfo> mods = new ArrayList<ModuleInfo> (nModules);
try {
for (int i = 0; i < nModules; i++) {
Class<?> cls = builtInModules[i];
ModuleInfo minfo = new ModuleInfo(cls.getName());
minfo.init = null; // Never used at present
minfo.params = getDefaultConfigParameters (cls);
mods.add(minfo);
}
}
catch (Exception e) {}
return mods;
}
/** We can't have a static method in an Interface and override it, so we
* have to get a bit ugly to fake static inheritance. The advantage of
* this is that only the Modules with non-null default config file
* parameters have to implement the defaultConfigParams static field. */
protected String[] getDefaultConfigParameters (Class<?> c) {
try {
Field dcpField = c.getField("defaultConfigParams");
return (String[]) dcpField.get(null);
}
catch (Exception e) {
return new String [] {};
}
}
/** The array of build-in modules. If a module is added to or removed from
* the build, this array must be changed. The Bytestream module is
* loaded by default and so is not listed here. */
private Class<?>[] builtInModules = {
AiffModule.class,
AsciiModule.class,
GifModule.class,
HtmlModule.class,
Jpeg2000Module.class,
JpegModule.class,
PdfModule.class,
TiffModule.class,
Utf8Module.class,
WaveModule.class,
XmlModule.class
};
}
@@ -1,194 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.util.*;
/**
* This class encapsulates information about format specification documents.
*
* @see DocumentType
*/
public class Document
{
private List _author;
private String _date;
private String _edition;
private String _enum;
private List _identifier;
private String _note;
private String _pages;
private List _publisher;
private String _title;
private DocumentType _type;
/**
* Creates a Document with a given title and one of the predefined
* DocumentTypes.
*/
public Document (String title, DocumentType type)
{
_title = title;
_type = type;
_author = new ArrayList ();
_identifier = new ArrayList ();
_publisher = new ArrayList ();
}
/**
* Returns a List of Agents, each representing an author of this
* Document.
* Returns an empty list if no authors have been listed.
*
* @see Agent
*/
public List getAuthor ()
{
return _author;
}
/**
* Returns the date of this Document
*/
public String getDate ()
{
return _date;
}
/**
* Returns informaton on the edition of this Document
*/
public String getEdition ()
{
return _edition;
}
/**
* Returns the enumeration (e.g., serial volume and number)
* of this Document
*/
public String getEnumeration ()
{
return _enum;
}
/**
* Returns the list of formal Identifiers for this Document.
* If no Identifiers are given, returns an empty list.
*/
public List getIdentifier ()
{
return _identifier;
}
/**
* Returns the note associated with this Document
*/
public String getNote ()
{
return _note;
}
/**
* Returns pagination information for this Document
*/
public String getPages ()
{
return _pages;
}
/**
* Returns a List of Agents, each representing a publisher of this
* Document. If no publishers are listed, returns an empty list.
*/
public List getPublisher ()
{
return _publisher;
}
/**
* Returns the title of this Document
*/
public String getTitle ()
{
return _title;
}
/**
* Returns one of the predefined DocumentTypes as the type of
* this Document
*/
public DocumentType getType ()
{
return _type;
}
/**
* Adds an author to the list of authors
*/
public void setAuthor (Agent author)
{
_author.add (author);
}
/**
* Sets the date of this Document
*/
public void setDate (String date)
{
_date = date;
}
/**
* Sets edition information for this Document
*/
public void setEdition (String edition)
{
_edition = edition;
}
/**
* Sets enumeration information (e.g., serial volume and number)
* for this Document
*/
public void setEnumeration (String enm)
{
_enum = enm;
}
/**
* Adds an Identifier to the list of identifiers
*/
public void setIdentifier (Identifier identifier)
{
_identifier.add (identifier);
}
/**
* Sets a note giving additional information about this Document
*/
public void setNote (String note)
{
_note = note;
}
/**
* Sets pagination information for this Document
*/
public void setPages (String pages)
{
_pages = pages;
}
/**
* Adds a publisher to the list of publishers
*/
public void setPublisher (Agent publisher)
{
_publisher.add (publisher);
}
}
@@ -1,61 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for a Document.
* Applications will not create or modify DocumentTypes, but will
* use one of the predefined DocumentType instances
* ARTICLE, BOOK, REPORT, RFC, STANDARD, WEB, or OTHER.
*
* @see Document
*
*/
public final class DocumentType
extends EnumerationType
{
/**
* Document type for a printed article.
*/
public static final DocumentType ARTICLE = new DocumentType ("Article");
/**
* Document type for an book.
*/
public static final DocumentType BOOK = new DocumentType ("Book");
/**
* Document type for a report.
*/
public static final DocumentType REPORT = new DocumentType ("Report");
/**
* Document type for an IETF Request for Comment.
*/
public static final DocumentType RFC = new DocumentType ("RFC");
/**
* Document type for a standards body publication.
*/
public static final DocumentType STANDARD = new DocumentType ("Standard");
/**
* Document type for a Web page.
*/
public static final DocumentType WEB = new DocumentType ("Web");
/**
* Document type that doesn't fit the other categories.
*/
public static final DocumentType OTHER = new DocumentType ("Other");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create DocumentTypes directly.
**/
private DocumentType (String value)
{
super (value);
}
}
@@ -1,134 +0,0 @@
/**********************************************************************
* JHOVE - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
/**
* Common methods for dump utilities.
*/
public class Dump
{
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
public Dump ()
{
}
/******************************************************************
* PUBLIC CLASS METHODS.
******************************************************************/
/**
* Return leading characters to pad out the byte offset to field width.
* @param os Byte offset
* @param width Field width
* @return String of leading zeros
*/
protected static String leading (int os, int width)
{
return leading (os, width, '0');
}
/**
* Return leading characters to pad out the byte offset to field width.
* @param os Byte offset
* @param width Field width
* @return String of leading zeros
*/
protected static String leading (long os, int width)
{
return leading (os, width, '0');
}
/**
* Return leading characters to pad out the byte offset to field width.
* @param os Byte offset
* @param width Field width
* @param pad Padding character
* @return String of leading characters
*/
protected static String leading (int os, int width, char pad)
{
return leading ((long) os, width, pad);
}
/**
* Return leading characters to pad out the byte offset to field width.
* @param os Byte offset
* @param width Field width
* @param pad Padding character
* @return String of leading characters
*/
protected static String leading (long os, int width, char pad)
{
String ss = Long.toString (os);
StringBuffer buffer = new StringBuffer ();
for (int j=0; j<width-ss.length (); j++) {
buffer.append (pad);
}
return buffer.toString ();
}
/**
* Return leading characters to pad out the string to field width.
* @param s String
* @param width Field width
* @return String of leading characters
*/
protected static String leading (String s, int width)
{
return leading (s, width, '0');
}
/**
* Return leading characters to pad out the string to field width.
* @param s String
* @param width Field width
* @param pad Padding character
* @return String of leading characters
*/
protected static String leading (String s, int width, char pad)
{
StringBuffer buffer = new StringBuffer ();
for (int j=0; j<width-s.length (); j++) {
buffer.append (pad);
}
return buffer.toString ();
}
/**
* Read and display a sequence of characters.
* @param stream Data input stream
* @param length Number of characters
* @return Character string
*/
protected static String readChars (DataInputStream stream, int length)
throws IOException
{
byte [] buffer = new byte [length];
for (int i=0; i<length; i++) {
buffer[i] = stream.readByte ();
}
return new String (buffer);
}
}
@@ -1,70 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* Enumerated data type. To create an emumeration, define a subclass
* of EnumerationType with a private constructor, and define as
* many <code>public final static</code> instances of the subclass,
* within the subclass definition, as there are enumeration cases.
* No other instances of an EnumerationType should ever be created,
* and the only operations which should ever be performed on an
* EnumerationType are assignment and equality testing.
*
*/
public abstract class EnumerationType
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Enumeration value. */
private String _value;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate an <tt>EnumerationType</tt> object.
* @param value Enumeration value
*/
protected EnumerationType (String value)
{
_value = value;
}
/**
* Private no-argument constructor to close off the default
* constructor.
*/
private EnumerationType ()
{
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Type equality test.
* @param enm Enumerated type
* @return True, if equal
*/
public boolean equals (EnumerationType enm)
{
return this == enm;
}
/**
* Return enumeration value.
* @return Value
*/
public String toString ()
{
return _value;
}
}
@@ -1,58 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates an error message from a Module, representing
* a problem in the content being analyzed.
*/
public class ErrorMessage
extends Message
{
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Create an ErrorMessage.
* @param message Human-readable string describing the problem.
*/
public ErrorMessage (String message)
{
super (message);
}
/**
* Create an ErrorMessage.
* @param message Human-readable string describing the problem.
* @param offset The offset in the file at which the problem
* was detected.
*/
public ErrorMessage (String message, long offset)
{
super (message, offset);
}
/**
* Create an ErrorMessage.
* @param message Human-readable string describing the problem.
*/
public ErrorMessage (String message, String subMessage)
{
super (message, subMessage);
}
/**
* Create an ErrorMessage.
* @param message Human-readable string describing the problem.
* @param subMessage Human-readable additional information.
* @param offset The offset in the file at which the problem
* was detected.
*/
public ErrorMessage (String message, String subMessage, long offset)
{
super (message, subMessage, offset);
}
}
@@ -1,60 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-4 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates information about external format signatures.
* The value of a Signature may be either a String or a byte array
* (stored as an int array to avoid signed byte problems).
*/
public class ExternalSignature
extends Signature
{
/**
* Creates an ExternalSignature given a string value, a type,
* and a use requirement.
*/
public ExternalSignature (String value, SignatureType type,
SignatureUseType use)
{
super (value, type, use);
}
/**
* Creates an ExternalSignature given a byte array, a type,
* and a use requirement.
*/
public ExternalSignature (int[] value, SignatureType type,
SignatureUseType use)
{
super (value, type, use);
}
/**
* Creates an ExternalSignature given a string value, a type,
* a use requirement, and a note.
*/
public ExternalSignature (String value, SignatureType type,
SignatureUseType use,
String note)
{
super (value, type, use, note);
}
/**
* Creates an ExternalSignature given a byte array, a type,
* a use requirement, and a note.
*/
public ExternalSignature (int[] value, SignatureType type,
SignatureUseType use,
String note)
{
super (value, type, use, note);
}
}
@@ -1,812 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.text.*;
import java.util.*;
import java.util.logging.*;
/**
* Abstract base class for Jhove output handlers.
* Output handlers should normally subclass HandlerBase.
*/
public abstract class HandlerBase
implements OutputHandler
{
/******************************************************************
* PUBLIC CLASS FIELDS.
******************************************************************/
/**
* A DateFormat for representing a Date in yyyy-MM-dd
* (e.g., 2003-07-31) format.
*/
public static SynchronizedDateFormat date =
new SynchronizedDateFormat ("yyyy-MM-dd");
/**
* A DateFormat for representing a Date in yyyy-MM-dd HH:mm:ss z
* (e.g., 2003-07-31 15:31:12 EDT) format.
*/
public static SynchronizedDateFormat dateTime =
new SynchronizedDateFormat ("yyyy-MM-dd HH:mm:ss z");
/**
* A DateFormat for representing a Date in ISO 8601
* (e.g., 2003-07-31T15:31:12-0400) format.
* We subclass SimpleDateFormat to make it thread-safe.
*/
public static SynchronizedDateFormat iso8601 =
new SynchronizedDateFormat ("yyyy-MM-dd'T'HH:mm:ssZ");
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** The application object */
protected App _app;
/** The Jhove engine */
protected JhoveBase _base;
/** Handler last modification date */
protected Date _date;
/** Character encoding for writer */
protected String _encoding;
/** Initialization value. */
protected String _init;
/** List of default parameters. */
protected List<String> _defaultParams;
/** JHOVE engine. */
protected JhoveBase _je;
/** Indentation level */
protected int _level;
/** Handler name */
protected String _name;
/** Handler note */
protected String _note;
/** Handler release description. */
protected String _release;
/** Handler-specific parameter. */
protected String _param;
/** Copyright notice */
protected String _rights;
/** Handler specification document list */
protected List<Document> _specification;
/** Handler vendor */
protected Agent _vendor;
/** Writer for doing output */
protected PrintWriter _writer;
/** Logger for a handler class. */
protected Logger _logger;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Constructors of all subclasses of HandlerBase should call
* this as a <code>super</code> constructor.
*
* @param name Name of the handler
* @param release Release identifier
* @param date Last modification date of the handler code,
* in the form of an array of three numbers.
* <code>date[0]</code> is the year,
* <code>date[1]</code> the month, and
* <code>date[2]</code> the day.
* @param note Additional information about the handler
* (may be null)
* @param rights Copyright notice for the handler
*/
protected HandlerBase (String name, String release, int [] date,
String note, String rights)
{
// Though we're actually in the jhove package, all the related
// action logically belongs in the handler package, so we name
// this logger accordingly.
_logger = Logger.getLogger ("edu.harvard.hul.ois.jhove.handler");
_logger.info ("Initializing " + name);
_name = name;
_release = release;
_encoding = "UTF-8";
Calendar calendar = new GregorianCalendar ();
calendar.set (date[0], date[1]-1, date[2]);
_date = calendar.getTime ();
_note = note;
_rights = rights;
_specification = new ArrayList<Document> ();
_level = -1;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Initialization methods.
******************************************************************/
/**
* Reset the handler. This needs to be called before each invocation.
*/
public void reset () {
_level = -1;
}
/**
* Set a a List of default parameters for the module.
*
* @param params A List whose elements are Strings.
* May be empty.
*/
public void setDefaultParams (List<String> params)
{
_defaultParams = params;
}
/**
* Applies the default parameters.
* Calling this clears any prior parameters.
*/
public void applyDefaultParams ()
throws Exception
{
resetParams ();
Iterator<String> iter = _defaultParams.iterator ();
while (iter.hasNext ()) {
String parm = iter.next ();
param (parm);
}
}
/** Reset parameter settings.
* Returns to a default state without any parameters.
* The default method clears the saved parameter.
*/
public void resetParams ()
throws Exception
{
_param = null;
}
/**
* Per-instantiation initialization.
* The default method does nothing.
*/
public void init (String init)
throws Exception
{
_init = init;
}
/**
* Per-action initialization.
* The default method does nothing.
*/
public void param (String param)
throws Exception
{
_param = param;
}
/******************************************************************
* Accessor methods.
******************************************************************/
/**
* Return the last modification date of this OutputHandler, as a
* Java Date object
*/
public final Date getDate ()
{
return _date;
}
/**
* Return the OutputHandler name
*/
public final String getName ()
{
return _name;
}
/**
* Return the OutputHandler note
*/
public final String getNote ()
{
return _note;
}
/**
* Return the release identifier
*/
public final String getRelease ()
{
return _release;
}
/**
* Return the copyright information string
*/
public final String getRights ()
{
return _rights;
}
/**
* Returns a list of <code>Document</code> objects (one for each
* specification document). The specification
* list is generated by the OutputHandler, and specifications cannot
* be added by callers.
*
* @see Document
*/
public final List<Document> getSpecification ()
{
return _specification;
}
/**
* Return the vendor information
*/
public final Agent getVendor ()
{
return _vendor;
}
/**
* Returns this handler's encoding.
*/
public String getEncoding ()
{
return _encoding;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/**
* Pass the associated App object to this Module.
* The App makes various services available.
*/
public final void setApp (App app)
{
_app = app;
}
/**
* Assigns the JHOVE engine object to provide services to this handler
*/
public final void setBase (JhoveBase je)
{
_je = je;
}
/**
* Assigns the encoding to be used by this OutputHandler
*/
public void setEncoding (String encoding)
{
_encoding = encoding;
}
/**
* Assigns a PrintWriter to do output for this OutputHandler
*/
public final void setWriter (PrintWriter writer)
{
_writer = writer;
}
/******************************************************************
* Serialization methods.
******************************************************************/
/**
* Callback allowing post-parse, pre-show analysis of object
* representation information.
* @param info Object representation information
*/
public void analyze (RepInfo info)
{
/* Do nothing, which is sufficient for most handlers. */
}
/**
* Callback indicating a directory is finished being processed.
*/
public void endDirectory ()
{
/* Do nothing, which is sufficient for most handlers. */
}
/**
* Callback to give the handler the opportunity to decide whether or
* not to process a file. Most handlers will always return true.
* @param filepath File pathname
*/
public boolean okToProcess (String filepath)
{
return true;
}
/**
* Outputs information about a Module
*/
public abstract void show (Module module);
/**
* Outputs the information contained in a RepInfo object
*/
public abstract void show (RepInfo info);
/**
* Outputs information about the OutputHandler specified
* in the parameter
*/
public abstract void show (OutputHandler handler);
/**
* Outputs minimal information about the application
*/
public abstract void show ();
/**
* Outputs detailed information about the application,
* including configuration, available modules and handlers,
* etc.
*/
public abstract void show (App app);
/**
* Do the initial output. This should be in a suitable format
* for including multiple files between the header and the footer.
*/
public abstract void showHeader ();
/**
* Do the final output. This should be in a suitable format
* for including multiple files between the header and the footer.
*/
public abstract void showFooter ();
/**
* Close the writer after all output has been done.
*/
public void close ()
{
_writer.close ();
}
/**
* Callback indicating a new directory is being processed.
* @param directory Directory path
*/
public void startDirectory (String directory)
{
/* Do nothing, which is sufficient for most handlers. */
}
/******************************************************************
* PRIVATE CLASS METHODS.
*
* XML methods.
******************************************************************/
/**
* Return the XML DOCTYPE instruction.
* @param root Root element of the DTD
* @param uri URI of the DTD
*/
protected static String doctype (String root, String uri)
{
return doctype (root, null, uri);
}
/**
* Return the XML DOCTYPE instruction.
* @param root Root element of the DTD
* @param name Public name of the DTD
* @param uri URI of the DTD
*/
protected static String doctype (String root, String name, String uri)
{
StringBuffer s = new StringBuffer ("<!DOCTYPE " + root);
if (name != null) {
s.append (" PUBLIC \"" + name + "\" \"");
}
else {
s.append (" SYSTEM \"");
}
s.append (uri + "\">");
return s.toString ();
}
/**
* Returns, as a String, an empty XML.
*
* @param tag XML tag
*/
protected static String element (String tag)
{
return "<" + tag + "/>";
}
/**
* Returns, as a String, an XML element with a given tag and content
*
* @param tag An XML tag
* @param content Content string. Characters requiring
* conversion to entitites will be converted.
*/
protected static String element (String tag, String content)
{
return elementStart (tag) + encodeContent (content) + elementEnd (tag);
}
/**
* Returns, as a String,
* an XML element with a given tag and attributes
*
* @param tag An XML tag
* @param attrs An array of String[2] elements, where for each
* element, attrs[i][0] is the attribute key and
* attrs[i][1] is the attribute value.
* Null values are skipped.
*/
protected static String element (String tag, String [][] attrs)
{
StringBuffer buffer = new StringBuffer ("<");
buffer.append (tag);
for (int i=0; i<attrs.length; i++) {
if (attrs[i][0] != null && attrs[i][1] != null) {
buffer.append (" ");
buffer.append (attrs[i][0]);
buffer.append ("=\"");
buffer.append (encodeValue (attrs[i][1]));
buffer.append ("\"");
}
}
buffer.append ("/>");
return buffer.toString ();
}
/**
* Returns, as a String,
* an XML element with a given tag, content and attributes
*
* @param tag An XML tag
* @param content Content string. Characters requiring
* conversion to entitites will be converted.
* @param attrs An array of String[2] elements, where for each
* element, attrs[i][0] is the attribute key and
* attrs[i][1] is the attribute value.
* Null values are skipped.
*
*/
protected static String element (String tag, String [][] attrs,
String content)
{
StringBuffer buffer = new StringBuffer ("<");
buffer.append (tag);
for (int i=0; i<attrs.length; i++) {
if (attrs[i][0] != null && attrs[i][1] != null) {
buffer.append (" ");
buffer.append (attrs[i][0]);
buffer.append ("=\"");
buffer.append (encodeValue (attrs[i][1]));
buffer.append ("\"");
}
}
buffer.append (">");
buffer.append (encodeContent (content));
buffer.append (elementEnd (tag));
return buffer.toString ();
}
/**
* Returns, as a String, the closing tag of an element.
* No checking is done that opening and closing tags match.
*
* @param tag An XML tag
*/
protected static String elementEnd (String tag)
{
return "</" + tag + ">";
}
/**
* Returns, as a String, the opening tag of an element.
*
* @param tag An XML tag
*/
protected static String elementStart (String tag)
{
return "<" + tag + ">";
}
/**
* Returns, as a String, the opening tag of an element with
* specified attributes.
*
* @param tag An XML tag
* @param attrs An array of String[2] elements, where for each
* element, attrs[i][0] is the attribute key and
* attrs[i][1] is the attribute value.
*/
protected static String elementStart (String tag, String [][] attrs)
{
StringBuffer buffer = new StringBuffer ("<");
buffer.append (tag);
for (int i=0; i<attrs.length; i++) {
buffer.append (" ");
buffer.append (attrs[i][0]);
buffer.append ("=\"");
buffer.append (encodeValue (attrs[i][1]));
buffer.append ("\"");
}
buffer.append (">");
return buffer.toString ();
}
/**
* Encodes a content String in XML-clean form, converting characters
* to entities as necessary and removing control characters disallowed
* by XML. The null string will be converted to an empty string.
*/
private static String encodeContent (String content)
{
if (content == null) {
content = "";
}
StringBuffer buffer = new StringBuffer (content);
/* Remove disallowed control characters from the content string. */
int n = buffer.length ();
for (int i=0; i<n; i++) {
char ch = buffer.charAt (i);
if ((0x00 <= ch && ch <= 0x08) || (0x0b <= ch && ch <= 0x0c) ||
(0x0e <= ch && ch <= 0x1f) || 0x7f == ch) {
buffer.deleteCharAt (i--);
n--;
}
}
n = 0;
while ((n = buffer.indexOf ("&", n)) > -1) {
buffer.insert (n+1, "amp;");
n +=5;
}
n = 0;
while ((n = buffer.indexOf ("<", n)) > -1) {
buffer.replace (n, n+1, "&lt;");
n += 4;
}
n = 0;
while ((n = buffer.indexOf (">", n)) > -1) {
buffer.replace (n, n+1, "&gt;");
n += 4;
}
return buffer.toString ();
}
/**
* Encodes an attribute value String in XML-clean form,
* converting quote characters to entities and removing control
* characters disallowed by XML.
*/
private static String encodeValue (String value)
{
StringBuffer buffer = new StringBuffer (value);
/* Remove disallowed control characters from the value string. */
int n = buffer.length ();
for (int i=0; i<n; i++) {
char ch = buffer.charAt (i);
if ((0x00 <= ch && ch <= 0x08) || (0x0b <= ch && ch <= 0x0c) ||
(0x0e <= ch && ch <= 0x1f) || 0x7f == ch) {
buffer.deleteCharAt (i--);
n--;
}
}
// [CC], escape &, < and > characters which are disallowed in xml
n = 0;
while ((n = buffer.indexOf ("&", n)) > -1) {
buffer.insert (n+1, "amp;");
n +=5;
}
n = 0;
while ((n = buffer.indexOf ("<", n)) > -1) {
buffer.replace (n, n+1, "&lt;");
n += 4;
}
n = 0;
while ((n = buffer.indexOf (">", n)) > -1) {
buffer.replace (n, n+1, "&gt;");
n += 4;
}
n = 0;
while ((n = buffer.indexOf ("\"", n)) > -1) {
// [LP] fix for invalid escaping, "" quotes were accidentally left in place.
buffer.replace (n, n+1, "&quot;");
n +=7;
}
return buffer.toString ();
}
/**
* Return a canonical XML declaration with default encoding.
*/
protected static String xmlDecl ()
{
return "<?xml version=\"1.0\"?>";
}
/**
* Return a canonical XML declaration with specified encoding.
*/
protected static String xmlDecl (String encoding)
{
return "<?xml version=\"1.0\" encoding=\"" +
encoding +
"\"?>";
}
/******************************************************************
* Nesting level methods.
******************************************************************/
/**
* Returns a String containing a number of spaces equal
* to the current indent level.
*/
protected static String getIndent (int level)
{
StringBuffer s = new StringBuffer ();
for (int i=0; i<level; i++) {
s.append (" ");
}
return s.toString ();
}
/******************************************************************
* Array methods.
******************************************************************/
/**
* Return String representation of an integer array.
*/
protected static String integerArray (int [] iarray)
{
return integerArray (iarray, ' ');
}
/**
* Return String representation of an integer array with
* specified separator.
*/
protected static String integerArray (int [] iarray, char separator)
{
StringBuffer buffer = new StringBuffer ();
for (int i=0; i<iarray.length; i++) {
if (i > 0) {
buffer.append (separator);
}
buffer.append (Integer.toString (iarray[i]));
}
return buffer.toString ();
}
/**
* Return String representation of an array of long with
* space separator.
*/
protected static String longArray (long [] larray)
{
StringBuffer buffer = new StringBuffer ();
for (int i=0; i<larray.length; i++) {
if (i > 0) {
buffer.append (" ");
}
buffer.append (Long.toString (larray[i]));
}
return buffer.toString ();
}
/**
* Return String representation of an array of Rational, each evaluated
* as a double, with space separator.
*/
protected static String rationalArray (Rational [] rarray)
{
StringBuffer buffer = new StringBuffer ();
for (int i=0; i<rarray.length; i++) {
if (i > 0) {
buffer.append (" ");
}
buffer.append (rarray[i].toDouble ());
}
return buffer.toString ();
}
/**
* Return String representation of an array of Rational, each as
* two integers, with space separator.
*/
protected static String rationalArray10 (Rational [] rarray)
{
StringBuffer buffer = new StringBuffer ();
for (int i=0; i<rarray.length; i++) {
if (i > 0) {
buffer.append (" ");
}
buffer.append (rarray[i].getNumerator ());
buffer.append (" ");
buffer.append (rarray[i].getNumerator ());
}
return buffer.toString ();
}
/**
* Return String representation of an array of double.
*/
protected static String doubleArray (double [] darray)
{
StringBuffer buffer = new StringBuffer ();
for (int i=0; i<darray.length; i++) {
if (i > 0) {
buffer.append (" ");
}
buffer.append (Double.toString (darray[i]));
}
return buffer.toString ();
}
/* Text formatting methods. */
/* Convert a date to the dateTime format used by the
* XML schema. This is ISO 8610 with a colon between
* the hour and minute fields of the time zone. Unfortunately,
* SimpleDateFormat generates the time zone without the
* colon; this also conforms to 8601, but doesn't conform
* to the schema, so we have to diddle it.
*/
protected String toDateTime (Date date)
{
String isoStr = iso8601.format (date);
// We can't directly use a SimpleDateFormat, because
// the 'z' field gives us the colonless time zone.
int len = isoStr.length ();
// Add the colon before the last two characters.
return isoStr.substring (0, len - 2) +
":" +
isoStr.substring (len - 2);
}
/** A DateFormat class to address an issue of thread safety. */
public static class SynchronizedDateFormat extends SimpleDateFormat
{
public SynchronizedDateFormat(String pattern) {
super(pattern);
}
public synchronized StringBuffer format(Date date,
StringBuffer toAppendTo, FieldPosition pos) {
return super.format(date, toAppendTo, pos);
}
}
}
@@ -1,64 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates information about an identifier
* for a specification document.
*/
public class Identifier
{
private IdentifierType _type;
private String _value;
private String _note;
/**
* Create an Identifier.
* @param value The text displayed for this Identifier.
* @param type The type of identification.
*/
public Identifier (String value, IdentifierType type)
{
_value = value;
_type = type;
}
/**
* Create an Identifier.
* @param value The text displayed for this Identifier.
* @param type The type of identification.
* @param note A note giving supplementary information.
*/
public Identifier (String value, IdentifierType type, String note)
{
this (value, type);
_note = note;
}
/**
* Return the identifier type.
*/
public IdentifierType getType ()
{
return _type;
}
/**
* Return the displayable string.
*/
public String getValue ()
{
return _value;
}
/**
* Return the note, which will be null if none was specified.
*/
public String getNote()
{
return _note;
}
}
@@ -1,115 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for an Identifier of a
* format specification document.
* Applications will not create or modify IdentifierTypes, but will
* use one of the predefined IdentifierType instances
* ANSI, DDC, DOI, ECMA, HANDLE, ISO, ISBN, LC, LCCN,
* NISO, PII, RFC, SICI, URI, URL, URN, or OTHER.
*
* @see Identifier
*/
public final class IdentifierType
extends EnumerationType
{
/**
* Identifier type for American National Standards Institute.
*/
public static final IdentifierType ANSI = new IdentifierType ("ANSI");
/**
* Identifier type for Dewey Decimal Classification.
*/
public static final IdentifierType DDC = new IdentifierType ("DDC");
/**
* Identifier type for Digital Object Identifier.
*/
public static final IdentifierType DOI = new IdentifierType ("DOI");
/**
* Identifier type for ECMA.
*/
public static final IdentifierType ECMA = new IdentifierType ("ECMA");
/**
* Identifier type for CNRI Handle.
*/
public static final IdentifierType HANDLE = new IdentifierType ("Handle");
/**
* Identifier type for International Standards Organization.
*/
public static final IdentifierType ISO = new IdentifierType ("ISO");
/**
* Identifier type for International Standard Book Number.
*/
public static final IdentifierType ISBN = new IdentifierType ("ISBN");
/**
* Identifier type for Library of Congress classification.
*/
public static final IdentifierType LC = new IdentifierType ("LC");
/**
* Identifier type for Library of Congress catalogue number.
*/
public static final IdentifierType LCCN = new IdentifierType ("LCCN");
/**
* Identifier type for NISO standard number.
*/
public static final IdentifierType NISO = new IdentifierType ("NISO");
/**
* Identifier type for Publisher Item Identifier.
*/
public static final IdentifierType PII = new IdentifierType ("PII");
/**
* Identifier type for IETF Request for Comment.
*/
public static final IdentifierType RFC = new IdentifierType ("RFC");
/**
* Identifier type for Serial Item and Contribution Identifier.
*/
public static final IdentifierType SICI = new IdentifierType ("SICI");
/**
* Identifier type for Uniform Resource Identifier.
*/
public static final IdentifierType URI = new IdentifierType ("URI");
/**
* Identifier type for Uniform Resource Locator.
*/
public static final IdentifierType URL = new IdentifierType ("URL");
/**
* Identifier type for Uniform Resource Name.
*/
public static final IdentifierType URN = new IdentifierType ("URN");
/**
* Identifier type for CCITT.
*/
public static final IdentifierType CCITT = new IdentifierType ("CCITT");
/**
* Identifier type for International Telecommunication Union.
*/
public static final IdentifierType ITU = new IdentifierType ("ITU");
/**
* Identifier type for Japan Electronics and Information Technology
* Industries Association.
*/
public static final IdentifierType JEITA = new IdentifierType ("JEITA");
/**
* Identifier type for whatever doesn't fit other categories.
*/
public static final IdentifierType OTHER = new IdentifierType ("Other");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create SignatureTypes directly.
**/
private IdentifierType (String value)
{
super (value);
}
}
@@ -1,62 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates an informational message from a Module, giving
* information (not necessarily a problem)
* about the content being analyzed or the way that Jhove
* deals with it.
*/
public class InfoMessage
extends Message
{
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Create an InfoMessage.
* @param message Human-readable string giving the information.
*/
public InfoMessage (String message)
{
super (message);
}
/**
* Create an InfoMessage.
* @param message Human-readable string giving the information.
* @param offset The offset in the file relevant to the
* situation being described
*/
public InfoMessage (String message, long offset)
{
super (message, offset);
}
/**
* Create an InfoMessage.
* @param message Human-readable string giving the information.
* @param subMessage Human-readable additional information.
*/
public InfoMessage (String message, String subMessage)
{
super (message, subMessage);
}
/**
* Create an InfoMessage.
* @param message Human-readable string giving the information.
* @param subMessage Human-readable additional information.
* @param offset The offset in the file relevant to the
* situation being described
*/
public InfoMessage (String message, String subMessage, long offset)
{
super (message, subMessage, offset);
}
}
@@ -1,140 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates information about internal format signatures.
* The value of a Signature may be either a String or a byte array
* (stored as an int array to avoid signed byte problems).
*/
public class InternalSignature
extends Signature
{
private boolean _hasFixedOffset;
private int _offset;
/**
* Creates an InternalSignature given a string value, a type,
* and a use requirement. An InternalSignature created with
* this constructor does not have a fixed byte offset.
*/
public InternalSignature (String value, SignatureType type,
SignatureUseType use)
{
super (value, type, use);
_hasFixedOffset = false;
}
/**
* Creates an InternalSignature given a byte array, a type,
* and a use requirement. An InternalSignature created with
* this constructor does not have a fixed byte offset.
*/
public InternalSignature (int[] value, SignatureType type,
SignatureUseType use)
{
super (value, type, use);
_hasFixedOffset = false;
}
/**
* Creates an InternalSignature given a string value, a type,
* a use requirement, and a byte offset. An InternalSignature
* created with this constructor has a fixed byte offset.
*/
public InternalSignature (String value, SignatureType type,
SignatureUseType use, int offset)
{
super (value, type, use);
_offset = offset;
_hasFixedOffset = true;
}
/**
* Creates an InternalSignature given a byte array, a type,
* a use requirement, and a byte offset. An InternalSignature
* created with this constructor has a fixed byte offset.
*/
public InternalSignature (int[] value, SignatureType type,
SignatureUseType use, int offset)
{
super (value, type, use);
_offset = offset;
_hasFixedOffset = true;
}
/**
* Creates an InternalSignature given a string value, a type,
* a use requirement, and a note. An InternalSignature created with
* this constructor does not have a fixed byte offset.
*/
public InternalSignature (String value, SignatureType type,
SignatureUseType use, String note)
{
super (value, type, use, note);
_hasFixedOffset = false;
}
/**
* Creates an InternalSignature given a byte array, a type,
* a use requirement, and a note. An InternalSignature created with
* this constructor does not have a fixed byte offset.
*/
public InternalSignature (int[] value, SignatureType type,
SignatureUseType use, String note)
{
super (value, type, use, note);
_hasFixedOffset = false;
}
/**
* Creates an InternalSignature given a string value, a type,
* a use requirement, a byte offset, and a note.
* An InternalSignature created with
* this constructor has a fixed byte offset.
*/
public InternalSignature (String value, SignatureType type,
SignatureUseType use, int offset,
String note)
{
super (value, type, use, note);
_offset = offset;
_hasFixedOffset = true;
}
/**
* Creates an InternalSignature given a string value, a type,
* a use requirement, a byte offset, and a note.
* An InternalSignature created with
* this constructor has a fixed byte offset.
*/
public InternalSignature (int[] value, SignatureType type,
SignatureUseType use, int offset,
String note)
{
super (value, type, use, note);
_offset = offset;
_hasFixedOffset = true;
}
/**
* Returns the byte offset. This value is meaningful only
* if this InternalSignature has a fixed byte offset.
*/
public int getOffset ()
{
return _offset;
}
/**
* Returns <code>true</code> if this InternalSignature
* has a fixed byte offset.
*/
public boolean hasFixedOffset ()
{
return _hasFixedOffset;
}
}
File diff suppressed because it is too large Load Diff
@@ -1,19 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
public class JhoveException
extends Exception
{
public JhoveException ()
{
super ();
}
public JhoveException (String message)
{
super (message);
}
}
@@ -1,96 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.lang.reflect.*;
//import com.apple.eio.FileManager;
/**
* Code specific to Macintosh Java. This class consists of static
* methods, and should not be instantiated. Its methods should be
* called only on the Macintosh OS X platform. It requires the
* package com.apple.eio.FileManager.
*
* @author Gary McGath
*
*/
public class MacStuff {
/** Private constructor to prevent instantiation.*/
private MacStuff ()
{
}
/**
* Determines if we're running on a Macintosh,
* so appropriate UI adjustments can be made. In accordance
* with Apple's recommendations, this checks for the existence
* of the mrj.version property rather than checking the os.name
* property.
*/
public static boolean isMacintosh ()
{
return (System.getProperty ("mrj.version") != null);
}
/**
* Returns true if a file has the given file type.
* This method uses FileManager in a dynamic way, so
* that it will merely throw a ClassNotFound exception
* if it fails.
*
* Currently this code isn't actually used, since the
* Jhove application is specified as checking only internal
* signatures. Should some future version or add-on
* code wish to use it, the code should
* look something like this:
*
* <code><pre>
* try {
* if (sig.getType() == SignatureType.FILETYPE &&
* MacStuff.isMacintosh ()) {
* if (!MacStuff.fileHasType(file, sig.getValueString())) {
* info.setConsistent (false);
* }
* }
* }
* catch (ClassNotFoundException e) {
* // Mac classes missing -- can't check filetype.
* }
* </pre></code>
*/
public static boolean fileHasType(File file, String type)
throws ClassNotFoundException
{
// Need to make this completely dynamic.
//return type.equals (FileManager.getFileType (file));
try {
if (type == null) {
return false;
}
Class fmclass = Class.forName ("com.apple.eio.FileManager");
Class[] params = new Class[1];
params[0] = Class.forName ("java.io.File");
Object[] args = new Object[1];
args[0] = file;
Method method = fmclass.getMethod ("getFileType", params);
String ftype = (String) method.invoke (null, args);
return (type.equals (ftype));
}
catch (ClassNotFoundException e) {
throw e;
}
catch (Exception f) {
return false;
}
}
}
@@ -1,131 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates a String to be displayed.
*/
public abstract class Message
{
/******************************************************************
* PUBLIC CLASS FIELDS.
******************************************************************/
/** Value indicating a null offset. */
public static final long NULL = -1;
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Message text. */
protected String _message;
/** Additional information. */
protected String _subMessage;
/** Byte offset to which message applies. */
protected long _offset;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Create a Message. This constructor cannot be invoked directly,
* since Message is abstract.
* @param message Human-readable string.
*/
protected Message (String message)
{
init (message, null, NULL);
}
/**
* Create a Message. This constructor cannot be invoked directly,
* since Message is abstract. The second argument
* adds secondary details to the primary message;
* the message will typically be displayed in the
* form "message:subMessage".
* @param message Human-readable string.
* @param subMessage Human-readable additional information.
*/
protected Message (String message, String subMessage)
{
init (message, subMessage, NULL);
}
/**
* Create a Message. This constructor cannot be invoked directly,
* since Message is abstract. The second argument
* adds secondary details to the primary message;
* the message will typically be displayed in the
* form "message:subMessage".
* @param message Human-readable string.
* @param offset Byte offset associated with the message.
*/
protected Message (String message, long offset)
{
init (message, null, offset);
}
/**
* Create a Message. This constructor cannot be invoked directly,
* since Message is abstract. The second argument
* adds secondary details to the primary message;
* the message will typically be displayed in the
* form "message:subMessage".
* @param message Human-readable string.
* @param subMessage Human-readable additional information.
* @param offset Byte offset associated with the message.
*/
protected Message (String message, String subMessage, long offset)
{
init (message, subMessage, offset);
}
/**
* Initialize the <tt>Message</tt> object.
* @param message Human-readable string.
* @param subMessage Human-readable additional information.
*/
private void init (String message, String subMessage, long offset)
{
_message = message;
_subMessage = subMessage;
_offset = offset;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/**
* Get the message string.
*/
public String getMessage ()
{
return _message;
}
/**
* Get the submessage string.
*/
public String getSubMessage ()
{
return _subMessage;
}
/**
* Return the offset to which the information is related.
*/
public long getOffset ()
{
return _offset;
}
}
@@ -1,264 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-4 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.util.*;
/**
* Public interface for Jhove format-specific plug-in modules.
*
* All format modules must implement the Module interface;
* usually the best way to do this will be to subclass ModuleBase.
* Modules must be declared in the configuration file and present
* in the Classpath to be recognized by Jhove.
*
* @see ModuleBase
*/
public interface Module
{
public static final int
MAXIMUM_VERBOSITY = 1,
MINIMUM_VERBOSITY = 2;
/**
* Per-instantiation initialization.
*
* @param init Initialization parameter. This is typically obtained
* from the configuration file.
*/
public void init (String init)
throws Exception;
/**
* Sets list of default parameters.
*
* @param params A List whose elements are Strings.
* May be empty.
*/
public void setDefaultParams (List<String> params);
/**
* Applies the default parameters.
*/
public void applyDefaultParams ()
throws Exception;
/** Reset parameter settings.
* Returns to a default state without any parameters.
*/
public void resetParams ()
throws Exception;
/**
* Per-action initialization.
* May be called multiple times.
*
* @param param Initialization parameter.
*/
public void param (String param)
throws Exception;
/**
* Pass the associated App object to this Module.
* The App makes various services available.
*/
public void setApp (App app);
/**
* Pass the JHOVE engine object to this Module.
*/
public void setBase (JhoveBase je);
/**
* Set the degree of verbosity desired from the module.
*
* @param verbosity The requested verbosity value.
*/
public void setVerbosity (int verbosity);
/**
* Return the name of this Module.
*/
public String getName ();
/**
* Return the release identifier
*/
public String getRelease ();
/**
* Return the last modification date of the Module, as a
* Java Date object
*/
public Date getDate ();
/**
* Return the vendor information
*/
public Agent getVendor ();
/**
* Return the copyright information string
*/
public String getRights ();
/**
* Return the Module note
*/
public String getNote ();
/**
* Return the array of format names supported by this Module
*/
public String [] getFormat ();
/**
* Return the array of MIME type strings for formats supported
* by this Module
*/
public String [] getMimeType ();
/**
* Return details as to the specific format versions or
* variants that are supported by this Module
*/
public String getCoverage ();
/**
* Return the List of Signatures recognized by this Module
*/
public List<Signature> getSignature ();
/**
* Returns a list of <code>Document</code> objects (one for each
* specification document of the format). The specification
* list is generated by the Module, and specifications cannot
* be added by callers.
*
* @see Document
*/
public List<Document> getSpecification ();
/**
* Return the string describing well-formedness criteria
*/
public String getWellFormedNote ();
/**
* Returns the RepInfo note
*/
public String getRepInfoNote ();
/**
* Returns the string describing validity criteria
*/
public String getValidityNote ();
/**
* Returns the random access flag (true if the module operates
* on random access files, false if it operates on streams)
*/
public boolean isRandomAccess ();
/**
* Returns <code>true</code> if the module supports a given
* named feature, and <code>false</code> if the feature is
* unsupported or unknown.
*/
public boolean hasFeature (String feature);
/**
* Returns the full list of features.
*/
public List<String> getFeatures ();
/**
* Returns the list of default parameters.
*/
public List<String> getDefaultParams ();
/**
* Parse the content of a stream digital object and store the
* results in RepInfo.
* A given Module will normally implement only one of the two
* parse methods, leaving the other as a stub.
*
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed.
* If multiple calls to <code>parse</code> are made
* on the basis of a nonzero value being returned,
* a new InputStream must be provided each time.
*
* @param info A fresh (on the first call) RepInfo object
* which will be modified
* to reflect the results of the parsing
* If multiple calls to <code>parse</code> are made
* on the basis of a nonzero value being returned,
* the same RepInfo object should be passed with each
* call.
*
* @param parseIndex Must be 0 in first call to <code>parse</code>. If
* <code>parse</code> returns a nonzero value, it must be
* called again with <code>parseIndex</code>
* equal to that return value.
*/
public int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException;
/**
* Parse the content of a random access digital object and store the
* results in RepInfo.
* A given Module will normally implement only one of the two
* parse methods, leaving the other as a stub.
*
* @param file A RandomAccessFile, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the parsing
*/
public void parse (RandomAccessFile file, RepInfo info)
throws IOException;
/**
* Check if the digital object conforms to this Module's
* internal signature information.
*
* @param file A File object for the object being parsed
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the test
*/
public void checkSignatures (File file,
InputStream stream,
RepInfo info)
throws IOException;
/**
* Check if the digital object conforms to this Module's
* internal signature information.
*
* @param file A File object for the object being parsed
* @param raf A RandomAccessFile, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the test
*/
public void checkSignatures (File file,
RandomAccessFile raf,
RepInfo info)
throws IOException;
/**
* Generates information about this Module.
* The format of the output depends on the OutputHandler.
*/
public void show (OutputHandler handler);
}
File diff suppressed because it is too large Load Diff
@@ -1,17 +0,0 @@
package edu.harvard.hul.ois.jhove;
/** A small class to hold information about a module. */
public class ModuleInfo {
public String clas;
public String init;
public String[] params;
public ModuleInfo (String className) {
clas = className;
}
public ModuleInfo (String className, String init) {
clas = className;
this.init = init;
}
}
File diff suppressed because it is too large Load Diff
@@ -1,118 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-4 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.util.*;
/**
* Module for identification of a document. "Identification"
* means determining, by querying modules successively, what the format
* of a document is. The Bytestream module is always queried last,
* so a document will by identified as a Bytestream if all else fails.
*/
public class ObjectIdentifier
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
private List _moduleList;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
public ObjectIdentifier (List moduleList)
{
_moduleList = moduleList;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Processing methods.
*
******************************************************************/
/**
* Perform identification on a file. The file is parsed by
* each of the modules in the module list until one
* declares that the file is well-formed. It is assumed
* that there is a module in the list (normally Bytestream
* at the end) which will always consider a file well-formed.
*/
public void identify (File file, RepInfo info,
String parm, boolean verbose,
boolean shortCheck)
throws IOException
{
/******************************************************
* Go through all modules, in the order in the config
* file, calling the parse method till we find one
* which matches.
******************************************************/
ListIterator modIter = _moduleList.listIterator();
while (modIter.hasNext ()) {
/* We need clean RepInfo for each run */
RepInfo info1;
info1 = (RepInfo) info.clone ();
Module mod = (Module) modIter.next ();
try {
if (!mod.hasFeature("edu.harvard.hul.ois.jhove.canValidate")) {
continue;
}
if (mod.isRandomAccess ()) {
RandomAccessFile raf =
new RandomAccessFile (file, "r");
mod.param (parm);
if (verbose) {
mod.setVerbosity (Module.MAXIMUM_VERBOSITY);
}
if (shortCheck) {
mod.checkSignatures (file, raf, info1);
}
else {
mod.parse (raf, info1);
}
raf.close ();
}
else {
InputStream stream = new FileInputStream (file);
mod.param (parm);
if (shortCheck) {
mod.checkSignatures (file, stream, info1);
}
else {
int parseIndex = mod.parse (stream, info1, 0);
while (parseIndex != 0) {
stream.close ();
stream = new FileInputStream (file);
parseIndex = mod.parse (stream, info1, parseIndex);
}
}
stream.close ();
}
}
catch (Exception e) {
/* The assumption is that in trying to analyze
the wrong type of file, the module may go
off its track and throw an exception, so we
just continue on to the next module.
*/
continue;
}
if (info1.getWellFormed () == RepInfo.TRUE) {
info.copy (info1);
break;
}
}
}
}
@@ -1,196 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
import java.util.*;
/**
* Public interface for Jhove output handlers.
* All output handlers must implement OutputHandler, and in
* normal cases should subclass HandlerBase.
*/
public interface OutputHandler
{
/**
* Reset the handler. This needs to be called before each invocation.
*/
public void reset ();
/**
* Callback allowing post-parse, pre-show analysis of object
* representation information.
* @param info Object representation information
*/
public void analyze (RepInfo info);
/**
* Callback indicating a directory is finished being processed.
* Most handlers will do nothing.
*/
public void endDirectory ();
/**
* Returns the name of this handler
*/
public String getName ();
/**
* Returns release information for this handler
*/
public String getRelease ();
/**
* Returns the last modification date of this handler
*/
public Date getDate ();
/**
* Returns a List of Document objects giving the format
* specification documentation
*
* @see Document
*/
public List<Document> getSpecification ();
/**
* Returns a List of Agent objects giving the vendor(s)
* of this handler.
*/
public Agent getVendor ();
/**
* Returns this handler's note
*/
public String getNote ();
/**
* Returns this handler's copyright information
*/
public String getRights ();
/**
* Returns this handler's encoding.
*/
public String getEncoding ();
/**
* Per-instantiation initialization.
*
* @param init Initialization parameter. This is typically obtained
* from the configuration file.
*/
public void init (String init)
throws Exception;
/**
* Callback to give the handler the opportunity to decide whether or
* not to process a file. Most handlers will always return true.
* @param filepath File pathname
*/
public boolean okToProcess (String filepath);
/**
* Sets list of default parameters.
*
* @param params A List whose elements are Strings.
* May be empty.
*/
public void setDefaultParams (List<String> params);
/**
* Applies the default parameters.
*/
public void applyDefaultParams ()
throws Exception;
/** Reset parameter settings.
* Returns to a default state without any parameters.
*/
public void resetParams ()
throws Exception;
/**
* Per-action initialization.
*
* @param param Initialization parameter.
*/
public void param (String param)
throws Exception;
/**
* Assigns an application object to provide services to this handler
*/
public void setApp (App app);
/**
* Assigns the JHOVE engine object to provide services to this handler
*/
public void setBase (JhoveBase je);
/**
* Assigns the encoding to be used by this OutputHandler
*/
public void setEncoding (String encoding);
/**
* Assigns a PrintWriter to do output for this OutputHandler
*/
public void setWriter (PrintWriter output);
/**
* Outputs information about a Module
*/
public void show (Module module);
/**
* Outputs the information contained in a RepInfo object
*/
public void show (RepInfo info);
/**
* Outputs information about the OutputHandler specified
* in the parameter
*/
public void show (OutputHandler handler);
/**
* Outputs minimal information about the application
*/
public void show ();
/**
* Outputs detailed information about the application,
* including configuration, available modules and handlers,
* etc.
*/
public void show (App app);
/**
* Do the initial output. This should be in a suitable format
* for including multiple files between the header and the footer.
*/
public void showHeader ();
/**
* Do the final output. This should be in a suitable format
* for including multiple files between the header and the footer.
*/
public void showFooter ();
/**
* Do appropriate finalization after all output is complete.
*/
public void close ();
/**
* Callback indicating a new directory is being processed.
* Most handlers will do nothing.
* @param directory Directory path
*/
public void startDirectory (String directory);
}
@@ -1,214 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.util.*;
/**
* This class encapsulates arbitrary format-specific properties.
* A Property's value can be a simple object or a structure.
* If it is a simple object, it has arity SCALAR. If it is
* a structure, its must be a Map, a List, a Set, or an array,
* with corresponding Arity. The simple object (in the case of
* arity SCALAR) or the components of the structure must have a
* type corresponding to one of the enumerations given by
* <code>PropertyType</code>.
*
* The components of a Property may themselves be Property
* objects, allowing nested structures.
*
* @see PropertyType
* @see PropertyArity
*/
public class Property
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
private String _name;
private PropertyType _type;
private PropertyArity _arity;
private Object _value;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates a Property with arity SCALAR.
*
* @param name The displayable name of the property
* @param type The type of property
* @param value The value of the property. The type of the
* parameter must agree with <code>type</code>.
*/
public Property (String name, PropertyType type, Object value)
{
init (name, type, PropertyArity.SCALAR, value);
}
/**
* Creates a Property with a given name, type, arity and value.
*
* @param name The displayable name of the property.
* @param type The type of property.
* @param arity The arity of the property.
* @param value The value of the property. If the arity is
* SCALAR, the type of this parameter must agree
* with <code>type</code>. Otherwise, the arity
* must agree with <code>arity</code>, and its
* components must agree with <code>type</code>.
*/
public Property (String name, PropertyType type, PropertyArity arity,
Object value)
{
init (name, type, arity, value);
}
private void init (String name, PropertyType type, PropertyArity arity,
Object value)
{
/* Some limited type checking. Checking for mismatched
types here may help avoid difficult chasing down
of the bugs such mismatches cause. */
if (value == null) {
throw new NullPointerException ("Null value for Property not permitted");
}
if (arity == PropertyArity.SCALAR) {
if (value instanceof List ||
value instanceof Map ||
value instanceof Set) {
throw new IncompatibleClassChangeError
("Wrong class for Scalar Property");
}
}
else if (arity == PropertyArity.MAP) {
if (!(value instanceof Map)) {
throw new IncompatibleClassChangeError
("Wrong class for Map Property");
}
}
else if (arity == PropertyArity.SET) {
if (!(value instanceof Set)) {
throw new IncompatibleClassChangeError
("Wrong class for Set Property");
}
}
else if (arity == PropertyArity.LIST) {
if (!(value instanceof List)) {
throw new IncompatibleClassChangeError
("Wrong class for List Property");
}
}
_name = name;
_type = type;
_arity = arity;
_value = value;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/**
* Returns the arity (type of structure) of this Property.
*/
public PropertyArity getArity ()
{
return _arity;
}
/**
* Return a property by its name, regardless of its position in the
* structural hierarchy of properties.
* @param name Property name
* @return Named property (or null)
*/
public Property getByName (String name)
{
if (_name.equals (name)) {
return this;
}
if (!_arity.equals (PropertyArity.SCALAR) &&
_type.equals (PropertyType.PROPERTY)) {
if (_arity.equals (PropertyArity.ARRAY)) {
Property [] array = (Property []) _value;
for (int i=0; i<array.length; i++) {
Property prop = array[i].getByName (name);
if (prop != null) {
return prop;
}
}
}
else if (_arity.equals (PropertyArity.LIST)) {
List list = (List) _value;
int len = list.size ();
for (int i=0; i<len; i++) {
Property prop = ((Property) list.get (i)).getByName (name);
if (prop != null) {
return prop;
}
}
}
else if (_arity.equals (PropertyArity.MAP)) {
Collection coll = ((Map) _value).values ();
Iterator iter = coll.iterator ();
while (iter.hasNext ()) {
Property prop = ((Property) iter.next ()).getByName (name);
if (prop != null) {
return prop;
}
}
}
else if (_arity.equals (PropertyArity.SET)) {
Iterator iter = ((Set) _value).iterator ();
while (iter.hasNext ()) {
Property prop = ((Property) iter.next ()).getByName (name);
if (prop != null) {
return prop;
}
}
}
}
return null;
}
/**
* Returns the displayable name of this Property.
*/
public String getName ()
{
return _name;
}
/**
* Returns the type of this Property.
* If the arity is other than SCALAR, the type refers to the
* compononents of the Property structure.
*/
public PropertyType getType ()
{
return _type;
}
/**
* Returns the Object which is the Property's value.
*/
public Object getValue ()
{
return _value;
}
}
@@ -1,60 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class represents the arity (structure type) of a Property.
* Applications will not create or modify PropertyArities, but will
* use one of the predefined PropertyArity instances
* ARRAY, LIST, MAP, SCALAR, or SET.
*
* @see Property
*/
public final class PropertyArity
extends EnumerationType
{
/**
* An Arity corresponding to a Java array. For the Java types
* <code>Integer</code>, <code>Boolean</code>, <code>Byte</code>,
* <code>Character</code>, <code>Double</code>,
* <code>Float</code>, <code>Long</code>, and
* <code>Short</code>, a Property with arity ARRAY is an array of primitive
* Java types rather than Objects (e.g., <code>int</code>
* rather than <code>Integer</code>).
*/
public static final PropertyArity ARRAY = new PropertyArity ("Array");
/**
* An Arity corresponding to java.util.List or any of its derived classes.
*/
public static final PropertyArity LIST = new PropertyArity ("List");
/**
* An Arity corresponding to java.util.Map or any of its derived classes.
*/
public static final PropertyArity MAP = new PropertyArity ("Map");
/**
* An Arity corresponding to a simple object, which must be of a
* type corresponding to one of the instances of
* <code>PropertyType</code>.
*/
public static final PropertyArity SCALAR = new PropertyArity ("Scalar");
/**
* An Arity corresponding to java.util.Set or any of its derived classes.
*/
public static final PropertyArity SET = new PropertyArity ("Set");
/**
* Applications will never create PropertyArities directly.
**/
private PropertyArity (String value)
{
super (value);
}
}
@@ -1,278 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.util.*;
/**
* A description of the location of a Property in a
* RepInfo object. Properties can be nested under other
* properties, in lists, maps, or subproperties. This class
* provides a general way to specify their location.
*
* For the moment, only paths by name are fully supported.
*
* @author Gary McGath
*
*/
public class PropertyPath {
private List pathInfo;
/**
* No-argument constructor.
* Creates an empty path.
*/
public PropertyPath ()
{
pathInfo = new LinkedList ();
}
/**
* Cloning constructor.
* This creates a new pathInfo list, whose elements
* are shared with the original PropertyPath's list.
*/
public PropertyPath (PropertyPath path)
{
pathInfo = new LinkedList ();
pathInfo.addAll (path.pathInfo);
}
/**
* String array constructor.
* This allows creation of a path in a common and simple
* case: a hierarchy of named subproperties. It's the
* equivalent of creating an empty PropertyPath and then
* calling addElement successively with each of the strings
* in the array.
*/
public PropertyPath (String[] pathArray)
{
pathInfo = new LinkedList ();
for (int i = 0; i < pathArray.length; i++) {
pathInfo.add (pathArray[i]);
}
}
/**
* Adds a property name to the path.
*/
public void addPropertyName (String str)
{
pathInfo.add (str);
// Now -- how do we add a key and distinguish it from a
// property name? We could define an internal type for
// keys. Do that for the moment and see what that leads
// to in retrieval. Alternatively, it isn't needed because
// we can see that a property has arity Map.
}
/**
* Adds a key to the path, for a property map.
*/
public void addPropertyKey (Object obj)
{
pathInfo.add (new PropertyKey (obj));
}
/**
* Adds an index to the path, for an indexed property.
*/
public void addPropertyIndex (int idx)
{
pathInfo.add (new Integer (idx));
}
/**
* Walk down the path and return the specified Property.
*
* @param info The RepInfo object to search
*
* @return The specified Property if found, otherwise null.
*/
public Property locateProperty (RepInfo info)
{
return locateProperty (info, false);
}
/**
* Walk down the path and return the specified Property.
*
* @param info The RepInfo object to search
* @param trace If <code>true</code>, write debugging information
* to standard output.
*
* @return The specified Property if found, otherwise null.
*/
public Property locateProperty (RepInfo info, boolean trace)
{
if (pathInfo.isEmpty ()) {
// An empty path can't reach any property
if (trace) {
System.out.println ("Empty property path");
}
return null;
}
Object obj = pathInfo.get(0);
if (!(obj instanceof String)) {
// The initial qualifier must be a property name
if (trace) {
System.out.println ("Not a property name");
}
return null;
}
String top = (String) obj;
if (trace) {
System.out.println ("Getting proprerty " + top);
}
Property prop = info.getProperty (top);
if (prop == null) {
// No property of that name in RepInfo
if (trace) {
System.out.println ("Property is null");
}
return null;
}
int pathLen = pathInfo.size ();
// Pass the CDR of the list to locateSubProperty.
return locateSubProperty (prop, pathInfo.subList (1, pathLen), trace);
}
/* Recursive function for extracting a subproperty of a property. */
private Property locateSubProperty (Property property, List path, boolean trace)
{
// If there's nothing left of the path, we're done.
if (path.isEmpty ()) {
return property;
}
List cdr = path.subList (1, path.size());
PropertyArity arity = property.getArity ();
PropertyType type = property.getType ();
Object val = property.getValue ();
if (trace) {
System.out.println ("Property arity = " + arity + ", type = " + type);
}
// If the type isn't PROPERTY, then there are no subproperties.
if (type != PropertyType.PROPERTY) {
if (trace) {
System.out.println ("Not a property, type is " + type.toString ());
}
return null;
}
Object obj = path.get (0);
if (obj instanceof String) {
Iterator iter;
// Iterate through the property and see if any of the elements
// are Properties that match the name.
String name = (String) obj;
if (trace) {
System.out.println ("Looking for subproperty " + name +
" arity= " + arity.toString ());
}
if (arity.equals (PropertyArity.SCALAR)) {
// There's just one shot at matching a scalar.
Property p = (Property) property.getValue ();
if (p.getName().equals (name)) {
return locateSubProperty (p, cdr, trace);
}
else {
return null;
}
}
else if (arity.equals (PropertyArity.ARRAY)) {
// We know it's an array of Properties, which saves much
// hair-tearing.
Property[] parray = (Property []) val;
for (int i = 0; i < parray.length; i++) {
Property p = parray[i];
if (p.getName ().equals (name)) {
return locateSubProperty (p, cdr, trace);
}
}
return null;
}
else if (arity.equals (PropertyArity.LIST)) {
iter = ((List) val).listIterator ();
return getIteratedSubProperty (iter, name, cdr, trace);
}
else if (arity.equals (PropertyArity.SET)) {
iter = ((Set) val).iterator ();
return getIteratedSubProperty (iter, name, cdr, trace);
}
else if (arity.equals (PropertyArity.MAP)) {
iter = ((Map) val).values().iterator();
return getIteratedSubProperty (iter, name, cdr, trace);
}
else {
// Should never happen, but keep compiler happy
//System.out.println ("Unknown arity");
return null;
}
}
else if (obj instanceof Integer) {
int idx = ((Integer) obj).intValue();
//System.out.println ("Property index = " + idx + ", arity= " + arity.toString ());
if (arity.equals (PropertyArity.LIST)) {
List propList = (List) val;
return locateSubProperty
((Property) propList.get (idx), cdr, trace);
}
else if (arity.equals (PropertyArity.ARRAY)) {
Property[] propArr = (Property []) val;
return locateSubProperty (propArr[idx], cdr, trace);
}
else {
// Other arities are not indexable
return null;
}
}
else if (obj instanceof PropertyKey) {
// This is applicable only to a Map.
if (arity != PropertyArity.MAP) {
return null;
}
return null; // I'm not sure this case is even meaningful
}
else {
// We should never get here
return null;
}
}
/* Walk through an Iterator, whose elements are Properties,
* and return the subproperty by <code>path</code> of the first element
* whose name matches <code>name</code>.
*/
private Property getIteratedSubProperty (Iterator iter,
String name,
List path,
boolean trace)
{
while (iter.hasNext ()) {
Property p = (Property) iter.next ();
if (p.getName ().equals (name)) {
return locateSubProperty (p, path, trace);
}
}
return null;
}
private class PropertyKey
{
public Object key;
public PropertyKey (Object obj)
{
key = obj;
}
}
}
@@ -1,110 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2009 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for an Property of
* some given content.
* Applications will not create or modify PropertyTypes, but will
* use one of the predefined PropertyType instances
* BOOLEAN, BYTE, CHARACTER, DATE, DOUBLE, FLOAT, INTEGER,
* LONG, OBJECT, PROPERTY, SHORT, STRING, RATIONAL, or
* NISOIMAGEMETADATA.
*
* @see Property
*/
public final class PropertyType
extends EnumerationType
{
/**
* Property type for a <code>Boolean</code> object, or a
* <code>boolean</code> if the Arity is Array.
*/
public static final PropertyType BOOLEAN = new PropertyType ("Boolean");
/**
* Property type for a <code>Byte</code> object, or a <code>byte</code>
* if the Arity is Array.
*/
public static final PropertyType BYTE = new PropertyType ("Byte");
/**
* Property type for a <code>Character</code> object, or a
* <code>char</code> if the Arity is Array.
*/
public static final PropertyType CHARACTER = new PropertyType("Character");
/**
* Property type for a <code>Date</code> object.
*/
public static final PropertyType DATE = new PropertyType ("Date");
/**
* Property type for a <code>Double</code> object, or
*a <code>double</code> if the Arity is Array.
*/
public static final PropertyType DOUBLE = new PropertyType ("Double");
/**
* Property type for a <code>Float</code> object, or a
* <code>float</code> if the Arity is Array.
*/
public static final PropertyType FLOAT = new PropertyType ("Float");
/**
* Property type for an <code>Integer</code> object, or an
* <code>integer</code> if the Arity is Array.
*/
public static final PropertyType INTEGER = new PropertyType ("Integer");
/**
* Property type for a <code>Long</code> object, or a
* <code>long</code> if the Arity is Array.
*/
public static final PropertyType LONG = new PropertyType ("Long");
/**
* Property type for an <code>Object</code>.
*/
public static final PropertyType OBJECT = new PropertyType ("Object");
/**
* Property type for an <code>AESAudioMetadata</code>.
*/
public static final PropertyType AESAUDIOMETADATA =
new PropertyType ("AESAudioMetadata");
/**
* Property type for a <code>NisoImageMetadata</code>.
*/
public static final PropertyType NISOIMAGEMETADATA =
new PropertyType ("NISOImageMetadata");
/**
* Property type for a <code>TextMDMetadata</code>.
*/
public static final PropertyType TEXTMDMETADATA =
new PropertyType ("TextMDMetadata");
/**
* Property type for a <code>Property</code> object.
*/
public static final PropertyType PROPERTY = new PropertyType ("Property");
/**
* Property type for a <code>Short</code> object, or a
* <code>short</code> if the Arity is Array.
*/
public static final PropertyType SHORT = new PropertyType ("Short");
/**
* Property type for a <code>String</code> object.
*/
public static final PropertyType STRING = new PropertyType ("String");
/**
* Property type for a <code>Rational</code> object.
*/
public static final PropertyType RATIONAL = new PropertyType ("Rational");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create PropertyTypes directly.
**/
private PropertyType (String value)
{
super (value);
}
}
@@ -1,261 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.*;
/**
* An InputStream layered on top of a RandomAccessFile.
* This is useful for a Module which has requirements that
* force it to use a RandomAccessFile, but is usually
* accessed sequentially.
*
* An RAFInputStream maintains its own position information
* in the file, so multiple RAFInputStreams in the same file
* will work without interference. However, this class is
* not thread-safe.
*
* @author Gary McGath
*
*/
public class RAFInputStream extends InputStream {
/* The file on which the stream is based */
private RandomAccessFile _raf;
/* Size of fileBuf */
private int fileBufSize;
/* Buffer for reading from the file */
private byte[] fileBuf;
/* Offset for reading next byte from filebuf */
private int fileBufPos;
/* Number of valid bytes in fileBuf */
private int fileBufBytes;
/* Position in file for next read from RandomAccessFile */
private long fileOffset;
/* EOF flag */
boolean eof;
/**
* Constructor with default buffer size.
* The stream starts at the current position of the
* RandomAccessFile.
*
* @param raf The file on which the
* stream is to be based.
*/
public RAFInputStream(RandomAccessFile raf) {
super();
_raf = raf;
fileBufSize = 65536; // pick a size, any size
init ();
}
/**
* Constructor with buffer size.
* The stream starts at the current position of the
* RandomAccessFile.
*
* @param raf The file on which the
* stream is to be based.
*
* @param bufferSize The buffer size to be used.
* If less than or equal to 0, the
* default buffer size is used.
*/
public RAFInputStream(RandomAccessFile raf, int bufferSize) {
super();
_raf = raf;
fileBufSize = (bufferSize <= 0 ? 65536 : bufferSize);
init ();
}
private void init ()
{
fileBufBytes = 0;
fileBufPos = 0;
fileBuf = new byte[fileBufSize];
try {
fileOffset = _raf.getFilePointer ();
}
catch (IOException e) {}
eof = false;
}
/**
* Reads a single byte from the file.
*/
public int read() throws IOException {
if (eof) {
return -1;
}
if (fileBufPos >= fileBufBytes) {
// Need to read another bufferful
_raf.seek (fileOffset);
fileBufBytes = _raf.read (fileBuf);
fileBufPos = 0;
if (fileBufBytes <= 0) {
// No more in file
eof = true;
return -1;
}
else {
fileOffset += fileBufBytes;
}
}
return ((int) fileBuf[fileBufPos++] & 0XFF);
}
/**
* Reads some number of bytes from the input stream and
* stores them into the buffer array b. The number of
* bytes actually read is returned as an integer.
*/
public int read (byte[] b) throws IOException
{
int bytesToRead = b.length;
int bytesRead = 0;
for (;;) {
// See how many bytes are available in fileBuf.
int fbAvail = fileBufBytes - fileBufPos;
if (fbAvail <= 0) {
// Need to read another bufferful
_raf.seek (fileOffset);
fileBufBytes = _raf.read (fileBuf);
fileBufPos = 0;
if (fileBufBytes <= 0) {
// No more in file -- return what we have
eof = true;
return bytesRead;
}
fbAvail = fileBufBytes;
fileOffset += fileBufBytes;
}
if (fbAvail > bytesToRead) {
// We have more than enough bytes.
fbAvail = bytesToRead;
}
for (int i = 0; i < fbAvail; i++) {
b[bytesRead++] = fileBuf[fileBufPos++];
bytesToRead--;
}
if (bytesToRead == 0) {
return bytesRead;
}
}
}
/**
* Reads up to len bytes of data from the input stream
* into an array of bytes. An attempt is made to read as
* many as len bytes, but a smaller number may be read,
* possibly zero. The number of bytes actually read is
* returned as an integer.
*
*/
public int read(byte[] b, int off, int len) throws IOException
{
int bytesToRead = len;
int bytesRead = 0;
for (;;) {
// See how many bytes are available in fileBuf.
int fbAvail = fileBufBytes - fileBufPos;
if (fbAvail <= 0) {
// Need to read another bufferful
_raf.seek (fileOffset);
fileBufBytes = _raf.read (fileBuf);
fileBufPos = 0;
if (fileBufBytes <= 0) {
// No more in file -- return what we have
eof = true;
return bytesRead;
}
fbAvail = fileBufBytes;
fileOffset += fileBufBytes;
}
if (fbAvail > bytesToRead) {
// We have more than enough bytes.
fbAvail = bytesToRead;
}
for (int i = 0; i < fbAvail; i++) {
b[off + bytesRead++] = fileBuf[fileBufPos++];
bytesToRead--;
}
if (bytesToRead == 0) {
return bytesRead;
}
}
}
/** Skips some number of bytes.
*
* @return The number of bytes actually skipped.
*/
public long skip (long n) throws IOException
{
// If the range of the skip lies
// within the current buffer, we just adjust
// the buffer offset.
int bytesLeft = fileBufBytes - fileBufPos;
if (bytesLeft > n) {
fileBufPos += (int) n;
}
else {
// doesn't fit within the buffer.
// Set up to seek to the new position.
//long curPos = _raf.getFilePointer ();
if (fileOffset + n - bytesLeft > _raf.length ()) {
fileOffset = _raf.length ();
}
//seek (curPos + n - bytesLeft);
else {
fileOffset += n - bytesLeft;
}
fileBufBytes = 0; // Invalidate current buffer
}
return n;
}
/**
* Returns the RandomAccessFile object.
*/
public RandomAccessFile getRAF ()
{
return _raf;
}
/** Positions the stream to a different point in the file.
* This invalidates the buffer.
*/
public void seek (long offset) throws IOException
{
_raf.seek (offset);
fileBufBytes = 0;
fileBufPos = 0;
eof = false;
}
/** Returns the current position in the file.
* What is reported is the position of the byte
* in the file which was last extracted from
* the buffer.
*/
public long getFilePos () throws IOException
{
return _raf.getFilePointer() -
(fileBufBytes - fileBufPos);
}
}
@@ -1,92 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* Class encapsulating RFC-1766 language codes.
*/
public final class RFC1766Lang
{
String _langCode;
/**
* Constructor.
*
* @param str The ASCII string for the language code.
*/
public RFC1766Lang (String str)
{
_langCode = str;
}
/**
* Returns the language code string.
*/
public String getLangCode ()
{
return _langCode;
}
/**
* Returns <code>true</code> if the language code
* string is syntactically compliant.
* The primary tag must be either a two-letter code,
* the letter i, or the letter x (case insensitive);
* no checking is done against registry lists.
*/
public boolean isSyntaxCorrect ()
{
int i;
if (_langCode == null) {
return false;
}
char[] chrs = _langCode.toLowerCase().toCharArray();
char firstChar = '\0';
int ntags = 0;
int taglength = 0;
for (i = 0; i < chrs.length; i++) {
char ch = chrs[i];
if (i == 0) {
firstChar = ch;
}
if (!Character.isLetter (ch) && ch != '-') {
return false;
}
if (ch == '-') {
taglength = 0;
// If this is the primary tag, do some checks
if (ntags++ == 0) {
if (taglength == 1) {
if (firstChar != 'i' && firstChar != 'x') {
return false;
}
else if (taglength != 2) {
return false;
}
}
}
}
else {
taglength++;
if (taglength > 8) {
return false;
}
}
}
// A dangling hyphen at the end isn't allowed.
if (taglength == 0) {
return false;
}
return true;
}
}
@@ -1,100 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates a number which is defined as the ratio
* of two 32-bit unsigned integers, in accordance with the TIFF
* specification.
*/
public class Rational
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Numerator of ratio. */
private long _numerator;
/** Denominator of ratio. */
private long _denominator;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* The arguments to this constructor are long in order to
* represent all possible 32-bit unsigned integers. Parameters
* greater than 2 ^ 32 - 1 are not meaningful.
* @param numerator numerator of the Rational value
* @param denominator denominator of the Rational value
*/
public Rational (long numerator, long denominator)
{
_numerator = numerator;
_denominator = denominator;
}
/**
* The arguments to the int constructor are treated as
* 32-bit unsigned integers.
* @param numerator numerator of the Rational value
* @param denominator denominator of the Rational value
*/
public Rational (int numerator, int denominator)
{
_numerator = (long) numerator & 0XFFFFFFFF;
_denominator = (long) denominator & 0XFFFFFFFF;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Returns the Numerator property.
*/
public long getNumerator ()
{
return _numerator;
}
/**
* Returns the Denominator property.
*/
public long getDenominator()
{
return _denominator;
}
/**
* Converts to a floating-point value (numerator/denominator).
* May throw an ArithmeticException.
**/
public double toDouble ()
{
return ((double) _numerator / (double) _denominator);
}
/**
* Converts to a long value (numerator/denominator).
* May throw an ArithmeticException.
**/
public long toLong ()
{
return (long) ((double) _numerator / (double) _denominator);
}
/**
* Represents the Rational as a String in the form of
* "numerator/denominator".
*/
public String toString ()
{
return Long.toString (_numerator) + "/" +
Long.toString (_denominator);
}
}
@@ -1,599 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2005 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.util.*;
/**
* This class encapsulates representation information, as defined
* by ISO/IEC 14721, about a content stream.
*
* @see <a href="http://wwwclassic.ccsds.org/documents/pdf/CCSDS-650.0-B-1.pdf">ISO/IEC
* 14721 (PDF)</a>
*/
public class RepInfo implements Cloneable
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** List of checksums. */
private List<Checksum> _checksum;
/** Consistency flag. */
private boolean _consistent;
/** Validity flag. A ternary variable which can have a value
* of TRUE, FALSE, or UNDETERMINED. */
private int _valid;
/** Values for _valid */
public final static int
TRUE = 1,
FALSE = 0,
UNDETERMINED = -1;
/** Creation date. */
private Date _created;
/** External representation information. */
private RepInfo _external;
/** Format identifier. */
private String _format;
/** Modification date. */
private Date _lastModified;
/** List of diagnostic and informative messages. */
private List<Message> _message;
/** MIME media type. */
private String _mimeType;
/** The module used to populate this representation information. */
private Module _module;
/** List of conforming format profiles. */
private List<String> _profile;
/** List of modules for which signature matches. */
private List<String> _sigMatch;
/** Associative map of module-specific representation information. */
private Map<String, Property> _property;
/** Object size. */
private long _size;
/** Object file pathname or URI. */
private String _uri;
/** Flag indicating _uri is a URL if true. */
private boolean _urlFlag;
/** Well-formed flag. A ternary variable which can have a value
* of TRUE, FALSE, or UNDETERMINED. */
private int _wellFormed;
/** Version of format which applies. */
private String _version;
/** Note. */
private String _note;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates a RepInfo with a URI reference
*
* @param uri Object file pathname or URI
*/
public RepInfo (String uri)
{
init (uri);
}
/**
* Creates a RepInfo with a URI reference and an external RepInfo.
*
* By default, urlFlag is <code>false</code>.
*
* @param uri Object file pathname or URI
* @param external External representation information
*/
public RepInfo (String uri, RepInfo external)
{
init (uri);
_external = external;
}
private void init (String uri)
{
_uri = uri;
_size = -1;
_wellFormed = TRUE;
_consistent = true;
_urlFlag = false;
_valid = TRUE;
_checksum = new ArrayList<Checksum> ();
_message = new ArrayList<Message> ();
_profile = new ArrayList<String> ();
_property = new TreeMap<String, Property> ();
_sigMatch = new ArrayList<String> ();
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Clones the RepInfo one level deep, making fresh copies
* of the checksum, message, profile, signature match,
* and property fields.
* The external RepInfo (if any) is not cloned, but
* is attached directly to the clone.
*/
public Object clone ()
{
RepInfo newri;
try {
newri = (RepInfo) super.clone ();
}
catch (CloneNotSupportedException e) {
return null; // should never happen
}
newri._checksum = new ArrayList<Checksum> (_checksum);
newri._message = new ArrayList<Message>(_message);
newri._profile = new ArrayList<String> (_profile);
newri._sigMatch = new ArrayList<String> (_sigMatch);
newri._property = new TreeMap<String, Property> (_property);
return (Object) newri;
}
/**
* Copies all the information out of the parameter object.
* This is a "shallow" copy; it is assumed that the parameter
* object is a temporary one that will not be further modified.
*/
public void copy (RepInfo info)
{
_checksum = info._checksum;
_consistent = info._consistent;
_created = info._created;
_external = info._external;
_format = info._format;
_lastModified = info._lastModified;
_message = info._message;
_mimeType = info._mimeType;
_profile = info._profile;
_property = info._property;
_size = info._size;
_uri = info._uri;
_urlFlag = info._urlFlag;
_wellFormed = info._wellFormed;
_valid = info._valid;
_version = info._version;
_note = info._note;
_module = info._module;
_sigMatch = info._sigMatch;
}
/******************************************************************
*
* Accessor methods.
******************************************************************/
/**
* Returns this object's list of Checksums
*/
public List<Checksum> getChecksum ()
{
return _checksum;
}
/**
* Returns the creation date stored in this object. A creation
* date is not automatically generated, but must be explicitly
* stored.
**/
public Date getCreated ()
{
return _created;
}
/**
* Return the format identifier
*/
public String getFormat ()
{
return _format;
}
/**
* Returns the last modified date stored in this object. A
* date is not automatically generated, but must be explicitly
* stored.
**/
public Date getLastModified ()
{
return _lastModified;
}
/**
* Returns the message list stored in this object
*/
public List<Message> getMessage ()
{
return _message;
}
/**
* Returns the MIME type string stored in this object
*/
public String getMimeType ()
{
return _mimeType;
}
/**
* Return the module.
*/
public Module getModule ()
{
return _module;
}
/**
* Returns the list of profiles (Strings) stored in this object
*/
public List<String> getProfile ()
{
return _profile;
}
/**
* Returns the Property map stored in this object. The
* Property map contains key-value pairs whose key is a
* <code>String</code> and whose value is a <code>Property</code>.
*/
public Map<String, Property> getProperty ()
{
return _property;
}
/**
* Returns a named Property from the Property map
*
* @param name The name of the Property.
*/
public Property getProperty (String name)
{
Property property = null;
if (_property.size () > 0) {
property = (Property) _property.get (name);
}
return property;
}
/**
* Returns the size property stored in this object.
*/
public long getSize ()
{
return _size;
}
/**
* Returns the URI property stored in this object.
*/
public String getUri ()
{
return _uri;
}
/**
* Returns a flag which, if <code>true</code>, indicates
* the object is a URL.
*/
public boolean getURLFlag ()
{
return _urlFlag;
}
/**
* Returns the value of the consistency flag.
*/
public boolean isConsistent ()
{
return _consistent;
}
/**
* Returns the value of the well-formed flag.
* Can return TRUE, FALSE, or UNDETERMINED.
*/
public int getWellFormed ()
{
return _wellFormed;
}
/**
* Returns the value of the validity flag.
* Can return TRUE, FALSE, or UNDETERMINED.
*/
public int getValid ()
{
return _valid;
}
/**
* Returns the version property stored in this object
*/
public String getVersion ()
{
return _version;
}
/**
* Returns the note property stored in this object
*/
public String getNote ()
{
return _note;
}
/**
* Returns the list of matching signatures.
* JhoveBase will make this value persistent across
* module invocations for a given document, so the list
* returned will reflect all modules that have looked
* at the document so far.
*/
public List<String> getSigMatch ()
{
return _sigMatch;
}
/**
* Return property by name, regardless of its position in the
* property hierarchy.
* @param name Property name
* @return Named property (or null)
*/
public Property getByName (String name)
{
Property prop = null;
Collection<Property> coll = _property.values ();
Iterator<Property> iter = coll.iterator ();
while (iter.hasNext ()) {
prop = (Property) iter.next ();
if ((prop = prop.getByName (name)) != null) {
break;
}
}
return prop;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/**
* Append a Checksum object to the checksum list.
*/
public void setChecksum (Checksum checksum)
{
_checksum.add (checksum);
}
/**
* Set the value of the consistency flag
*/
public void setConsistent (boolean consistent)
{
_consistent = consistent;
}
/**
* Set the creation date
*/
public void setCreated (Date created)
{
_created = created;
}
/**
* Set the format identifier
*/
public void setFormat (String format)
{
_format = format;
}
/**
* Set the last modified date
*/
public void setLastModified (Date lastModified)
{
_lastModified = lastModified;
}
/**
* Append a Message object to the message list
*/
public void setMessage (Message message)
{
_message.add (message);
}
/**
* Set the MIME type string
*/
public void setMimeType (String mimeType)
{
_mimeType = mimeType;
}
/**
* Add the module.
*/
public void setModule (Module module)
{
_module = module;
}
/**
* Append a profile String to the profile list
*/
public void setProfile (String profile)
{
_profile.add (profile);
}
/**
* Add a Property to the property map. The name of the Property
* becomes its key in the map.
*/
public void setProperty (Property property)
{
_property.put (property.getName (), property);
}
/**
* Set the size property
*/
public void setSize (long size)
{
_size = size;
}
/**
* Set the flag to indicate whether this is a URL (true)
* or a file (false)
*/
public void setURLFlag (boolean flag)
{
_urlFlag = flag;
}
/**
* Set the well-formed flag
*
* @param wellFormed Boolean argument that maps to
* an integer value:
* true maps to TRUE, and false to FALSE.
*/
public void setWellFormed (boolean wellFormed)
{
_wellFormed = wellFormed ? TRUE : FALSE;
if (!wellFormed) {
_consistent = false;
_valid = FALSE;
}
}
/**
* Set the wellFormed flag.
* Setting wellFormed to false forces the consistent and
* valid flags to be false as well.
*/
public void setWellFormed (int wellFormed)
{
_wellFormed = wellFormed;
if (wellFormed == FALSE) {
_consistent = false;
_valid = FALSE;
}
if (wellFormed == UNDETERMINED) {
_valid = UNDETERMINED;
}
}
/**
* Set the validity flag
*
* @param valid Boolean argument that maps to
* an integer value:
* true maps to TRUE, and false to FALSE.
*/
public void setValid (boolean valid)
{
_valid = valid ? TRUE : FALSE;
}
/**
* Set the validity flag
*
* @param valid Permitted values are TRUE, FALSE, AND
* UNDETERMINED. The effect of using
* other values is undefined.
*/
public void setValid (int valid)
{
_valid = valid;
}
/**
* Set the version string
*/
public void setVersion (String version)
{
_version = version;
}
/**
* Set the note string
*/
public void setNote (String note)
{
_note = note;
}
/** Adds the name of a module, signifying that the document
* signature matched the module's requirements.
* JhoveBase will make this value persistent across
* module invocations for a given document.
*/
public void setSigMatch (String modname)
{
_sigMatch.add (modname);
}
/** Adds a list of module names, signifying that the document
* signature matched the module's requirements.
* Any previous list is lost.
* JhoveBase will make this value persistent across
* module invocations for a given document.
*/
public void setSigMatch (List<String> modnames)
{
_sigMatch = modnames;
}
/******************************************************************
* Serialization methods.
******************************************************************/
/**
* Output the information in this object. The format and
* destination of the output are determined by the
* OutputHandler.
*/
public void show (OutputHandler handler)
{
handler.analyze (this);
handler.show (this);
}
}
@@ -1,157 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class encapsulates information about format signatures,
* both internal and external.
* The value of a Signature may be either a String or a byte array
* (stored as an int array to avoid signed byte problems).
*/
public abstract class Signature
{
private int[] _value;
private String _stringValue;
private SignatureType _type;
private SignatureUseType _use;
private String _note;
/**
* A Signature cannot be created directly; this constructor
* can be called as the superclass constructor from a subclass.
* This constructor uses a String value.
*/
protected Signature (String value, SignatureType type,
SignatureUseType use)
{
this (new int[value.length ()], type, use);
int len = value.length ();
_stringValue = value;
for (int i = 0; i < len; i++) {
_value[i] = value.charAt(i);
}
}
/**
* A Signature cannot be created directly; this constructor
* can be called as the superclass constructor from a subclass.
* This constructor uses a byte array (stored as an int array) value.
*/
protected Signature (int[] value, SignatureType type,
SignatureUseType use)
{
_value = value;
_type = type;
_use = use;
_stringValue = null;
}
/**
* A Signature cannot be created directly; this constructor
* can be called as the superclass constructor from a subclass.
* This constructor uses a String value and allows specification
* of a note.
*/
protected Signature (String value, SignatureType type,
SignatureUseType use,
String note)
{
this (new int[value.length ()], type, use, note);
int len = value.length ();
for (int i = 0; i < len; i++) {
_value[i] = value.charAt(i);
}
_stringValue = value;
}
/**
* A Signature cannot be created directly; this constructor
* can be called as the superclass constructor from a subclass.
* This constructor uses a byte array (stored as an int array) value
* and allows specification of a note.
*/
protected Signature (int[] value, SignatureType type,
SignatureUseType use,
String note)
{
this (value, type, use);
_note = note;
}
/**
* Returns the type of this Signature
*/
public SignatureType getType ()
{
return _type;
}
/**
* Returns the use requirement for this Signature
*/
public SignatureUseType getUse ()
{
return _use;
}
/**
* Returns the byte array value for this Signature.
* If this Signature was constructed from a String, it
* returns the characters of the String as the bytes of
* the array.
*/
public int[] getValue ()
{
return _value;
}
/**
* Returns the note specified for this Signature, or null
* if no note was specified.
*/
public String getNote ()
{
return _note;
}
/**
* Returns true if this Signature's value was provided as a
* String, false if as an array.
*/
public boolean isStringValue ()
{
return (_stringValue != null);
}
/**
* Returns the string value of this Signature. Returns null
* if this Signature was constructed with an array.
*/
public String getValueString ()
{
return _stringValue;
}
/**
* Returns the value of this Signature as a hexadecimal string.
* The length of the string is twice the length of the array
* or string from which this Signature was created, and all
* alphabetic characters are lower case.
*/
public String getValueHexString ()
{
StringBuffer valBuf = new StringBuffer ("0x");
for (int i = 0; i < _value.length; i++) {
/* Make each byte exactly two digits */
int b = _value[i];
if (b < 16) {
valBuf.append ('0');
}
valBuf.append (Integer.toHexString (b));
}
return valBuf.toString ();
}
}
@@ -1,54 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated types for a Signature in a module.
* Applications will not create or modify SignatureTypes, but will
* use one of the predefined SignatureType instances
* EXTENSION, FILETYPE, or MAGIC.
*
* @see Signature
*/
public final class SignatureType
extends EnumerationType
{
/******************************************************************
* PUBLIC STATIC INSTANCES.
******************************************************************/
/**
* Signature type for a file extension, i.e., a sequence of
* characters following a period character in a file name.
*/
public static final SignatureType EXTENSION =
new SignatureType ("File extension");
/**
* Signature type for a Macintosh OS file type. This applies
* only to Mac OS files, and is always a four-character code.
*/
public static final SignatureType FILETYPE =
new SignatureType ("Mac OS file type");
/**
* Signature type for a "magic number" stored in the file.
*/
public static final SignatureType MAGIC =
new SignatureType ("Magic number");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create SignatureTypes directly.
**/
private SignatureType (String value)
{
super (value);
}
}
@@ -1,55 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
/**
* This class defines enumerated use types for a Signature in a module.
* These give information on whether a signature is required in
* valid content.
* Applications will not create or modify SignatureUseTypes, but will
* use one of the predefined SignatureUseType instances
* MANDATORY, MANDATORY_IF_APPLICABLE, or OPTIONAL.
*
* @see Signature
*/
public final class SignatureUseType
extends EnumerationType
{
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
/**
* Use type for a required signature.
*/
public static final SignatureUseType MANDATORY =
new SignatureUseType ("Mandatory");
/**
* Use type for a conditionally required signature.
*/
public static final SignatureUseType MANDATORY_IF_APPLICABLE =
new SignatureUseType ("Mandatory if applicable");
/**
* Use type for an optional signature.
*/
public static final SignatureUseType OPTIONAL =
new SignatureUseType ("Optional");
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Applications will never create SignatureUseTypes directly.
**/
private SignatureUseType (String value)
{
super (value);
}
}
@@ -1,438 +0,0 @@
package edu.harvard.hul.ois.jhove;
import java.nio.charset.Charset;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
/**
* Encapsulation of the textMD metadata for text files.
* See http://www.loc.gov/standards/textMd for more information.
*
* @author Thomas Ledoux
*
*/
public class TextMDMetadata {
/**
* textMD namespace and version
*/
public static final String NAMESPACE = "info:lc/xmlns/textMD-v3";
public static final String DEFAULT_LOCATION =
"http://www.loc.gov/standards/textMD/textMD-v3.01a.xsd";
public static final String VERSION = "3.0";
/**
* Uses enumerated values of 'big', 'little', and 'middle' endian.
*/
public static final String[] BYTE_ORDER = {
"big", "little", "middle"
};
public static final int BYTE_ORDER_BIG = 0;
public static final int BYTE_ORDER_LITTLE = 1;
public static final int BYTE_ORDER_MIDDLE= 2;
/**
* Uses enumerated values of 'CR', 'LF' and 'CR/LF' for the idenntification of the linebreak.
*/
public static final String[] LINEBREAK = {
"CR", "LF", "CR/LF"
};
public static final int LINEBREAK_CR = 0;
public static final int LINEBREAK_LF = 1;
public static final int LINEBREAK_CRLF = 2;
/**
* Array of textMD charsets unknown by java.nio.charset.Charsets
*/
protected static final String[] UNKNOWN_JAVA_CHARSET =
new String[] {
"ISO-10646-UTF-1", "ISO_646.basic:1983", "INVARIANT", "BS_4730", "NATS-SEFI",
"NATS-SEFI-ADD", "NATS-DANO", "NATS-DANO-ADD", "SEN_850200_B", "SEN_850200_C", "ISO-2022-CN-EXT",
"JIS_C6220-1969-jp", "JIS_C6220-1969-ro", "IT", "PT", "ES", "greek7-old", "latin-greek",
"DIN_66003", "NF_Z_62-010_(1973)", "latin-greek-1", "ISO_5427", "JIS_C6226-1978", "BS_viewdata",
"INIS", "INIS-8", "INIS-cyrillic", "ISO_5427:1981", "ISO_5428:1980", "GB_1988-80", "GB_2312-80",
"NS_4551-1", "NS_4551-2", "NF_Z_62-010", "videotex-suppl", "PT2", "ES2", "MSZ_7795.3", "greek7",
"ASMO_449", "iso-ir-90", "JIS_C6229-1984-a", "JIS_C6229-1984-b", "JIS_C6229-1984-b-add",
"JIS_C6229-1984-hand", "JIS_C6229-1984-hand-add", "JIS_C6229-1984-kana", "ISO_2033-1983",
"ANSI_X3.110-1983", "T.61-7bit", "ECMA-cyrillic", "CSA_Z243.4-1985-1", "CSA_Z243.4-1985-2",
"CSA_Z243.4-1985-gr", "ISO_8859-6-E", "ISO_8859-6-I", "T.101-G2", "ISO_8859-8-E", "ISO_8859-8-I",
"CSN_369103", "JUS_I.B1.002", "ISO_6937-2-add", "IEC_P27-1", "JUS_I.B1.003-serb",
"JUS_I.B1.003-mac", "greek-ccitt", "NC_NC00-10:81", "ISO_6937-2-25", "GOST_19768-74",
"ISO_8859-supp", "ISO_10367-box", "ISO-8859-10", "latin-lap", "DS_2089", "us-dk", "dk-us",
"KSC5636", "ISO-10646-UCS-4", "DEC-MCS", "hp-roman8", "macintosh", "IBM038", "IBM274", "IBM275",
"IBM281", "IBM290", "IBM423", "IBM851", "IBM880", "IBM891", "IBM903", "IBM904", "IBM905",
"EBCDIC-AT-DE", "EBCDIC-AT-DE-A", "EBCDIC-CA-FR", "EBCDIC-DK-NO", "EBCDIC-DK-NO-A", "EBCDIC-FI-SE",
"EBCDIC-FI-SE-A", "EBCDIC-FR", "EBCDIC-IT", "EBCDIC-PT", "EBCDIC-ES", "EBCDIC-ES-A", "EBCDIC-ES-S",
"EBCDIC-UK", "EBCDIC-US", "UNKNOWN-8BIT", "MNEMONIC", "MNEM", "VISCII", "VIQR", "IBM00924",
"UNICODE-1-1", "SCSU", "UTF-7", "CESU-8", "UNICODE-1-1-UTF-7", "ISO-8859-14", "ISO-8859-16",
"Extended_UNIX_Code_Fixed_Width_for_Japanese", "ISO-10646-UCS-Basic", "ISO-10646-Unicode-Latin1",
"ISO-10646-J-1", "ISO-Unicode-IBM-1268", "ISO-Unicode-IBM-1276", "ISO-Unicode-IBM-1264",
"ISO-Unicode-IBM-1265", "ISO-8859-1-Windows-3.0-Latin-1", "ISO-8859-1-Windows-3.1-Latin-1",
"ISO-8859-2-Windows-Latin-2", "ISO-8859-9-Windows-Latin-5", "Adobe-Standard-Encoding",
"Ventura-US", "Ventura-International", "PC8-Danish-Norwegian", "PC8-Turkish", "IBM-Symbols",
"HP-Legal", "HP-Pi-font", "HP-Math8", "Adobe-Symbol-Encoding", "HP-DeskTop", "Ventura-Math",
"Microsoft-Publishing", "HZ-GB-2312", };
/**
* Set of unknown charsets in Java
*/
protected static Set setOfUnknownJavaCharset;
/**
* Map from ISO 639/2 T to ISO 639/2 B
*/
protected static Map<String, String> fromISO_639_2_T2B;
public static final String CHARSET_ASCII = "US-ASCII";
public static final String CHARSET_UTF8 = "UTF-8";
public static final String CHARSET_ISO8859_1 = "ISO-8859-1";
/**
* To represent the unknown
*/
public static final int NILL = -1;
static {
setOfUnknownJavaCharset = new HashSet(Arrays.asList(UNKNOWN_JAVA_CHARSET));
// Map to transform from the terminology code to the bibliographic one
fromISO_639_2_T2B = new HashMap<String, String>();
fromISO_639_2_T2B.put("sqi", "alb");
fromISO_639_2_T2B.put("hye", "arm");
fromISO_639_2_T2B.put("eus", "baq");
fromISO_639_2_T2B.put("mya", "bur");
fromISO_639_2_T2B.put("zho", "chi");
fromISO_639_2_T2B.put("ces", "cze");
fromISO_639_2_T2B.put("nld", "dut");
fromISO_639_2_T2B.put("fra", "fre");
fromISO_639_2_T2B.put("kat", "geo");
fromISO_639_2_T2B.put("deu", "ger");
fromISO_639_2_T2B.put("ell", "gre");
fromISO_639_2_T2B.put("isl", "ice");
fromISO_639_2_T2B.put("mkd", "mac");
fromISO_639_2_T2B.put("mri", "mao");
fromISO_639_2_T2B.put("msa", "may");
fromISO_639_2_T2B.put("fas", "per");
fromISO_639_2_T2B.put("ron", "rum");
fromISO_639_2_T2B.put("slk", "slo");
fromISO_639_2_T2B.put("bod", "tib");
fromISO_639_2_T2B.put("cym", "wel");
}
/**
charset
Usage: The character set employed by the text. Controlled vocab using IANA names for character sets.
Attributes: none.
Contains: none.
Contained by: character_info.
*/
private String charset;
/**
byte_order
Usage: Byte order, primarily useful for cases where it's not clear just by specifying an IANA character set.
Uses enumerated values of big, little, and middle' endian.
Attributes: none.
Contains: none.
Contained by: character_info.
*/
private int byte_order = NILL;
/**
byte_size
Usage: The size of an individual byte within the expressed as a number of bits (as integer). This does not necessarily equal the character size, as a character may have more than one, or a variable number of bytes per character.
Attributes: none.
Contains: none.
Contained by: character_info.
*/
private String byte_size;
/**
character_size
Usage: The size of an individual character within the character set as a number of bytes of the size expressed in the byte_size. In the case of variable encodings, such as UTF-8 for Unicode, the character_size element should state "variable" and also identify the specific variable character set encoding in the encoding attribute.
Attributes: encoding.
Contains: none.
Contained by: character_info.
*/
private String character_size;
/**
linebreak
Usage: How line breaks are represented in current file (which may differ from how they were originally encoded). Either carriage return, line feed, or carriage return/line feed.
Attributes: none.
Contains: none.
Contained by: character_info.
*/
private int linebreak = NILL;
/**
language
Usage: Language(s) used in work. Use ISO 639-2 codes, which are enumerated in the schema as valid text values.
Attributes: none.
Contains: none.
Contained by: textMD.
*/
private String language;
/**
markup_basis
Usage: The metalanguage used to create the markup language, such as SGML, XML, GML, etc.
Attributes: version.
Contains: none.
Contained by: textMD.
*/
private String markup_basis;
/**
version
Usage: Used to record the version number (as a string) for a given piece of software, a markup language, or a schema version.
*/
private String markup_basis_version;
/**
markup_language
Usage: Markup language employed on the text (i.e., the specific schema or dtd). May be a URI for schema or dtd, but not mandatory.
Attributes: version.
Contains: none.
Contained by: textMD.
*/
private String markup_language;
/**
version
Usage: Used to record the version number (as a string) for a given piece of software, a markup language, or a schema version.
*/
private String markup_language_version;
/**
* @return the charset
*/
public String getCharset() {
return charset;
}
/**
* @param charset the charset to set
*/
public void setCharset(String charset) {
this.charset = toTextMDCharset(charset);
}
/**
* @return the byte_order
*/
public int getByte_order() {
return byte_order;
}
public String getByte_orderString() {
if (byte_order == NILL) {
return BYTE_ORDER[BYTE_ORDER_BIG]; // default !!!
}
return BYTE_ORDER[byte_order];
}
/**
* @param byte_order the byte_order to set
*/
public void setByte_order(int byte_order) {
this.byte_order = byte_order;
}
/**
* @return the byte_size
*/
public String getByte_size() {
return byte_size;
}
/**
* @param byte_size the byte_size to set
*/
public void setByte_size(String byte_size) {
this.byte_size = byte_size;
}
/**
* @return the character_size
*/
public String getCharacter_size() {
return character_size;
}
/**
* @param character_size the character_size to set
*/
public void setCharacter_size(String character_size) {
this.character_size = character_size;
}
/**
* @return the linebreak
*/
public int getLinebreak() {
return linebreak;
}
/**
* @return the linebreak in String form
*/
public String getLinebreakString() {
if (linebreak == NILL) {
return LINEBREAK[LINEBREAK_CRLF]; // default !!!
}
return LINEBREAK[linebreak];
}
/**
* @param linebreak the linebreak to set
*/
public void setLinebreak(int linebreak) {
this.linebreak = linebreak;
}
/**
* @return the language
*/
public String getLanguage() {
return language;
}
/**
* @param language the language to set
*/
public void setLanguage(String language) {
this.language = toISO_639_2(language);
}
/**
* @return the markup_basis
*/
public String getMarkup_basis() {
return markup_basis;
}
/**
* @param markup_basis the markup_basis to set
*/
public void setMarkup_basis(String markup_basis) {
this.markup_basis = markup_basis;
}
/**
* @return the markup_basis_version
*/
public String getMarkup_basis_version() {
return markup_basis_version;
}
/**
* @param markup_basis_version the markup_basis_version to set
*/
public void setMarkup_basis_version(String markup_basis_version) {
this.markup_basis_version = markup_basis_version;
}
/**
* @return the markup_language
*/
public String getMarkup_language() {
return markup_language;
}
/**
* @param markup_language the markup_language to set
*/
public void setMarkup_language(String markup_language) {
this.markup_language = markup_language;
}
/**
* @return the markup_language_version
*/
public String getMarkup_language_version() {
return markup_language_version;
}
/**
* @param markup_language_version the markup_language_version to set
*/
public void setMarkup_language_version(String markup_language_version) {
this.markup_language_version = markup_language_version;
}
/**
* Transform a given charset in the "authorized" list given in the textMD schema enumeration.
* From the schema documentation on charset (http://www.loc.gov/standards/textMD/elementSet/index.html#element_charset).
* The character set employed by the text. Controlled vocab using IANA names for character sets:
* http://www.iana.org/assignments/character-sets.
* The problem arises because the java Charset uses the (preferred MIME name) where textMD uses the Name ...
* @param srcCharset charset from the file
* @return normalized charset
*/
public static String toTextMDCharset(String srcCharset) {
if (srcCharset == null) return null;
Charset cs = null;
String textMDCharset = null;
try {
cs = Charset.forName(srcCharset);
textMDCharset = cs.name();
} catch (Exception e) {
// Try a unknown one
if (setOfUnknownJavaCharset.contains(srcCharset)) {
textMDCharset = srcCharset;
} else {
// Downgrade to default
textMDCharset = CHARSET_ISO8859_1;
}
}
if (textMDCharset != null) {
return textMDCharset;
} else {
// Downgrade to default
return CHARSET_ISO8859_1;
}
}
/**
* Transform a language to the ISO_639-2 language (only enumeration allowed in textMD schema).
* @param srcLang language in the file
* @return normalized language in 3 letters (except qaa-qtz)
*/
public static String toISO_639_2(String srcLang) {
if (srcLang == null) return null;
if ("qaa-qtz".equals(srcLang)) return srcLang;
String textMDLang = null;
if (srcLang.length() == 3) {
textMDLang = srcLang;
}
else if (srcLang.length() == 2) {
try {
Locale loc = new Locale(srcLang);
textMDLang = loc.getISO3Language();
} catch (Exception e) {
// Unknown language
}
}
else if (srcLang.length() > 3) {
// Just try with the first 2 characters
try {
Locale loc = new Locale(srcLang.substring(0, 2));
srcLang = loc.getISO3Language();
} catch (Exception e) {
// Unknown language
}
}
if (textMDLang != null && textMDLang.length() == 3) {
// From ISO 639-2/T to ISO 639-2/B
if (fromISO_639_2_T2B.containsKey(textMDLang)) {
textMDLang = (String)fromISO_639_2_T2B.get(textMDLang);
}
return textMDLang;
}
return null;
}
}
@@ -1,62 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import edu.harvard.hul.ois.jhove.module.Utf8Block;
import java.util.*;
/**
*
* @author Gary McGath
*
*/
public class Utf8BlockMarker {
private boolean[] blocksUsed;
public Utf8BlockMarker () {
blocksUsed = new boolean [Utf8Block.unicodeBlock.length];
}
public void markBlock (int code) {
for (int i=0; i<Utf8Block.unicodeBlock.length; i++) {
if (Utf8Block.unicodeBlock[i].getStart () <= code &&
Utf8Block.unicodeBlock[i].getEnd () >= code) {
blocksUsed[i] = true;
break;
}
}
}
/** Returns a Property listing the blocks that have been
* marked as used. If no blocks have been marked,
* returns null. */
public Property getBlocksUsedProperty (String name)
{
List<String> block = new ArrayList<String> (blocksUsed.length);
for (int i=0; i<blocksUsed.length; i++) {
if (blocksUsed[i]) {
block.add (Utf8Block.unicodeBlock[i].getName ());
}
}
if (block.isEmpty ()) {
return null;
}
else {
return new Property (name,
PropertyType.STRING,
PropertyArity.LIST,
block);
}
}
/** Clears all marked blocks. */
public void reset ()
{
for (int i=0; i<Utf8Block.unicodeBlock.length; i++) {
blocksUsed[i] = false;
}
}
}
@@ -1,161 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
*
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import java.io.IOException;
import java.io.InputStream;
/**
* This class serves to provide an InputStream for an XML
* fragment (e.g., embedded XMP metadata).
*
* @author Gary McGath
*/
public class XMLWrapperStream extends InputStream
{
private String xmlDecl;
private String rootStart;
private String rootEnd;
private int strIndex;
private int state;
/* Values for state */
private final static int DECL = 0,
ROOT_START = 1,
CONTENT = 2,
ROOT_END = 3;
private InputStream _wrappedStream;
/**
* Constructor.
*
* @param wrappedStream The stream which this stream will subsume.
* @param rootName The name of the root element. May be null
* if <code>wrappedStream</code> already contains a root element.
* @param version The XML version. Should be null or "1.0" unless
* there's a really good reason.
* @param encoding The name of the character encoding. May be null.
* @param standalone The value of the <code>standalone</code> attribute. May be null.
*/
public XMLWrapperStream (InputStream wrappedStream,
String rootName,
String version,
String encoding,
String standalone)
{
_wrappedStream = wrappedStream;
xmlDecl = "<?xml version='";
if (version == null) {
xmlDecl += "1.0'";
}
else {
xmlDecl += version + "'";
}
if (encoding != null) {
xmlDecl += " encoding='" + encoding + "' ";
}
if (standalone != null) {
xmlDecl += " standalone='" + standalone + "' ";
}
xmlDecl += "?>";
state = DECL;
strIndex = 0;
if (rootName != null) {
rootStart = "<" + rootName + ">";
rootEnd = "</" + rootName + ">";
}
else {
rootStart = "";
rootEnd = "";
}
}
/**
* Constructor. Equivalent to
* <code>XMLWrapperStream (wrappedStream, null, null, null, null)</code>
*
* @param wrappedStream The stream which this stream will subsume.
*/
public XMLWrapperStream (InputStream wrappedStream)
{
this (wrappedStream, null, null, null, null);
}
/**
* Constructor. Equivalent to
* <code>XMLWrapperStream (wrappedStream, rootName, null, null, null)</code>
*
* @param wrappedStream The stream which this stream will subsume.
* @param rootName The name of the root element. May be null.
*/
public XMLWrapperStream (InputStream wrappedStream, String rootName)
{
this (wrappedStream, rootName, null, null, null);
}
/**
* Get a byte. Successive calls will return the
* XML declaration, then the wrapped stream.
*
* @see java.io.InputStream#read()
*/
public int read() throws IOException
{
int retval;
if (state == DECL) {
if (strIndex >= xmlDecl.length()) {
// We have finished the declaration string now.
state = ROOT_START;
strIndex = 0;
}
else {
// We haven't finished returning the declaration string.
return (int) xmlDecl.charAt(strIndex++);
}
}
// Each state can fall through to the next
if (state == ROOT_START) {
if (strIndex >= rootStart.length()) {
// We have finished the root element start now.
state = CONTENT;
}
else {
return (int) rootStart.charAt(strIndex++);
}
}
if (state == CONTENT) {
// Metadata alleged stream doesn't look remotely like metadata --
// probably looking at wrong part of file!
retval = _wrappedStream.read ();
if (retval == -1) {
state = ROOT_END;
strIndex = 0;
}
else {
return retval;
}
}
// Must be ROOT_END if it gets here
if (strIndex >= rootEnd.length()) {
// We have finished the root element end and the document now.
return -1;
}
else {
return (int) rootEnd.charAt(strIndex++);
}
}
}
@@ -1,175 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
import org.xml.sax.*;
/**
* This class encapsulates XMP metadata within a file. It makes use
* of an InputStream as a data source.
*
* This differs from normal XML handling in that it's necessary to
* process the xpacket processing instruction in order to determine
* the encoding of the XML. the processingInstruction function looks
* for xpacket, and throws a special SAXException if it's necessary
* to change encoding.
*
* We don't actually extract any information from the XMP, but
* simply check it for well-formedness. By convention, XMPHandler
* should be invoked on an XMPSource (TBW), which provides the
* ability to recapture the InputStream from which the XMP was
* obtained and put it into a property once it's verified here.
*
*/
public class XMPHandler extends org.xml.sax.helpers.DefaultHandler {
/* URI strings */
private final static String xmpBasicSchema =
"http://ns.adobe.com/xap/1.0/";
// private final static String xmpRightsSchema =
// "http://ns.adobe.com/xap/1.0/rights/";
// private final static String dublinCoreSchema =
// "http://purl.org/dc/elements/1.1/";
// private final static String adobePDFSchema =
// "http://ns.adobe.com/pdf/1.3/";
// private final static String photoshopSchema =
// "http://ns.adobe.com/photoshop/1.0/";
private int curStructType;
/* Values which may be assigned to curStructType */
private final static int
UNASSIGNED = 0,
BAG = 1,
ALT = 2,
SEQ = 3;
private boolean pdfaCompliant;
public XMPHandler ()
{
super ();
pdfaCompliant = true; // compliance is presumed till disproven
}
/** Returns true if no violations of PDF/A compliance have been found,
* false if a problem was detected. */
public boolean isPdfaCompliant () {
return pdfaCompliant;
}
public void processingInstruction (String target, String data)
throws SAXException
{
if ("xpacket".equals (target)) {
// We assume that the data will be non-endian (i.e., simply
// a stream of bytes) unless we find a valid endian code.
boolean bigEndian = false;
boolean noEndian = true;
// a Processing Instruction can't really have attributes,
// so we have to parse the data string ourselves. The order
// of the attributes is guaranteed, fortunately.
int idx = data.indexOf ("begin=");
idx = data.indexOf ('"', idx + 1);
if (data.length () >= idx + 2) {
int char1 = (int) data.charAt (idx + 1);
int char2 = (int) data.charAt (idx + 2);
if (char1 == 0XFF && char2 == 0XFE) {
noEndian = false;
bigEndian = false;
}
else if (char1 == 0XFE && char2 == 0XFF) {
noEndian = false;
bigEndian = true;
}
// EF BB B8 signifies UTF-8, but that's the default anyway.
}
// Check the bytes attribute. We don't do anything with it except
// note that it isn't allowed with PDF/A.
idx = data.indexOf("bytes=");
if (idx > 0) {
pdfaCompliant = false;
}
// Next find encoding, which is optional.
idx = data.indexOf ("encoding=");
if (idx > 0) {
pdfaCompliant = false; // not allowed in PDF/A
idx = data.indexOf ('"', idx + 1);
int encEnd = data.indexOf ('"', idx + 1 );
String encoding = data.substring (idx + 1, encEnd);
// Throw a SAXException which consists of
// "ENC=<endian>,<enc>", where
// endian is either 'B' (big), 'L' (little) or space (none), and
// enc is the encoding attribute.
// This is an expected exception, not an error.
String exText = "ENC=";
if (noEndian) {
exText += " ,";
}
else if (bigEndian) {
exText += "B,";
}
else {
exText += "L,";
}
exText += encoding;
throw new SAXException (exText);
}
}
}
/**
* Catches the start of an element and, if it's one we care
* about, sets state information.
*/
public void startElement (String namespaceURI, String localName,
String rawName, Attributes atts)
throws SAXException
{
//System.out.println (namespaceURI); // Just for debugging and placeholding
if (xmpBasicSchema.equals (namespaceURI)) {
if ("Bag".equals (rawName)) {
curStructType = BAG;
}
else if ("Seq".equals (rawName)) {
curStructType = SEQ;
}
else if ("Alt".equals (rawName)) {
curStructType = ALT;
}
}
}
/**
* Catches the end of an element.
*/
public void endElement (String namespaceURI, String localName,
String rawName)
throws SAXException
{
if (xmpBasicSchema.equals (namespaceURI)) {
// Check for the end of an XMP structure
if ("Bag".equals (rawName)||
"Seq".equals (rawName) ||
"Alt".equals (rawName)) {
curStructType = UNASSIGNED;
}
}
}
/** Catch a fatal error. This is put here because the default
* behavior is to report a "fatal error" to standard output,
* which is harmless but scary.
*/
public void fatalError(SAXParseException exception)
throws SAXException
{
}
}
@@ -1,118 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove;
//import java.io.InputStream;
import java.io.Reader;
import org.xml.sax.InputSource;
import java.io.IOException;
/**
* Class for providing an InputSource to XMPHandler.
* Only an InputSource based on a Reader is supported.
*
* Each module can provide its own subclass of XMPSource.
* The subclass must provide a way to
* reset to the beginning and reread the data when makeProperty
* is called.
*
* @author Gary McGath
*
*/
public abstract class XMPSource extends InputSource {
/** The Reader on which the InputSource is based. */
protected Reader _reader;
/**
* Constructor with Reader.
*/
public XMPSource(Reader rdr) {
super(rdr);
_reader = rdr;
}
/**
* Generates a property from the underlying data.
* The beginning and ending processing instructions are
* stripped out.
*/
public Property makeProperty () throws IOException
{
boolean maybePI = false; // set to true for ?xpacket partial match
boolean seenStart = false; // set to true after initial xpacket
resetReader (); // go back to the beginning
StringBuffer textBuf = new StringBuffer ();
StringBuffer xpacBuf = new StringBuffer ();
for (;;) {
int ch = _reader.read();
if (ch < 0) {
break;
}
if (maybePI) {
xpacBuf.append((char) ch);
if ("<?xpacket".equals (xpacBuf.toString ())) {
if (!seenStart) {
// This is the starting xpacket. Clear away
// everything to the closing ?>
seenStart = true;
xpacBuf.setLength (0);
maybePI = false;
int prevCh = 0;
for (;;) {
ch = _reader.read ();
if (ch < 0 ||
(prevCh == (int) '?' && ch == (int) '>')) {
break;
}
prevCh = ch;
}
}
else {
// This is the ending xpacket. Discard it.
// We're done.
break;
}
}
if (!"<?xpacket".startsWith(xpacBuf.toString ())) {
// Not an xpacket -- clear xpacBuf into the prop string
maybePI = false;
textBuf.append (xpacBuf);
xpacBuf.setLength (0);
}
}
else {
if (ch == '<') {
// This could be the start of an <?xpacket?>.
// Start buffering.
maybePI = true;
xpacBuf.append ((char) ch);
}
else {
// Just plain text. Append it.
textBuf.append ((char) ch);
}
}
}
// Some XMP's end with lots of white space, so give
// it a trim before returning.
return new Property ("XMP",
PropertyType.STRING,
textBuf.toString ().trim ());
}
/**
* Causes reading to begin from the start again.
* Typically this means creating a new value for
* _reader that will start over.
*/
protected abstract void resetReader ();
}
@@ -1,79 +0,0 @@
<project name="Jhove" default="api-dist" basedir=".">
<description>API build file
Jhove - JSTOR/Harvard Object Validation Environment
Version 1.0 2004-02-03
Copyright 2004 by JSTOR and the President and Fellows of Harvard College
</description>
<!-- set global properties for this build -->
<property name="root" location="../../../../.."/>
<property name="bin" location="${root}/../bin"/>
<property name="classpath" value="${root}"/>
<property name="jar" location="${bin}/jhove.jar"/>
<property name="manifest" location="META-INF"/>
<target name="api-init" description="Create the time stamp">
<tstamp>
<format property="timestamp.isoformat" pattern="yyyy-MM-dd'T'HH:mm:ss"/>
</tstamp>
</target>
<target name="api" depends="api-init" description="Compile the API">
<javac compiler="javac1.5" target="1.5" srcdir="." debug="${dbg}" classpath="${classpath}"/>
</target>
<target name="api-manifest" depends="api">
<mkdir dir="${manifest}"/>
<manifest file="${manifest}/MANIFEST.MF">
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Built-On" value="${timestamp.isoformat}"/>
<section name="JHOVE">
<attribute name="Version" value="1.0"/>
<attribute name="Package" value="Framework API"/>
<attribute name="Rights" value="Copyright 2004 by JSTOR and the President and Fellows of Harvard College"/>
</section>
</manifest>
</target>
<target name="api-dist" depends="api-manifest,module,handler"
description="Create the distribution">
<mkdir dir="${bin}"/>
<jar destfile="${jar}" basedir="${root}" index="true"
manifest="${manifest}/MANIFEST.MF"
includes="edu/harvard/hul/ois/jhove/*.class"/>
</target>
<target name="module">
<ant dir="module" target="module-dist" inheritAll="false">
<property name="dbg" value="${dbg}"/>
</ant>
</target>
<target name="handler">
<ant dir="handler" target="handler-dist" inheritAll="false">
<property name="dbg" value="${dbg}"/>
</ant>
</target>
<target name="api-clean" description="Delete backup files">
<delete>
<fileset dir="." includes="*~ \#*"/>
</delete>
<ant dir="handler" target="handler-clean" inheritAll="false"/>
<ant dir="module" target="module-clean" inheritAll="false"/>
<delete>
<fileset dir="./viewer" includes="*~ \#*"/>
</delete>
</target>
<target name="api-cleanclass" description="Delete backup and class files">
<delete>
<fileset dir="." includes="*.class*"/>
</delete>
<ant dir="handler" target="handler-cleanclass" inheritAll="false"/>
<ant dir="module" target="module-cleanclass" inheritAll="false"/>
<delete>
<fileset dir="./viewer" includes="*.class*"/>
</delete>
</target>
</project>
@@ -1,438 +0,0 @@
/**********************************************************************
* Audit output handler
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.handler;
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.handler.audit.*;
import java.util.*;
/**
* JHOVE audit output handler, derived from the standard JHOVE XML
* handler. It is expected that this class will be used as the parent for
* other, more interesting output handlers. Subclasses should override the
* implementations of the Impl methods, e.g., endDirectoryImpl ().
* @see <a href="http://hul.harvard.edu/jhove/using.html#xml-hul">JHOVE
* XML output handler</a>
*/
public class AuditHandler
extends XmlHandler
{
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
/** Audit output handler name. */
private static final String NAME = "Audit";
/** Audit output handler release ID. */
private static final String RELEASE = "1.1";
/** Audit output handler release date. */
private static final int [] DATE = {2005, 04, 22};
/** Audit output handler informative note. */
private static final String NOTE =
"This output handler is derived from the standard JHOVE XML output " +
"handler. It is intended to be used as the parent class for other, " +
"more interesting handlers.";
/** Audit output handler rights statement. */
private static final String RIGHTS =
"Copyright 2004-2005 by the President and Fellows of Harvard College. " +
"Released under the GNU LGPL license";
/*****************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Home directory of the audit. */
protected String _home;
/** Number of files processed by MIME type. */
protected Map _mimeType;
/** State map. */
protected Map _stateMap;
/** State stack. */
protected Stack _stateStack;
/** Initial time. */
protected long _t0;
/** Number of files audited. */
protected int _nAudit;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate a <tt>AudiHandler</tt> object.
*/
public AuditHandler ()
{
super (NAME, RELEASE, DATE, NOTE, RIGHTS);
/* Define the standard output handler properties. */
_name = NAME;
_release = RELEASE;
Calendar calendar = new GregorianCalendar ();
calendar.set (DATE[0], DATE[1]-1, DATE[2]);
_date = calendar.getTime ();
_note = NOTE;
_rights = RIGHTS;
/* Initialize the handler. */
_mimeType = new TreeMap ();
_stateMap = new TreeMap ();
_stateStack = new Stack ();
_nAudit = 0;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Callback indicating a directory is finished being processed.
* Prop the state stack and place the current directory file count
* into the directory hash.
*/
public final void endDirectory ()
{
AuditState state = (AuditState) _stateStack.pop ();
_stateMap.put (state.getDirectory (), state);
endDirectoryImpl (state);
}
/**
* Local extension to the standard callback indicating a directory is
* finished being processed.
* @param state Audit handler state
*/
public void endDirectoryImpl (AuditState state)
{
}
/**
* Determine whether or not to process the file.
* @param filepath File pathname
*/
public final boolean okToProcess (String filepath)
{
AuditState state = (AuditState) _stateStack.peek ();
boolean ok = okToProcessImpl (filepath, state);
if (!ok) {
state.setNotProcessed (state.getNotProcessed () + 1);
}
return ok;
}
/**
* Local extension to standard callback that determines whether or not
* to process the file.
* @param filepath File pathname
* @param state Audit handler state
*/
public boolean okToProcessImpl (String filepath, AuditState state)
{
return true;
}
/**
* Outputs the information contained in a RepInfo object
* @param info Object representation information
*/
public void show (RepInfo info)
{
AuditState state = (AuditState) _stateStack.peek ();
/* If the file is not found, then no module is assigned in the
* RepInfo object.
*/
if (info.getModule () == null) {
state.setNotFound (state.getNotFound () + 1);
_writer.println ("<!-- file not found or not readable: " +
info.getUri () + " -->");
}
else {
String mime = info.getMimeType ();
AuditCount count = (AuditCount) _mimeType.get (mime);
if (count == null) {
count = new AuditCount ();
}
int valid = info.getValid ();
if (valid == RepInfo.TRUE) {
state.setValid (state.getValid () + 1);
count.setValid (count.getValid () + 1);
}
else {
state.setWellFormed (state.getWellFormed () + 1);
count.setWellFormed (count.getWellFormed () + 1);
}
_mimeType.put (mime, count);
}
showImpl (info, state);
}
/**
* Local extension to the standard callback that outputs the
* information contained in a RepInfo object
* @param info Object representation information
* @param state Audit handler state
*/
public void showImpl (RepInfo info, AuditState state)
{
String status = null;
String mime = info.getMimeType ();
if (mime != null) {
if (info.getWellFormed () == RepInfo.TRUE) {
if (info.getValid () == RepInfo.TRUE) {
status = "valid";
}
else {
status = "well-formed";
}
}
else {
status = "not well-formed";
}
}
else {
status = "not found";
}
/* Retrieve the MD5 checksum, if available. */
String md5 = null;
List list = info.getChecksum ();
int len = list.size ();
for (int i=0; i<len; i++) {
Checksum checksum = (Checksum) list.get (i);
if (checksum.getType ().equals (ChecksumType.MD5)) {
md5 = checksum.getValue ();
break;
}
}
if (_nAudit == 0) {
String margin = getIndent (++_level);
String [][] attrs = { {"home", _home} };
_writer.println (margin + elementStart ("audit", attrs));
}
String margn2 = getIndent (_level) + " ";
String uri = info.getUri ();
/* Change the URI to a relative path by removing the home
* directory prefix.
*/
int n = uri.indexOf (_home);
if (n == 0) {
uri = uri.substring (_home.length () + 1);
}
String [][] attrs = { {"mime", mime }, {"status", status},
{"md5", md5} };
_writer.println (margn2 + element ("file", attrs, uri));
_nAudit++;
}
/**
* Do the final output. This should be in a suitable format for
* including multiple files between the header and the footer, and
* the XML of the header and footer must balance out.
*/
public void showFooter ()
{
AuditState state = (AuditState) _stateStack.pop ();
if (state.getTotal () > 0 || state.getNotFound () > 0) {
_stateMap.put (state.getDirectory (), state);
}
showFooterImpl (state);
// super.showFooter ();
_writer.println ("<!-- Summary by MIME type:");
int nTotal = 0;
int nValid = 0;
int nWellFormed = 0;
Set keys = _mimeType.keySet ();
Iterator iter = keys.iterator ();
while (iter.hasNext ()) {
String mime = (String) iter.next ();
AuditCount count = (AuditCount) _mimeType.get (mime);
int total = count.getTotal ();
int valid = count.getValid ();
int wellFormed = count.getWellFormed ();
_writer.println (mime + ": " + total + " (" + valid + "," +
wellFormed + ")");
nTotal += total;
nValid += valid;
nWellFormed += wellFormed;
}
_writer.println ("Total: " + nTotal + " (" + nValid + "," +
nWellFormed + ")");
_writer.println ("-->");
_writer.println ("<!-- Summary by directory:");
nTotal = 0;
nValid = 0;
nWellFormed = 0;
int nNotFound = 0;
int nNotProcessed = 0;
keys = _stateMap.keySet ();
iter = keys.iterator ();
while (iter.hasNext ()) {
String directory = (String) iter.next ();
state = (AuditState) _stateMap.get (directory);
int total = state.getTotal ();
int valid = state.getValid ();
int wellFormed = state.getWellFormed ();
int notFound = state.getNotFound ();
int notProcessed = state.getNotProcessed ();
_writer.println (directory + ": " + total + " (" + valid + "," +
wellFormed + ") + " + notProcessed + "," +
notFound);
nTotal += total;
nValid += valid;
nWellFormed += wellFormed;
nNotFound += notFound;
nNotProcessed += notProcessed;
}
_writer.println ("Total: " + nTotal + " (" + nValid + "," +
nWellFormed + ") + " + nNotProcessed + "," +
nNotFound);
_writer.println ("-->");
/* Update the elapsed time. */
long dt = (System.currentTimeMillis () - _t0 + 999) / 1000;
long ss = dt % 60;
long dm = dt / 60;
long mm = dm % 60;
long hh = dm / 60;
_writer.println ("<!-- Elapsed time: " + hh + ":" +
(mm > 9 ? "" : "0") + mm + ":" +
(ss > 9 ? "" : "0") + ss + " -->");
_writer.flush ();
}
/**
* Local extension to the standard callback that does the final output.
* This should be in a suitable format for
* including multiple files between the header and the footer, and
* the XML of the header and footer must balance out.
* @param state Audit handler state
*/
public void showFooterImpl (AuditState state)
{
if (_nAudit > 0) {
String margin = getIndent (_level--);
_writer.println (margin + elementEnd ("audit"));
}
super.showFooter ();
}
/**
* Do the initial output. This should be in a suitable format for
* including multiple files between the header and the footer, and
* the XML of the header and footer must balance out.
*/
public void showHeader ()
{
/* Initialize the handler. */
_mimeType = new TreeMap ();
_stateMap = new TreeMap ();
_stateStack = new Stack ();
_nAudit = 0;
_t0 = System.currentTimeMillis ();
/* Instantiate a state object and initialize with the values
* of the global configuration file.
*/
AuditState state = showHeaderImpl (".");
_stateStack.push (state);
_home = state.getDirectory ();
}
/**
* Local extension to the standard callback that does the initial output.
* This should be in a suitable format for including multiple files
* between the header and the footer, and the XML of the header and footer
* must balance out.
* @param directory Current directory filepath
*/
public AuditState showHeaderImpl (String directory)
{
super.showHeader ();
return new AuditState (directory);
}
/**
* Callback indicating a new directory is being processed.
*
* Additional state information can be added to the AuditState object
* in the showHeaderImpl() method before it is pushed onto the stack.
*/
public void startDirectory (String directory)
{
try {
AuditState state = (AuditState)
((AuditState) _stateStack.peek ()).clone (directory);
startDirectoryImpl (state);
_stateStack.push (state);
}
catch (CloneNotSupportedException e) {
e.printStackTrace (System.err);
System.exit (-1);
}
}
/**
* Local extension to the standard callback indicating a new directory
* is being processed.
* @param state Audit handler state
*/
public void startDirectoryImpl (AuditState state)
{
}
}
@@ -1,12 +0,0 @@
Manifest-Version: 1.0
Ant-Version: Apache Ant 1.8.3
Created-By: 1.6.0_37-b06-434-10M3909 (Apple Inc.)
Built-By: gmcgath
Built-On: 2012-12-09T08:40:55
Name: JHOVE
Version: 1.0
Package: Handlers
Rights: Copyright 2004 by JSTOR and the President and Fellows of Harva
rd College
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,109 +0,0 @@
/**********************************************************************
* Audit output handler
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.handler.audit;
//import java.io.*;
/**
* Count object for the JHOVE Audit output handler.
*/
public class AuditCount
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/** Number of files not processed. */
protected int _notProcessed;
/** Number of valid files. */
protected int _valid;
/** Number of well-formed files. */
protected int _wellFormed;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate a <tt>AuditCount</tt> object.
*/
public AuditCount ()
{
_notProcessed = 0;
_valid = 0;
_wellFormed = 0;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Accessor methods.
******************************************************************/
/** Returns the total number of files not processed. */
public int getNotProcessed ()
{
return _notProcessed;
}
/** Returns the total number of valid or well-formed
files. */
public int getTotal ()
{
return _valid + _wellFormed;
}
/** Returns the total number of valid files. */
public int getValid ()
{
return _valid;
}
/** Returns the total number of well-formed files. */
public int getWellFormed ()
{
return _wellFormed;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/** Sets the count of files that are not processed. */
public void setNotProcessed (int notProcessed)
{
_notProcessed = notProcessed;
}
/** Sets the count of valid files. */
public void setValid (int valid)
{
_valid = valid;
}
/** Sets the count of well-formed files. */
public void setWellFormed (int wellFormed)
{
_wellFormed = wellFormed;
}
}
@@ -1,126 +0,0 @@
/**********************************************************************
* Audit output handler
* Copyright 2004 by the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.handler.audit;
import java.io.*;
/**
* State object for the JHOVE Audit output handler.
*/
public class AuditState
extends AuditCount
implements Cloneable
{
/******************************************************************
* PROTECTED INSTANCE FIELDS.
******************************************************************/
/** Directory pathname. */
protected String _directory;
/** Number of files not found. */
protected int _notFound;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate a <tt>AuditState</tt> object.
*/
public AuditState (String directory)
{
super ();
init (directory);
}
/**
* Initializes to a specified directory and clears counters.
*/
protected void init (String directory)
{
try {
File file = new File (directory);
_directory = file.getCanonicalPath ();
}
catch (Exception e) {
_directory = directory;
}
_notFound = 0;
_valid = 0;
_wellFormed = 0;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
******************************************************************/
/**
* Creates and returns a copy of this object.
*/
public Object clone (String directory)
throws CloneNotSupportedException
{
AuditState state = (AuditState) super.clone ();
state.init (directory);
return state;
}
/******************************************************************
* Accessor methods.
******************************************************************/
/** Returns the directory path. */
public String getDirectory ()
{
return _directory;
}
/** Returns the number of files not found. */
public int getNotFound ()
{
return _notFound;
}
/******************************************************************
* Mutator methods.
******************************************************************/
/** Sets the directory path. */
public void setDirectory (String directory)
{
try {
File file = new File (directory);
_directory = file.getCanonicalPath ();
}
catch (Exception e) {
_directory = directory;
}
}
/** Sets the count of files not found. */
public void setNotFound (int notFound)
{
_notFound = notFound;
}
}
@@ -1,32 +0,0 @@
<project name="Jhove" basedir=".">
<description>Audit output handler build file
Jhove - JSTOR/Harvard Object Validation Environment
Version 1.0 2004-12-09
Copyright 2004 by JSTOR and the President and Fellows of Harvard College
</description>
<!-- set global properties for this build -->
<property name="root" location="../../../../../../.."/>
<property name="bin" location="${root}/../bin"/>
<property name="classpath" value="${bin}/jhove.jar:${root}"/>
<target name="audit-init" description="Create the time stamp">
<tstamp/>
</target>
<target name="audit" depends="audit-init" description="Compile the Audit output handler">
<javac compiler="javac1.5" target="1.5" srcdir="." debug="{$dbg}" classpath="${classpath}"/>
</target>
<target name="audit-clean" description="Delete backup files" >
<delete>
<fileset dir="." includes="*~ \#*"/>
</delete>
</target>
<target name="audit-cleanclass" description="Delete backup and class files">
<delete>
<fileset dir="." includes="*.class*"/>
</delete>
</target>
</project>
@@ -1,24 +0,0 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<!--
Jhove - JSTOR/Harvard Object Validation Environment
Copyright 2004 by JSTOR and the President and Fellows of Harvard College
-->
</head>
<body bgcolor="white">
Contains the supporting classes for the JHOVE audit output handler.
<br>
<!-- Put @see and @since tags down here. -->
</body>
</html>
@@ -1,65 +0,0 @@
<project name="Jhove" default="handler-dist" basedir=".">
<description>Handler build file
Jhove - JSTOR/Harvard Object Validation Environment
Version 1.0 2004-02-03
Copyright 2004 by JSTOR and the President and Fellows of Harvard College
</description>
<!-- set global properties for this build -->
<property name="root" location="../../../../../.."/>
<property name="bin" location="${root}/../bin"/>
<property name="classpath" value="${bin}/jhove.jar:${root}"/>
<property name="jar" location="${bin}/jhove-handler.jar"/>
<property name="manifest" location="META-INF"/>
<target name="handler-init" description="Create the time stamp">
<tstamp>
<format property="timestamp.isoformat" pattern="yyyy-MM-dd'T'HH:mm:ss"/>
</tstamp>
</target>
<target name="handler" depends="handler-init"
description="Compile the handlers">
<javac compiler="javac1.5" target="1.5" srcdir="." debug="${dbg}" classpath="${classpath}"/>
<ant dir="audit" target="audit" inheritAll="false">
<property name="dbg" value="${dbg}"/>
</ant>
</target>
<target name="handler-manifest" depends="handler">
<mkdir dir="${manifest}"/>
<manifest file="${manifest}/MANIFEST.MF">
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Built-On" value="${timestamp.isoformat}"/>
<section name="JHOVE">
<attribute name="Version" value="1.0"/>
<attribute name="Package" value="Handlers"/>
<attribute name="Rights" value="Copyright 2004 by JSTOR and the President and Fellows of Harvard College"/>
</section>
</manifest>
</target>
<target name="handler-dist" depends="handler-manifest"
description="Create the distribution">
<mkdir dir="${bin}"/>
<jar destfile="${jar}" basedir="${root}" index="true"
manifest="${manifest}/MANIFEST.MF"
includes="edu/harvard/hul/ois/jhove/handler/*.class,
edu/harvard/hul/ois/jhove/handler/audit/*.class"/>
</target>
<target name="handler-clean" description="Delete backup files">
<delete>
<fileset dir="." includes="*~ \#*"/>
</delete>
<ant dir="audit" target="audit-clean" inheritAll="false"/>
</target>
<target name="handler-cleanclass"
description="Delete backup and class files">
<delete>
<fileset dir="." includes="*.class*"/>
</delete>
<ant dir="audit" target="audit-cleanclass" inheritAll="false"/>
</target>
</project>
@@ -1,25 +0,0 @@
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<!--
Jhove - JSTOR/Harvard Object Validation Environment
Copyright 2003 by JSTOR and the President and Fellows of Harvard College
-->
</head>
<body bgcolor="white">
Contains the main classes for JHOVE output handlers.
All module classes are subclasses of <code>HandlerBase</code>.
<br>
<!-- Put @see and @since tags down here. -->
</body>
</html>
@@ -1,118 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- JHOVE - JSTOR/Harvard Object Validation Environment -->
<!-- Copyright 2004-2007 by JSTOR and the President and Fellows of Harvard College -->
<!-- JHOVE configuration file schema, Rel. 1.4 (2007-04-10) -->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
targetNamespace="http://hul.harvard.edu/ois/xml/ns/jhove/jhoveConfig"
elementFormDefault="qualified" attributeFormDefault="unqualified">
<xs:element name="jhoveConfig">
<xs:annotation>
<xs:documentation>The JHOVE configuration file consists of the JHOVE home directory, optional temporary directory and buffer size, an unbounded, ordered set of modules, followed by an optional unbounded set of output handlers. It has a single required attribute, version, which specifies the schema version to which a configuration file conforms.</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="jhoveHome" type="xs:string">
<xs:annotation>
<xs:documentation>JHOVE home directory.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="defaultEncoding" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>Default character encoding used by the output handlers.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="tempDirectory" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>Optional directory in which temporary files are created. Default value is the current working directory.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="bufferSize" type="xs:nonNegativeInteger" minOccurs="0">
<xs:annotation>
<xs:documentation>Optional buffer size used for buffered I/O. Defeault value is 131072.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="mixVersion" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>The version of the MIX schema to be used in metadata output. Default value is "0.2".</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="sigBytes" type="xs:nonNegativeInteger" minOccurs="0">
<xs:annotation>
<xs:documentation>The number of initial bytes to search when matching internal signatures.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="module" maxOccurs="unbounded">
<xs:annotation>
<xs:documentation>A module consists of a fully package qualified class name, followed by an optional initialization parameter.</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="class" type="xs:string">
<xs:annotation>
<xs:documentation>A class consists of the fully package qualified class name for the module. Note that the class must be found on the Java CLASSPATH at the time that JHOVE is invoked.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="init" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>A module initialization value is passed to the module once at the time it is first instantiated.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="param" type="xs:string" minOccurs="0" maxOccurs="unbounded">
<xs:annotation>
<xs:documentation>A module parameter value that is passed to the module immediately prior to each invocation of the parse() method.</xs:documentation>
</xs:annotation>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="outputHandler" minOccurs="0" maxOccurs="unbounded">
<xs:annotation>
<xs:documentation>An output handler consists of a fully package qualified class name.</xs:documentation>
</xs:annotation>
<xs:complexType>
<xs:sequence>
<xs:element name="class" type="xs:string">
<xs:annotation>
<xs:documentation>A class consists of the fully package qualified class name for the output handler. Note that the class must be found on the Java CLASSPATH at the time that JHOVE is invoked.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="init" type="xs:string" minOccurs="0">
<xs:annotation>
<xs:documentation>A handler initialization value that is passed to the class at the time it is first instantiated.</xs:documentation>
</xs:annotation>
</xs:element>
<xs:element name="param" type="xs:string" minOccurs="0" maxOccurs="unbounded">
<xs:annotation>
<xs:documentation>A handler parameter value that is passed to the module immediately prior to each invocation of the showHeader() method.</xs:documentation>
</xs:annotation>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="logLevel" minOccurs="0">
<xs:annotation>
<xs:documentation>Logging level.</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="OFF"/>
<xs:enumeration value="CONFIG"/>
<xs:enumeration value="INFO"/>
<xs:enumeration value="WARNING"/>
<xs:enumeration value="SEVERE"/>
<xs:enumeration value="FINE"/>
<xs:enumeration value="FINER"/>
<xs:enumeration value="FINEST"/>
<xs:enumeration value="ALL"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
<xs:attribute name="version" type="xs:string" use="required">
<xs:annotation>
<xs:documentation>The version attribute specifies the schema version to which a configuration file conforms.</xs:documentation>
</xs:annotation>
</xs:attribute>
</xs:complexType>
</xs:element>
</xs:schema>
@@ -1,763 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2007 by JSTOR and the President and Fellows of Harvard College
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.aiff.*;
import edu.harvard.hul.ois.jhove.module.iff.*;
import java.io.*;
import java.util.*;
/**
* Module for identification and validation of AIFF files.
* Supports AIFF and AIFF-C.
*
* @author Gary McGath
*/
public class AiffModule
extends ModuleBase
{
/******************************************************************
* PRIVATE Instance FIELDS.
******************************************************************/
/* Checksummer object */
protected Checksummer _ckSummer;
/* Input stream wrapper which handles checksums */
protected ChecksumInputStream _cstream;
/* Data input stream wrapped around _cstream */
protected DataInputStream _dstream;
/* Top-level metadata property */
protected Property _metadata;
/* Top-level property list */
protected List _propList;
/* AES audio metadata to go into AIFF metadata */
protected AESAudioMetadata _aesMetadata;
/* List of Annotation Chunk properties */
protected List _annotationList;
/* List of MIDI Chunk properties */
protected List _midiList;
/* List of Saxel properties */
protected List _saxelList;
/* Bytes remaining to be read. */
protected long bytesRemaining;
/* Flag to check for multiple sound chunks */
protected boolean soundChunkSeen;
/* Flag to check for exactly one format version chunk */
protected boolean formatVersionChunkSeen;
/* Flag to check for exactly one instrument chunk */
protected boolean instrumentChunkSeen;
/* Flag to check for exactly one common chunk */
protected boolean commonChunkSeen;
/* Flag to check for exactly one comments chunk */
protected boolean commentsChunkSeen;
/* Flag to check for exactly one name chunk */
protected boolean nameChunkSeen;
/* Flag to check for exactly one author chunk */
protected boolean authorChunkSeen;
/* Flag to check for exactly one copyright chunk */
protected boolean copyrightChunkSeen;
/* Flag to check for exactly one marker chunk */
protected boolean markerChunkSeen;
/* Flag to check for exactly one audio recording chunk */
protected boolean audioRecChunkSeen;
/* Flag to note that first sample offset has been recorded */
protected boolean firstSampleOffsetMarked;
/* File type */
protected int fileType;
/* Endianness for current file (not necessarily same as _bigEndian) */
protected boolean thisFileBigEndian;
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
/* Chunk data orientation is big-endian. */
private static final boolean BIGENDIAN = true;
/* Values for fileType */
public final static int
AIFFTYPE = 1,
AIFCTYPE = 2;
/* Fixed value for first 4 bytes */
private static final int[] sigByte =
{ 0X46, 0X4F, 0X52, 0X4D };
private static final String NAME = "AIFF-hul";
private static final String RELEASE = "1.3";
private static final int [] DATE = {2006, 9, 5};
private static final String [] FORMAT = {
"AIFF", "Audio Interchange File Format"
};
private static final String COVERAGE = "AIFF 1.3, AIFF-C";
private static final String [] MIMETYPE = {
"audio/x-aiff", "application/aiff"
};
private static final String WELLFORMED =
"Magic number: \"FORM\" at byte offset 0; \"AIFF\"" +
"(for AIFF) or \"AIFC\" (for AIFF-C) at offset 8; one Form chunk " +
"containing one Common chunk and at most one Sound Data chunk " +
"(if numSampleFrames > 0); at most one instance of each of the " +
"following optional chunks: Marker, Instrument, Audio Recording, " +
"Comments, Name, Author, Copyright; all chunks required by a given " +
"profile exist in the file; all chunk structures are well-formed: " +
"a four ASCII character ID, followed by a 32 signed integer size, " +
"followed by a size byte data block (if size is odd, then the data " +
"block includes a final padding byte of value 0x00); and no data " +
"exist before the first byte of the chunk or after the " +
"last byte of the last chunk";
private static final String VALIDITY = "The file is well-formed";
private static final String REPINFO =
"Properties capturing the technical attributes of the audio " +
"from all chunks";
private static final String NOTE = null;
private static final String RIGHTS = "Copyright 2004-2007 by JSTOR and " +
"the President and Fellows of Harvard College. " +
"Released under the GNU Lesser General Public License.";
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiates an <tt>AiffModule</tt> object.
*/
public AiffModule ()
{
super (NAME, RELEASE, DATE, FORMAT, COVERAGE, MIMETYPE, WELLFORMED,
VALIDITY, REPINFO, NOTE, RIGHTS, false);
Agent agent = new Agent ("Harvard University Library",
AgentType.EDUCATIONAL);
agent.setAddress ("Office for Information Systems, " +
"90 Mt. Auburn St., " +
"Cambridge, MA 02138");
agent.setTelephone ("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
Document doc = new Document ("Audio Interchange File Format: " +
"\"AIFF\", A Standard for Sampled Sound " +
"Files, Version 1.3", DocumentType.REPORT);
agent = new Agent ("Apple Computer, Inc.",
AgentType.COMMERCIAL);
agent.setAddress ("1 Infinite Loop, Cupertino, CA 95014");
agent.setTelephone("(408) 996-1010");
agent.setWeb ("http://www.apple.com/");
doc.setAuthor (agent);
doc.setDate ("1989-01-04");
doc.setIdentifier (new Identifier ("http://developer.apple.com/documentation/QuickTime/INMAC/SOUND/imsoundmgr.30.htm#pgfId=3190",
IdentifierType.URL));
_specification.add (doc);
doc = new Document ("Audio Interchange File Format AIFF-C: " +
"A revision to include compressed audio data",
DocumentType.REPORT);
doc.setAuthor (agent);
doc.setDate ("1991-08-26");
doc.setNote ("*** DRAFT ***"); // Asterisks as in the printed document
_specification.add (doc);
Signature sig = new ExternalSignature ("AIFF", SignatureType.FILETYPE,
SignatureUseType.OPTIONAL);
_signature.add (sig);
sig = new ExternalSignature ("AIFC", SignatureType.FILETYPE,
SignatureUseType.OPTIONAL);
_signature.add (sig);
sig = new ExternalSignature (".aif", SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add (sig);
sig = new ExternalSignature (".aifc", SignatureType.EXTENSION,
SignatureUseType.OPTIONAL,
"For AIFF-C profile");
_signature.add (sig);
sig = new InternalSignature ("FORM", SignatureType.MAGIC,
SignatureUseType.MANDATORY, 0);
_signature.add (sig);
sig = new InternalSignature ("AIFF", SignatureType.MAGIC,
SignatureUseType.OPTIONAL, 8,
"For AIFF profile");
_signature.add (sig);
sig = new InternalSignature ("AIFC", SignatureType.MAGIC,
SignatureUseType.OPTIONAL, 0,
"For AIFF-C profile");
_signature.add (sig);
_bigEndian = true;
}
/**
* Parses the content of a purported AIFF digital object and stores the
* results in RepInfo.
*
*
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the parsing
* @param parseIndex Must be 0 in first call to <code>parse</code>. If
* <code>parse</code> returns a nonzero value, it must be
* called again with <code>parseIndex</code>
* equal to that return value.
*/
public int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException
{
initParse ();
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setModule (this);
_aesMetadata.setPrimaryIdentifier(info.getUri());
if (info.getURLFlag ()) {
_aesMetadata.setOtherPrimaryIdentifierType("URI");
}
else {
_aesMetadata.setPrimaryIdentifierType(AESAudioMetadata.FILE_NAME);
}
/* We may have already done the checksums while converting a
temporary file. */
_ckSummer = null;
if (_je != null && _je.getChecksumFlag () &&
info.getChecksum ().size () == 0) {
_ckSummer = new Checksummer ();
_cstream = new ChecksumInputStream (stream, _ckSummer);
_dstream = getBufferedDataStream (_cstream, _je != null ?
_je.getBufferSize () : 0);
}
else {
_dstream = getBufferedDataStream (stream, _je != null ?
_je.getBufferSize () : 0);
}
try {
// Check the start of the file for the right opening bytes
for (int i = 0; i < 4; i++) {
int ch = readUnsignedByte(_dstream, this);
if (ch != sigByte[i]) {
info.setMessage(new ErrorMessage ("Document does not start with AIFF FORM Chunk", 0));
info.setWellFormed (RepInfo.FALSE);
return 0;
}
}
/* If we got this far, take note that the signature is OK. */
info.setSigMatch(_name);
// Get the length of the Form chunk. This includes all
// the subsequent chunks in the file, but excludes the
// header ("FORM" and the length itself).
// bytesRemaining = readUnsignedInt (_dstream, _bigEndian, this);
bytesRemaining = readUnsignedInt (_dstream, BIGENDIAN, this);
// Read the file type.
if (!readFileType (info)) {
return 0;
}
while (bytesRemaining > 0) {
if (!readChunk (info)) {
break;
}
}
}
catch (EOFException e) {
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("Unexpected EOF", _nByte));
return 0;
}
if (!commonChunkSeen) {
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("Document does not contain a Common Chunk"));
}
if (fileType == AIFCTYPE && !formatVersionChunkSeen) {
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("AIFF-C document must contain a Format Version Chunk"));
}
if (info.getWellFormed () != RepInfo.TRUE) {
return 0;
}
/* This file looks OK. */
if (_ckSummer != null){
/* We may not have actually hit the end of file. If we're calculating
* checksums on the fly, we have to read and discard whatever is
* left, so it will get checksummed. */
for (;;) {
try {
int n = skipBytes (_dstream, 2048, this);
if (n == 0) {
break;
}
}
catch (Exception e) {
break;
}
}
info.setSize (_cstream.getNBytes ());
info.setChecksum (new Checksum (_ckSummer.getCRC32 (),
ChecksumType.CRC32));
String value = _ckSummer.getMD5 ();
if (value != null) {
info.setChecksum (new Checksum (value, ChecksumType.MD5));
}
if ((value = _ckSummer.getSHA1 ()) != null) {
info.setChecksum (new Checksum (value, ChecksumType.SHA1));
}
}
if (fileType == AIFFTYPE) {
info.setProfile("AIFF");
}
else if (fileType == AIFCTYPE) {
info.setProfile ("AIFF-C");
}
_aesMetadata.setByteOrder (thisFileBigEndian ? AESAudioMetadata.BIG_ENDIAN :
AESAudioMetadata.LITTLE_ENDIAN);
// Most properties were added by the Chunks. The Annotations, Saxel
// and MIDIData properties could have come from multiple chunks,
// and these were added to lists which we now make into Properties
// if there's anything to report.
if (!_annotationList.isEmpty ()) {
_propList.add (new Property ("Annotations",
PropertyType.PROPERTY,
PropertyArity.LIST,
_annotationList));
}
if (!_midiList.isEmpty ()) {
_propList.add (new Property ("MIDIData",
PropertyType.PROPERTY,
PropertyArity.LIST,
_midiList));
}
if (!_saxelList.isEmpty ()) {
_propList.add (new Property ("Saxels",
PropertyType.PROPERTY,
PropertyArity.LIST,
_saxelList));
}
info.setProperty (_metadata);
return 0;
}
/** Sets the endian-ness. <code>true</code> indicates
* big-endian, and <code>false</code> means little-endian.
* This is needed because chunk data can change the
* usual little-endian byte order to big-endian.
*/
public void setEndian (boolean bigEndian)
{
thisFileBigEndian = bigEndian;
}
/** Adds a Property to the AIFF metadata. */
public void addAiffProperty (Property prop)
{
_propList.add (prop);
}
/** Adds an Annotation Property to the annotation list.
* This will get put into an Annotations Property. */
public void addAnnotation (Property prop)
{
_annotationList.add (prop);
}
/** Adds a Saxel Property to the saxel list.
* This will get put into a Saxels Property. */
public void addSaxel (Property prop)
{
_saxelList.add (prop);
}
/** Adds a MIDI Property to the MIDI list.
* This will get put into a MIDIData Property. */
public void addMidi (Property prop)
{
_midiList.add (prop);
}
/**
* Initializes the state of the module for parsing.
*/
protected void initParse()
{
super.initParse ();
thisFileBigEndian = _bigEndian;
_propList = new LinkedList ();
_metadata = new Property ("AIFFMetadata",
PropertyType.PROPERTY,
PropertyArity.LIST,
_propList);
firstSampleOffsetMarked = false;
_aesMetadata = new AESAudioMetadata ();
_aesMetadata.setAnalogDigitalFlag("FILE_DIGITAL");
_aesMetadata.setFormat ("AIFF"); // Further data may modify this value
_aesMetadata.setSpecificationVersion("1.3 (1989-01-04)"); // Ditto
_aesMetadata.setAudioDataEncoding ("PCM");
// _aesMetadata.setBitrateReduction ("PCM", "", "", "",
// "LOSSY", "UNKNOWN", "FIXED");
// Per Bugzilla 772, default is to omit bitrateReduction info
_aesMetadata.clearBitrateReduction();
_aesMetadata.setUse ("OTHER", "JHOVE_validation");
_aesMetadata.setDirection ("NONE");
_propList.add (new Property ("AESAudioMetadata",
PropertyType.AESAUDIOMETADATA,
_aesMetadata));
// Create a List for accumulating properties from Annotation Chunks
_annotationList = new LinkedList ();
// Create a List for accumulating properties from MIDI Chunks
_midiList = new LinkedList ();
// Create a List for accumulating properties from SAXL chunks
_saxelList = new LinkedList ();
// Most chunk types are allowed to occur only once,
// and a few must occur exactly once.
// Clear flags for whether they have been seen.
soundChunkSeen = false;
commonChunkSeen = false;
markerChunkSeen = false;
formatVersionChunkSeen = false;
instrumentChunkSeen = false;
commentsChunkSeen = false;
nameChunkSeen = false;
authorChunkSeen = false;
audioRecChunkSeen = false;
copyrightChunkSeen = false;
}
/** One-argument version of <code>readUnsignedInt</code>. */
public long readUnsignedInt (DataInputStream stream)
throws IOException
{
return readUnsignedInt (stream, BIGENDIAN, this);
}
/** One-argument version of <code>readUnsignedShort</code>.
*/
public int readUnsignedShort (DataInputStream stream)
throws IOException
{
return readUnsignedShort (stream, BIGENDIAN, this);
}
/** One-argument version of <code>readSignedShort</code>.
*/
public int readSignedShort (DataInputStream stream)
throws IOException
{
return readSignedShort (stream, true, this);
}
/** This reads an 80-bit SANE number, aka IEEE 754
* extended double.
*/
public double read80BitDouble (DataInputStream stream)
throws IOException
{
byte[] buf = new byte[10];
readByteBuf(_dstream, buf, this);
ExtDouble xd = new ExtDouble (buf);
return xd.toDouble();
}
/**
* Reads 4 bytes and concatenates them into a String.
* This pattern is used for ID's of various kinds.
*/
public String read4Chars(DataInputStream stream) throws IOException
{
StringBuffer sbuf = new StringBuffer(4);
for (int i = 0; i < 4; i++) {
int ch = readUnsignedByte(stream, this);
sbuf.append((char) ch);
}
return sbuf.toString();
}
/** Reads a Pascal string.
* A Pascal string is one whose count is given in the first
* byte. The count is exclusive of the count byte itself.
* A Pascal string can have a maximum of 255 characters.
* If the count of a Pascal string is even (meaning the total
* number of bytes is odd), there will be a pad byte to skip,
* so that the next item can start on an even boundary.
*
* We assume the string will be in ASCII or Macintosh encoding.
*/
public String readPascalString (DataInputStream stream) throws IOException
{
int byteCnt = readUnsignedByte (stream, this);
byte[] byteBuf = new byte[byteCnt];
readByteBuf (_dstream, byteBuf, this);
if ((byteCnt & 1) == 0) {
skipBytes (_dstream, 1, this);
}
return new String (byteBuf, "MacRoman");
}
/** Converts a Macintosh-style timestamp (seconds since
* January 1, 1904) into a Java date. The timestamp is
* treated as a time in the default localization.
* Depending on that localization,
* there may be some variation in the exact hour of the date
* returned, e.g., due to daylight savings time.
*
*/
public Date timestampToDate (long timestamp)
{
Calendar cal = Calendar.getInstance ();
cal.set (1904, 0, 1, 0, 0, 0);
// If we add the seconds directly, we'll truncate the long
// value when converting to int. So convert to hours plus
// residual seconds.
int hours = (int) (timestamp / 3600);
int seconds = (int) (timestamp - (long) hours * 3600L);
cal.add (Calendar.HOUR_OF_DAY, hours);
cal.add (Calendar.SECOND, seconds);
return cal.getTime ();
}
/** Returns the filetype, which is AIFFTYPE or AIFCTYPE. */
public int getFileType ()
{
return fileType;
}
/** Marks the first sample offset as the current byte position,
* if it hasn't already been marked.
* The SSND chunk offset value must be added to the current
* byte offset for a correct value.
*/
public void markFirstSampleOffset (long offset)
{
if (!firstSampleOffsetMarked) {
firstSampleOffsetMarked = true;
_aesMetadata.setFirstSampleOffset (_nByte + offset);
}
}
/** Reads the file type.
* Broken out from parse().
* If it is not a valid file type, returns false.
*/
protected boolean readFileType (RepInfo info) throws IOException
{
String typ = read4Chars (_dstream);
bytesRemaining -= 4;
if ("AIFF".equals (typ)) {
fileType = AIFFTYPE;
return true;
}
else if ("AIFC".equals (typ)) {
fileType = AIFCTYPE;
_aesMetadata.setFormat ("AIFF-C");
_aesMetadata.setSpecificationVersion ("Draft 1991-08-26");
return true;
}
else {
info.setMessage (new ErrorMessage
("File type in Form Chunk is not AIFF or AIFC", _nByte));
info.setWellFormed (RepInfo.FALSE);
return false;
}
}
/** Reads an AIFF Chunk.
*
*/
protected boolean readChunk (RepInfo info) throws IOException
{
Chunk chunk = null;
ChunkHeader chunkh = new ChunkHeader (this, info);
if (!chunkh.readHeader(_dstream)) {
return false;
}
int chunkSize = (int) chunkh.getSize ();
bytesRemaining -= chunkSize + 8;
String id = chunkh.getID ();
if ("FVER".equals (id)) {
if (formatVersionChunkSeen) {
dupChunkError (info, "Format Version");
}
chunk = new FormatVersionChunk (this, chunkh, _dstream);
formatVersionChunkSeen = true;
}
else if ("APPL".equals (id)) {
chunk = new ApplicationChunk (this, chunkh, _dstream);
// Any number of application chunks is ok
}
else if ("COMM".equals (id)) {
if (commonChunkSeen) {
dupChunkError (info, "Common");
}
chunk = new CommonChunk (this, chunkh, _dstream);
commonChunkSeen = true;
}
else if ("SSND".equals (id)) {
// Watch for multiple sound chunks
if (soundChunkSeen) {
dupChunkError (info, "Sound");
}
else {
chunk = new SoundDataChunk (this, chunkh, _dstream);
soundChunkSeen = true;
}
}
else if ("COMT".equals (id)) {
if (commentsChunkSeen) {
dupChunkError (info, "Comments");
}
chunk = new CommentsChunk (this, chunkh, _dstream);
commentsChunkSeen = true;
}
else if ("INST".equals (id)) {
if (instrumentChunkSeen) {
dupChunkError (info, "Instrument");
}
chunk = new InstrumentChunk (this, chunkh, _dstream);
instrumentChunkSeen = true;
}
else if ("MARK".equals (id)) {
if (markerChunkSeen) {
dupChunkError (info, "Marker");
}
else {
chunk = new MarkerChunk (this, chunkh, _dstream);
markerChunkSeen = true;
}
}
else if ("MIDI".equals (id)) {
chunk = new MidiChunk (this, chunkh, _dstream);
// Any number of MIDI chunks are allowed
}
else if ("NAME".equals (id)) {
if (nameChunkSeen) {
dupChunkError (info, "Name");
}
else {
chunk = new NameChunk (this, chunkh, _dstream);
nameChunkSeen = true;
}
}
else if ("AUTH".equals (id)) {
if (authorChunkSeen) {
dupChunkError (info, "Author");
}
else {
chunk = new AuthorChunk (this, chunkh, _dstream);
authorChunkSeen = true;
}
}
else if ("(c) ".equals (id)) {
if (copyrightChunkSeen) {
dupChunkError (info, "Copyright");
}
else {
chunk = new CopyrightChunk (this, chunkh, _dstream);
copyrightChunkSeen = true;
}
}
else if ("AESD".equals (id)) {
if (audioRecChunkSeen) {
dupChunkError (info, "Audio Recording");
}
else {
chunk = new AudioRecChunk (this, chunkh, _dstream);
audioRecChunkSeen = true;
}
}
else if ("SAXL".equals (id)) {
chunk = new SaxelChunk (this, chunkh, _dstream);
// Multiple saxel chunks are ok
}
else if ("ANNO".equals (id)) {
chunk = new AnnotationChunk (this, chunkh, _dstream);
// Multiple annotations are OK
}
else {
info.setMessage (new InfoMessage
("Chunk type '" + id + "' ignored", _nByte));
}
if (chunk != null) {
try {
if (!chunk.readChunk (info)) {
return false;
}
}
catch (JhoveException e) {
info.setMessage(new ErrorMessage (e.getMessage()));
info.setWellFormed (false);
return false;
}
}
else {
// Other chunk types are legal, just skip over them
skipBytes (_dstream, chunkSize, this);
}
if ((chunkSize & 1) != 0) {
// Must come out to an even byte boundary
skipBytes (_dstream, 1, this);
--bytesRemaining;
}
return true;
}
/** Returns the module's AES metadata. */
public AESAudioMetadata getAESMetadata ()
{
return _aesMetadata;
}
/* Factor out the reporting of duplicate chunks. */
protected void dupChunkError (RepInfo info, String chunkName)
{
info.setMessage (new ErrorMessage
("Multiple " + chunkName + " Chunks not permitted",
_nByte));
info.setValid (false);
}
}
@@ -1,432 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2007 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
import java.util.*;
/**
* Module for analysis of content as an ASCII stream.
*/
public class AsciiModule
extends ModuleBase
{
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
private static final String NAME = "ASCII-hul";
private static final String RELEASE = "1.3";
private static final int [] DATE = {2006, 9, 5};
private static final String [] FORMAT = {
"ASCII", "US-ASCII", "ANSI X3.4", "ISO 646"
};
private static final String COVERAGE = null;
private static final String [] MIMETYPE = {"text/plain; charset=US-ASCII"};
private static final String WELLFORMED = "An ASCII object is well-formed "+
"if each byte is between 0x00 and 0x7F";
private static final String VALIDITY = null;
private static final String REPINFO = "Additional representation information includes: line ending and control characters";
private static final String NOTE = null;
private static final String RIGHTS = "Copyright 2003-2007 by JSTOR and " +
"the President and Fellows of Harvard College. " +
"Released under the GNU Lesser General Public License.";
private static final int CR = 0x0d; // '\r'
private static final int LF = 0x0a; // '\n'
/* Mnemonics for control characters (0-1F) */
private static final String controlCharMnemonics[] = {
"NUL (0x00)", "SOH (0x01)", "STX (0x02)", "ETX (0x03)",
"EOT (0x04)", "ENQ (0x05)", "ACK (0x06)", "BEL (0x07)",
"BS (0x08)", "TAB (0x09)", "LF (0x0A)", "VT (0x0B)",
"FF (0x0C)", "CR (0x0D)", "SO (0x0E)", "SI (0x0F)",
"DLE (0x10)", "DC1 (0x11)", "DC2 (0x12)", "DC3 (0x13)",
"DC4 (0x14)", "NAK (0x15)", "SYN (0x16)", "ETB (0x17)",
"CAN (0x18)", "EM (0x19)", "SUB (0x1A)", "ESC (0x1B)",
"FS (0x1C)", "GS (0x1D)", "RS (0x1E)", "US (0x1F)"
};
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/* Input stream wrapper which handles checksums */
protected ChecksumInputStream _cstream;
/* Data input stream wrapped around _cstream */
protected DataInputStream _dstream;
protected boolean _lineEndCR;
protected boolean _lineEndLF;
protected boolean _lineEndCRLF;
protected int _prevChar;
protected Map<Integer, String> _controlCharMap;
/* Flag to know if the property TextMDMetadata is to be added */
protected boolean _withTextMD = false;
/* Hold the information needed to generate a textMD metadata fragment */
protected TextMDMetadata _textMD;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates an AsciiModule.
*/
public AsciiModule ()
{
super (NAME, RELEASE, DATE, FORMAT, COVERAGE, MIMETYPE, WELLFORMED,
VALIDITY, REPINFO, NOTE, RIGHTS, false);
Agent agent = new Agent ("Harvard University Library",
AgentType.EDUCATIONAL);
agent.setAddress ("Office for Information Systems, " +
"90 Mt. Auburn St., " +
"Cambridge, MA 02138");
agent.setTelephone ("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
Document doc = new Document ("Information technology -- ISO 7-bit " +
"coded character set for information " +
"interchange", DocumentType.STANDARD);
agent = new Agent ("ISO", AgentType.STANDARD);
agent.setAddress ("1, rue de Varembe, Casa postale 56, " +
"CH-1211, Geneva 20, Switzerland");
agent.setTelephone ("+41 22 749 01 11");
agent.setFax ("+41 22 733 34 30");
agent.setEmail ("iso@iso.ch");
agent.setWeb ("http://www.iso.org/");
doc.setPublisher (agent);
doc.setDate ("1991");
doc.setIdentifier (new Identifier ("ISO/IEC 646:1991",
IdentifierType.ISO));
_specification.add (doc);
doc = new Document ("Information Systems -- Coded Character Sets " +
"7-Bit American National Standard Code for " +
"Information Interchange (7-Bit ASCII)",
DocumentType.STANDARD);
agent = new Agent ("ANSI", AgentType.STANDARD);
agent.setAddress ("1819 L Street, NW, Washington, DC 20036");
agent.setTelephone ("+1 (202) 293-8020");
agent.setFax ("+1 (202) 293-9287");
agent.setEmail ("info@ansi.org");
agent.setWeb ("http://www.ansi.org/");
doc.setPublisher (agent);
doc.setDate ("1986-12-30");
doc.setIdentifier (new Identifier ("ANSI X3.4-1986",
IdentifierType.ANSI));
_specification.add (doc);
doc = new Document ("7-Bit coded Character Set",DocumentType.STANDARD);
doc.setEdition ("6th");
doc.setDate ("1991-12");
agent = new Agent ("ECMA", AgentType.STANDARD);
agent.setAddress ("114 Rue du Rhone, CH-1204 Geneva, Switzerland");
agent.setTelephone ("+41 22 849.60.00");
agent.setFax ("+41 22 849.60.01");
agent.setEmail ("helpdesk@ecma.ch");
agent.setWeb ("http://www.ecma-international.org/");
doc.setPublisher (agent);
doc.setIdentifier (new Identifier ("ECMA-6", IdentifierType.ECMA));
doc.setIdentifier (new Identifier ("http://www.ecma-international." +
"org/publications/files/ecma-st/" +
"Ecma-006.pdf",IdentifierType.URL));
_specification.add (doc);
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Parsing methods.
******************************************************************/
/**
* Parse the content of a stream digital object and store the
* results in RepInfo.
*/
public final int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException
{
// Test if textMD is to be generated
if (_defaultParams != null) {
Iterator iter = _defaultParams.iterator ();
while (iter.hasNext ()) {
String param = (String) iter.next ();
if (param.toLowerCase ().equals ("withtextmd=true")) {
_withTextMD = true;
}
}
}
initParse ();
info.setModule (this);
// No line end types have been discovered.
_lineEndCR = false;
_lineEndLF = false;
_lineEndCRLF = false;
_prevChar = 0;
_controlCharMap = new HashMap<Integer, String> ();
_textMD = new TextMDMetadata();
boolean printableChars = false;
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
/* We may have already done the checksums while converting a
temporary file. */
Checksummer ckSummer = null;
if (_je != null && _je.getChecksumFlag () &&
info.getChecksum ().size () == 0) {
ckSummer = new Checksummer ();
_cstream = new ChecksumInputStream (stream, ckSummer);
_dstream = getBufferedDataStream (_cstream, _je != null ?
_je.getBufferSize () : 0);
}
else {
_dstream = getBufferedDataStream (stream, _je != null ?
_je.getBufferSize () : 0);
}
boolean eof = false;
_nByte = 0;
while (!eof) {
try {
int ch = readUnsignedByte (_dstream, this);
/* Only byte values 0x00 through 0x7f are valid. */
if (ch > 0x7f) {
ErrorMessage error =
new ErrorMessage ("Invalid character",
"Character = " + ((char) ch) +
" (0x" + Integer.toHexString (ch) +
")", _nByte - 1);
info.setMessage (error);
info.setWellFormed (RepInfo.FALSE);
return 0;
}
/* Track what control characters are used. */
if (ch < 0X20 && ch != 0X0D && ch != 0X0A) {
_controlCharMap.put (new Integer (ch),
controlCharMnemonics [ch]);
}
else if (ch == 0X7F) {
_controlCharMap.put (new Integer (ch), "DEL (0x7F)");
}
/* Determine the line ending type(s). */
checkLineEnd(ch);
/* Only byte values 0x20 through 0x7e are printable. */
if (0x20 <= ch && ch <= 0x7e) {
printableChars = true;
}
_prevChar = ch;
}
catch (EOFException e) {
eof = true;
/* Catch line endings at very end. */
checkLineEnd(0);
}
}
/* The object is well-formed ASCII. */
if (ckSummer != null){
info.setSize (_cstream.getNBytes ());
info.setChecksum (new Checksum (ckSummer.getCRC32 (),
ChecksumType.CRC32));
String value = ckSummer.getMD5 ();
if (value != null) {
info.setChecksum (new Checksum (value, ChecksumType.MD5));
}
if ((value = ckSummer.getSHA1 ()) != null) {
info.setChecksum (new Checksum (value, ChecksumType.SHA1));
}
}
/* Only non-zero-length files are well-formed ASCII.
*/
if (_nByte == 0) {
info.setMessage (new ErrorMessage ("Zero-length file"));
info.setWellFormed (RepInfo.FALSE);
return 0;
}
/* Add the textMD information */
_textMD.setCharset(TextMDMetadata.CHARSET_ASCII);
_textMD.setByte_order(
_bigEndian?TextMDMetadata.BYTE_ORDER_BIG:TextMDMetadata.BYTE_ORDER_LITTLE);
_textMD.setByte_size("8");
_textMD.setCharacter_size("1");
/* Create a metadata property for the module-specific
* info. (4-Feb-04) */
List<Property> metadataList = new ArrayList<Property> (2);
/* Set property reporting line ending type */
if (_lineEndCR || _lineEndLF || _lineEndCRLF) {
ArrayList<String> propArray = new ArrayList<String>(3);
if (_lineEndCR) {
propArray.add("CR");
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_CR);
}
if (_lineEndLF) {
propArray.add("LF");
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_LF);
}
if (_lineEndCRLF) {
propArray.add("CRLF");
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_CRLF);
}
Property property = new Property("LineEndings",
PropertyType.STRING, PropertyArity.LIST, propArray);
metadataList.add(property);
}
/* Set property reporting control characters used */
if (!_controlCharMap.isEmpty ()) {
LinkedList<String> propList = new LinkedList<String> ();
String mnem;
for (int i = 0; i < 0X20; i++) {
mnem = (String) _controlCharMap.get (new Integer (i));
if (mnem != null) {
propList.add (mnem);
}
}
/* need to check separately for DEL */
mnem = (String) _controlCharMap.get (new Integer (0X7F));
if (mnem != null) {
propList.add (mnem);
}
Property property = new Property ("ControlCharacters",
PropertyType.STRING,
PropertyArity.LIST,
propList);
metadataList.add (property);
}
if (_withTextMD) {
Property property = new Property ("TextMDMetadata",
PropertyType.TEXTMDMETADATA, PropertyArity.SCALAR, _textMD);
metadataList.add (property);
}
/* Add the ASCII-specific metadata, if it exists. */
if (metadataList.size () > 0) {
info.setProperty (new Property ("ASCIIMetadata",
PropertyType.PROPERTY,
PropertyArity.LIST,
metadataList));
}
if (!printableChars) {
info.setMessage (new InfoMessage ("No printable characters"));
}
return 0;
}
/**
* Check if the digital object conforms to this Module's
* internal signature information.
* An ASCII file has no "signature," so in cases like this we just
* check the beginning of the file as a plausible guess. This really
* proves nothing, since a text file could have a single accented
* character dozens of kilobytes into it. But oh well.
*
* @param file A File object for the object being parsed
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the test
*/
public void checkSignatures (File file,
InputStream stream,
RepInfo info)
throws IOException
{
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setModule (this);
JhoveBase jb = getBase();
int sigBytes = jb.getSigBytes();
int bytesRead = 0;
boolean eof = false;
DataInputStream dstream = new DataInputStream (stream);
while (!eof && bytesRead < sigBytes) {
try {
int ch = readUnsignedByte (dstream, this);
++bytesRead;
/* Only byte values 0x00 through 0x7f are valid. */
if (ch > 0x7f) {
info.setWellFormed (false);
return;
}
}
catch (EOFException e) {
eof = true;
}
}
// Reject an empty file.
if (bytesRead == 0) {
info.setWellFormed (false);
return;
}
// Do this only after being sure it's OK, as this property
// is sticky.
info.setSigMatch(_name);
}
/******************************************************************
* PRIVATE INSTANCE METHODS.
******************************************************************/
/* Accumulate information about line endings. ch is the
current character, and _prevChar the one before it. */
protected void checkLineEnd (int ch)
{
if (ch == LF) {
if (_prevChar == CR) {
_lineEndCRLF = true;
}
else {
_lineEndLF = true;
}
}
else if (_prevChar == CR) {
_lineEndCR = true;
}
}
}
@@ -1,167 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2007 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import edu.harvard.hul.ois.jhove.*;
import java.io.*;
/**
* Module for analysis of content as a byte stream.
* This is the module of last resort, accepting any content as
* valid and well-formed.
*/
public final class BytestreamModule
extends ModuleBase
{
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/* Input stream wrapper which handles checksums */
protected ChecksumInputStream _cstream;
/* Data input stream wrapped around _cstream */
protected DataInputStream _dstream;
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
private static final String NAME = "BYTESTREAM";
private static final String RELEASE = "1.3";
private static final int [] DATE = {2007, 4, 10};
private static final String [] FORMAT = {"bytestream"};
private static final String COVERAGE = null;
private static final String [] MIMETYPE = {"application/octet-stream"};
private static final String WELLFORMED = "All bytestreams are well-formed";
private static final String VALIDITY = null;
private static final String REPINFO = null;
private static final String NOTE = "This is the default format";
private static final String RIGHTS = "Copyright 2003-2007 by JSTOR and " +
"the President and Fellows of Harvard College. " +
"Released under the GNU Lesser General Public License.";
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Creates a BytestreamModule.
*/
public BytestreamModule ()
{
super (NAME, RELEASE, DATE, FORMAT, COVERAGE, MIMETYPE, WELLFORMED,
VALIDITY, REPINFO, NOTE, RIGHTS, false);
Agent agent = new Agent ("Harvard University Library",
AgentType.EDUCATIONAL);
agent.setAddress ("Office for Information Systems, " +
"90 Mt. Auburn St., " +
"Cambridge, MA 02138");
agent.setTelephone ("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
}
/******************************************************************
* PUBLIC INSTANCE METHODS.
*
* Parsing methods.
******************************************************************/
/**
* Parse the content of a stream digital object and store the
* results in RepInfo.
* Any arbitrary bytestream is considered well-formed.
*/
public final int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException
{
initParse ();
info.setModule (this);
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
/* We may have already done the checksums while converting a
temporary file. */
Checksummer ckSummer = null;
if (_je != null && _je.getChecksumFlag () &&
info.getChecksum ().size () == 0) {
ckSummer = new Checksummer ();
_cstream = new ChecksumInputStream (stream, ckSummer);
_dstream = getBufferedDataStream (_cstream, _je != null ?
_je.getBufferSize () : 0);
}
else {
_dstream = getBufferedDataStream (stream, _je != null ?
_je.getBufferSize () : 0);
}
boolean eof = false;
_nByte = 0;
byte[] byteBuf = new byte[4096];
while (!eof) {
try {
// int ch = readUnsignedByte (_dstream, this);
// All the calculations are done down in ChecksumInputStream
int n = readByteBuf (_dstream, byteBuf, this);
if (n <= 0) {
break;
}
}
catch (EOFException e) {
eof = true;
}
}
info.setSize (_nByte);
if (_nByte == 0) {
info.setMessage (new InfoMessage ("Zero-length file"));
}
if (ckSummer != null) {
info.setChecksum (new Checksum (ckSummer.getCRC32 (),
ChecksumType.CRC32));
String value = ckSummer.getMD5 ();
if (value != null) {
info.setChecksum (new Checksum (value, ChecksumType.MD5));
}
if ((value = ckSummer.getSHA1 ()) != null) {
info.setChecksum (new Checksum (value, ChecksumType.SHA1));
}
}
return 0;
}
/**
* Check signature. Bytestreams have no signatures, but since any
* byte stream is considered a valid Bytestream, return immediately
* doing nothing. The RepInfo._consistent flag will remain true.
*/
public void checkSignatures (File file, InputStream stream,
RepInfo info)
throws IOException
{
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setModule (this);
info.setSigMatch(_name);
}
}
@@ -1,993 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2003-2007 by JSTOR and the President and Fellows of Harvard College
*
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import java.io.*;
import java.util.*;
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.gif.*;
import org.xml.sax.XMLReader;
import org.xml.sax.SAXException;
import javax.xml.parsers.SAXParserFactory;
/**
* Module for identification and validation of GIF files.
*
* @author Gary McGath
*
*/
public class GifModule extends ModuleBase
{
/******************************************************************
* DEBUGGING FIELDS.
* All debugging fields should be set to false for release code.
******************************************************************/
/* Set to true to allow application identifiers to be case-insensitive. */
private static final boolean debug_appIdentCaseInsens = false;
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
private static final String NAME = "GIF-hul";
private static final String RELEASE = "1.3";
private static final int [] DATE = {2006, 9, 5};
private static final String [] FORMAT = {"GIF",
"Graphics Interchange Format"};
private static final String COVERAGE = "GIF87a, GIF89a";
private static final String [] MIMETYPE = {"image/gif"};
private static final String WELLFORMED = "A GIF file is well-formed if " +
"it has a header block; a sequence of properly formed control, " +
"graphic-rendering, and special purpose blocks; and a trailer block";
private static final String VALIDITY = "A GIF file is valid if " +
"well-formed, has at most one global color map, and at most one " +
"graphic control extension preceding an image descriptor or a plain " +
"text extension";
private static final String REPINFO = "Additional representation " +
"information includes: NISO Z39.87 Digital Still Image Technical " +
"Metadata, and block-specific metadata";
private static final String NOTE = "'GIF' and 'Graphics Interchange " +
"Format' are trademarks of " +
"Compuserve Interactive Services Inc.";
private static final String RIGHTS = "Copyright 2003-2007 by JSTOR and " +
"the President and Fellows of Harvard College. " +
"Released under the GNU Lesser General Public License.";
/* Block type values */
private static final int
EXT_BLOCK = 0X21,
APPLICATION_EXT = 0XFF,
COMMENT_EXT = 0XFE,
GRAPHIC_CONTROL_EXT = 0XF9,
IMAGE_DESC = 0X2C,
PLAIN_TEXT_EXT = 0X01,
TRAILER = 0X3B;
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/* First 6 bytes of file */
protected byte _sig[];
/* Checksummer object */
protected Checksummer _ckSummer;
/* XMP property */
protected Property _xmpProp;
/* Input stream wrapper which handles checksums */
protected ChecksumInputStream _cstream;
/* Data input stream wrapped around _cstream */
protected DataInputStream _dstream;
/* Flag for presence of global color table */
protected boolean _globalColorTableFlag;
/* Size of global color table */
protected int _globalColorTableSize;
/* Count of graphic control extensions preceding
* something to modify */
protected int _gceCounter;
/* Top-level metadata property */
protected Property _metadata;
/* Blocks list property */
protected List _blocksList;
/* Total count of graphic and plain text extension blocks */
protected int _numGraphicBlocks;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate a <tt>GifModule</tt> object.
*/
public GifModule ()
{
super (NAME, RELEASE, DATE, FORMAT, COVERAGE, MIMETYPE, WELLFORMED,
VALIDITY, REPINFO, NOTE, RIGHTS, false);
Agent agent = new Agent ("Harvard University Library",
AgentType.EDUCATIONAL);
agent.setAddress ("Office for Information Systems, " +
"90 Mt. Auburn St., " +
"Cambridge, MA 02138");
agent.setTelephone ("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
Document doc = new Document ("GIF (Graphics Interchange Format): A " +
"standard defining a mechanism for the " +
"storage and transmission of raster-" +
"based graphics information",
DocumentType.REPORT);
agent = new Agent ("Compuserve Interactive Services Inc.",
AgentType.COMMERCIAL);
agent.setAddress ("5000 Arlington Centre Blvd., Columbus, OS 43220");
agent.setTelephone ("(614) 457-8600");
agent.setWeb ("http://www.compuserve.com/");
doc.setAuthor (agent);
doc.setDate ("1987-06-15");
doc.setIdentifier (new Identifier ("http://www.w3.org/Graphics/GIF/spec-gif87.txt",
IdentifierType.URL));
_specification.add (doc);
doc = new Document ("Graphics Interchange Format",
DocumentType.REPORT);
doc.setEdition ("Version 89a");
doc.setAuthor (agent);
doc.setDate ("1987-06-15");
doc.setIdentifier (new Identifier ("http://www.w3.org/Graphics/GIF/spec-gif89a.txt",
IdentifierType.URL));
_specification.add (doc);
Signature sig = new InternalSignature ("GIF", SignatureType.MAGIC,
SignatureUseType.MANDATORY, 0);
_signature.add (sig);
sig = new InternalSignature ("87a", SignatureType.MAGIC,
SignatureUseType.MANDATORY_IF_APPLICABLE,
3, "For version 87a");
_signature.add (sig);
sig = new InternalSignature ("89a", SignatureType.MAGIC,
SignatureUseType.MANDATORY_IF_APPLICABLE,
3, "For version 89a");
_signature.add (sig);
sig = new ExternalSignature (".gif", SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add (sig);
_bigEndian = false;
}
/******************************************************************
* Parsing methods.
******************************************************************/
/**
* Check if the digital object conforms to this Module's
* internal signature information.
*
* @param file A File object for the object being parsed
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the test
*/
public void checkSignatures (File file,
InputStream stream,
RepInfo info)
throws IOException
{
int sigBytes[] = { 'G', 'I', 'F', '8', '*', 'a' };
int i;
int ch;
try {
_dstream = null;
_dstream = getBufferedDataStream (stream, _je != null ?
_je.getBufferSize () : 0);
for (i = 0; i < 4; i++) {
ch = readUnsignedByte(_dstream, this);
if (ch != sigBytes[i]) {
info.setWellFormed (false);
return;
}
}
/* Byte 4 can be either 7 or 9 */
ch = readUnsignedByte (_dstream, this);
if (ch != (int) '7' && ch != (int) '9') {
info.setWellFormed (false);
return;
}
ch = readUnsignedByte (_dstream, this);
if (ch != sigBytes[5]) {
info.setWellFormed (false);
return;
}
info.setModule (this);
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setSigMatch(_name);
}
catch (Exception e) {
// Reading a very short file may take us here.
info.setWellFormed (false);
return;
}
}
/**
* Parse the content of a purported GIF stream digital object and store the
* results in RepInfo.
*
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the parsing
* @param parseIndex Must be 0 in first call to <code>parse</code>. If
* <code>parse</code> returns a nonzero value, it must be
* called again with <code>parseIndex</code>
* equal to that return value.
*/
public int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException
{
initParse ();
info.setModule (this);
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
_blocksList = new LinkedList ();
Property _blocks = new Property ("Blocks",
PropertyType.PROPERTY,
PropertyArity.LIST,
_blocksList);
// We may have already done the checksums while converting a
// temporary file.
_ckSummer = null;
if (_app != null && _je.getChecksumFlag () &&
info.getChecksum ().size () == 0) {
_ckSummer = new Checksummer ();
_cstream = new ChecksumInputStream (stream, _ckSummer);
_dstream = getBufferedDataStream (_cstream, _app != null ?
_je.getBufferSize () : 0);
}
else {
_dstream = getBufferedDataStream (stream, _app != null ?
_je.getBufferSize () : 0);
}
if (!readSig (info)) {
return 0;
}
/* If we got this far, take note that the signature is OK. */
info.setSigMatch(_name);
if (!readLSD (info)) {
return 0;
}
boolean moreToCome = true;
while (moreToCome) {
moreToCome = readBlock (info);
if (info.getWellFormed () == RepInfo.FALSE) {
return 0;
}
}
if (_ckSummer != null){
/* We may not have actually hit the end of file. If we're calculating
* checksums on the fly, we have to read and discard whatever is
* left, so it will get checksummed. */
for (;;) {
try {
int n = skipBytes (_dstream, 2048, this);
if (n == 0) {
break;
}
}
catch (Exception e) {
break;
}
}
info.setSize (_cstream.getNBytes ());
info.setChecksum (new Checksum (_ckSummer.getCRC32 (),
ChecksumType.CRC32));
String value = _ckSummer.getMD5 ();
if (value != null) {
info.setChecksum (new Checksum (value, ChecksumType.MD5));
}
if ((value = _ckSummer.getSHA1 ()) != null) {
info.setChecksum (new Checksum (value, ChecksumType.SHA1));
}
}
Property metaArray[];
if (_xmpProp != null) {
// Making this an array rather than a list is a pain, but it's policy
metaArray = new Property[3];
}
else {
metaArray = new Property[2];
}
_metadata = new Property ("GIFMetadata",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
metaArray);
metaArray[1] = _blocks; // this comes after GraphicRenderingBlocks,
// which we can't calculate yet
metaArray[0] = new Property ("GraphicRenderingBlocks",
PropertyType.INTEGER,
new Integer (_numGraphicBlocks));
if (_xmpProp != null) {
metaArray[2] = _xmpProp;
}
info.setProperty (_metadata);
return 0;
}
/**
* Initializes the state of the module for parsing.
*/
protected void initParse ()
{
super.initParse ();
_sig = new byte[6];
_globalColorTableFlag = false;
_globalColorTableSize = 0;
_gceCounter = 0;
_numGraphicBlocks = 0;
}
/* Read the 6-byte signature. */
protected boolean readSig (RepInfo info) throws IOException
{
boolean eof = false;
String badHeader = "Invalid GIF header";
int nbyt = 0;
while (nbyt < 6) {
try {
int ch = readUnsignedByte (_dstream, this);
if (nbyt < 6) {
_sig[(int) nbyt] = (byte) ch;
}
nbyt++;
//if (_ckSummer != null) {
// _ckSummer.update (ch);
//}
}
catch (EOFException e) {
eof = true;
info.setMessage(new ErrorMessage (badHeader, 0));
info.setWellFormed (RepInfo.FALSE);
return false;
}
}
String sigStr = new String (_sig);
if ("GIF89a".equals (sigStr)) {
info.setVersion ("89a");
info.setProfile ("GIF 89a");
}
else if ("GIF87a".equals (sigStr)) {
info.setVersion ("87a");
info.setProfile ("GIF 87a");
}
else {
info.setMessage(new ErrorMessage (badHeader, 0));
info.setWellFormed (RepInfo.FALSE);
return false;
}
return true;
}
/* Read the Logical Screen Descriptor. */
protected boolean readLSD (RepInfo info) throws IOException
{
Vector propVec = new Vector (8);
/* GIF data is always little-endian. */
int width = readUnsignedShort (_dstream);
propVec.add (new Property ("LogicalScreenWidth",
PropertyType.INTEGER,
new Integer (width)));
int height = readUnsignedShort (_dstream);
propVec.add (new Property ("LogicalScreenHeight",
PropertyType.INTEGER,
new Integer (height)));
int packedFields = readUnsignedByte (_dstream, this);
_globalColorTableFlag = (packedFields & 0X80) != 0;
int bitsPerColor = ((packedFields & 0X70) >> 4) + 1;
propVec.add (new Property ("ColorResolution",
PropertyType.INTEGER,
new Integer (bitsPerColor)));
boolean sortFlag = (packedFields & 0X8) != 0;
int rawGlobalColorTableSize = packedFields & 0X7;
if (_globalColorTableFlag) {
_globalColorTableSize = 3 *
(1 << (rawGlobalColorTableSize + 1));
}
int bgColorIndex = readUnsignedByte (_dstream, this);
propVec.add (new Property ("BackgroundColorIndex",
PropertyType.INTEGER,
new Integer (bgColorIndex)));
int pixAspectRatio = readUnsignedByte (_dstream, this);
// The pixel aspect ratio is turned into a real aspect
// ratio by a formula, but we just report the raw number.
propVec.add (new Property ("PixelAspectRatio",
PropertyType.SHORT,
new Short ((short) pixAspectRatio)));
propVec.add (addByteProperty ("GlobalColorTableFlag",
_globalColorTableFlag ? 1 : 0,
GifStrings.GLOBAL_COLOR_TABLE_FLAG));
propVec.add (addByteProperty ("GlobalColorTableSortFlag",
sortFlag ? 1 : 0,
GifStrings.COLOR_TABLE_SORT_FLAG));
propVec.add (new Property ("GlobalColorTableSize",
PropertyType.SHORT,
new Short ((short) rawGlobalColorTableSize)));
Property prop = new Property ("LogicalScreenDescriptor",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
vectorToPropArray (propVec));
_blocksList.add (prop);
// Make a property with the global color table, if present
if (_globalColorTableFlag) {
short[] gctArray = new short[_globalColorTableSize];
for (int i = 0; i < _globalColorTableSize; i++) {
gctArray[i] = (short) _dstream.readUnsignedByte ();
}
_blocksList.add (new Property ("GlobalColorTable",
PropertyType.SHORT,
PropertyArity.ARRAY,
gctArray));
}
return true;
}
/* Read Graphic blocks, Special blocks, and the trailer.
* Return false if we get an error that prevents further
* progress, or if we encounter the Trailer. */
protected boolean readBlock (RepInfo info) throws IOException
{
int type;
try {
type = readUnsignedByte (_dstream, this);
}
catch (EOFException e) {
// The spec isn't fully clear on whether a trailer is
// required, but seems to imply it is.
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("End of file reached without encountering Trailer block",
_nByte));
return false;
}
try {
switch (type) {
case EXT_BLOCK:
return readExtBlock (info);
case IMAGE_DESC:
return readImage (info);
case TRAILER:
return false; // end of file
default:
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("Unknown data block type",
"Type = " + type, _nByte));
return false;
}
}
catch (EOFException e) {
// An EOF in the middle of a block is definitely a problem
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("Unexpected end of file", _nByte));
return false;
}
}
/* Read an extension block.
*/
protected boolean readExtBlock (RepInfo info) throws IOException
{
int subtype = readUnsignedByte (_dstream, this);
switch (subtype) {
case APPLICATION_EXT:
return readAppExtension (info);
case COMMENT_EXT:
return readCommentExtension (info);
case GRAPHIC_CONTROL_EXT:
return readGraphicsCtlBlock (info);
case PLAIN_TEXT_EXT:
return readPlainTextExtension (info);
default:
info.setWellFormed (RepInfo.FALSE);
info.setMessage (new ErrorMessage
("Unknown extension block type",
"Type = " + subtype,
_nByte));
return false;
}
}
/* Read an image descriptor and the subsequent data.
* We are positioned just after the type byte of the
* image descriptor.
*/
protected boolean readImage (RepInfo info) throws IOException
{
++_numGraphicBlocks;
Vector propVec = new Vector (7);
NisoImageMetadata niso = new NisoImageMetadata ();
Property nisoProp = new Property ("NisoImageMetadata",
PropertyType.NISOIMAGEMETADATA, niso);
// GIF doesn't have a lot of options, so several
// NISO properties are constants.
niso.setMimeType ("image/gif");
niso.setByteOrder ("little-endian");
niso.setCompressionScheme(5); // LZW
niso.setColorSpace (3); // palette color
niso.setOrientation(1); // normal
niso.setBitsPerSample (new int[] {8});
_gceCounter = 0;
int leftPos = readUnsignedShort (_dstream);
propVec.add (new Property ("ImageLeftPosition",
PropertyType.INTEGER,
new Integer (leftPos)));
int topPos = readUnsignedShort (_dstream);
propVec.add (new Property ("ImageTopPosition",
PropertyType.INTEGER,
new Integer (topPos)));
int width = readUnsignedShort (_dstream);
niso.setImageWidth (width);
int height = readUnsignedShort (_dstream);
niso.setImageLength (height);
int packedFields = readUnsignedByte (_dstream, this);
int interlaceFlag = (packedFields & 0X40) >> 6;
propVec.add (addByteProperty ("InterlaceFlag",
interlaceFlag,
GifStrings.INTERLACE_FLAG));
int localColorTableFlag = (packedFields & 0X80) >> 7;
propVec.add (addByteProperty ("LocalColorTableFlag",
localColorTableFlag,
GifStrings.LOCAL_COLOR_TABLE_FLAG));
int sortFlag = (packedFields & 0X20) >> 5;
propVec.add (addByteProperty ("LocalColorTableSortFlag",
sortFlag,
GifStrings.COLOR_TABLE_SORT_FLAG));
int localColorTableSize = 0;
int rawLocalColorTableSize = packedFields & 0X7;
propVec.add (new Property ("LocalColorTableSize",
PropertyType.SHORT,
new Short ((short) rawLocalColorTableSize)));
propVec.add (nisoProp);
if (localColorTableFlag != 0) {
localColorTableSize =
3 * (1 << (rawLocalColorTableSize + 1));
skipBytes (_dstream, localColorTableSize, this);
}
Property prop = new Property ("ImageDescriptor",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
vectorToPropArray (propVec));
_blocksList.add (prop);
// Skip over the LZW minimum code size
int minCodeSize = readUnsignedByte (_dstream, this);
// Now read sub-blocks till we get one of zero size.
for (;;) {
int blockSize = readUnsignedByte (_dstream, this);
if (blockSize == 0) {
break;
}
skipBytes (_dstream, blockSize, this);
}
return true;
}
/* Read an application extension block and fill in the appropriate
* properties */
protected boolean readAppExtension (RepInfo info)
throws IOException
{
int blockSize = readUnsignedByte (_dstream, this);
if (blockSize != 11) {
info.setMessage (new ErrorMessage
("Wrong application extension block size",
_nByte));
info.setWellFormed (RepInfo.FALSE);
return false;
}
Vector propVec = new Vector (3);
StringBuffer appIdent = new StringBuffer ();
int i;
for (i = 0; i < 8; i++) {
appIdent.append((char) readUnsignedByte (_dstream, this));
}
propVec.add (new Property ("ApplicationIdentifier",
PropertyType.STRING,
appIdent.toString ()));
short appAuth[] = new short[3];
for (i = 0; i < 3; i++) {
appAuth[i] = (short) readUnsignedByte (_dstream, this);
}
propVec.add (new Property ("ApplicationAuthenticationCode",
PropertyType.SHORT,
PropertyArity.ARRAY,
appAuth));
int appDataSize = 0;
// We are interested in the application extension for XMP.
if ((appIdent.toString ().equals ("XMP Data") ||
(debug_appIdentCaseInsens &&
appIdent.toString().toLowerCase().equals ("xmp data"))) &&
appAuth[0] == (short) 'X' &&
appAuth[1] == (short) 'M' &&
appAuth[2] == (short) 'P') {
appDataSize = readXMP ();
}
else {
// Zip through the application data blocks, totalling their size
for (;;) {
int subBlockSize = readUnsignedByte (_dstream, this);
appDataSize += subBlockSize + 1;
if (subBlockSize == 0) {
break;
}
skipBytes (_dstream, subBlockSize, this);
}
}
propVec.add (new Property ("ApplicationDataSize",
PropertyType.INTEGER,
new Integer (appDataSize)));
Property prop = new Property ("ApplicationExtension",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
vectorToPropArray(propVec));
_blocksList.add (prop);
return true;
}
/* Read an application extension block and fill in the appropriate
* properties. A comment extension should, by recommendation,
* contain ASCII, but actually can contain anything. Nulls
* are skipped, but everything else is included as is. */
protected boolean readCommentExtension (RepInfo info)
throws IOException
{
StringBuffer buf = new StringBuffer ();
for (;;) {
int subBlockSize = readUnsignedByte (_dstream, this);
if (subBlockSize == 0) {
break;
}
for (int i = 0; i < subBlockSize; i++) {
int ch = readUnsignedByte (_dstream, this);
if (ch != 0) {
buf.append ((char) ch);
}
}
}
Property prop = new Property ("CommentExtension",
PropertyType.STRING,
buf.toString ());
return true;
}
/* Read an application extension block and fill in the appropriate
* properties */
protected boolean readPlainTextExtension (RepInfo info)
throws IOException
{
++_numGraphicBlocks;
_gceCounter = 0;
int blockSize = readUnsignedByte (_dstream, this);
if (blockSize != 12) {
info.setMessage (new ErrorMessage
("Wrong plain text extension block size",
_nByte));
info.setWellFormed (RepInfo.FALSE);
return false;
}
// A plain text extension requires a global color table
if (!_globalColorTableFlag) {
info.setMessage (new ErrorMessage
("Plain text extension requires global color table",
_nByte));
info.setValid (false);
}
Vector propVec = new Vector (9);
int textLeft = readUnsignedShort (_dstream);
propVec.add (new Property ("TextGridLeftPosition",
PropertyType.INTEGER,
new Integer (textLeft)));
int textTop = readUnsignedShort (_dstream);
propVec.add (new Property ("TextGridTopPosition",
PropertyType.INTEGER,
new Integer (textTop)));
int textGWidth = readUnsignedShort (_dstream);
propVec.add (new Property ("TextGridWidth",
PropertyType.INTEGER,
new Integer (textGWidth)));
int textGHeight = readUnsignedShort (_dstream);
propVec.add (new Property ("TextGridHeight",
PropertyType.INTEGER,
new Integer (textGHeight)));
int charCWidth = readUnsignedByte (_dstream, this);
propVec.add (new Property ("CharacterCellWidth",
PropertyType.SHORT,
new Short ((short) charCWidth)));
int charCHeight = readUnsignedByte (_dstream, this);
propVec.add (new Property ("CharacterCellHeight",
PropertyType.SHORT,
new Short ((short) charCHeight)));
int textFgIdx = readUnsignedByte (_dstream, this);
propVec.add (new Property ("TextForegroundColorIndex",
PropertyType.SHORT,
new Short ((short) textFgIdx)));
int textBgIdx = readUnsignedByte (_dstream, this);
propVec.add (new Property ("TextBackgroundColorIndex",
PropertyType.SHORT,
new Short ((short) textBgIdx)));
// Read the text data. The GIF recommendation states that
// characters less than 0X20 or greater than 0XF7 (why F7?
// It's on at least 2 independent copies of the spec, so
// apparently it's not a typo, or else is a well-entrenched one)
// should be represented as spaces.
StringBuffer buf = new StringBuffer ();
for (;;) {
int subBlockSize = readUnsignedByte (_dstream, this);
if (subBlockSize == 0) {
break;
}
for (int i = 0; i < subBlockSize; i++) {
int ch = readUnsignedByte (_dstream, this);
if (ch >= 0X20 || ch <= 0XF7) {
buf.append ((char) ch);
}
else {
buf.append (' ');
}
}
}
propVec.add (new Property ("PlainTextData",
PropertyType.STRING,
buf.toString ()));
Property prop = new Property ("PlainTextExtension",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
vectorToPropArray(propVec));
_blocksList.add (prop);
return true;
}
/* Read a graphics control block and fill in the
* relevant properties */
protected boolean readGraphicsCtlBlock (RepInfo info)
throws IOException
{
Vector propVec = new Vector (5);
if (++_gceCounter > 1) {
info.setMessage (new ErrorMessage
("Multiple graphics control blocks for one image",
_nByte));
info.setWellFormed (RepInfo.FALSE);
}
int blockSize = readUnsignedByte (_dstream, this);
if (blockSize != 4) {
info.setMessage (new ErrorMessage
("Wrong graphics control block size",
_nByte));
info.setWellFormed (RepInfo.FALSE);
return false;
}
int packedFields = readUnsignedByte (_dstream, this);
int dispMethod = (packedFields & 0X1C) >> 3;
propVec.add (addByteProperty ("DisposalMethod",
dispMethod,
GifStrings.GCE_DISPOSAL_METHOD));
int userInputFlag = (packedFields & 2) >> 1;
propVec.add (addByteProperty ("UserInputFlag",
userInputFlag,
GifStrings.GCE_USER_INPUT_FLAG));
int transparencyFlag = packedFields & 1;
propVec.add (addByteProperty ("TransparencyFlag",
transparencyFlag,
GifStrings.GCE_TRANSPARENCY_FLAG));
int delayTime = readUnsignedShort(_dstream);
propVec.add (new Property ("DelayTime",
PropertyType.INTEGER,
new Integer (delayTime)));
int transIndex = readUnsignedByte (_dstream, this);
propVec.add (new Property ("TransparentColorIndex",
PropertyType.SHORT,
new Short ((short) transIndex)));
// Skip the block terminator.
readUnsignedByte (_dstream, this);
Property prop = new Property ("GraphicControlExtension",
PropertyType.PROPERTY,
PropertyArity.ARRAY,
vectorToPropArray(propVec));
_blocksList.add (prop);
return true;
}
/* Read and process an XMP block and return the number of
* bytes read. When we reach a 0 byte, we've hit the
* end of the "magic" trailer.
*/
protected int readXMP () throws IOException
{
// Read bytes till we get to the trailer. Annoyingly,
// we don't know how big the byte buffer has to be,
// so we build a List of fixed-size buffers. We don't add
// curBuf to bufList till it's full.
List bufList = new LinkedList ();
final int bufsiz = 4096;
byte[] curBuf = new byte[bufsiz];
int curBufOff = 0;
// Fill up buffers till we hit a null.
for (;;) {
int ch = readUnsignedByte (_dstream, this);
if (ch == 0) {
// Read past second null, which concludes trailer
readUnsignedByte (_dstream, this);
break;
}
if (curBufOff == bufsiz) {
bufList.add (curBuf);
curBuf = new byte[bufsiz];
curBufOff = 0;
}
curBuf[curBufOff++] = (byte) ch;
}
// Consolidate the buffers into one big buffer.
// The magic trailer is 258 bytes long, of which 256
// bytes were actually read into the buffers, so we
// set our target to 256 bytes less than the total.
int appDataSize = bufList.size() * bufsiz + curBufOff + 2;
int totalSize = appDataSize - 258;
byte[] bigBuf = new byte[totalSize];
int bigBufOff = 0;
ListIterator iter = bufList.listIterator();
int i;
l1:
while (iter.hasNext ()) {
byte[] buf = (byte []) iter.next ();
for (i = 0; i < bufsiz; i++) {
bigBuf[bigBufOff++] = buf[i];
if (bigBufOff >= totalSize) {
break l1;
}
}
}
// Finally curBuf gets added
for (i = 0; i < curBufOff; i++) {
if (bigBufOff >= totalSize) {
break;
}
bigBuf[bigBufOff++] = curBuf[i];
}
// OK. All that was just to get the XMP into one big byte
// buffer. Now process it.
final String badMetadata = "Invalid or ill-formed XMP metadata";
try {
ByteArrayInputStream strm =
new ByteArrayInputStream (bigBuf);
ByteArrayXMPSource src = new ByteArrayXMPSource (strm, "UTF-8");
// Create an InputSource to feed the parser.
SAXParserFactory factory =
SAXParserFactory.newInstance();
factory.setNamespaceAware (true);
XMLReader parser = factory.newSAXParser ().getXMLReader ();
XMPHandler handler = new XMPHandler ();
parser.setContentHandler (handler);
parser.setErrorHandler (handler);
// We have to parse twice. The first time, we may get
// an encoding change as part of an exception thrown. If this
// happens, we create a new InputSource with the encoding, and
// continue.
try {
parser.parse (src);
_xmpProp = src.makeProperty ();
return appDataSize;
}
catch (SAXException se) {
String msg = se.getMessage ();
if (msg != null && msg.startsWith ("ENC=")) {
String encoding = msg.substring (5);
try {
//The only permitted encoding is UTF-8, but
//that may come under various aliased names,
//so we assume the encoding is legitimate.
src = new ByteArrayXMPSource (strm, encoding);
parser.parse (src);
}
catch (UnsupportedEncodingException uee) {
return appDataSize;
}
}
_xmpProp = src.makeProperty ();
return appDataSize;
}
}
catch (Exception e) {
return appDataSize;
}
}
protected Property addByteProperty (String name, int value,
String [] labels)
{
if (!_je.getShowRawFlag ()) {
try {
return new Property (name, PropertyType.STRING, labels[value]);
}
catch (Exception e) {
// fall through
}
}
return new Property (name, PropertyType.BYTE, new Byte ((byte) value));
}
/* GIF is always little-endian, so readUnsignedShort can
* unambiguously drop its endian argument */
protected int readUnsignedShort (DataInputStream stream)
throws IOException
{
return readUnsignedShort (stream, false, this);
}
}
@@ -1,765 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2007 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import java.io.*;
import java.util.*;
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.html.*;
/**
* Module for identification and validation of HTML files.
*
* HTML is different from most of the other documents in that
* sloppy construction is practically assumed in the specification.
* This module attempt to report as many errors as possible and
* recover reasonably from errors. To do this, there is more
* heuristic behavior built into this module than into the more
* straightforward ones.
*
* XHTML is recognized by this module, but is handed off to the
* XML module for processing. If the XML module is missing (which
* it shouldn't be if you've installed the JHOVE application without
* modifications), this won't be able to deal with XHTML files.
*
* HTML should be placed ahead of XML in the module order. If the
* XML module sees an XHTML file first, it will recognize it as XHTML,
* but won't be able to report the complete properties.
*
* The HTML module uses code created with the JavaCC parser generator
* and lexical analyzer generator. There is apparently a bug in
* JavaCC which causes blank lines not to be counted in certain cases,
* causing lexical errors to be reported with incorrect line numbers.
*
* @author Gary McGath
*
*/
public class HtmlModule extends ModuleBase {
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
private static final String NAME = "HTML-hul";
private static final String RELEASE = "1.3";
private static final int [] DATE = {2006, 9, 5};
private static final String [] FORMAT = {
"HTML"
};
private static final String COVERAGE = "HTML 3.2, HTML 4.0 Strict," +
"HTML 4.0 Transitional, HTML 4.0 Frameset, " +
"HTML 4.01 Strict, HTML 4.01 Transitional, HTML 4.01 Frameset" +
"XHTML 1.0 Strict, XHTML 1.0 Transitional, XHTML 1.0 Frameset" +
"XHTML 1.1";
private static final String [] MIMETYPE = {
"text/html"
};
private static final String WELLFORMED = "An HTML file is well-formed " +
"if it meets the criteria defined in the HTML 3.2 specification " +
"(W3C Recommendation, 14-Jan-1997), " +
"the HTML 4.0 specification (W3C Recommendation, 24-Apr-1998, " +
"the HTML 4.01 specification (W3C Recommendation, 24-Dec-1999, " +
"the XHTML 1.0 specification (W3C Recommendation, 26-Jan-2000, " +
"revised 1-Aug-2002, " +
"or the XHTML 1.1 specification (W3C Recommendation, 31-May-2001";
private static final String VALIDITY = "An HTML file is valid if it is " +
"well-formed and has a valid DOCTYPE declaration.";
private static final String REPINFO = "Languages, title, META tags, " +
"frames, links, scripts, images, citations, defined terms, " +
"abbreviations, entities, Unicode entity blocks";
private static final String NOTE = "";
private static final String RIGHTS = "Copyright 2004-2007 by JSTOR and " +
"the President and Fellows of Harvard College. " +
"Released under the GNU Lesser General Public License.";
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/* Input stream wrapper which handles checksums */
protected ChecksumInputStream _cstream;
/* Data input stream wrapped around _cstream */
protected DataInputStream _dstream;
/* Doctype extracted from document */
protected String _doctype;
/* Constants for the recognized flavors of HTML */
public static final int
HTML_3_2 = 1,
HTML_4_0_STRICT = 2,
HTML_4_0_FRAMESET = 3,
HTML_4_0_TRANSITIONAL = 4,
HTML_4_01_STRICT = 5,
HTML_4_01_FRAMESET = 6,
HTML_4_01_TRANSITIONAL = 7,
XHTML_1_0_STRICT = 8,
XHTML_1_0_TRANSITIONAL = 9,
XHTML_1_0_FRAMESET = 10,
XHTML_1_1 = 11;
/* Profile names, matching the above indices */
private static final String[] profileNames =
{
null,
null, // there are no profiles for HTML 3.2
"Strict",
"Frameset",
"Transitional",
"Strict",
"Frameset",
"Transitional",
"Strict",
"Frameset",
"Transitional",
null // there are no profiles for XHTML 1.1
};
/* Version names, matching the above indices */
private static final String[] versionNames =
{
null,
"HTML 3.2",
"HTML 4.0",
"HTML 4.0",
"HTML 4.0",
"HTML 4.01",
"HTML 4.01",
"HTML 4.01",
"XHTML 1.0",
"XHTML 1.0",
"XHTML 1.0",
"XHTML 1.1"
};
/* Flag to know if the property TextMDMetadata is to be added */
protected boolean _withTextMD = false;
/* Hold the information needed to generate a textMD metadata fragment */
protected TextMDMetadata _textMD;
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate an <tt>HtmlModule</tt> object.
*/
public HtmlModule ()
{
super (NAME, RELEASE, DATE, FORMAT, COVERAGE, MIMETYPE, WELLFORMED,
VALIDITY, REPINFO, NOTE, RIGHTS, false);
Agent agent = new Agent ("Harvard University Library",
AgentType.EDUCATIONAL);
agent.setAddress ("Office for Information Systems, " +
"90 Mt. Auburn St., " +
"Cambridge, MA 02138");
agent.setTelephone ("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
/* HTML 3.2 spec */
Document doc = new Document ("HTML 3.2 Reference Specification",
DocumentType.REPORT);
Agent w3cAgent = new Agent ("Word Wide Web Consortium", AgentType.NONPROFIT);
w3cAgent.setAddress ("Massachusetts Institute of Technology, " +
"Computer Science and Artificial Intelligence Laboratory, " +
"32 Vassar Street, Room 32-G515, " +
"Cambridge, MA 02139");
w3cAgent.setTelephone ("(617) 253-2613");
w3cAgent.setFax ("(617) 258-5999");
w3cAgent.setWeb ("http://www.w3.org/");
doc.setPublisher (w3cAgent);
Agent dRaggett = new Agent ("Dave Raggett", AgentType.OTHER);
doc.setAuthor(dRaggett);
doc.setDate ("1997-01-14");
doc.setIdentifier(new Identifier
("http://www.w3c.org/TR/REC-html32-19970114", IdentifierType.URL));
_specification.add (doc);
/* HTML 4.0 spec */
doc = new Document ("HTML 4.0 Specification", DocumentType.REPORT);
doc.setPublisher (w3cAgent);
doc.setAuthor(dRaggett);
Agent leHors = new Agent ("Arnaud Le Hors", AgentType.OTHER);
doc.setAuthor(leHors);
Agent jacobs = new Agent ("Ian Jacobs", AgentType.OTHER);
doc.setAuthor(jacobs);
doc.setDate ("1998-04-24");
doc.setIdentifier(new Identifier
("http://www.w3.org/TR/1998/REC-html40-19980424/", IdentifierType.URL));
_specification.add (doc);
/* HTML 4.01 spec */
doc = new Document ("HTML 4.01 Specification", DocumentType.REPORT);
doc.setPublisher (w3cAgent);
doc.setAuthor(dRaggett);
doc.setAuthor(leHors);
doc.setAuthor(jacobs);
doc.setDate ("1999-12-24");
doc.setIdentifier(new Identifier
("http://www.w3.org/TR/1999/REC-html401-19991224/", IdentifierType.URL));
_specification.add (doc);
/* XHTML 1.0 spec */
doc = new Document ("XHTML(TM) 1.0 The Extensible HyperText Markup Language " +
"(Second Edition)", DocumentType.REPORT);
doc.setPublisher (w3cAgent);
doc.setDate ("01-08-2002");
doc.setIdentifier (new Identifier
("http://www.w3.org/TR/xhtml1/", IdentifierType.URL));
_specification.add (doc);
/* XHTML 1.1 spec */
doc = new Document (" XHTML(TM) 1.1 - Module-based XHTML",
DocumentType.REPORT);
doc.setPublisher (w3cAgent);
doc.setDate ("31-05-2001");
doc.setIdentifier (new Identifier
("http://www.w3.org/TR/2001/REC-xhtml11-20010531/",
IdentifierType.URL));
_specification.add (doc);
/* XHTML 2.0 spec -- NOT included yet; this is presented in "conditionalized-out"
* form just as a note for future expansion. */
if (false) {
doc = new Document ("XHTML 2.0, W3C Working Draft", DocumentType.OTHER);
doc.setPublisher (w3cAgent);
doc.setDate ("22-07-2004");
doc.setIdentifier (new Identifier
("http://www.w3.org/TR/2004/WD-xhtml2-20040722/", IdentifierType.URL));
_specification.add (doc);
}
Signature sig = new ExternalSignature (".html", SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add (sig);
sig = new ExternalSignature (".htm", SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add (sig);
}
/**
* Parse the content of a purported HTML stream digital object and store the
* results in RepInfo.
*
*
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed.
* If multiple calls to <code>parse</code> are made
* on the basis of a nonzero value being returned,
* a new InputStream must be provided each time.
*
* @param info A fresh (on the first call) RepInfo object
* which will be modified
* to reflect the results of the parsing
* If multiple calls to <code>parse</code> are made
* on the basis of a nonzero value being returned,
* the same RepInfo object should be passed with each
* call.
*
* @param parseIndex Must be 0 in first call to <code>parse</code>. If
* <code>parse</code> returns a nonzero value, it must be
* called again with <code>parseIndex</code>
* equal to that return value.
*/
public int parse (InputStream stream, RepInfo info, int parseIndex)
throws IOException
{
if (parseIndex != 0) {
// Coming in with parseIndex = 1 indicates that we've determined
// this is XHTML; so we invoke the XML module to parse it.
// If parseIndex is 100, this is the first invocation of the
// XML module, so we call it with 0; otherwise we call it with
// the value of parseIndex.
if (isXmlAvailable ()) {
edu.harvard.hul.ois.jhove.module.XmlModule xmlMod =
new edu.harvard.hul.ois.jhove.module.XmlModule ();
if (parseIndex == 100) {
parseIndex = 0;
}
xmlMod.setApp (_app);
xmlMod.setBase (_je);
xmlMod.setXhtmlDoctype(_doctype);
return xmlMod.parse (stream, info, parseIndex);
}
else {
// The XML module shouldn't be missing from any installation,
// but someone who really wanted to could remove it. In
// that case, you deserve what you get.
info.setMessage ( new ErrorMessage
("XML-HUL module required to validate XHTML documents"));
info.setWellFormed (false); // Treat it as completely wrong
return 0;
}
}
else {
/* parseIndex = 0, first call only */
_doctype = null;
}
// Test if textMD is to be generated
if (_defaultParams != null) {
Iterator iter = _defaultParams.iterator ();
while (iter.hasNext ()) {
String param = (String) iter.next ();
if (param.toLowerCase ().equals ("withtextmd=true")) {
_withTextMD = true;
}
}
}
initParse ();
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setModule (this);
if (_textMD == null || parseIndex == 0) {
_textMD = new TextMDMetadata();
}
/* We may have already done the checksums while converting a
temporary file. */
Checksummer ckSummer = null;
if (_je != null && _je.getChecksumFlag () &&
info.getChecksum ().size () == 0) {
ckSummer = new Checksummer ();
_cstream = new ChecksumInputStream (stream, ckSummer);
_dstream = getBufferedDataStream (_cstream, _je != null ?
_je.getBufferSize () : 0);
}
else {
_dstream = getBufferedDataStream (stream, _je != null ?
_je.getBufferSize () : 0);
}
ParseHtml parser = null;
HtmlMetadata metadata = null;
HtmlCharStream cstream = null;
try {
cstream = new HtmlCharStream (_dstream, "ISO-8859-1");
parser = new ParseHtml (cstream);
}
catch (UnsupportedEncodingException e) {
info.setMessage (new ErrorMessage
("Internal error: " + e.getMessage ()));
info.setWellFormed (false);
return 0; // shouldn't happen!
}
int type = 0;
try {
List elements = parser.HtmlDoc ();
if (elements.isEmpty ()) {
// Consider an empty document bad
info.setWellFormed (false);
info.setMessage (new ErrorMessage
("Document is empty"));
return 0;
}
type = checkDoctype (elements);
if (type < 0) {
info.setWellFormed (false);
info.setMessage (new ErrorMessage
("DOCTYPE is not HTML"));
return 0;
}
/* Check if there is at least one html, head, body or title tag.
* A plain text document
* might be interpreted as a single PCDATA, which is in some
* ethereal sense well-formed HTML, but it's pointless to consider
* it such. It might also use angle brackets as a text delimiter,
* and that shouldn't count as HTML either. */
boolean hasElements = false;
Iterator iter = elements.iterator ();
while (iter.hasNext ()) {
Object o = iter.next ();
if (o instanceof JHOpenTag) {
String name = ((JHOpenTag) o).getName ();
if ("html".equals (name) ||
"head".equals (name) ||
"body".equals (name) ||
"title".equals (name)) {
hasElements = true;
}
break;
}
}
if (!hasElements) {
info.setMessage (new ErrorMessage ("Document contains no html, head, body or title tags"));
info.setWellFormed (false);
return 0;
}
// CRLF from HtmlCharStream ...
String lineEnd = cstream.getKindOfLineEnd();
if (lineEnd == null) {
info.setMessage(new InfoMessage("Not able to determine type of end of line"));
_textMD.setLinebreak(TextMDMetadata.NILL);
} else if (lineEnd.equalsIgnoreCase("CR")) {
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_CR);
} else if (lineEnd.equalsIgnoreCase("LF")) {
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_LF);
} else if (lineEnd.equalsIgnoreCase("CRLF")) {
_textMD.setLinebreak(TextMDMetadata.LINEBREAK_CRLF);
}
if (type == 0) {
/* If we can't find a doctype, it still might be XHTML
* if the elements start with an XML declaration and
* the root element is "html" */
switch (seemsToBeXHTML (elements)) {
case 0: // Not XML
break; // fall through
case 1: // XML but not HTML
info.setMessage (new ErrorMessage
("Document has XML declaration but no DOCTYPE; " +
"probably XML rather than HTML"));
info.setWellFormed (false);
return 0;
case 2: // probably XHTML
return 100;
}
info.setMessage (new ErrorMessage
("Unrecognized or missing DOCTYPE declaration; " +
"validation continuing as HTML 3.2"));
info.setValid (false);
// But keep going
}
HtmlDocDesc docDesc = null;
switch (type) {
case HTML_3_2:
default:
docDesc = new Html3_2DocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("3.2");
break;
case HTML_4_0_FRAMESET:
docDesc = new Html4_0FrameDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.0");
break;
case HTML_4_0_TRANSITIONAL:
docDesc = new Html4_0TransDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.0");
break;
case HTML_4_0_STRICT:
docDesc = new Html4_0StrictDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.0");
break;
case HTML_4_01_FRAMESET:
docDesc = new Html4_01FrameDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.01");
break;
case HTML_4_01_TRANSITIONAL:
docDesc = new Html4_01TransDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.01");
break;
case HTML_4_01_STRICT:
docDesc = new Html4_01StrictDocDesc ();
_textMD.setMarkup_basis("HTML");
_textMD.setMarkup_basis_version("4.01");
break;
case XHTML_1_0_STRICT:
case XHTML_1_0_TRANSITIONAL:
case XHTML_1_0_FRAMESET:
case XHTML_1_1:
// Force a second call to parse as XML. 100 is a
// magic code for the first XML call.
return 100;
}
_textMD.setMarkup_language(_doctype);
if (docDesc == null) {
info.setMessage (new InfoMessage ("Code for appropriate HTML version not available yet:" +
"substituting HTML 3.2"));
docDesc = new Html3_2DocDesc ();
}
docDesc.validate (elements, info);
metadata = docDesc.getMetadata ();
// Try to get the charset from the meta Content
if (metadata.getCharset() != null) {
_textMD.setCharset(metadata.getCharset());
} else {
_textMD.setCharset(TextMDMetadata.CHARSET_ISO8859_1);
}
String textMDEncoding = _textMD.getCharset();
if (textMDEncoding.indexOf("UTF") != -1) {
_textMD.setByte_order(
_bigEndian?TextMDMetadata.BYTE_ORDER_BIG:TextMDMetadata.BYTE_ORDER_LITTLE);
_textMD.setByte_size("8");
_textMD.setCharacter_size("variable");
}
else {
_textMD.setByte_order(
_bigEndian?TextMDMetadata.BYTE_ORDER_BIG:TextMDMetadata.BYTE_ORDER_LITTLE);
_textMD.setByte_size("8");
_textMD.setCharacter_size("1");
}
}
catch (ParseException e) {
Token t = e.currentToken;
info.setMessage (new ErrorMessage ("Parse error",
"Line = " + t.beginLine + ", column = " + t.beginColumn));
info.setWellFormed (false);
}
catch (TokenMgrError f) {
info.setMessage (new ErrorMessage ("TokenMgrError: " +
f.getLocalizedMessage()));
info.setWellFormed (false);
}
if (info.getWellFormed() == RepInfo.FALSE) {
return 0;
}
if (type != 0) {
if (profileNames[type] != null) {
info.setProfile (profileNames[type]);
}
info.setVersion (versionNames[type]);
}
if (metadata != null) {
Property property = metadata.toProperty (_withTextMD?_textMD:null);
if (property != null) {
info.setProperty (property);
}
}
if (ckSummer != null){
info.setSize (_cstream.getNBytes ());
info.setChecksum (new Checksum (ckSummer.getCRC32 (),
ChecksumType.CRC32));
String value = ckSummer.getMD5 ();
if (value != null) {
info.setChecksum (new Checksum (value, ChecksumType.MD5));
}
if ((value = ckSummer.getSHA1 ()) != null) {
info.setChecksum (new Checksum (value, ChecksumType.SHA1));
}
}
return 0;
}
/**
* Check if the digital object conforms to this Module's
* internal signature information.
*
* HTML is one of the most ill-defined of any open formats, so
* checking a "signature" really means using some heuristics. The only
* required tag is TITLE, but that could occur well into the file. So we
* look for any of three strings -- taking into account case-independence
* and white space -- within the first sigBytes bytes, and call that
* a signature check.
*
* @param file A File object for the object being parsed
* @param stream An InputStream, positioned at its beginning,
* which is generated from the object to be parsed
* @param info A fresh RepInfo object which will be modified
* to reflect the results of the test
*/
public void checkSignatures (File file,
InputStream stream,
RepInfo info)
throws IOException
{
info.setFormat (_format[0]);
info.setMimeType (_mimeType[0]);
info.setModule (this);
char[][] sigtext = new char[3][];
sigtext[0] = "<!DOCTYPE HTML".toCharArray();
sigtext[1] = "<HTML".toCharArray();
sigtext[2] = "<TITLE".toCharArray();
int[] sigstate = {0, 0, 0};
JhoveBase jb = getBase();
int sigBytes = jb.getSigBytes();
int bytesRead = 0;
boolean eof = false;
DataInputStream dstream = new DataInputStream (stream);
while (!eof && bytesRead < sigBytes) {
try {
int ch = readUnsignedByte (dstream, this);
char chr = Character.toUpperCase((char) ch);
++bytesRead;
if (Character.isWhitespace (chr)) {
continue; // ignore all whitespace
}
for (int i = 0; i < 3; i++) {
int ss = sigstate[i];
char[] st = sigtext[i];
if (chr == st[ss]) {
++sigstate[i];
if (sigstate[i] == st.length) {
// One of the sig texts matches!
info.setSigMatch(_name);
return;
}
}
else sigstate[i] = 0;
}
}
catch (EOFException e) {
eof = true;
}
}
// If we fall through, there was no sig match
info.setWellFormed (false);
return;
}
/* Check if there is a DOCTYPE at the start of the elements
* list. If there is, return the appropriate version string.
* If the DOCTYPE says it isn't HTML, trust it and call this
* document ill-formed by returning -1.
* If there is no DOCTYPE, or an unrecognized one, return 0.
*/
protected int checkDoctype (List elements)
{
JHElement firstElem = (JHElement) elements.get (0);
if (firstElem instanceof JHXmlDecl && elements.size() >= 2) {
firstElem = (JHElement) elements.get (1);
}
if (!(firstElem instanceof JHDoctype)) {
return 0; // no DOCTYPE found
}
List dt = ((JHDoctype) firstElem).getDoctypeElements ();
if (dt.size () < 3) {
return 0;
}
try {
// Is DOCTYPE case sensitive? Assume not.
String str = ((String) dt.get(0)).toUpperCase ();
if (!"HTML".equals (str)) {
// It's not HTML
return -1;
}
str = ((String) dt.get (1)).toUpperCase ();
if (!"PUBLIC".equals (str)) {
return 0;
}
str = stripQuotes (((String) dt.get (2)).toUpperCase ());
_doctype = str;
if ("-//W3C//DTD HTML 3.2 FINAL//EN".equals (str) ||
"-//W3C//DTD HTML 3.2//EN".equals (str)) {
return HTML_3_2;
}
else if ("-//W3C//DTD HTML 4.0//EN".equals (str)) {
return HTML_4_0_STRICT;
}
else if ("-//W3C//DTD HTML 4.0 TRANSITIONAL//EN".equals (str)) {
return HTML_4_0_TRANSITIONAL;
}
else if ("-//W3C//DTD HTML 4.0 FRAMESET//EN".equals (str)) {
return HTML_4_0_FRAMESET;
}
else if ("-//W3C//DTD HTML 4.01//EN".equals (str)) {
return HTML_4_01_STRICT;
}
else if ("-//W3C//DTD HTML 4.01 TRANSITIONAL//EN".equals (str)) {
return HTML_4_01_TRANSITIONAL;
}
else if ("-//W3C//DTD HTML 4.01 FRAMESET//EN".equals (str)) {
return HTML_4_01_FRAMESET;
}
}
catch (Exception e) {
// Really shouldn't happen, but if it does we've got
// a bad doctype
return 0;
}
return 0;
}
/* See if this document, even if it lacks a doctype, is most likely
* XHTML. The test is that the document starts with an XML declaration
* and has "html" for its first tag.
*
* Returns:
* 0 if there's no XML declaration
* 1 if there's an XML declaration but no html tag; in this
* case it's probably some other kind of XML
* 2 if there's an XML declaration and an html tag
*
*/
protected int seemsToBeXHTML (List elements)
{
JHElement elem;
try {
elem = (JHElement) elements.get (0);
if (!(elem instanceof JHXmlDecl)) {
return 0;
}
Iterator iter = elements.iterator ();
while (iter.hasNext ()) {
elem = (JHElement) iter.next ();
if (elem instanceof JHOpenTag) {
JHOpenTag tag = (JHOpenTag) elem;
return ("html".equals (tag.getName()) ? 2 : 1);
}
}
}
catch (Exception e) {
return 0; // document must be really empty
}
return 1;
}
/* Remove quotes from the beginning and end of a string. If it doesn't
* have quotes in both places, leave it alone. */
protected String stripQuotes(String str) {
int len = str.length ();
if (str.charAt (0) == '"' &&
str.charAt (len - 1) == '"') {
return str.substring(1, len - 1);
}
else {
return str;
}
}
/* Checks if the XML module is available.
*/
protected static boolean isXmlAvailable ()
{
try {
Class.forName ("edu.harvard.hul.ois.jhove.module.XmlModule");
return true;
}
catch (Exception e) {
return false;
}
}
}
@@ -1,878 +0,0 @@
/**********************************************************************
* Jhove - JSTOR/Harvard Object Validation Environment
* Copyright 2004-2007 by JSTOR and the President and Fellows of Harvard College
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation; either version 2 of the License, or (at
* your option) any later version.
*
* This program is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
* USA
**********************************************************************/
package edu.harvard.hul.ois.jhove.module;
import edu.harvard.hul.ois.jhove.*;
import edu.harvard.hul.ois.jhove.module.jpeg2000.*;
import java.io.*;
import java.util.*;
/**
* Module for identification and validation of JPEG 2000 files.
*
* Code is included for JPX, but is almost entirely untested due
* to the lack of available sample files that use more than a tiny
* fraction of the features. The current version of the module
* is based on the typo-laden and inconsisent "JPEG 2000 Part II
* Final Committee Draft" (7 December 2000).
* The final standard (5 May 2004) has just reached our hands,
* and this code will be reviewed against it and revised accordingly
* in the near future. (All opinions expressed in this
* paragraph are those of the programmer.)
*
* JPEG 2000 format is not JPEG format, and isn't compatible
* with it. As with JPEG, JPEG 2000 is not in itself
* a file format. It can be encapsulated in JP2 or JPX format,
* which are recognized here.
*
* @author Gary McGath
*/
public class Jpeg2000Module extends ModuleBase {
/*
* Some general notes on JPEG 2000 parsing:
* The format started out as a straightforward stream format, which
* could be parsed as a stream from beginning to end. But JPX throws
* us several curves, requiring random access. The Cross Reference
* Box is a direction to replace itself with another box somewhere
* else in the file (or even in one or more entirely different files!), which
* may be scattered through different parts of the file even if it
* doesn't spread to other files. The Fragment Table Box does the
* same for codestreams. In addition, the Binary Filter Box allows
* boxes to be hidden within its compressed or encrypted data. All this
* makes the concept of "parent box" trickier than it is in unextended
* JP2, since parenthood and containment are not necessarily the same thing.
*
* The way I've chosen deal with this rather chimerical design is to
* have a RandomAccessFile for a base, but have each Box based on a
* DataInputStream. Not every Box has its own separate stream -- that
* would be wasteful -- but there can be multiple box streams active at
* once. Each box (JP2Box) has a parent (BoxHolder), which can be
* another JP2Box or a TopLevelBoxHolder. The extension to multiple
* files isn't supported here, since the job of Jhove is to validate
* individual files, but it wouldn't be difficult to add.
*
* Every BoxHolder (hence every box) implements the Iterator interface,
* so that a box can get its subboxes in a way which is blind to the
* details of their encoding.
*/
/******************************************************************
* PRIVATE CLASS FIELDS.
******************************************************************/
private static final String NAME = "JPEG2000-hul";
private static final String RELEASE = "1.3";
private static final int[] DATE = { 2007, 1, 8 };
private static final String[] FORMAT = { "JPEG 2000", "JP2", "JPX" };
private static final String COVERAGE = "JP2 (ISO/IEC 15444-1:2000/" +
"ITU-T Rec. T.800 (200)), JPX (ISO/IEC 15444-2:2004)";
private static final String[] MIMETYPE = { "image/jp2", "image/jpx" };
private static final String WELLFORMED =
"The required Signature and File Type box structures are the first " +
"two boxes in the file; all boxes required by a given profile exist " +
"in the file; all box structures are well-formed (a four byte " +
"unsigned integer Box Length, followed by a four byte unsigned " +
"integer Box type, followed by a eight byte unsigned integer Box " +
"Length, followed by the Box Contents); no data exist before the " +
"first byte of the first box or after the last byte of the last box";
private static final String VALIDITY = "The file is well-formed";
private static final String REPINFO =
"Properties capturing the technical attributes of the JPEG 2000 " +
"image from all boxes";
private static final String NOTE = null;
private static final String RIGHTS =
"Copyright 2004-2007 by JSTOR and the "
+ "President and Fellows of Harvard College. "
+ "Released under the GNU Lesser General Public License.";
/******************************************************************
* PRIVATE INSTANCE FIELDS.
******************************************************************/
/* NISO image metadata for the current image */
protected NisoImageMetadata _niso;
/* NISO image metadata for default image values */
protected NisoImageMetadata _defaultNiso;
/* DataInputSream for reading the file */
protected DataInputStream _dstream;
/* RandomAccessFile for reading the file */
protected RandomAccessFile _raf;
/* RAFInputStream underlying the DataInputStream */
protected RAFInputStream _rafStream;
/* Properties which are global to the file */
protected List<Property> _propList;
/* List of codestreams. An entry can be created by
* either a codestream or a codestream header, depending
* on which is seen first. The elements of the List
* are Codestream objects. */
protected List<Codestream> codestreams;
/* List of Binary Filter properties. */
protected List<Property> binaryFilterProps;
/* List of Association properties. */
protected List<Property> associationProps;
/* List of Digital Signature properties. */
protected List<Property> digitalSigProps;
/* Number of Contiguous Codestreams seen.
* May be less than or equal to the size of codestreams. */
protected int nCodestreams;
/* Number of Codestream headers seen */
protected int nCodestreamHeaders;
/* List of Color Spec properties */
protected List<Property> colorSpecs;
/* List of UUIDBox properties */
protected List<Property> uuids;
/* List of Compositing Layer properties */
protected List<Property> composLayers;
/* List of UUID Info properties */
protected List<Property> uuidInfos;
/* List of data (String) extracted from XML boxes */
protected List<String> xmlList;
/* Flag for JP2 headerbox detection */
protected boolean jp2HdrSeen;
/* Flag for Reader Requirements detection */
protected boolean rreqSeen;
/* Flag for Color Specification detection */
protected boolean colorSpecSeen;
/* Flag for Image Header Box detection */
protected boolean imageHeaderSeen;
/* Flag for JP2 compliance */
protected boolean jp2Compliant;
/* Flag for JPX compliance */
protected boolean jpxCompliant;
/* The Codestream currently being worked on. This
* is an element of codestreams */
protected Codestream curCodestream;
/* Flag which is true when we are reading a
* BinaryFilterBox, which needs special handling
* for byte counts. */
protected boolean filterMode;
/* Fixed value for first 12 bytes */
private static final int[] sigByte =
{
0X00,
0X00,
0X00,
0X0C,
0X6A,
0X50,
0X20,
0X20,
0X0D,
0X0A,
0X87,
0X0A };
/******************************************************************
* CLASS CONSTRUCTOR.
******************************************************************/
/**
* Instantiate a <tt>JpegModule</tt> object.
*/
public Jpeg2000Module() {
super(
NAME,
RELEASE,
DATE,
FORMAT,
COVERAGE,
MIMETYPE,
WELLFORMED,
VALIDITY,
REPINFO,
NOTE,
RIGHTS,
true);
Agent agent =
new Agent("Harvard University Library", AgentType.EDUCATIONAL);
agent.setAddress(
"Office for Information Systems, "
+ "90 Mt. Auburn St., "
+ "Cambridge, MA 02138");
agent.setTelephone("+1 (617) 495-3724");
agent.setEmail("jhove-support@hulmail.harvard.edu");
_vendor = agent;
Document doc = new Document ("Information technology -- " +
"JPEG 2000 image coding system -- Part 1: Code coding system",
DocumentType.STANDARD);
Agent isoAgent = new Agent ("ISO", AgentType.STANDARD);
isoAgent.setAddress ("1, rue de Varembe, Casa postale 56, " +
"CH-1211, Geneva 20, Switzerland");
isoAgent.setTelephone ("+41 22 749 01 11");
isoAgent.setFax ("+41 22 733 34 30");
isoAgent.setEmail ("iso@iso.ch");
isoAgent.setWeb ("http://www.iso.org/");
doc.setAuthor (isoAgent);
doc.setIdentifier (new Identifier ("ISO/IEC 15444-1:2000",
IdentifierType.ISO));
doc.setDate ("2002-07-31");
_specification.add (doc);
doc = new Document ("Information technology -- " +
"JPEG 2000 image coding system -- " +
"Part 2: Extensions",
DocumentType.STANDARD);
doc.setAuthor (isoAgent); // ISO agent
doc.setIdentifier (new Identifier ("ISO/IEC 15444-2:2004",
IdentifierType.ISO));
doc.setDate ("2004-05-15");
_specification.add (doc);
doc = new Document ("MIME Type Registrations for JPEG 2000 " +
"(ISO/IEC 15444) RFC 3745",
DocumentType.RFC);
Agent ietfAgent = new Agent ("IETF", AgentType.STANDARD);
ietfAgent.setWeb ("http://www.ietf.org");
doc.setPublisher (ietfAgent);
agent = new Agent ("D. Singer",
AgentType.OTHER);
doc.setAuthor(agent);
agent = new Agent ("R. Clark",
AgentType.OTHER);
doc.setAuthor(agent);
agent = new Agent ("D. Lee",
AgentType.OTHER);
doc.setAuthor(agent);
doc.setDate ("2004-04");
Identifier ident = new Identifier
("http://www.ietf.org/rfc/rfc3745.txt",
IdentifierType.URL);
doc.setIdentifier (ident);
_specification.add (doc);
doc = new Document ("ITU-T Rec. T.800 (2002), Information " +
"technology -- JPEG 2000 image coding system: " +
"Core coding system",
DocumentType.STANDARD);
Agent ituAgent = new Agent ("ITU", AgentType.STANDARD);
ituAgent.setAddress("ITU, Place des Nations, " +
"CH-1211 Geneva 20 Switzerland");
ituAgent.setTelephone("+41 22 730 51 11");
ituAgent.setFax("+41 22 730 6500");
ituAgent.setEmail ("itumail@itu.int");
ituAgent.setWeb ("http://www.itu.int/home/");
doc.setAuthor (ituAgent);
doc.setDate ("2002-08");
ident = new Identifier ("ITU-T Rec. T.800 (2002)",
IdentifierType.ITU);
doc.setIdentifier (ident);
_specification.add (doc);
Signature sig =
new InternalSignature (sigByte, SignatureType.MAGIC,
SignatureUseType.MANDATORY, 0,
"");
_signature.add (sig);
sig =
new ExternalSignature(
".jp2",
SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add(sig);
sig =
new ExternalSignature(
".jpx",
SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add(sig);
sig =
new ExternalSignature(
".jpf",
SignatureType.EXTENSION,
SignatureUseType.OPTIONAL);
_signature.add(sig);
// Macintosh signature for JP2 files
sig =
new ExternalSignature(
"jp2 ",
SignatureType.FILETYPE,
SignatureUseType.OPTIONAL);
_signature.add(sig);
// Macintosh signature for JPX files
sig =
new ExternalSignature(
"jpx ",
SignatureType.FILETYPE,
SignatureUseType.OPTIONAL);
_signature.add(sig);
_bigEndian = true;
}
/**
* Parse the content of a stream digital object and store the
* results in RepInfo.
*
* This module is based on a RandomAccessFile because of the
* requirements of the (so far) rarely used fragmented codestream
* feature. Since just about everything else can be done with
* an InputStream, we use a RAFInputStream except on the occasions
* when random access is needed. We pass the module as the <code>counted</code>
* argment to all read calls, so that we can compute relative
* positions in the stream based on _nByte.
*
* @param raf A RandomAccessFile to be parsed.
*
* @param info A fresh (on the first call) RepInfo object
* which will be modified
* to reflect the results of the parsing
* If multiple calls to <code>parse</code> are made
* on the basis of a nonzero value being returned,
* the same RepInfo object should be passed with each
* call.
*/
public final void parse(RandomAccessFile raf, RepInfo info)
throws IOException {
initParse();
_rafStream = new RAFInputStream (raf,
_je != null ? _je.getBufferSize() : 0);
_dstream = new DataInputStream
(_rafStream);
info.setFormat(_format[0]);
info.setMimeType(_mimeType[0]);
info.setModule(this);
_propList = new ArrayList<Property>(12);
Property metadata =
new Property(
"JPEG2000Metadata",
PropertyType.PROPERTY,
PropertyArity.LIST,
_propList);
_raf = raf;
// A JPEG 2000 file consists of a series of "boxes."
// The signature box must match the signature bytes.
int i = 0;
boolean badhdr = false;
try {
for (i = 0; i < 12; i++) {
int ch;
ch = readUnsignedByte(_dstream, this);
if (ch != sigByte[i]) {
badhdr = true;
break;
}
}
}
catch (IOException e) {
badhdr = true;
}
if (badhdr) {
info.setMessage (new ErrorMessage ("No JPEG 2000 header", i));
info.setWellFormed(false);
return;
}
/* If we got this far, take note that the signature is OK. */
info.setSigMatch(_name);
// Next check the file type box.
if (!readFileTypeBox(info)) {
return;
}
// Go through the rest of the boxes.
if (!readBoxes(info)) {
return;
}
if (info.getWellFormed() == RepInfo.FALSE) {
return;
}
// File has been read successfully; do final processing.
info.setProperty(metadata);
// Calculate checksums, if necessary.
if (_je != null && _je.getChecksumFlag ()) {
if (info.getChecksum ().size () == 0) {
Checksummer ckSummer = new Checksummer ();
calcRAChecksum (ckSummer, raf);
setChecksums (ckSummer, info);
}
}
// Reader Requirements box is mandatory for JPX
if (!rreqSeen || info.getValid() != RepInfo.TRUE) {
jpxCompliant = false;
}
if (!imageHeaderSeen || !colorSpecSeen || info.getValid() != RepInfo.TRUE) {
jp2Compliant = false;
}
if (jp2Compliant) {
info.setProfile("JP2");
}
if (jpxCompliant) {
info.setProfile ("JPX");
String mime = _mimeType[1];
info.setMimeType(mime);
curCodestream.getNiso().setMimeType (mime);
// This doesn't deal with a case where some
// codestreams are JP2 and others are JPX;
// can that happen?
_defaultNiso.setMimeType (mime);
}
if (!colorSpecs.isEmpty()) {
_propList.add(
new Property(
"ColorSpecs",
PropertyType.PROPERTY,
PropertyArity.LIST,
colorSpecs));
}
if (!binaryFilterProps.isEmpty ()) {
_propList.add(
new Property(
"BinaryFilters",
PropertyType.PROPERTY,
PropertyArity.LIST,
binaryFilterProps));
}
if (!associationProps.isEmpty ()) {
_propList.add(
new Property(
"Associations",
PropertyType.PROPERTY,
PropertyArity.LIST,
associationProps));
}
if (!digitalSigProps.isEmpty ()) {
_propList.add(
new Property(
"DigitalSignatures",
PropertyType.PROPERTY,
PropertyArity.LIST,
digitalSigProps));
}
if (!uuids.isEmpty()) {
_propList.add(
new Property(
"UUIDs",
PropertyType.PROPERTY,
PropertyArity.LIST,
uuids));
}
if (!composLayers.isEmpty ()) {
_propList.add(
new Property(
"CompositingLayers",
PropertyType.PROPERTY,
PropertyArity.LIST,
composLayers));
}
if (!uuidInfos.isEmpty()) {
_propList.add(
new Property(
"UUIDInfoBoxes",
PropertyType.PROPERTY,
PropertyArity.LIST,
uuidInfos));
}
if (!codestreams.isEmpty()) {
List<Property> csProps = new ArrayList<Property>(codestreams.size());
ListIterator<Codestream> csIter = codestreams.listIterator();
while (csIter.hasNext()) {
Codestream cs = (Codestream) csIter.next();
csProps.add(cs.makeProperty());
}
_propList.add(
new Property(
"Codestreams",
PropertyType.PROPERTY,
PropertyArity.LIST,
csProps));
}
if (!xmlList.isEmpty()) {
_propList.add(
new Property(
"XML",
PropertyType.STRING,
PropertyArity.LIST,
xmlList));
}
return;
}
/**
* Return the current position in the module.
* This is somewhat optimistically named; it should
* be trusted only for relative positions when no
* seek operations occur in between calls to getFilePos.
*/
public long getFilePos() {
// return _nByte;
try {
return _rafStream.getFilePos ();
}
catch (IOException e) {
// really shouldn't happen
return 0;
}
}
/** Seek to a new file position. */
public void seek (long pos) throws IOException {
_rafStream.seek (pos);
}
/** Returns the default NisoImageMetadata object. */
public NisoImageMetadata getDefaultNiso() {
return _defaultNiso;
}
/** Returns the current NisoImageMetadata object.
* If curCodestream is null, returns the default
* NisoImageMetadata, otherwise returns the
* NisoImageMetadata of curCodestream.
*/
public NisoImageMetadata getCurrentNiso() {
if (curCodestream == null) {
return _defaultNiso;
} else {
return curCodestream.getNiso();
}
}
/** Returns the nth <code>Codestream</code>, creating it if necessary,
* and make it the current one. The value of <code>nCodestreams</code>
* or <code>nCodestreamHeaders</code>
* is not affected, even if a new <code>Codestream</code> is created. */
public Codestream getCodestream(int n) {
Codestream cs;
if (n < codestreams.size()) {
cs = (Codestream) codestreams.get(n);
} else {
cs = new Codestream();
cs.setDefaultNiso(_defaultNiso);
codestreams.add(cs);
}
curCodestream = cs;
return cs;
}
/** Returns the codestream count.
* Because items may be added to the codestreams list
* when only the header is seen, this may be less than
* the size of the codestreams list. It will never be
* more. */
public int getNCodestreams() {
return nCodestreams;
}
/** Returns the codestream header count. */
public int getNCodestreamHeaders() {
return nCodestreamHeaders;
}
public boolean isJP2HdrSeen() {
return jp2HdrSeen;
}
/** Sets the codestream count. This affects only the
* variable, not the size of the codestreams list.
* The codestream count should never be set to a value
* larger than the codestreams list; it signifies the
* number of elements of the list for which codestream
* boxes have actually been seen.
*/
public void setNCodestreams(int n) {
nCodestreams = n;
}
/** Sets the codestream header count. */
public void setNCodestreamHeaders(int n) {
nCodestreamHeaders = n;
}
/** Set the flag indicating that a JP2 header
* has been seen. */
public void setJP2HdrSeen(boolean b) {
jp2HdrSeen = b;
}
/** Set the flag indicating the reader requirements box
* has been seen. */
public void setRReqSeen (boolean b) {
rreqSeen = b;
}
/** Set the flag indicating the color specification box
* has been seen. */
public void setColorSpecSeen (boolean b) {
colorSpecSeen = b;
}
/** Set the flag indicating the color specification box
* has been seen. */
public void setImageHeaderSeen (boolean b) {
imageHeaderSeen = b;
}
/** Sets a flag indicating JP2 compliance. If the
* flag is set to <code>true</code>, and the JPX
* compliance flag is also true, set the MIME type
* to "image/jpx". */
public void setJP2Compliant(boolean b) {
jp2Compliant = b;
if (jp2Compliant && jpxCompliant) {
_defaultNiso.setMimeType(MIMETYPE[1]);
}
}
/** Sets a flag indicating JPX compliance. */
public void setJPXCompliant(boolean b) {
jpxCompliant = b;
}
/** Adds a property to the JPEG2000 metadata.*/
public void addProperty(Property p) {
_propList.add(p);
}
/** Adds a color spec property to the metadata. */
public void addColorSpec(Property p) {
colorSpecs.add(p);
}
/** Adds a binary filter property to the metadata. */
public void addBinaryFilterProp (Property p) {
binaryFilterProps.add (p);
}
/** Adds an association property to the metadata. */
public void addAssociationProp (Property p) {
associationProps.add (p);
}
/** Adds a digital signature property to the metadata. */
public void addDigitalSignatureProp (Property p)
{
digitalSigProps.add (p);
}
/** Adds a UUID property to the list of UUID
* properties. Called from the UUIDBox.
*/
public void addUUID(Property p) {
uuids.add(p);
}
/** Adds a UUIDInfo property to the list of UUIDInfo
* properties. Called from UUIDInfoBox.
*/
public void addUUIDInfo(Property p) {
uuidInfos.add(p);
}
/** Adds a Compositing Layer property to the list
* of Compositing Layer properties. Called
* from the ComposLayerHdrBox. */
public void addComposLayer (Property p) {
composLayers.add (p);
}
/** Adds an XML string to the list of XML properties.
* Called from XMLBox. */
public void addXML(String s) {
xmlList.add(s);
}
/**
* Reads 4 bytes and concatenates them into a String.
*/
public String read4Chars(DataInputStream stream) throws IOException {
StringBuffer sbuf = new StringBuffer(4);
for (int i = 0; i < 4; i++) {
int ch = readUnsignedByte(stream, this);
sbuf.append((char) ch);
}
return sbuf.toString();
}
/** One-argument version of <code>readUnsignedShort</code>.
* JPEG2000 is always big-endian, so readUnsignedShort can
* unambiguously drop its endian argument. */
public int readUnsignedShort(DataInputStream stream) throws IOException {
return readUnsignedShort(stream, true, this);
}
/** One-argument version of <code>readUnsignedInt</code>.
* JPEG2000 is always big-endian, so readUnsignedInt can
* unambiguously drop its endian argument. */
public long readUnsignedInt(DataInputStream stream) throws IOException {
return readUnsignedInt(stream, true, this);
}
/** One-argument version of <code>readSignedLong</code>.
* JPEG2000 is always big-endian, so readSignedLong can
* unambiguously drop its endian argument. */
public long readSignedLong(DataInputStream stream) throws IOException {
return readSignedLong(stream, true, this);
}
/**
* Initializes the state of the module for parsing.
*/
protected void initParse() {
super.initParse();
colorSpecs = new LinkedList<Property>();
binaryFilterProps = new LinkedList<Property> ();
associationProps = new LinkedList<Property> ();
digitalSigProps = new LinkedList<Property> ();
uuids = new LinkedList<Property>();
uuidInfos = new LinkedList<Property>();
composLayers = new LinkedList<Property> ();
xmlList = new LinkedList<String>();
//uuidList = new LinkedList ();
codestreams = new LinkedList<Codestream>();
curCodestream = null;
nCodestreams = 0;
nCodestreamHeaders = 0;
jp2HdrSeen = false;
//paletteSeen = false;
//cmSeen = false;
rreqSeen = false;
filterMode = false;
_defaultNiso = new NisoImageMetadata();
_defaultNiso.setByteOrder("big-endian");
_defaultNiso.setMimeType(MIMETYPE[0]);
// Compliance flags are innocent till proven guilty
jp2Compliant = true;
jpxCompliant = true;
}
/* Dispatcher for reading boxes. Because of the existence
* of filter boxes, any box can come from either _dstream
* or from a separate input stream that is being undeflated.
* This dispatcher always works from _dstream. We must
* pass a stream to every box function, to allow these
* multiple sources.
*/
protected boolean readBoxes(RepInfo info) throws IOException {
// From here on, boxes may occur with some freedom
// of order. Apparently the only indication that
// we're done is an end-of-file condition.
TopLevelBoxHolder bh = new TopLevelBoxHolder (this, _raf, info,
_dstream);
while (bh.hasNext ()) {
JP2Box box = (JP2Box) bh.next ();
// TopLevelBoxHolder.next() may not be reliable about detecting
// that no more boxes are left.
if (box == null) {
break;
}
if (!box.readBox ()) {
return false;
}
}
return true;
}
/* Read the file type box. This assumes that we are actually
* positioned at the file type box, and reads its header.
* If we get something other than a file type box, the file
* is not well-formed.
*/
protected boolean readFileTypeBox(RepInfo info) throws IOException {
BoxHeader hdr = new BoxHeader(this, _dstream);
hdr.readHeader();
// 8 bytes have been read
if (!"ftyp".equals(hdr.getType())) {
info.setMessage(
new ErrorMessage(
"Expected File Type Box, got " + hdr.getType(),
_nByte));
info.setWellFormed(false);
return false;
}
FileTypeBox box = new FileTypeBox(_raf);
box.setBoxHeader(hdr);
box.setDataInputStream(_dstream);
box.setRandomAccessFile (_raf);
box.setModule(this);
box.setRepInfo(info);
if (!box.readBox()) {
return false;
}
return true;
}
/* This is called for any box that isn't recognized, or
* by placeholder methods for boxes that haven't yet
* been coded. */
protected boolean skipOverBox(
BoxHeader hdr,
RepInfo info,
DataInputStream dstrm)
throws IOException {
if (hdr.getLength() != 0) {
skipBytes(dstrm, (int) hdr.getDataLength(), this);
}
return true;
}
}
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More