better handling of ligatures: fixes #58

OCRmyPDF.sh

@@ -46,7 +46,6 @@ Usage: OCRmyPDF.sh  [-h] [-v] [-g] [-k] [-d] [-c] [-i] [-o dpi] [-f] [-l languag
 -C : Pass an additional configuration file to the tesseract OCR engine.
      (this option can be used more than once)
      Note 1: The configuration file must be available in the "tessdata/configs" folder of your tesseract installation
-     Note 2: The folder "./tess-cfg" contains useful tesseract configuration files
 inputfile  : PDF file to be OCRed
 outputfile : The PDF/A file that will be generated 
 --------------------------------------------------------------------------------------

src/hocrTransform.py

@@ -1,4 +1,5 @@
 #!/usr/local/bin/python2
+# coding: utf-8
 ##############################################################################
 # Copyright (c) 2013-14: fritz-hh from Github (https://github.com/fritz-hh)
 #
@@ -171,6 +172,17 @@ class hocrTransform():
 		"""
 		return float(pxl)/self.dpi*inch
 
+	def replace_unsupported_chars(self, str):
+		"""
+		Given an input string, returns the corresponding string that:
+		- is available in the helvetica facetype
+		- does not contain any ligature (to allow easy search in the PDF file)
+		"""		
+		# The 'u' before the character to replace indicates that it is a unicode character
+		str=str.replace(u"fl","fl")
+		str=str.replace(u"fi","fi")
+		return str
+		
 	def to_pdf(self, outFileName, imageFileName, showBoundingboxes, fontname="Helvetica"):
 		"""
 		Creates a PDF file with an image superimposed on top of the text.
@@ -217,6 +229,9 @@ class hocrTransform():
 		for elem in self.hocr.findall(".//%sspan[@class='%s']" % (self.xmlns, elemclass)):
 
 			elemtxt=self._get_element_text(elem).rstrip()
+			
+			elemtxt=self.replace_unsupported_chars(elemtxt)
+			
 			if len(elemtxt) == 0:
 				continue
 

tess-cfg/README.md

@@ -1,12 +0,0 @@
-##############################################################################
-# Readme
-#
-# Copyright (c) 2013-14: fritz-hh from Github (https://github.com/fritz-hh)
-##############################################################################
-
-The file(s) located in this folder are tesseract configuration files.
-(Tesseract configuration files enable to tune the behaviour of tesseract)
-
-If needed, these files should be copied to the "tessdata/configs" folder of your tesseract installation.
-
-To request OCRmyPDF.sh to use a configuration file, please use the -C option

tess-cfg/no_ligature

@@ -1,9 +0,0 @@
-##############################################################################
-# tesseract config file provided for OCRmyPDF
-#
-# prevents tesseract to detect ligatures, as ligatures are not displayed correctly in the final PDF file
-# but are replaced by a black square
-#
-# Copyright (c) 2013-14: fritz-hh from Github (https://github.com/fritz-hh)
-##############################################################################
-tessedit_char_blacklist fifl