Hyphenation does not not work with 1.2.x

[Jaizkibel]

After switching to 1.2.2, the hyphenation does not work anymore.

The reason seems to be the UniversalDetector class which does not recognize the encoding of the XML files from the itext-hyph_xml.jar

Code snippet from class SimpleXMLParser:

public static void parse(SimpleXMLDocHandler doc, InputStream in) throws IOException {
        byte b4[] = new byte[4];
        int count = in.read(b4);
        if (count != 4)
            throw new IOException(MessageLocalization.getComposedMessage("insufficient.length"));
        UniversalDetector detector = new UniversalDetector(null);
        detector.handleData(b4, 0, count);
        String encoding = detector.getDetectedCharset();     // <--- returns null

        String decl = null;
        if (encoding.equals("UTF-8")) {    // <--- NPE
            ...

so the XML is not parsed and the hyphenation not done, because the HyphenationTree is empty

[andreasrosdal]

Thanks for the bugreport. Are you able to contribute a solution to this bug?

[andreasrosdal]

Can you please submit code to reproduce the problem?

[Jaizkibel]

This is how to reproduce (see also "iText in Action, 1 ed., page 120f)

1. put itext-hyph-xml.jar in the classpath
   This JAR contains XML-Files with the hyphenation rules for different countries

2. Instantiate a HyphenationAuto object:

HyphenationAuto hyphen = new HyphenationAuto("en", null, 2, 2);

The constructor parses the "en.xml" file and builds a HyphenTree object that should be visible in a debugger afer execution of the previous line.

This works fine in OpenPDF 1.1.0:

With OpenPDF 1.2.2 the HyphenTree is null, because the encoding of the en.xml file coud not be identified.

[andreasrosdal]

I have attempted a quick-fix for this. Can you please try the latest master version of OpenPDF?

This is the commit: cf834ab

[Jaizkibel]

Thanks for the patch.
Will try it out.

But it gonna take some time.
Busy at work and my private pc crashed :-(

[Jaizkibel]

Did a test after adding itext-hyph-xml.jar to the dependencies:

import static org.junit.Assert.*;

public class HypenationPdfTest {
  @Test
  public void testHyphenation() {
    // english
    HyphenationAuto hyphen = new HyphenationAuto("en", null, 2, 2);
    Hyphenator h = hyphen.hyphenator;

    Hyphenation s = h.hyphenate("incomprehensibilities");
    System.out.println(s);
    assertEquals(s.toString(), "in-com-pre-hens-ib-il-it-ies");

    // german
    hyphen = new HyphenationAuto("de", null, 2, 2);
    h = hyphen.hyphenator;

    s = h.hyphenate("Leistungsscheinziffer");
    System.out.println(s);
    assertEquals(s.toString(), "Lei-stungs-schein-zif-fer");

    s = h.hyphenate("Transaktionsdatum");
    System.out.println(s);
    assertEquals(s.toString(), "Trans-ak-ti-ons-da-tum");

    // with umlauts
    s = h.hyphenate("überschlagsmäßig");
    System.out.println(s);
    assertEquals(s.toString(), "über-schlags-mä-ßig");

  }

}

So everything works fine here.

[andreasrosdal]

Thanks. So this issue is fixed now?
Please reopen if the problem still exists.
I'll create a new release soon.

[andreasrosdal]

Please give the latest release of OpenPDF a try. Version 1.2.3.

[Jaizkibel]

Hyphenation works in 1.2.3 as expected.

Thanks a lot!

[arnthom]

Sorry to add this to the issue, but where do I find itext-hyph-xml.jar to add hyphenation to openPdf?
Is there a difference between iText's hyphenation jar and the one by openPDF? Or is it possible to use the one by iText without (licensing) problems?
Thanks a lot!

Edit: finally I found one with the right path "com/lowagie/text/pdf/hyphenation/hyph/" ... and found, that I was using the wrong juniversalchardet jar.
UniversalDetector.detectCharsetFromBOM is not available in older releases.
NOw hyphenation works as expected.