Uploaded image for project: 'WebDanica'
  1. WebDanica
  2. WEBDAN-210

Language detection fails: java.lang.IllegalArgumentException: Probability must be <= 1 but was Infinity

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • None
    • None
    • ANALYSIS
    • None
    • webdanicasprint - sommer 2017

    Description

      org.apache.pig.backend.executionengine.ExecException: Exception while executing [POUserFunc (Name: POUserFunc(dk.kb.webdanica.core.criteria.CombinedCombo)[chararray] - scope-48 Operator Key: scope-48) children: null at []]: java.lang.IllegalArgumentException: Probability must be <= 1 but was Infinity
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:364) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:404) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:321) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:280) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:275) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:65) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) ~[hadoop-core-1.0.4.jar:na]
              at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[hadoop-core-1.0.4.jar:na]
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) ~[hadoop-core-1.0.4.jar:na]
              at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ~[hadoop-core-1.0.4.jar:na]
      Caused by: java.lang.IllegalArgumentException: Probability must be <= 1 but was Infinity
              at com.optimaize.langdetect.DetectedLanguage.<init>(DetectedLanguage.java:46) ~[language-detector-0.6-SNAPSHOT.jar:na]
              at com.optimaize.langdetect.LanguageDetectorImpl.sortProbability(LanguageDetectorImpl.java:257) ~[language-detector-0.6-SNAPSHOT.jar:na]
              at com.optimaize.langdetect.LanguageDetectorImpl.getProbabilities(LanguageDetectorImpl.java:139) ~[language-detector-0.6-SNAPSHOT.jar:na]
              at org.apache.tika.langdetect.OptimaizeLangDetector.detectAll(OptimaizeLangDetector.java:166) ~[tika-langdetect-1.13.jar:1.13]
              at org.apache.tika.language.detect.LanguageDetector.detectAll(LanguageDetector.java:230) ~[tika-core-1.13.jar:1.13]
              at dk.kb.webdanica.core.criteria.C4.computeNewC4(C4.java:33) ~[webdanica-core-1.1-RC8.jar:0e69e3f6cd9c66db199de04b5d864e5dbf4b7980]
              at dk.kb.webdanica.core.criteria.CombinedCombo.exec(CombinedCombo.java:368) ~[webdanica-core-1.1-RC8.jar:0e69e3f6cd9c66db199de04b5d864e5dbf4b7980]
              at dk.kb.webdanica.core.criteria.CombinedCombo.exec(CombinedCombo.java:116) ~[webdanica-core-1.1-RC8.jar:0e69e3f6cd9c66db199de04b5d864e5dbf4b7980]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNext(POUserFunc.java:326) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.getNextString(POUserFunc.java:426) ~[pig-0.16.0-core-h1.jar:na]
              at org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.getNext(PhysicalOperator.java:341) ~[pig-0.16.0-core-h1.jar:na]
              ... 9 common frames omitted
      17:30:45.788 [main] ERROR o.a.p.t.p.mapreduce.MRPigStatsUtil - 1 map reduce job(s) failed!
      Details at logfile: /opt/workflows/automatic-workflow/pig_1503502231455.log
      ERROR: pig call failed with exitcode 2
      

      Attachments

        Issue Links

          Activity

            People

              svc Søren Vejrup Carlsen (Inactive)
              svc Søren Vejrup Carlsen (Inactive)
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: