Uploaded image for project: 'NetarchiveSuite'
  1. NetarchiveSuite
  2. NAS-2015

OutOfMemoryError : Java heap space when trying to upload seeds from a large file (55 MB)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Major
    • I51, 3.20.0
    • 3.16.1
    • None
    • None
    • SB/KB
    • Hide

      Try to add seeds from a file to selective harvest. A candidate file (55 MB seed list) is located on KB internal network:
      \\SQL-1\KB-Faelles\Netarkivet\FacebookProfilerSamletUdg.txt

      Show
      Try to add seeds from a file to selective harvest. A candidate file (55 MB seed list) is located on KB internal network: \\SQL-1\KB-Faelles\Netarkivet\FacebookProfilerSamletUdg.txt

    Description

      Trying to upload 55 MB seed list with facebook URL's. The file is located here :
      \\SQL-1\KB-Faelles\Netarkivet\FacebookProfilerSamletUdg.txt

      After 1 minut I get :

      java.lang.OutOfMemoryError: Java heap space
      at java.util.Arrays.copyOf(Arrays.java:2882)
      at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
      at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
      at java.lang.StringBuffer.append(StringBuffer.java:224)
      at java.util.logging.SimpleFormatter.format(SimpleFormatter.java:67)
      at java.util.logging.StreamHandler.publish(StreamHandler.java:179)
      at java.util.logging.FileHandler.publish(FileHandler.java:555)
      at java.util.logging.Logger.log(Logger.java:458)
      at java.util.logging.Logger.doLog(Logger.java:480)
      at java.util.logging.Logger.logp(Logger.java:596)
      at org.apache.commons.logging.impl.Jdk14Logger.log(Jdk14Logger.java:91)
      at org.apache.commons.logging.impl.Jdk14Logger.warn(Jdk14Logger.java:242)
      at dk.netarkivet.common.utils.DBUtils.setClobMaxLength(DBUtils.java:622)
      at dk.netarkivet.harvester.datamodel.DomainDBDAO.insertSeedlist(DomainDBDAO.java:741)
      at dk.netarkivet.harvester.datamodel.DomainDBDAO.updateSeedlists(DomainDBDAO.java:423)
      at dk.netarkivet.harvester.datamodel.DomainDBDAO.update(DomainDBDAO.java:280)
      at dk.netarkivet.harvester.datamodel.PartialHarvest.addSeeds(PartialHarvest.java:463)
      at dk.netarkivet.harvester.webinterface.EventHarvest.addConfigurationsFromSeedsFile(EventHarvest.java:198)
      at org.apache.jsp.Definitions_002dadd_002devent_002dseeds_jsp._jspService(org.apache.jsp.Definitions_002dadd_002devent_002dseeds_jsp:187)
      at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:389)
      at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
      at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
      at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
      at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
      at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
      at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
      at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
      at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
      at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)

      Attachments

        Activity

          People

            svc Søren Vejrup Carlsen (Inactive)
            svc Søren Vejrup Carlsen (Inactive)
            Colin Rosenthal Colin Rosenthal
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: