Details
-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
3.8
-
None
Description
When requesting for uncompressed indices using the new CreateIndex tool
I get the error seen below. It seems that it assumes that all indices are compressed during transmission.
The createIndex currently only works for indices of type dedup:
java -Dsettings.common.cacheDir=/tmp/cache -Dsettings.common.environmentName=QUICKSTART -Dsettings.common.remoteFile.port=5000 dk.netarkivet.archive.tools.CreateIndex -t cdx -l 1
May 19 2009 5:54:56 PM dk.netarkivet.archive.indexserver.FileBasedCache <init>
INFO: Metadata cache for 'CDX' uses directory '/tmp/cache/CDX'
Creating CDX index for ids: [1]
May 19 2009 5:54:56 PM dk.netarkivet.archive.indexserver.distribute.IndexRequestClient cacheData
INFO: Requesting an index of type 'CDX' for the jobs [1]
May 19 2009 5:54:56 PM dk.netarkivet.common.utils.Settings getAll
WARNING: The list of loaded data settings is empty.Is this OK?
May 19 2009 5:54:56 PM dk.netarkivet.common.distribute.JMSConnectionSunMQ <init>
INFO: Creating instance of dk.netarkivet.common.distribute.JMSConnectionSunMQ
May 19 2009 5:54:56 PM dk.netarkivet.common.distribute.JMSConnection initConnection
INFO: Initializing a JMS connection of type 'class dk.netarkivet.common.distribute.JMSConnectionSunMQ' to Broker at localhost:7676.
May 19 2009 5:54:57 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Logging to org.slf4j.impl.JCLLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
May 19 2009 5:54:57 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: jetty-6.1.6
May 19 2009 5:54:57 PM org.slf4j.impl.JCLLoggerAdapter info
INFO: Started SocketConnector@0.0.0.0:5000
May 19 2009 5:54:57 PM dk.netarkivet.archive.indexserver.distribute.IndexRequestClient cacheData
WARNING: IOFailure during unzipping of index
dk.netarkivet.common.exceptions.IOFailure: Error ungzipping './tests/commontempdir/remotefile-unzip16976.gz'
at dk.netarkivet.common.utils.ZipUtils.gunzipFile(ZipUtils.java:300)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.unzipAndDeleteRemoteFile(IndexRequestClient.java:258)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.cacheData(IndexRequestClient.java:194)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.cacheData(IndexRequestClient.java:59)
at dk.netarkivet.archive.indexserver.FileBasedCache.cache(FileBasedCache.java:164)
at dk.netarkivet.archive.indexserver.FileBasedCache.getIndex(FileBasedCache.java:229)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.getIndex(IndexRequestClient.java:59)
at dk.netarkivet.archive.tools.CreateIndex.main(CreateIndex.java:114)
Caused by: java.io.IOException: Not in GZIP format
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:137)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68)
at dk.netarkivet.common.utils.LargeFileGZIPInputStream.<init>(LargeFileGZIPInputStream.java:56)
at dk.netarkivet.common.utils.ZipUtils.gunzipFile(ZipUtils.java:296)
... 7 more
May 19 2009 5:54:57 PM dk.netarkivet.archive.indexserver.FileBasedCache getIndex
INFO: Requested index of type 'CDX' data '[1]' not available. Retrying with available subset '[]'
May 19 2009 5:54:57 PM dk.netarkivet.archive.indexserver.distribute.IndexRequestClient cacheData
INFO: Requesting an index of type 'CDX' for the jobs []
May 19 2009 5:54:57 PM dk.netarkivet.archive.indexserver.distribute.IndexRequestClient cacheData
WARNING: IOFailure during unzipping of index
dk.netarkivet.common.exceptions.IOFailure: Error ungzipping './tests/commontempdir/remotefile-unzip16977.gz'
at dk.netarkivet.common.utils.ZipUtils.gunzipFile(ZipUtils.java:300)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.unzipAndDeleteRemoteFile(IndexRequestClient.java:258)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.cacheData(IndexRequestClient.java:194)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.cacheData(IndexRequestClient.java:59)
at dk.netarkivet.archive.indexserver.FileBasedCache.cache(FileBasedCache.java:164)
at dk.netarkivet.archive.indexserver.FileBasedCache.getIndex(FileBasedCache.java:229)
at dk.netarkivet.archive.indexserver.distribute.IndexRequestClient.getIndex(IndexRequestClient.java:59)
at dk.netarkivet.archive.tools.CreateIndex.main(CreateIndex.java:114)
Caused by: java.io.EOFException
at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:207)
at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197)
at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58)
at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68)
at dk.netarkivet.common.utils.LargeFileGZIPInputStream.<init>(LargeFileGZIPInputStream.java:56)
at dk.netarkivet.common.utils.ZipUtils.gunzipFile(ZipUtils.java:296)
... 7 more