[NAS-2235] Test data for TEST9 is too old Created: 12/Aug/13  Updated: 17/Sep/15  Resolved: 12/Aug/13

Status: Closed
Project: NetarchiveSuite
Component/s: Test
Affects Version/s: None
Fix Version/s: 4.2

Type: Bug Priority: Minor
Reporter: Colin Rosenthal Assignee: Søren Vejrup Carlsen (Inactive)
Resolution: Fixed  
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File 1-cache.tar.bz2    

 Description   

In the first part of TEST9:

[test@kb-test-adm-001 TEST9]$ java -Ddk.netarkivet.settings.file=$HOME/$TESTX/conf/settings_ArcRepositoryApplication.xml -Dsettings.common.applicationInstanceId=record dk.netarkivet.archive.tools.GetRecord /tmp/$TESTX/test-index http://www.pligtaflevering.dk/online/vejledning.pdf > x.pdf
log4j:WARN No appenders could be found for logger (dk.netarkivet.archive.arcrepository.distribute.JMSArcRepositoryClient).
log4j:WARN Please initialize the log4j system properly.
An error occurred during processing.
Output (if any) is not OK
Exception message is:
NetarkivetException while performing ARCArchiveAccess.lookup
Stack trace:
dk.netarkivet.common.exceptions.IOFailure: NetarkivetException while performing ARCArchiveAccess.lookup
	at dk.netarkivet.archive.tools.GetRecord$GetRecordTool.run(GetRecord.java:146)
	at dk.netarkivet.common.tools.ToolRunnerBase.runIt(ToolRunnerBase.java:84)
	at dk.netarkivet.common.tools.ToolRunnerBase.setupAndRunIt(ToolRunnerBase.java:94)
	at dk.netarkivet.common.tools.ToolRunnerBase.checkArgsSetupAndRun(ToolRunnerBase.java:110)
	at dk.netarkivet.common.tools.ToolRunnerBase.internalRunTheTool(ToolRunnerBase.java:119)
	at dk.netarkivet.common.tools.ToolRunnerBase.runTheTool(ToolRunnerBase.java:73)
	at dk.netarkivet.archive.tools.GetRecord.main(GetRecord.java:69)
Caused by: dk.netarkivet.common.exceptions.IOFailure: Unable to find/open index /tmp/TEST9/test-index
	at dk.netarkivet.common.distribute.arcrepository.ARCLookup.setIndex(ARCLookup.java:125)
	at dk.netarkivet.archive.tools.GetRecord$GetRecordTool.run(GetRecord.java:137)
	... 6 more
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource: ChecksumIndexInput(NIOFSIndexInput(path="/tmp/TEST9/test-index/segments"))): -1 (needs to be between -9 and -11). This version of Lucene only supports indexes created with release 3.0 and later.
	at org.apache.lucene.codecs.lucene3x.Lucene3xSegmentInfoReader.readLegacySegmentInfo(Lucene3xSegmentInfoReader.java:128)
	at org.apache.lucene.codecs.lucene3x.Lucene3xSegmentInfoReader.readLegacyInfos(Lucene3xSegmentInfoReader.java:54)
	at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:345)
	at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:56)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:812)
	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:66)
	at dk.netarkivet.common.distribute.arcrepository.ARCLookup.setIndex(ARCLookup.java:121)
	... 7 more


 Comments   
Comment by Colin Rosenthal [ 13/Aug/13 ]

I have replaced the test data on kb-prod-udv-001 with Sørens migrated index and confirmed that it works.

Comment by Colin Rosenthal [ 13/Aug/13 ]

The same error turns up in the Pure Viewerproxy section of TEST9 of course.

Comment by Colin Rosenthal [ 13/Aug/13 ]

If we need an IndexServer to generate the index, and the pure ArcRepository instance doesn't include an IndexServer, then this isn't a very realistic test scenario. I suggest we replace this step with running a batch job from the GUI. This will confirm that the ArcRepository is functioning correctly, which is the only goal of the test.

Comment by Søren Vejrup Carlsen (Inactive) [ 12/Aug/13 ]

The lucene index migrated to Lucene 3.6 format

Comment by Søren Vejrup Carlsen (Inactive) [ 12/Aug/13 ]

The method is found in NAS-2202.
Have attached the output from the conversion as a attachment

Comment by Søren Vejrup Carlsen (Inactive) [ 12/Aug/13 ]

This is actually a tool, we could also for generating data for our unittests.
The reason for this issue is, that lucene 4.X does not accept old indices from before Lucene 3, and the Indices are probably in Lucene 2.X format.

However I have already done the migration for the indices used by the unitest, so I'm assigning this issue to me.

Comment by Søren Vejrup Carlsen (Inactive) [ 12/Aug/13 ]

The TODO stems from me, but I don't have a tool for this.

I believe, that our testdata have been copied from indices generated by the Indexserver.

Comment by Colin Rosenthal [ 12/Aug/13 ]

The real bug is the comment "TODO: Mention how to make an luceneindex for your stored arcfiles."
in https://sbforge.org/display/NASDOC321/Tools+in+the+Archive+Module#ToolsintheArchiveModule-dknetarkivetarchivetoolsGetRecord !

Does anyone know how and want to share the information with the rest of us?

Generated at Sat Apr 20 04:59:35 CEST 2024 using Jira 9.4.15#940015-sha1:bdaa9cbecfb6791ea579749728cab771f0dfe90b.