Loading...

XML

Word

Printable

Details

Type: Sub-task
Resolution: Fixed
Priority: Major
Fix Version/s: I53, 4.0
Affects Version/s: None
Component/s: Wayback
Labels:
None

Verification:

Hide

Using NAS w/ WARC writing enabled, make an event harvest combining two sites, www.netarkivet.dk and www.pligtaflevering.dk

Install standalone wayback, using proxy mode, and using NetarchiveResourceStore as ResourceStore, and pointing to settingsfile that uses LocalArcRepository.

Generate CDX using wayback-tool 'warc-indexer'

To be completed!

Show
Using NAS w/ WARC writing enabled, make an event harvest combining two sites, www.netarkivet.dk and www.pligtaflevering.dk Install standalone wayback, using proxy mode, and using NetarchiveResourceStore as ResourceStore, and pointing to settingsfile that uses LocalArcRepository. Generate CDX using wayback-tool 'warc-indexer' To be completed!

Description

We need to change the dk.netarkivet.wayback.NetarchiveResourceStore so it handles warc-records and not just arc-records fetched from the bitarchive.

We have already made it possible to access WARC-data from wayback using the NetarchiveCacheResourceStore, but this class seems to be too inefficient for large scale use, as this latter class fetches whole archive-files from the bitarchive.

Attachments

Issue Links

mentioned in: Page Loading...

Activity

People

Assignee:: Nicholas Clarke (Inactive)

Reporter:: Søren Vejrup Carlsen (Inactive)

Watchers:: 1 Start watching this issue

Dates

Created:: 20/Jun/12 2:25 PM

Updated:: 16/Feb/16 6:28 PM

Resolved:: 23/Jan/13 4:02 PM