Created HDFS based file-store for the HadoopReferencePillar

Activity

CR-BITMAG-384 13

Keyboard shortcuts  
  • Summarize the review outcomes (optional)
     
    #permalink

    Details

    Warning: no files are visible, they have all been filtered.
    Participant Role Time Spent Comments Latest Comment
    Author 1h 3m 5 The method is used quite rarely, but when it is called, t...
    Reviewer - Complete 17m 4 Should we get these info dynamically, or should we cache ...
    Reviewer - 69% reviewed 21m 4 Jeg er ret sikker på at Asger Askov Blekinge kan sætte de...
    Total   1h 42m 13  
    #permalink

    Objectives

    • Created HDFS based file-store for the HadoopReferencePillar. Probably needs some settings and tests.
    • Fixed issue with dependencies, and also made Intellij re-organize imports...
    • HdfsArchive Unit test hasFile
    • Test logback makes hadoop shut up
    • hadoop minicluster only scoped for test
    • Minor changes to hadoopArchive
    • better path handling in HdfsArchive
    • Smaller fixes as discussed on Zoom
    • Merge branch 'HadoopReferencePillar' of https://github.com/bitrepository/reference into HadoopReferencePillar
    • failable methods return boolean rather than disregard outcome
    • Hdfs pillar is configurable
    • HDFS Pillar model test
    • Readded part of the ReferenceSetting, which made some tests fail. Also made the Hadoop test skip, when it cannot prepare the test.
    • Bumped C3P0 patch version to fix github alert.

    Branches in review

    #permalink

    Issues Raised From Comments

    Key Summary State Assignee
    #permalink

    General Comments

    There are no general comments on this review.
    /bitrepository-core/.../filestore/DefaultFileInfo.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../filestore/FileInfo.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../utils/ChecksumUtils.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../http/HttpFileExchange.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../protocol/LocalFileExchange.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../utils/StreamUtilsTest.java Changed
    Open in IDE #permalink
    /bitrepository-core/.../bitrepository-devel/ReferenceSettings.xml Changed 2
    Open in IDE #permalink
    /bitrepository-core/.../bitrepository-devel/core-site.xml Added 2
    Open in IDE #permalink
    /bitrepository-core/.../bitrepository-devel/hdfs-site.xml Added
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../messagehandler/GetFileRequestHandler.java Changed
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../checksumdatabase/ExtractedFileIDsResultSet.java Changed
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../filearchive/ReferenceArchive.java Changed
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../hadooparchive/CollectionHdfsManager.java Added 1
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../hadooparchive/HadoopFileInfo.java Added 3
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../hadooparchive/HdfsArchive.java Added
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../store/FileStorageModel.java Changed
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../hadooparchive/HdfsArchiveTest.java Added
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../store/FullHadoopPillarModelTest.java Added 1
    Open in IDE #permalink
    /bitrepository-reference-pillar/.../resources/logback-test.xml Added
    Open in IDE #permalink
    /bitrepository-reference-pillar/pom.xml Changed
    Open in IDE #permalink
    /bitrepository-reference-settings/.../xsd/ReferenceSettings.xsd Changed
    Open in IDE #permalink
    /bitrepository-service/pom.xml Changed
    Open in IDE #permalink
    /pom.xml Changed 4
    Open in IDE #permalink

    Review updated: Reload | Ignore | Collapse

    You cannot reload the review while writing a comment.

    Log time