[NAS-2672] Add tool for verifying if ftp server setup is NetarchiveSuite compliant Created: 20/Sep/17  Updated: 20/Jul/18

Status: Reopened
Project: NetarchiveSuite
Component/s: Common
Affects Version/s: 5.2.2, 5.3.1
Fix Version/s: 5.5.1

Type: Improvement Priority: Minor
Reporter: Søren Vejrup Carlsen (Inactive) Assignee: Søren Vejrup Carlsen (Inactive)
Resolution: Unresolved  
Labels: None
Remaining Estimate: Not Specified
Time Spent: 28m
Original Estimate: Not Specified

Issue Links:
Spawned
spawned NAS-2699 Make deploy check mail and ftp-config... Open
External reference:

https://sbprojects.statsbiblioteket.dk/jira/browse/NARK-1349

 Description   

A tool for verifying if your ftp server setup is NetarchiveSuite compliant
is useful when using FTPRemoteFile as your chosen RemoteFile implementation.

Note that HTTPRemoteFile is normally used in a quickstart installation of NetarchiveSuite

 Comments   
Comment by Colin Rosenthal [ 28/May/18 ]

Is this tested? Otherwise I am postponing it to a future release.

Comment by Colin Rosenthal [ 24/Apr/18 ]

I tried to guess my way to a test of this tool - it "worked" but failed to validate:

[devel@kb-test-adm-001 TEST5B]$ java -cp lib/netarchivesuite-common-core.jar -Ddk.netarkivet.settings.file=conf/settings_HarvestJobManagerApplication.xml dk.netarkivet.common.tools.FTPValidator
File 'FTPValidator_file1.xml' should not exist already on server. Trying to delete it
10:24:46.448 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Writing 'FTPValidator_file1.xml' as 'FTPValidator_file1.xml-1880-1524558286446' on ftp-server kb-test-har-003.kb.dk
10:24:46.488 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:46.488 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:46.507 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Completed writing the file 'FTPValidator_file1.xml-1880-1524558286446'
10:24:46.508 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Checksum of 'FTPValidator_file1.xml-1880-1524558286446' is:ba3847129319312c0de3b349d9d453da
10:24:46.508 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:46.508 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Ftp logout
10:24:46.741 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:46.742 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:46.745 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:46.753 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Writing 'FTPValidator_file1.xml' as 'FTPValidator_file1.xml-32952-1524558286753' on ftp-server kb-test-har-003.kb.dk
10:24:46.976 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:46.976 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:46.979 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Completed writing the file 'FTPValidator_file1.xml-32952-1524558286753'
10:24:46.979 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Checksum of 'FTPValidator_file1.xml-32952-1524558286753' is:ba3847129319312c0de3b349d9d453da
10:24:46.979 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:46.979 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Ftp logout
10:24:47.214 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:47.215 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:47.223 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:47.225 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - Deleting file 'FTPValidator_file1.xml-32952-1524558286753' from ftp server
10:24:47.455 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:47.455 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:47.456 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:47.456 [main] DEBUG dk.netarkivet.common.distribute.FTPRemoteFile - File 'FTPValidator_file1.xml-32952-1524558286753' was deleted from ftp server. Cleanup finished.
10:24:47.688 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - w/ DataTimeout (ms): 0
10:24:47.688 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Logged onto ftp://devel:********@kb-test-har-003.kb.dk:21
10:24:47.691 [main] WARN dk.netarkivet.common.distribute.FTPRemoteFile - Append operation from 'FTPValidator_file1.xml-32952-1524558286753' failed: Error 550: '550 FTPValidator_file1.xml-32952-1524558286753: No such file or directory
'
10:24:47.691 [main] DEBUG dk.netarkivet.common.distribute.FTPConnectionManager - Trying to log out.
10:24:47.697 [main] WARN dk.netarkivet.common.distribute.AbstractRemoteFile - Could not retrieve the file 'FTPValidator_file1.xml' on first attempt. Will retry up to '3' times.
dk.netarkivet.common.exceptions.IOFailure: Append operation from 'FTPValidator_file1.xml-32952-1524558286753' failed: Error 550: '550 FTPValidator_file1.xml-32952-1524558286753: No such file or directory
'
Comment by Colin Rosenthal [ 24/Apr/18 ]

Is it documented in the Tools Manual?

Comment by Søren Vejrup Carlsen (Inactive) [ 30/Jan/18 ]

This solution has still not been been merged with the master branch

Generated at Sat Apr 27 00:22:39 CEST 2024 using Jira 9.4.15#940015-sha1:bdaa9cbecfb6791ea579749728cab771f0dfe90b.