Modifier and Type | Field and Description |
---|---|
static String |
ARC_EXTENSION
Extension used for ARC files, including separator .
|
static String |
ARC_GZIPPED_EXTENSION
Extension used for gzipped ARC files, including separator .
|
static String |
ARC_PATTERN
Pattern matching ARC files, including separator.
|
static FilenameFilter |
ARCS_FILTER
A filter that matches arc files, that is any file that ends on .arc or .arc.gz in any case.
|
static String |
CDX_EXTENSION
Extension used for CDX files, including separator .
|
static FilenameFilter |
CDX_FILE_FILTER
A FilenameFilter accepting a file if and only if its name (transformed to lower case) ends on ".cdx".
|
static int |
MAX_IDS_IN_FILENAME
Maximum number of IDs we will put in a filename.
|
static String |
OPEN_ARC_PATTERN
Pattern matching open ARC files, including separator .
|
static FilenameFilter |
OPEN_ARCS_FILTER
A filter that matches files left open by a crashed Heritrix process.
|
static String |
OPEN_WARC_PATTERN
Pattern matching open WARC files, including separator .
|
static FilenameFilter |
OPEN_WARCS_FILTER
A filter that matches warcfiles left open by a crashed Heritrix process.
|
static String |
WARC_ARC_PATTERN
Pattern matching WARC and ARC files, including separator.
|
static String |
WARC_EXTENSION
Extension used for WARC files, including separator .
|
static String |
WARC_GZIPPED_EXTENSION
Extension used for gzipped WARC files, including separator .
|
static String |
WARC_PATTERN
Pattern matching WARC files, including separator.
|
static FilenameFilter |
WARCS_ARCS_FILTER
A filter that matches warc and arc files, that is any file that ends on .warc, .warc.gz, .arc or .arc.gz in any
case.
|
static FilenameFilter |
WARCS_FILTER
A filter that matches warc files, that is any file that ends on .warc or .warc.gz in any case.
|
Constructor and Description |
---|
FileUtils() |
Modifier and Type | Method and Description |
---|---|
static void |
appendToFile(File file,
String... lines)
Append the given lines to a file.
|
static void |
copyDirectory(File from,
File to)
Copy an entire directory from one location to another.
|
static void |
copyFile(File from,
File to)
Copy file from one location to another.
|
static long |
countLines(File file)
Count the number of lines in a file.
|
static boolean |
createDir(File dir)
Check if the directory exists and is writable and create it if needed.
|
static File |
createUniqueTempDir(File inDir,
String prefix)
Creates a new temporary directory with a unique name.
|
static String |
formatFilename(String filename)
Returns a valid filename for most filesystems.
|
static <T extends Comparable<T>> |
generateFileNameFromSet(Set<T> IDs,
String suffix)
Given a set, generate a reasonable file name from the set.
|
static long |
getBytesFree(File f)
Returns the number of bytes free on the file system calling the FreeSpaceProvider class defined by the setting
CommonSettings.FREESPACE_PROVIDER_CLASS (a.k.a.
|
static InputStream |
getEphemeralInputStream(File file)
Create an InputStream that reads from a file but removes the file when all data has been read.
|
static List<File> |
getFilesRecursively(String dir,
List<File> files,
String type)
Retrieves all files whose names ends with 'type' from directory 'dir' and all its subdirectories.
|
static String |
getHumanReadableFileSize(File aFile)
Get a humanly readable representation of the file size.
|
static File |
getResourceFileFromClassPath(String filePath)
Loads an file from the class path (for retrieving a file from '.jar').
|
static File |
getTempDir()
Get the location of the standard temporary directory.
|
static FilenameFilter |
getXmlFilesFilter()
Return a filter that only accepts XML files (ending with .xml), irrespective of their location.
|
static boolean |
hasFiles(File aDir) |
static void |
makeSortedFile(File unsortedFile,
File sortedOutput)
Sort a file into another.
|
static File |
makeValidFileFromExisting(String filename)
Makes a valid file from filename passed in String.
|
static void |
moveFile(File fromFile,
File toFile)
Attempt to move a file using rename, and if that fails, move the file by copy-and-delete.
|
static byte[] |
readBinaryFile(File file)
Read an entire file, byte by byte, into a byte array, ignoring any locale issues.
|
static String |
readFile(File file)
Load file content into text string.
|
static String |
readLastLine(File file)
Read the last line in a file.
|
static List<String> |
readListFromFile(File file)
Read all lines from a file into a list of strings.
|
static String |
relativeTo(File theFile,
File theDir) |
static boolean |
remove(File f)
Remove a file.
|
static void |
removeLineFromFile(String line,
File file)
Remove a line from a given file.
|
static boolean |
removeRecursively(File f)
Remove a file and any subfiles in case of directories.
|
static void |
sortCDX(File file,
File toFile)
Sort a CDX file according to our standard for CDX file sorting.
|
static void |
sortCrawlLog(File file,
File toFile)
Sort a crawl.log file according to the url.
|
static void |
sortCrawlLogOnTimestamp(File file,
File toFile)
Sort a crawl.log file according to the timestamp.
|
static void |
sortFile(File file,
File toFile)
Sort a file using UNIX sort.
|
static void |
writeBinaryFile(File file,
byte[] b)
Write an entire byte array to a file, ignoring any locale issues.
|
static void |
writeCollectionToFile(File file,
Collection<String> collection)
Writes a collection of strings to a file, each string on one line.
|
static void |
writeFileToStream(File f,
OutputStream out)
Write the entire contents of a file to a stream.
|
static void |
writeStreamToFile(InputStream in,
File f)
Write the contents of a stream into a file.
|
public static final String CDX_EXTENSION
public static final String ARC_EXTENSION
public static final String ARC_GZIPPED_EXTENSION
public static final String WARC_EXTENSION
public static final String WARC_GZIPPED_EXTENSION
public static final String ARC_PATTERN
public static final String OPEN_ARC_PATTERN
public static final String WARC_PATTERN
public static final String OPEN_WARC_PATTERN
public static final String WARC_ARC_PATTERN
public static final FilenameFilter CDX_FILE_FILTER
public static final FilenameFilter OPEN_ARCS_FILTER
public static final FilenameFilter OPEN_WARCS_FILTER
public static final FilenameFilter ARCS_FILTER
public static final FilenameFilter WARCS_FILTER
public static final FilenameFilter WARCS_ARCS_FILTER
public static final int MAX_IDS_IN_FILENAME
public FileUtils()
public static boolean removeRecursively(File f)
f
- A file to completely and utterly remove.SecurityException
- If a security manager exists and its SecurityManager.checkDelete(java.lang.String)
method denies delete access to the filepublic static boolean remove(File f)
f
- A file to completely and utterly remove.ArgumentNotValid
- if f is null.SecurityException
- If a security manager exists and its SecurityManager.checkDelete(java.lang.String)
method denies delete access to the filepublic static String formatFilename(String filename)
filename
- the filename to format correctlypublic static List<File> getFilesRecursively(String dir, List<File> files, String type)
dir
- Path of base directoryfiles
- Initially, an empty list (e.g. an ArrayList)type
- The extension/ending of the files to retrieve (e.g. ".xml", ".ARC")public static String readFile(File file) throws IOException
file
- The file to loadIOException
- If any IO trouble occurs while reading the file, or the file cannot be found.public static void copyFile(File from, File to)
from
- original to copyto
- destination of copyIOFailure
- if an io error occurs while copying file, or the original file does not exist.public static void copyDirectory(File from, File to) throws IOFailure
from
- Original directory (or file, for that matter) to copy.to
- Destination directory, i.e. the 'new name' of the copy of the from directory.IOFailure
- On IO trouble copying files.public static byte[] readBinaryFile(File file) throws IOFailure, IndexOutOfBoundsException
file
- A file to be read.IOFailure
- on IO trouble reading the file, or the file does not existIndexOutOfBoundsException
- If the file is too large to be in an array.public static void writeBinaryFile(File file, byte[] b)
file
- The file to write the data tob
- The byte array to write to the fileIOFailure
- If an exception occurs during the writing.public static FilenameFilter getXmlFilesFilter()
public static List<String> readListFromFile(File file)
file
- The file to read from.IOFailure
- on trouble reading the file, or if the file does not existpublic static void writeCollectionToFile(File file, Collection<String> collection)
file
- A file to write to. The contents of this file will be overwritten.collection
- The collection to write. The order it will be written in is unspecified.IOFailure
- if any error occurs writing to the file.ArgumentNotValid
- if file or collection is null.public static void makeSortedFile(File unsortedFile, File sortedOutput)
unsortedFile
- A file to sortsortedOutput
- The file to sort intopublic static void removeLineFromFile(String line, File file)
line
- The full line to removefile
- The file to remove the line from. This file will be rewritten in full, and the entire contents will
be kept in memoryUnknownID
- If the file does not existpublic static boolean createDir(File dir) throws PermissionDenied
dir
- The directory to createArgumentNotValid
- If dir is null or its name is the empty stringPermissionDenied
- If directory cannot be created for any reason, or is not writable.public static long getBytesFree(File f)
f
- a given filepublic static String relativeTo(File theFile, File theDir)
theFile
- A file to make relativetheDir
- A directorypublic static long countLines(File file)
file
- the file to readIOFailure
- If an error occurred while reading the filepublic static InputStream getEphemeralInputStream(File file)
file
- A file to read. This file will be deleted when the inputstream is closed, finalized, reaches
end-of-file, or when the VM closes.IOFailure
- If an error occurs in creating the ephemeral input streampublic static File makeValidFileFromExisting(String filename) throws IOFailure
filename
- The file to create the File object fromIOFailure
- if file cannot be created.public static void writeFileToStream(File f, OutputStream out)
f
- A file to write to the stream.out
- The stream to write to.IOFailure
- If any error occurs while writing the file to a streampublic static void writeStreamToFile(InputStream in, File f)
in
- A stream to read from. This stream is not closed by this method.f
- The file to write the stream contents into.IOFailure
- If any error occurs while writing the stream to a filepublic static File getTempDir()
public static void moveFile(File fromFile, File toFile)
fromFile
- The sourcetoFile
- The targetpublic static <T extends Comparable<T>> String generateFileNameFromSet(Set<T> IDs, String suffix)
T
- The type of objects, that the Set IDs argument contains.IDs
- A set of IDs.suffix
- A suffix. May be empty string.public static void sortCrawlLog(File file, File toFile)
file
- The file containing the unsorted data.toFile
- The file that the sorted data can be put into.IOFailure
- if there were errors running the sort process, or if the file does not exist.public static void sortCrawlLogOnTimestamp(File file, File toFile)
file
- The file containing the unsorted data.toFile
- The file that the sorted data can be put into.IOFailure
- if there were errors running the sort process, or if the file does not exist.public static void sortCDX(File file, File toFile)
file
- The raw unsorted CDX file.toFile
- The file that the result will be put into.IOFailure
- If the file does not exist, or could not be sortedpublic static void sortFile(File file, File toFile)
file
- the file that you want to sort.toFile
- The destination file.public static File createUniqueTempDir(File inDir, String prefix)
inDir
- The directory where the temporary directory should be created.prefix
- The prefix of the directory name, for identification purposes.ArgumentNotValid
- if inDir is not an existing directory that can be written to.IOFailure
- if a free name couldn't be found within a reasonable number of tries.public static String readLastLine(File file)
file
- input file to read last line from.ArgumentNotValid
- on null argument, or file is not a readable file.IOFailure
- on IO trouble reading file.public static void appendToFile(File file, String... lines)
file
- A file to append to.lines
- The lines to write.public static File getResourceFileFromClassPath(String filePath) throws IOFailure
filePath
- The path of the file.IOFailure
- If resource cannot be retrieved from the class path.public static String getHumanReadableFileSize(File aFile)
aFile
- a File objectCopyright © 2005–2016 The Royal Danish Library, the Danish State and University Library, the National Library of France and the Austrian National Library.. All rights reserved.