public class NASFetchDNS extends org.archive.modules.fetcher.FetchDNS
Modifier and Type | Field and Description |
---|---|
protected boolean |
acceptDefinedHosts
Look for hosts in the hosts file/text value before doing a DNS lookup.
|
protected Map<String,String> |
hosts
Map of hosts that override the normal DNS lookup.
|
protected org.archive.io.ReadSource |
hostsFile
Text from which to load hosts
|
protected org.archive.io.ReadSource |
hostsSource
Text from which to look for hosts.
|
Constructor and Description |
---|
NASFetchDNS() |
Modifier and Type | Method and Description |
---|---|
boolean |
getAcceptDefinedHosts() |
protected void |
getHosts(org.archive.io.ReadSource hostsSource)
Run through the lines in a
ReadSource and add all valid host lines encountered. |
org.archive.io.ReadSource |
getHostsFile() |
org.archive.io.ReadSource |
getHostsSource() |
protected void |
innerProcess(org.archive.modules.CrawlURI curi) |
protected void |
reload()
Clear loaded hosts of reload from hosts file and value text.
|
void |
setAcceptDefinedHosts(boolean acceptDefinedHosts) |
void |
setHostsFile(org.archive.io.ReadSource hostsFile) |
void |
setHostsSource(org.archive.io.ReadSource hostsSource) |
static int |
tokenize(String str,
String[] tokensArr)
Split input string into tokens.
|
getAcceptNonDnsResolves, getDigestAlgorithm, getDigestContent, getDNSRecord, getFirstARecord, getServerCache, isQuadAddress, recordDNS, setAcceptNonDnsResolves, setDigestAlgorithm, setDigestContent, setServerCache, setUnresolvable, shouldProcess, storeDNSRecord
doCheckpoint, finishCheckpoint, flattenVia, fromCheckpointJson, getBeanName, getEnabled, getKeyedProperties, getRecordedSize, getShouldProcessRule, getURICount, hasHttpAuthenticationCredential, innerProcessResult, innerRejectProcess, isRunning, isSuccess, process, report, setBeanName, setEnabled, setRecoveryCheckpoint, setShouldProcessRule, start, startCheckpoint, stop, toCheckpointJson
protected boolean acceptDefinedHosts
protected org.archive.io.ReadSource hostsFile
protected org.archive.io.ReadSource hostsSource
public NASFetchDNS()
public boolean getAcceptDefinedHosts()
public void setAcceptDefinedHosts(boolean acceptDefinedHosts)
public org.archive.io.ReadSource getHostsFile()
public void setHostsFile(org.archive.io.ReadSource hostsFile)
public org.archive.io.ReadSource getHostsSource()
public void setHostsSource(org.archive.io.ReadSource hostsSource)
protected void innerProcess(org.archive.modules.CrawlURI curi)
innerProcess
in class org.archive.modules.fetcher.FetchDNS
protected void reload()
protected void getHosts(org.archive.io.ReadSource hostsSource)
ReadSource
and add all valid host lines encountered.hostsSource
- hosts file or value textpublic static int tokenize(String str, String[] tokensArr)
str
- split input string into tokenstokensArr
- supply a string array to be filled with tokensCopyright © 2005–2016 The Royal Danish Library, the Danish State and University Library, the National Library of France and the Austrian National Library.. All rights reserved.