Søren Vejrup Carlsen

replace with isTreatDnsLikeHttp()

replace with isTreatDnsLikeHttp()

Note that this means that fetching of prerequisites is always allowed

Note that this means that fetching of prerequisites is always allowed

All followup committed, now

All followup committed, now

agreed

agreed

When is the former statement true?

When is the former statement true?

No follow-up required

No follow-up required

NAS-2752: We always look for duplicationmigration records during indexing
NAS-2752: We always look for duplicationmigration records during indexing
Have now moved it to a finally clause

Have now moved it to a finally clause

I have now changed the code to only catch an Exception in the last catch phrase

I have now changed the code to only catch an Exception in the last catch phrase

It might be a good idea to renew the jmsConnection object in the same go: jmsConnection = null; qSession = null; and then add a if (jmsConnection == null) { jmsConnection = JMSConnectionFactor...

It might be a good idea to renew the jmsConnection object in the same go:

jmsConnection = null;
qSession = null;

and then add a

 
if (jmsConnection == null) {
  jmsConnection = JMSConnectionFactory.getInstance();
}

before the "if (qSession == null)" clause

NAS-2751: The HarvestStatusReceiver.getCount method has a memoryLeak
NAS-2751: The HarvestStatusReceiver.getCount method has a memoryLeak
Should really delete the method createQueueBrowser(ChannelID queueID) which creates a QueueSession every time the method is called unless of course we should cache a QueueBrowser for every queueID,...

Should really delete the method createQueueBrowser(ChannelID queueID) which creates a QueueSession every time the method is called
unless of course we should cache a QueueBrowser for every queueID, we see?

This change is strictly not necessary anymore. It was part of my attempt at having a separate JMSConnection for the QueueController rather than just using the JMSConnectionFactory.getInstance() method

This change is strictly not necessary anymore.
It was part of my attempt at having a separate JMSConnection for the QueueController rather than just using
the JMSConnectionFactory.getInstance() method

All follow-up to this issue has been pushed to branch NAS-2733

All follow-up to this issue has been pushed to branch NAS-2733

Note that the new "old" filtering method gives you a Domain X is not registered! error, if domain X is not registered in the Domains table of the harvestdatabase

Note that the new "old" filtering method gives you a

 Domain X is not registered! 

error, if domain X is not registered in the Domains table of the harvestdatabase

In the latest commit, I have renabled the old filtering method using lookup in the database, and made this the default method. There is now a setting for the filteringMethod used by the Harveststat...

In the latest commit, I have renabled the old filtering method using lookup in the database, and made this the default method.
There is now a setting for the filteringMethod used by the Harveststatus-running.jsp page:
settings.harvester.webinterface.runningjobsFilteringMethod

If this value is set to "cachedLogs", the filtering is done by searching in the cached crawllogs
if this value is set to "database", the filtering is done by searching in the database
The latter is the default value in the settings

Fixed in the latest commit

Fixed in the latest commit

Fixed in the latest commit

Fixed in the latest commit

Fixed in the latest commit to branch NAS-2733

Fixed in the latest commit to branch NAS-2733

The searching for jobs harvesting a domain used the code in 5.2.2 used the ./harvester/harvester-core/src/main/java/dk/netarkivet/harvester/webinterface/FindRunningJobQuery.java and this code in th...

The searching for jobs harvesting a domain used the code in 5.2.2
used the ./harvester/harvester-core/src/main/java/dk/netarkivet/harvester/webinterface/FindRunningJobQuery.java
and this code in the Harveststatus-running.jsp

 FindRunningJobQuery findJobQuery = new FindRunningJobQuery(request);
    Long[] jobIdsForDomain = findJobQuery.getRunningJobIds();

and further down:

<% if (jobIdsForDomain.length > 0) { %>
<br/>
<table class="selection_table_small">
<tr>
    <th><fmt:message key="running.jobs.finder.table.jobId"/></th>
</tr>
<% for (long jobId : jobIdsForDomain) {
    String jobDetailsLink = "Harveststatus-jobdetails.jsp?"
       + Constants.JOB_PARAM + "=" + jobId;
%>
<tr><td><a href="<%=jobDetailsLink%>"><%=jobId%></a></td></tr>
<% } %>
</table>
<% } else {

    //after using the search button "searchDone" !=null
    String searchDone = request.getParameter("searchDone");
    if (searchDone != null) { %>
    	 <fmt:message key="table.job.no.jobs"/>

<% } %>
<% } %>
<% } %>
<%
 HTMLUtils.generateFooter(out);
%>


Currently the method FindRunningJobQuery(ServletRequest req) is no longer called in Harveststatus-running.jsp

And also when using the new search-formula, jobs not found in the cached logs are not shown at all.
In the old version, the jobs matching the search were shown in a table of their own

However we could introduce the old method again without too much work.
But what do we want here?

But in that case it is only useful, if caching is enabled for all the H3 hosts

But in that case it is only useful, if caching is enabled for all the H3 hosts

After Sara's comments last Friday, I still would like to know what the initial requirements for this feature was. According to BNF, they would have preferred a filter, that searched in the harvest-...

After Sara's comments last Friday, I still would like to know what the initial requirements for this feature was.
According to BNF, they would have preferred a filter, that searched in the harvest-database instead of in the cached crawl-logs.
As it is - now - it is very confusing

I think it should possible to disable this feature.

There was no logging at all for the NASEnvironment class. And I didn't want to waste any time figuring out how logging should work in NICL's framework. So I've added my own framework. So, if you th...

There was no logging at all for the NASEnvironment class. And I didn't want to waste any time figuring out how logging should work in NICL's framework.
So I've added my own framework.
So, if you think the methodName writeLog is confusing, you're welcome to find another name for it.

Merge these two loglines

Merge these two loglines

We should create the directory represented by tempPath, if it doesn't exist instead of falling back to /tmp

We should create the directory represented by tempPath, if it doesn't exist instead of falling back to /tmp

Work on NAS-2733 - now searching for the domain in the cached logs. Only does lookup, if the crawlog...
Work on NAS-2733 - now searching for the domain in the cached logs. Only does lookup, if the crawlog...
This value is always set to "", even if we just searched for jobs harvesting a specific domain. We should replace "" with the value of searchedDomainName if it is not null Created https://sbforge....

This value is always set to "", even if we just searched for jobs harvesting a specific domain.
We should replace "" with the value of searchedDomainName if it is not null

Created https://sbforge.org/jira/browse/NAS-2735 for this