Easily verified by generating jobs for a broadcrawl using production data (TEST7).
Alternately, ingest 30000 domains into an empty database, and then schedule a broadcrawl.
All harvesters should be shutdown in this test.
[NAS-2005] Mail sent whenever scheduling is skipped because the previous scheduling is still running Created: 05/Dec/11 Updated: 30/Jul/12 Resolved: 29/Mar/12 |
|
| Status: | Resolved |
| Project: | NetarchiveSuite |
| Component/s: | None |
| Affects Version/s: | 3.17.0 |
| Fix Version/s: | I50, 3.18.3, 3.19.0 |
| Type: | Bug | Priority: | Major |
| Reporter: | Colin Rosenthal | Assignee: | Søren Vejrup Carlsen (Inactive) |
| Resolution: | Fixed | ||
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Inspector: | |
| External reference: | |
| Verification: | Easily verified by generating jobs for a broadcrawl using production data (TEST7). |
| Description |
|
In TEST7 I am trying to create jobs for a snapshot harvest but the HarvestJobManager is giving me errors of the form WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running and no jobs are being created. |
| Comments |
| Comment by Søren Vejrup Carlsen (Inactive) [ 30/Jul/12 ] |
|
Problem finally fixed in |
| Comment by Søren Vejrup Carlsen (Inactive) [ 24/Feb/12 ] |
|
The current solution is broken, because it also creates a notification for snapshot harvests, which are only run once. |
| Comment by Søren Vejrup Carlsen (Inactive) [ 14/Dec/11 ] |
|
As it negates the implementation of |
| Comment by Søren Vejrup Carlsen (Inactive) [ 13/Dec/11 ] |
|
This actually negates issue |
| Comment by Søren Vejrup Carlsen (Inactive) [ 05/Dec/11 ] |
|
How often are we sending these messages? |
| Comment by Colin Rosenthal [ 05/Dec/11 ] |
|
I think it's a minor issue that the harvest job name doesn't appear in the log message when a job is generated so you can't grep for it: WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running FINE: Creating Job 136664 (state = NEW, HD = 155, priority = LOWPRIORITY, forcemaxcount = -1, forcemaxbytes = 100000, forcemaxrunningtime = 0, orderxml = default_orderxml, numconfigs = 10000) FINE: Generated job: 'Job 136664 (state = NEW, HD = 155, priority = LOWPRIORITY, forcemaxcount = -1, forcemaxbytes = 100000, forcemaxrunningtime = 0, orderxml = default_orderxml, numconfigs = 10000)' 25217155.dk:defaultconfig 155.dk:defaultconfig 36700155.dk:defaultconfig 155mbit.dk:defaultconfig 31555555.dk:defaultconfig 22215545.dk:defaultconfig 1553kbhv.dk:defaultconfig 1553k�benhavnv.dk:defaultconfig 1553københavnv.dk:defaultconfig WARNING: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running SEVERE: Mailing netarkivet error: Not creating jobs for harvestdefinition #155 (test7) as the previous scheduling is still running So grepping on the job-number shows jobs being created but grepping on the harvest name just gives a list of "SEVERE" log messages. |
| Comment by Colin Rosenthal [ 05/Dec/11 ] |
|
After restarting HarvestJobManager the harvest is now scheduling jobs. This issue may be hard to reproduce or may not have been a misunderstanding caused by the unnecessarily high severity of the log messages ??? |