Commits · master · Gerhard Gossen / archive-crawler

Jan 12, 2018
- Force spec name to spec file basename · c66b9de7
  Gerhard Gossen authored 7 years ago
  
  c66b9de7
- Print crawl configs · bd08880d
  Gerhard Gossen authored 7 years ago
  
  bd08880d
- Prevent overwriting of existing CSV output files · 1c32699e
  Gerhard Gossen authored 7 years ago
  
  1c32699e
- Report more accurate progress · 772527f7
  Gerhard Gossen authored 7 years ago
  
  772527f7
- Update .gitignore · c165da19
  Gerhard Gossen authored 7 years ago
  
  c165da19
- Remove outdated UI URL path segment · 813bea0c
  Gerhard Gossen authored 7 years ago
  
  813bea0c
- Include more parameters in displayed crawl names · de227f73
  Gerhard Gossen authored 7 years ago
  
  de227f73
Jan 05, 2018
- Catch exception for invalid dates · c40d2f16
  Gerhard Gossen authored 7 years ago
  
  c40d2f16
Jun 15, 2017
- Fix usage text · 7dd73767
  Gerhard Gossen authored 7 years ago
  
  7dd73767
Apr 11, 2017
- Add method to extract seed URLs from CDX files · 50207ebd
  Gerhard Gossen authored 8 years ago
  
  50207ebd
- Only use time similarity if content similarity is above threshold · baf7c8e9
  Gerhard Gossen authored 8 years ago
  
  baf7c8e9
Apr 03, 2017
- Normalize scores using reference document similarity · 75dec058
  Gerhard Gossen authored 8 years ago
  
  75dec058
Feb 14, 2017
- Update dependencies · 4e0caf72
  Gerhard Gossen authored 8 years ago
  
  4e0caf72
Feb 13, 2017
- Code cleanuo · a974072e
  Gerhard Gossen authored 8 years ago
  
  a974072e
Feb 07, 2017
- Add data & tools for web page date extraction evaluation · 89757d12
  Gerhard Gossen authored 8 years ago
  
  89757d12
- Add wikinews categories for evaluation topics · 88807231
  Gerhard Gossen authored 8 years ago
  
  88807231
- Improve web page date extraction · 87cce025
  Gerhard Gossen authored 8 years ago
  
  87cce025
- Store date and relevance interval for analysed snapshots · 4a6e4039
  Gerhard Gossen authored 8 years ago
  
  4a6e4039
- Skip evaluation using different snapshot counts for time-sensitive WMs · f9a82cfb
  Gerhard Gossen authored 8 years ago
  
  f9a82cfb
- Add Wikinews categories as seed URL sources · bb8b1492
  Gerhard Gossen authored 8 years ago
  
  bb8b1492
- Increase threshold for number keywords · 9e11747e
  Gerhard Gossen authored 8 years ago
  
  9e11747e
- Unwrap some archive URLs when creating crawl spec · ee3259f8
  Gerhard Gossen authored 8 years ago
  
  ee3259f8
- Improve display of metrics in UI · f7154258
  Gerhard Gossen authored 8 years ago
  
  f7154258
Jan 20, 2017
- Include modified date in ZIP metadata file · 49a61880
  Gerhard Gossen authored 8 years ago
  
  49a61880
- Formatting · f9bafcde
  Gerhard Gossen authored 8 years ago
  
  f9bafcde
- Replace JodaTime with java.time · fd51b98f
  Gerhard Gossen authored 8 years ago
  
  fd51b98f
- Check multiple snapshots of page and pick best one · d963cb79
  Gerhard Gossen authored 8 years ago
  
  d963cb79
Jul 21, 2016
- Fix classpath after cluster config change · 96fb3195
  Gerhard Gossen authored 8 years ago
  
  96fb3195
- Store estimated modification time · de18611c
  Gerhard Gossen authored 8 years ago
  
  de18611c
Jul 20, 2016
- Fix regexp for month in modification date estimation · 8fbbff72
  Gerhard Gossen authored 8 years ago
  
  8fbbff72
Apr 18, 2016
- Store referrer URL and crawl time in CrawlUrl · 56214a44
  Gerhard Gossen authored 9 years ago
  
  56214a44
- Use ZonedDateTime instead of LocalDateTime · c2fe97d9
  Gerhard Gossen authored 9 years ago
  
  c2fe97d9
Mar 02, 2016
- Add properties file · ca28fa73
  Gerhard Gossen authored 9 years ago
  
  ca28fa73
Mar 01, 2016
- Add description of zip output format · ab8db4d4
  Gerhard Gossen authored 9 years ago
  
  ab8db4d4
- Add filtering by minimum relevance · 5208f446
  Gerhard Gossen authored 9 years ago
  
  5208f446
Feb 29, 2016
- Add ResultStorer that produces .zip files of results · ecf8d9ae
  Gerhard Gossen authored 9 years ago
  
  ecf8d9ae
Feb 22, 2016
- Extract base classes for YARN client & app master · e330ad0a
  Gerhard Gossen authored 9 years ago
  
  e330ad0a
Feb 10, 2016
- Update to Java 8 · b6cc1069
  Gerhard Gossen authored 9 years ago
  
  b6cc1069
Feb 09, 2016
- Prevent NPE for Urls · 7c4de43c
  Gerhard Gossen authored 9 years ago
  
  7c4de43c
- Create crawl spec version without keywords · e4a43c6e
  Gerhard Gossen authored 9 years ago
  
  e4a43c6e