Skip to content

Conversation

JedGrabman
Copy link
Contributor

Requirements:
This PR requires an update from evalcast-killcards to evalcast.

Memory related fixes:
1.) Update from evalcast-killcards to evalcast. Evalcast does not have memoisation, which will reduce memory requirements
2.) Only retrieve forecasts designated as "primary". Reduces memory requirements and aligns evaluation with Reich Lab documentation

Additionally, fixes a bug when processing forecast_dates for a forecaster that has no associated forecasts.

Memory related fixes:
1.) Update from evalcast-killcards to evalcast. Evalcast does not have memoisation, which will reduce memory requirements
2.) Only retrieve forecasts designated as "primary". Reduces memory requirements and aligns evaluation with Reich Lab documentation

Additionally, fixes a bug when processing forecast_dates for a forecaster that has no associated forecasts.
@tildechris
Copy link
Contributor

Error while doing make score_forecast:

Calls to the `evalcast` functions `download_signal()` and
`get_covidhub_predictions()` are memoized in memory by default.
To cache to disk instead use `memoise::cache_filesystem()`. Ex:

 db <- memoise::cache_filesystem("path/to/dir/.rcache")
 download_signal <<- memoise::memoise(download_signal, cache = db)

Error in get_covidhub_forecaster_names(designations = "primary") :
  unused argument (designations = "primary")
Calls: create_prediction_cards
Execution halted

@tildechris
Copy link
Contributor

This is probably because you need to update https://github.com/cmu-delphi/covidcast-docker with the new evalcast package.

@JedGrabman
Copy link
Contributor Author

@tildechris I've created https://github.com/cmu-delphi/covidcast-docker/pull/1 to address the docker issue, but I don't have permission to set reviewers.

@kateharwood kateharwood merged commit 9dae9c8 into dev Mar 18, 2021
@kateharwood kateharwood deleted the dev-memory-fixes branch March 30, 2021 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants