Skip to content

Fix exclusion error on stats processing and misplaced ENDLOCK logging entry #1710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 3, 2020
Merged

Fix exclusion error on stats processing and misplaced ENDLOCK logging entry #1710

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c81aed1
Fix misplaced ENDLOCK message and optimize lock in scraper
jamescowens Jun 2, 2020
8328006
Correct stats processing exclusion logic
jamescowens Jun 2, 2020
a24214e
Exclude VerifiedBeacons from Quorum ResolvePartsFor
jamescowens Jun 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/neuralnet/quorum.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1181,8 +1181,12 @@ class SuperblockValidator
// If this project does not exist in the superblock, skip the
// attempt to associate its parts:
//
// Skip the BeaconList, which is always part 0, and skip
// the VerifiedBeacons "project", which is matched by project
// name.
if (!project_option
|| entry.part1 < 1
|| entry.project == "VerifiedBeacons"
|| entry.part1 >= (int)manifest.vParts.size())
{
continue;
Expand Down
18 changes: 12 additions & 6 deletions src/scraper/scraper.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2069,10 +2069,17 @@ bool ProcessProjectRacFileByCPID(const std::string& project, const fs::path& fil
_log(logattribute::INFO, "ENDLOCK", "cs_TeamIDMap");
}

LOCK(cs_VerifiedBeacons);
_log(logattribute::INFO, "LOCK", "cs_VerifiedBeacons");
ScraperVerifiedBeacons ScraperVerifiedBeacons;

ScraperVerifiedBeacons& ScraperVerifiedBeacons = GetVerifiedBeacons();
// Minimize lock time
{
LOCK(cs_VerifiedBeacons);
_log(logattribute::INFO, "LOCK", "cs_VerifiedBeacons");

ScraperVerifiedBeacons = GetVerifiedBeacons();

_log(logattribute::INFO, "ENDLOCK", "cs_VerifiedBeacons");
}

std::string line;
stringbuilder builder;
Expand Down Expand Up @@ -2154,7 +2161,6 @@ bool ProcessProjectRacFileByCPID(const std::string& project, const fs::path& fil
else
builder.append(line);

_log(logattribute::INFO, "ENDLOCK", "cs_VerifiedBeacons");
}

if (bfileerror)
Expand Down Expand Up @@ -3252,7 +3258,7 @@ ScraperStatsAndVerifiedBeacons GetScraperStatsByConvergedManifest(const Converge
ScraperStats mProjectScraperStats;

// Do not process the BeaconList or VerifiedBeacons as a project stats file.
if (project != "BeaconList" || project != "VerifiedBeacons")
if (project != "BeaconList" && project != "VerifiedBeacons")
{
_log(logattribute::INFO, "GetScraperStatsByConvergedManifest", "Processing stats for project: " + project);

Expand Down Expand Up @@ -3370,7 +3376,7 @@ ScraperStatsAndVerifiedBeacons GetScraperStatsFromSingleManifest(CScraperManifes
ScraperStats mProjectScraperStats;

// Do not process the BeaconList or VerifiedBeacons as a project stats file.
if (project != "BeaconList" || project != "VerifiedBeacons")
if (project != "BeaconList" && project != "VerifiedBeacons")
{
_log(logattribute::INFO, "GetScraperStatsFromSingleManifest", "Processing stats for project: " + project);

Expand Down