Concurrency for issue generation #536
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This separates cross-linking of duplicates from the process of turning the issue text and issue resolution into HTML, as suggested in comments in the code.
The remaining functions access the section_db and so can't be const yet. Ideally we would avoid making any changes to that section_db after parsing the XML files and converting the issue text to HTML. That would allow all functions that generate HTML files to be const, which would allow them to run in parallel. Generating the individual HTML files currently takes about 25% of the total time to generate the lists, but in theory in should be possible to generate those files concurrently. Also make the paper_title_attr function take a const reference to the lwg::metadata object. It needs to lookup paper titles in the map, but it can use map::find instead of map::operator[], so that it doesn't need a non-const reference.
By using four threads to create the issues lists we reduce the total runtime from 1.5s to 1s. We could do much better if we could generate the individual HTML files for each issue in parallel, or even just do that in parallel with creating the main lists. Currently we have to finish generating the main lists before starting to generate the individual issues.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reduce time to generate issues lists from 1.5s to 1s by using
std::asyncto generate the lists.