Skip to content

Migrating to Golang based collectors instead of parallel-ssh and also…#219

Open
venksrin09 wants to merge 1 commit into
mainfrom
go_cluster_mon
Open

Migrating to Golang based collectors instead of parallel-ssh and also…#219
venksrin09 wants to merge 1 commit into
mainfrom
go_cluster_mon

Conversation

@venksrin09

Copy link
Copy Markdown
Collaborator

For larger number of nodes, parallel-ssh and asyncio are very slow and often take over 20 mins to fetch the commands and do not seem to work well for over 100 nodes. So based on Ignatious suggestions, I have migrated to Golang and was able to test with the 160 nodes in OSSCI. Also made one more bug fix.

  • Migrating all collectors to use Golang binary instead of parallel-ssh. This is having x8 to x10 performance improvement from before.
  • Accept any key file names instead of standard id_rsa, edcs etc.
  • By default, do not print the command outputs in log files as it has an impact on performance, so 2 changes have been done, we print only the top 5 lines of the commands and that also only when the application is started in debug mode (full_rebuild.sh --debug)

Test Plan

Validated with the 165 nodes from OSSCI and all the initial counters show up in UI in less than a minute compared to 20 mins before.

Test Result

Validated all the tabs - metrics, logs

Submission Checklist

… support for any user defined key file names, we had a bug and only accepting standard key file names like id_rsa
@venksrin09 venksrin09 requested a review from cijohnson June 7, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant