Avoid starting and stopping service components on policy change

[ycombinator]

What does this PR do?

This PR identifies Service Runtime components with only their input type; the output ID is not longer used.

Why is it important?

Service Runtime components are intended to be kept running (via a service) for as long as possible. We should only start or stop them if they are being explicitly added or removed, respectively, from the component model. If only their configuration is being updated, we should not stop and start the component.

If a component's ID changes between the last and current component models, Elastic Agent will as the component's service to stop and then start itself. Prior to this PR, service components' ID were determined by their input type and output ID. Therefore, if a service component's output were changed, it would cause the service to restart. This is undesirable behavior, as services should be kept running as long as possible.

With the changes in this PR, we no longer consider the output ID when generating service components' IDs. If a service component's output is changed, it's ID remains the same between the last and current component models. Elastic Agent does not stop and start the component's service but simply passes the configuration change to it (which it was doing prior to this PR anyway).

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

None.

How to test this PR locally

1. Using the Fleet UI, create three Agent policies:
   • default: containing only the system integration
   • tp-es: containing the Elastic Defend integration, with tamper protection enabled, and using the Elasticsearch output.
   • tp-ls: containing the Elastic Defend integration, with tamper protection enabled, and using the Logstash output. Note that you will need to create the Logstash output in Fleet > Settings.
2. Enroll an Elastic Agent in the tp-es policy and verify the agent is healthy and shipping data.
3. Assign the Agent to the tp-ls policy.
4. Check the Agent logs and make sure the Endpoint component is not being stopped and started. Concretely, check that there is no log entry for stopping endpoint service runtime.
5. Check the Endpoint logs (located under /opt/Elastic/Endpoint/state/log/ on Linux) and make sure that Endpoint has connected to Logstash (or has attempted to and failed if there is no actual Logstash endpoint listening).
6. Assign the Agent to the default policy.
7. Check the Agent logs and make sure the Endpoint component is stopped and uninstalled. Concretely, check that there is a log entry for stopping endpoint service runtime, followed by uninstall endpoint service, followed by Stopped: endpoint service runtime.

Related issues

• Resolves Cannot successfully change output type or name in tamper protected agent polices that contain Elastic Defend #11266

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[mergify]

This pull request does not have a backport label. Could you fix it @ycombinator? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elasticmachine]

💔 Build Failed

• Buildkite Build
• Commit: d7bf91b

Failed CI Steps

• Win2022:sudo:fleet
• x86_64:sudo: default
• x86_64:sudo: default
• x86_64:sudo: default - platform-ingest-elastic-agent-debian-13-1762801856

History

• 💔 Build #31949 failed 1951fec
• 💚 Build #31882 succeeded 8787cfc
• 💛 Build #31841 was flaky a71bf32
• 💛 Build #31839 was flaky 18d8022

cc @ycombinator