feat(sdk): Add plugins - #1436
Conversation
213835b to
65d811d
Compare
| #[builder(field)] | ||
| worker_plugins: Vec<ErasedWorkerPlugin>, |
There was a problem hiding this comment.
Does this make sense here? Would it be possible to have a worker plugin add itself to the worker options and infinite loop? 😆
There was a problem hiding this comment.
It is private so the only way that a plugin could mess with the list would be to rebuild from scratch via *options = WorkerOptions::new(...).worker_plugins(vec![AnotherPlugin]).build().
Even if a user did that we wouldn't recurse since we take the plugin list before we start applying it. Added a unit test verifying this along with a warning comment.
There was a problem hiding this comment.
Well I was less concerned about the infinite loop thing and more just wondering why they actually need to get stored here. Doesn't the plugin apply the options stuff earlier on via the client? Or I guess there's another worker-only path that I probably just didn't quite grok in my review
There was a problem hiding this comment.
Yeah, we still allow for worker plugins to be explicitly configured on the worker directly instead of the client. Could force everything through the client and remove this though if that's desired.
| /// | ||
| /// **Experimental:** This API may change or be removed. | ||
| #[derive(Clone)] | ||
| pub struct ClientAndWorkerPlugin { |
There was a problem hiding this comment.
Yeah I think in theory people could avoid needing to know about this directly by impl
impl From<T> for ClientAndWorkerPlugin where T: ClientPlugin + WorkerPlugin
And then accept anything that Into<ClientAndWorkerPlugin>
Or, maybe better, do
trait ClientAndWorkerPlugin { ... }; impl<T> ClientAndWorkerPlugin for T where T: ClientPlugin + WorkerPlugin {...}
There was a problem hiding this comment.
What would the be the trait bounds in the client crate for accepting a ClientAndWorkerPlugin? The trait would need to be defined in the client crate, but the blanket impl needs to be in the SDK so we have access to WorkerPlugin, but won't be able to because of the ophan rule.
We could do this via a trait extension defined in the SDK crate that adds a combined_plugin<ClientPlugin + WorkerPlugin> to client options? Still some concern about user misuse of passing a combined plugin to client_plugin not realizing the woker half of the plugin is getting dropped.
There was a problem hiding this comment.
Ah, I see what you mean. Yeah, that being the case I can see why this makes sense.
91a9469 to
08c3d48
Compare
Sushisource
left a comment
There was a problem hiding this comment.
My last comment isn't blocking. Might be good to have @tconley1428 do a pass.
| /// | ||
| /// Worker plugin registrations are captured before this method is called. Altering plugins in | ||
| /// this method does not change which plugins are applied. | ||
| fn configure_worker_options(&self, _options: &mut WorkerOptions) -> Result<(), PluginError> { |
There was a problem hiding this comment.
In other languages this also provides a mechanism for running around the worker lifetime
| #[builder(field)] | ||
| data_converter: Option<DataConverter>, | ||
| #[builder(field)] | ||
| client_interceptors: Vec<Arc<dyn ClientInterceptor>>, |
There was a problem hiding this comment.
I have found separate client/worker interceptor lists to be tricky to use when you have an interceptor which does both (unless that's not a thing in rust) - instead I took both as a single list.
There was a problem hiding this comment.
A type could implement both. Hard thing is we would need to add a new enum with each variant having a different Arc<dyn *Interceptor> combination. End users would need to manually construct these as we couldn't provide conversions that would distinguish between types that are implement one interceptor or those that implement multiple.
| /// **Experimental:** This API may change or be removed. | ||
| #[derive(Clone, bon::Builder)] | ||
| #[builder(state_mod(vis = "pub"))] | ||
| pub struct SimplePlugin { |
There was a problem hiding this comment.
This will be more limited than other simple plugins, which allow you to see the previous value before setting using functions. That's especially common around data converter manipulation.
| #[builder(field)] | ||
| workflows: WorkflowDefinitions, | ||
|
|
||
| #[builder(field)] |
There was a problem hiding this comment.
I don't follow why adding plugins added new interceptor options here. How did worker interceptors work before?
There was a problem hiding this comment.
Previously they were registered by modifying an already constructed worker. First commit of this PR switched so all interceptors are set via the WorkerOptions instead of modifying already created workers.
| client_plugin_names: HashSet<String>, | ||
|
|
||
| #[builder(field)] | ||
| plugins_applied: bool, |
There was a problem hiding this comment.
What are these things for? They sound like internal state, but this seems like a public configuration object.
There was a problem hiding this comment.
Was able to remove the worker one by being a little more careful around the worker lifecycle. Still need the client one as the client in activity context is constructed. This bit of state isn't visible to end users.
| /// Use [`ClientAndWorkerPlugin`] for plugins that target both clients and workers. | ||
| /// | ||
| /// **Experimental:** This API may change or be removed. | ||
| pub trait WorkerPlugin: Send + Sync + 'static { |
There was a problem hiding this comment.
Does the rust SDK have a replayer?
09ba165 to
5b1913a
Compare
What was changed
Prefactors
Debugimplementations toClient/WorkerWorkerOptionsinstead of directly on the Worker. Allow multipleWorkerInterceptors per Worker instead of only a single one. This is done in the first commit of the PR.Main Changes
Added
ClientPluginandWorkerPlugintraits which allows users to implement plugins. The stored types areErasedClientPluginandErasedWorkerPlugin. The worker plugin is just a namedArc<dyn WorkerPlugin>, butErasedClientPlugincarriesWorkerPluginDatawhich is essentially a namedAnythat carries worker plugins. This is needed since the client crate doesn't have access to the types necessary to define the worker plugin trait. On the actual worker plugin consumption side, we only respect the privatePropagatedWorkerPlugin.Client/worker only plugins will be registered via
ClientOptions::client_plugin(plugin: impl ClientPlugin)/WorkerOptions::worker_plugin(plugin: impl WorkerPlugin).Dual plugins are registered using
ClientAndWorkerPluginstruct which enabled propagating plugins that from aClientto anyWorkerthat is constructed with that client. These plugins are used withClientOptions::plugin. e.g.Why?
SDK Parity
I do not love the special type for dual plugins, but I didn't find a better solution that didn't encourage misuse.
We could get rid of the special
client_plugin/worker_pluginconstructors and force users to build the erased types themselves if we want to remove some surface area here. We could offer blanket implementation conversion for any plugin implementers into the erased type, but that opens the door for misuse of dual plugins e.g.Checklist
Closes [Feature Request] Support Plugins #1356
How was this tested:
Added unit tests around error propagation. Very basic integration tests just verifying plugin fields are respected.
Any docs updates needed?
Plugin docs can now be updated to include Rust samples once this is released.