I think there are a few more details on what such a scheme would look like in this thread: Coprocessors or modules
It also sounds like as a constructive proof that you could do that that Wavefront already have such a system set up. It is probably the easiest way to do something like what you’ve described without a new API to push down operations.