That would be extermely complicated and would require additional support from every game. Streaming right now takes every frame, compresses it and forwards it,

That would be extermely complicated and would require additional support from every game. Streaming right now takes every frame, compresses it and forwards it, as simple as that and works for everything.