This is a great reply! The greatest difficulty I face is convincing my data engineers to stop reinventing the wheel and leverage existing (and appropriate!) too

This is a great reply! The greatest difficulty I face is convincing my data engineers to stop reinventing the wheel and leverage existing (and appropriate!) tools. Writing code to directly manage events and batch jobs should be a thing of the past by now. Pick a tool. Configure your jobs, retry policy, etc. and be done with it.