Have you considered message queues? You could push updates to a queue (with reasonable max size limits to avoid overflow). This queue could be exposed through w

Have you considered message queues? You could push updates to a queue (with reasonable max size limits to avoid overflow). This queue could be exposed through websockets or by using AWS SQS. Your customers would populate the older data using your existing API, but then switch to the queue for new updates.