I have used card database files before with success. https://cr.yp.to/cdb.html https://cr.yp.to/cdb.html Have your process regularly update the CDB file from a

I have used card database files before with success. https://cr.yp.to/cdb.html https://cr.yp.to/cdb.html Have your process regularly update the CDB file from a blob store like S3. Any deltas can be pulled from S3 or you can use a message bus if the changes are small. Every so often pull the latest CDB down and start aggregating deltas again. CDB performs great and can scale to multiple GBs.