I came here to say the same thing. We use to rip through anywhere from ~100M-3B rows of data every day.. updates to this dataset are time sensitive so we use

I came here to say the same thing. We use to rip through anywhere from ~100M-3B rows of data every day.. updates to this dataset are time sensitive so we use it to finish the job as quickly as possible.