I think some of this might be misinformed. I know Hive can definitely query other datasources like traditional SQL databases, redis, cassandra, hbase, elastics

I think some of this might be misinformed. I know Hive can definitely query other datasources like traditional SQL databases, redis, cassandra, hbase, elasticsearch, etc, etc. I thought Impala had some bit of support for this as well, though I'm less familiar with it. And SparkSQL can be run on a multi-tenant cluster with automatic resource allocation - Mesos, YARN, or Kubernetes.