You may want to try something similar to Python Polars scan_csv for lazy evaluation of same schema csv directories. It also supports a SQL context where you can

You may want to try something similar to Python Polars scan_csv for lazy evaluation of same schema csv directories. It also supports a SQL context where you can use a subset of Ansi Sql instead of learning the functional api to start.