I work on collecting (ETL) data of European power data sources and supply them to clients via web service (in almost real time). For the client i work on - wrot

I work on collecting (ETL) data of European power data sources and supply them to clients via web service (in almost real time). For the client i work on - wrote cloud based distributed ETL system to collect data from approx 100 importers and exporters of power, gas, wind, nuclear. Its quiet possible to collect this data in real time as EU regulations asks each of them to provide data on their websites. For some of the websites - i use selenium which is much better than using webrequest via http/s