better365

14 ms·

1. 1.▲
   by better365 2y ago

   That's fair. Actually, Last is usually called last_k(n), so that you can specify the number of the values in the result array. For example, if the input column is page_view_id and n = 300, it will return the last 300 page_view_id as an
2. 2.▲
   by better365 2y ago

   We utilize a lambda architecture, which incorporates the concept of precomputed tables as well. Those precomputed tables store intermediate representation of the final results. These precomputed tables are capable of providing snapshot or d
3. 3.▲
   by better365 2y ago

   True. But it is not necessary to reinvent the wheel for engineers. :)
4. 4.▲
   by better365 2y ago

   For offline computation, it is okay with the table with 5bn rows. But for online serving, it would be really challenging to serve the features at a few milliseconds. But even for offline computation, for the same computation logic, the code
5. 5.▲
   by better365 2y ago

   For correctness, yes, it works as well as the SQL version. And the aggregation can be extensible for other operations easily. For example, we have an operation of last, which is not even available in standard SQL.