6 ms·
150,000 rows of data, where will I store it all?!
by actusual 3mo ago
150,000 rows of data, where will I store it all?!
- nojito 3mo agoThe biggest misconception that people have when modeling using tabular data is that more data = better model.
- mvcalder 3mo agoThere's a story sharing the front page right now about scaling laws, so it's not an unreasonable assumption. https://news.ycombinator.com/item?id=48689744 https://news.ycombinator.com/item?id=48689744
- estetlinus 3mo agoBig Data is terrifying and only solvable with opaque deep learning
- piokoch 3mo agoYou don't need to. Just sample 1% of data, let the model do the feature engineering, ask model model to replicate everything in Pandas (or R, or whatever else), so you can run it for a full dataset.