kootenpv

34 ms·

1.
by kootenpv 3y ago
On phone I typed with a capital letter the correct word but it was considered wrong.
2.
by kootenpv 4y ago
I have 2 popular python projects, one with 4.9k and one with 2.3k stars and I don't qualify :/ https://github.com/kootenpv If anyone knows why pls let me know
3.
by kootenpv 7y ago
Models are cached and not large so the setup time is very low. I'll time it when I get to a PC
4.
by kootenpv 7y ago
Perfect interview questions to get to the juice details haha! The problem is that choice of compression is very much dependent on the sample size, so this is why just choosing the algorithm based on running benchmarks on the sample will be
5.
by kootenpv 7y ago
Yea, I thought parameters per compression algorithm should indeed be added in a next version :) more compute but definitely an improvement. I think pandas doesn't offer zstd as option with csv, but I'll check once more. EDIT: inde
6.
by kootenpv 7y ago
Yea zstd is really amazing... if I would choose a single one all the time it'd be zstd for sure.
7.
by kootenpv 7y ago
I can give you this... the larger the data the more useful parquet and compression on it will be...
8.
by kootenpv 7y ago
Yea I am aware of meta hyperparameter approach for ML, except they only focus on accuracy instead of also including train/prediction times in to the equation :) That's what I was referring to! (you can save A LOT of compute and zo
9.
by kootenpv 7y ago
Yea, look at jekyll in combination with github pages. You can see my blog for example ( https://vks.ai ), the code is hosted here: https://github.com/kootenpv/kootenpv.github.io
10.
by kootenpv 7y ago
There's TimescaleDB built on top of PostgreSQL.
11.
by kootenpv 7y ago
Indeed, but this is for lossless compression :)
12.
by kootenpv 7y ago
Spot on (I briefly touch on this in the article)! This is why I try to work with cheap-to-compute features. I used to calculate how unique all values were, but ended up taking a sample instead to speed that part up for large data!
13.
by kootenpv 7y ago
Yea - that is related to genetic programming. That, and using auto-encoders for e.g. image compression are known approaches in "AI". I'm particularly proud of this meta approach and I am actually thinking this could become hu
14.
by kootenpv 7y ago
I like mine for the fact that it is static and hosted on github pages with a not too bad design. https://vks.ai/
15.
by kootenpv 7y ago
shrynk is not doing any compression itself, it's using ML to do a "meta" approach: it computes features on your data like how many rows, how many columns, how much duplication etc, and then predicts which existing compression
16.
by kootenpv 7y ago
I'm a long time fan of your blog :O
17.
by kootenpv 7y ago
Definitely! I really optimized for "no development time spent" and was just using pandas to extract html tables into csv and just store them :-). 2 lines of code really. I had no idea I would have it running for so long. It was re
18.
Using machine learning to choose compression algorithms(vks.ai)
125 points by kootenpv 7y ago|58 comments
19.
IRL Silicon Valley TV-Show: Using Machine Learning to Compress – Shrynk
5 points by kootenpv 7y ago|0 comments
20.
by kootenpv 7y ago
Privacy & Personal Analytics I am working on an open-source implementation that is based on the fundamental idea of privacy. The consumers should have full control eventually. For now only Google etc have all the information on us, but
21.
Created a fast and customizable NLP tokenizer in Python using a novel approach(github.com)
2 points by kootenpv 7y ago|0 comments
22.
Textsearch – an NLP library for fast configurable search(vks.ai)
2 points by kootenpv 7y ago|0 comments
23.
by kootenpv 7y ago
I'm happy to share with you the result of this project, it certainly has been a lot of fun! First of all it was shocking how many messages we had over such a long time (more than 300k!)... and it was very interesting to see the patter
24.
Tricked my wife talking to a neural bot trained on our messages on Facebook(youtube.com)
2 points by kootenpv 7y ago|1 comments
25.
by kootenpv 9y ago
The best article I have read in a long time.
26.
by kootenpv 9y ago
Are you saying there is some library already existing? I'd be curious to try it then.
27.
by kootenpv 9y ago
Yes, these things should ideally be taken into account. I have been contacted by a company that is actually working on it: http://sourced.tech/
28.
by kootenpv 9y ago
Haha! That's gonna be scary.
29.
by kootenpv 9y ago
Yea, I also have a similar project in the making: https://github.com/kootenpv/xtoy . This one does optimisation of finding a machine learning model using evolutionary search, but mainly focuses on just taking any kind
30.
by kootenpv 9y ago
You are totally right: I was mixing up with another project I'm working on where I am using seq2seq (using only tensorflow) :) I will update the text of the repo. Thank you!

More ›