EdwardRaff

13 ms·

1.
by EdwardRaff 7y ago
Oh, I totally agree with everything you've said! Some collaborators I'm chatting with have more niche NLP applications where larger n would be valuable though. I don't want to go into too much detail yet since it's their
2.
by EdwardRaff 7y ago
I want to open source it, but the government owns the code and bureaucracy is slow. We've talked about it, but it will move at its own pace. We think our tool will provide features that can improve AV systems to be better and more accu
3.
by EdwardRaff 7y ago
Top-k for which phase? Most of the work we actually don't use any fancy sketching. Just a GIANT table where we count hashes, and ignore collisions, making it super fast to count. Then we just use Quick-Select to find the top-k counts o
4.
by EdwardRaff 7y ago
Thats true in many cases! For our malware work and research, we've gotten a lot of consistent push-back that n-grams needed to be larger. A lot of older work (with much smaller datasets) said 15-20 grams were best, but that couldn&#x27
5.
by EdwardRaff 7y ago
Thanks! Yea, we "cheat" by restricting ourselves to the top-k & distributional assumptions. For our case, the low frequency grams are never used, so it makes sense. >So if I understand correctly... the application here is t
6.
by EdwardRaff 7y ago
Bioinformatics is one of the application areas we thought this might be useful! I've chatted with a few, and it appears the applications right _now_ might be more limited due to the need for exact matches. It's something we are th
7.
by EdwardRaff 7y ago
Paper author here, happy to answer questions! I'll try and pop in as I have time in the day :)
8.
by EdwardRaff 9y ago
We have looked into it, it's just not in this paper. It actually works better on other file formats. PDFs are really easy to do with even simpler techniques, no parsing is needed. Modern office docs need to be unzipped first, but that&
9.
by EdwardRaff 9y ago
At a very high level, yes. But the same could be said for anybody in the AI-AV space. At a more technical level, the approach we take in this paper (and most of my research) is fairly orthogonal to what most AV vendors are doing. Even com
10.
by EdwardRaff 9y ago
Brushing up on your math will definitely help! But it's not strictly necessary. It really depends on how deep you want to go. For the Neural Network type stuff we used in this work, I would recommend Michael Nielsen's awesome webs
11.
by EdwardRaff 9y ago
Really, the benign-vs-malicious question is an oversimplification. But thats what we have data for, and what most people focus on. The reality is there is a big gray area between the two classes. Some cases are really hard to determine, and
12.
by EdwardRaff 9y ago
I'm not sure I understand the question. We didn't use any data from ClamAV.
13.
by EdwardRaff 9y ago
I've tried. They are fairly unresponsive. Right now my advisor is trying to get an academic license to their system.
14.
by EdwardRaff 9y ago
Feel free to send an email if you have any questions when trying it! Since it is a static technique we don't expect it to become quite as good as what you could get with a dynamic approach, but we've been happy with our results th
15.
by EdwardRaff 9y ago
I'd be cautious about making broad statements regarding how AI based AVs tend to behave. It's a new space with a lot of competitors, of varying degrees of quality, with varying degrees of actual AI in use. Not all of them are ev
16.
by EdwardRaff 9y ago
Hi, paper author here! The dataset is small by AV standards, but we aren't an AV company. We can only use as much as real AV companies are willing to share with us. If you'd like to share more, we would be happy to take it :) The
17.
by EdwardRaff 9y ago
This is something we are looking at! It is a harder problem to create adversarial examples in the malware space, because you can't make arbitrary changes and have the code still work. Endgame has a great paper on this problem, and show
18.
by EdwardRaff 9y ago
Paper author here! A lot of that issue comes from people using bad datasets. One of our first papers was about that ( http://www.readcube.com/articles/10.1007/s11416-016-0283-1?a... ), and showed that using the