markchen90

7 ms·

1.
by markchen90 6y ago
I didn't notice any obvious visual differences, but I'm also not an expert on adversarial examples. The transformer models were similarly susceptible to attacks, but while adversarial examples transferred well within a model class
2.
by markchen90 6y ago
Author here. You're absolutely right that "understanding" is a fuzzy word. As you pointed out, part of reason we hold this belief is that the model can generate diverse samples and successfully complete out-of-distribution in
3.
by markchen90 6y ago
Author here. I ran some early experiments a while ago, and it looked like adversarial examples for convnet classifiers didn't transfer to transformer classifiers and vice versa. Definitely worth looking more into!
4.
by markchen90 6y ago
I don't think this is true. The ResNet was born at Microsoft, DQN was born at Deepmind, the Transformer was born at Google, and GPT2 was born at OpenAI. I'm obviously biased since I work at an industry AI lab, but we both have imp
5.
by markchen90 7y ago
These analogies need to be taken with a grain of salt. For instance, king - man + woman actually returns king (since woman - man ~= 0). It's only queen when you ban the words king, man, or woman (since the embeddings of queen and king
6.
by markchen90 7y ago
The article discusses effects on doctoral students, not masters or undergrads. STEM PhD programs at top schools are a lot less pay-to-play.
7.
by markchen90 10y ago
Are you serious? This class is intended for students with deep knowledge of CS theory who want to learn more about recent data structures research. 6.006 and 6.046 are much more suitable if you aren't formally trained in computer scien
8.
by markchen90 10y ago
I took this class in 2010. Most assignments consisted of proving one or two theorems from a recent research paper.