jeeceebees

6 ms·

1.
by jeeceebees 2y ago
You can just mask the output probabilities for each token based on which options are valid according to a grammar. There are quite a few open source implementations of this e.g. https://github.com/outlines-dev/outlines
2.
by jeeceebees 5y ago
I think this is a property spheres. It seems to me that any two spheres that are touching have a straight line from one center to the other center exactly through the point of contact. Try thinking of just two spheres and adding more in ste
3.
by jeeceebees 5y ago
How does the performance between GPU programs written with std::par compare to those written in CUDA? Do you happen to know of any online resources that show a comparison of the kernel code and performance of the two frameworks on common ta
4.
by jeeceebees 5y ago
There is a lot of evidence that these token-based models work with multi-modal data. In fact, several groups have proposed different multi-modal transformer architectures already (e.g. [1] or [2]), although I don't believe anyone has s
5.
by jeeceebees 5y ago
Having more memory to address means you need more circuits that direct your reads/writes to the right place. Travelling a longer distance / through more complicated routes means the latency for each request will be higher on avera
6.
by jeeceebees 5y ago
The hidden state isn't storing the samples of your loop (or a compressed version of your loop). It's encoding a representation of how the output will change based on what the current state and input are. This might be strongly dep
7.
by jeeceebees 5y ago
LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state size because it's just learning a nonlinear func
8.
by jeeceebees 6y ago
I think most human creativity is built around a seed of inspiration from outside sources. In my experience serendipity and happy accidents are exactly what leads to the most creative and interesting outcomes. This is a system that’s able to
9.
by jeeceebees 6y ago
I think the larger models get, the more incentive there is for researchers to look into pruning/distilling them for practical use. GPT-1,2,3 et al. have all shown that larger is better. While in the short term this means people will si
10.
by jeeceebees 6y ago
It's YYMM.ID I believe Last year may was 1905.xxxxxx
11.
by jeeceebees 7y ago
Have all novel qubits gotten nobel prizes so far? I think there's more than enough room in between bullshit and nobel prize. This looks very promising, but as always the devil is in the details. The next steps are multiqubit gates, the
12.
by jeeceebees 7y ago
I don't really understand why I'd use gin. From the example ipynbs it looks like pretty much the same amount of code but in a gin file and then it spookily fills in parameters for you in python. Why is this useful?
13.
by jeeceebees 7y ago
As far as I understand, it's because what it's simulating is a logical qubit which is different from the very noisy, almost instantaneously collapsing physical qubits present in current quantum computers. Software simulates what&#
14.
by jeeceebees 7y ago
Of course there is. All the building blocks that people are mix and matching in networks nowadays were introduced at some point. The paper that introduced batch norm, adaptive instance norm, attention heads, or any module used in a network
15.
by jeeceebees 7y ago
I think the approach is really cool but the processing time required is too much for this to be very useful at the moment. On a 1080 Ti it takes 45-90 minutes to train networks for the various tasks on 256px images (depending on some qualit
16.
by jeeceebees 7y ago
I had already completed this course before Stefan started his conquest of the CS bachelors courses, however, I did have some other courses that he taught. I think he has a great knack for explaining things in a way that doesn't make th
17.
by jeeceebees 7y ago
Detect when there are multiple apps that are asking for new permissions simultaneously and then ask the user if they want to go to the "bulk permission editing page" to manage them. On that page there could be a grid of apps vs pe
18.
by jeeceebees 7y ago
This is the same idea that underlies style transfers and metrics like the FID (which is used to judge generative networks' outputs on their similarity to the test set). The idea is that the activations within an image recognition netwo
19.
by jeeceebees 7y ago
I think your camera example is a false equivalence that makes this seem as if it's not a computation. The camera is not running the same algorithm as the renderer and so you're comparing different things. The experiment used a cla
20.
by jeeceebees 7y ago
I don't think it's a non-event even if you're only interested in factoring. This proves the underlying principle of a quantum speedup is a physical reality. It might be something people already take for granted if they'r
21.
by jeeceebees 7y ago
Would be even nicer if it could be trained on unpaired datasets (ala CycleGAN https://arxiv.org/abs/1703.10593 ).
22.
by jeeceebees 8y ago
The author notes that the analysis is based on 1300 songs from the Billboard top 100 in the last 2 years. I think it's safe to say the effect of metal songs using phrygian is negligible.
23.
by jeeceebees 8y ago
Distill[1] is another example of interactive scientific papers (with a focus on machine learning). But is there really a good reason to not just keep these in browser? I don't really know if there's much value in reading these loc
24.
by jeeceebees 9y ago
Accompanying jupyter notebook: https://colab.research.google.com/drive/1e9o2NB2GDDjadptGr3r...
25.
WaveGAN: Synthesizing Audio with GANs(wavegan-v1.s3-website-us-east-1.amazonaws.com)
2 points by jeeceebees 9y ago|1 comments
26.
by jeeceebees 9y ago
Alpha Go Zero is actually more similar to a GAN: https://arxiv.org/abs/1711.09091
27.
by jeeceebees 9y ago
I'm quite partial to this implementation of deep dream: https://github.com/ProGamerGov/Protobuf-Dreamer Feel like something similar should be possible with a style transfer as well. Also: https://distil
28.
by jeeceebees 9y ago
I think the interesting part would be example based audio synthesis. Could you replace a synthesizer with a neural network which, when fed examples, would allow you to generate sounds / explore some latent space between the examples. F
29.
by jeeceebees 9y ago
If you created a 2-way pegged bitcoin ERC20 token, you could use something like EthLend.
30.
by jeeceebees 9y ago
They also added compelling generated samples back into the database. A sort of evolution based on the original sounds thay decided to seed it with.

More ›