scv119

6 ms·

1.
by scv119 3y ago
The tweet is referring to a paper that fine tunes Chinese dataset on english base model. I'm not surprised with LoRA's poor result in this setup.
2.
What we talk about when we talk about system design(maheshba.bitbucket.io)
259 points by scv119 3y ago|35 comments
3.
by scv119 3y ago
I believe you can slightly change the flash attention kernel to implement the same kernel of this page attention, since both of them work on the key/value cache at block level.
4.
by scv119 3y ago
Pretty cool stuff and the results are amazing. Hoping we will see virtual memory get standardized in pytorch or cuda.
5.
by scv119 5y ago
We wanted to ensure everyone on the team has explored the full design space, and everyone agreed that the chosen solution is the best we can think of, given the current information and constraints. Once we have done that, everyone is aligne
6.
by scv119 5y ago
Co-creator of Delos here, Let me give a concrete example for [18]. We have a "StorageEngine" interface that wraps around RocksDB, i.e. "RocksDbStorageEngine" to provide key-value interfaces; however we are worried that s
7.
by scv119 5y ago
I guess that a hard question for user facing API: you need to support both the old and the new features during the migration. In our case, this design principle mostly applied to our internal APIs. Particularly, we designed our internal sub
8.
by scv119 5y ago
The first engineer on the Delos team here. This design decision gave us an edge: we delivered the first production cluster in 8 months by layering over existing systems ; and we smoothly migrated all underlying implementations in the follow
9.
Paxos Made Abstract(maheshba.bitbucket.io)
2 points by scv119 5y ago|0 comments
10.
42 Things I learned from building a production database(maheshba.bitbucket.io)
3 points by scv119 5y ago|0 comments
11.
by scv119 8y ago
Agreed, technology-wise FDB and TIDB look almost the same.
12.
by scv119 12y ago
Sorry, there is no DECENT internet connection in China