msackman

8 ms·

1.
by msackman 4y ago
The software industry seems to fetishize complexity. It's extremely harmful, yet often appealing, and I don't know how we can avoid it.
2.
Complexity and Software Engineering(wellquite.org)
1 points by msackman 4y ago|1 comments
3.
GoshawkDB 0.3 Released. Object Capabilities and Collections Library(goshawkdb.io)
5 points by msackman 10y ago|0 comments
4.
by msackman 10y ago
Paxos: I've been trying to look for that. Having cloned the code and grepped for paxos I'm getting no hits. Where is the paxos implementation?
5.
GoshawkDB 0.2 released(goshawkdb.io)
3 points by msackman 10y ago|0 comments
6.
by msackman 11y ago
So GoshawkDB doesn't have an SQL engine currently, so in that way it's probably not comparable with F1. GoshawkDB stores and accesses an object graph. Hopefully the howtos on the website will help people get into the mindset. I&#x
7.
by msackman 11y ago
Heh, definitely! I could ramble for hours about how there are so many disincentives to share anything until launch - the whole industry really isn't set up to actually make progress. However, multiple independent co-invention of stuff
8.
by msackman 11y ago
I'm the author of GoshawkDB. I've just been watching the talk on this at https://www.youtube.com/watch?v=yE3eMxYJDiE . GoshawkDB has a very similar design wrt the messaging and replication design. In fact, in some
9.
by msackman 11y ago
Not quite. Only F+1 are required per object to vote. As I explain in the blog post ( https://goshawkdb.io/blog.html#20151224 ), the remaining F can be sent after the outcome is known.
10.
by msackman 11y ago
Well I have no idea of how Cassandra works, so I can't comment on that. GoshawkDB gets every object touched in the transaction to vote on the outcome of the transaction, in parallel, and guarantees that if two transactions touch the sa
11.
by msackman 11y ago
Yes, you're right. Now where that paper finishes is pretty much where GoshawkDB starts. Because with GoshawkDB, the server controls the object UUId, you can push as much entropy in there as necessary so the problem identified in the pa
12.
by msackman 11y ago
For those interested in a bit more detail how GoshawkDB actually works, I've added a (slightly rushed) blog post on the topic, which covers transaction lifecycle. https://goshawkdb.io/blog.html#20151224
13.
by msackman 11y ago
I don't think that's the problem though. The problem that I'm thinking of is that when a cluster grows in size, due to the use of consistent hashing, there'll be a set of objects that need to move between nodes. Calculat
14.
by msackman 11y ago
Excellent points well made :) I think I may well change the description then as you suggest, and once I have clients that support the same API model as things like ZODB then I'd describe it as something like "document store that c
15.
by msackman 11y ago
Thanks for the link. Whilst I'm not arguing with your point, I believe I've never used the term "object database" to describe GoshawkDB, only "object store". I guess I'm struggling to find a more accurate
16.
by msackman 11y ago
Having just had a quick look at both, the only point I'm willing to make is that ignite's distributed transactions use 2PC. https://apacheignite.readme.io/docs/transactions 2PC, at least in its usual forms, i
17.
by msackman 11y ago
So that I understand, what is the definition of object store?
18.
by msackman 11y ago
> As I understand it, in this hypothetical case, the majority of the system would work fine if 3 nodes fail, but there could also be some percentage of transactions which are unfulfillable due to this scenario while there should be many
19.
by msackman 11y ago
Ok, so F=2, so 2F+1 is 5. So client 1 creates a txn that writes to object x. x has replicas on A-E, but not F and G. So client 1's txn gets sent to F+1 drawn from A-E. In your scenario A and B are down so that only leaves C, D and E. S
20.
by msackman 11y ago
So ZODB has really great integration with Python: the fact that you can just subclass Persistent and get persistent objects is a really nice model. That type of model is certainly the goal for GoshawkDB but I certainly accept your point tha
21.
by msackman 11y ago
> Considering more than one exact txn I imagine will hit a single specific node often, at large scale with a single mode down even if that means 5% of transactions block, you are basically growing a queue of waiting work indefinitely wit
22.
by msackman 11y ago
Not quite: Paxos Synod is merely used as the replacement for 2PC as 2PC does not work. So that's just achieving consensus on the txn votes for each node. It's actually vector clocks (and improvements thereof) that manage the depen
23.
by msackman 11y ago
> With your situation #1, what if this is very common transaction and therefore you have 100 of these all waiting. What about 1000, 5000 etc. what system resources are used to let these transactions wait indefinitely ( if I understand yo
24.
by msackman 11y ago
Hi, I'm the author of GoshawkDB. Thanks for your questions - they're certainly not answered on the website so I'll do my best to answer them here and incorporate them into the website later. > Curious how this works at sca
25.
by msackman 11y ago
Hi, I'm the author of GoshawkDB. I'm no Cassandra expert, but I'm certainly under the impression that Cassandra does not offer full transactions - lightweight txns only that are atomic only for a single row.
26.
by msackman 11y ago
Yup, on the whole, in order to increase performance, you don't want more and more nodes to be contacted. Remember that every txn has to have a minimum of F+1 replicas vote on each txn, so if F is really large, although they vote in par
27.
by msackman 11y ago
Hi. I'm the author of GoshawkDB. Thanks for your questions. > It isn't true. There're plenty of storages with same strong consistency model (tolerates F failures of 2F+1 nodes), among them are Cassandra with lightweight tr
28.
by msackman 11y ago
> There is no requirement for weave that the password is human readable. It can be supplied through a file, so you can happily dd if=/dev/random of=/my/weave/passwd bs=1k count=1 to create a suitable weave passwo
29.
by msackman 11y ago
DH is used to ensure that every connection between every pair of weave router nodes that ever gets established uses a unique session key. Yes, the public key is exchanged in the clear, and yes, that is MITMable. But as you say, the subseque
30.
Analysing and improving consistent hashing(wellquite.org)
1 points by msackman 13y ago|0 comments

More ›