aboytsov

4 ms·

1.
by aboytsov 12y ago
San Francisco, CA - FULL-TIME, INTERN, VISA Stuph ( https://angel.co/stuph , https://stuph.co ) is building its founding team! We are a stealth startup creating a consumer web product fueled by data mining and info
2.
by aboytsov 12y ago
Stuph ( https://angel.co/stuph , https://stuph.co ) is building its founding team! We are a stealth startup creating a consumer product fueled by data mining and information extraction. We are well funded and backe
3.
by aboytsov 12y ago
San Francisco, CA [will relocate/sponsor visa] Stuph ( https://angel.co/stuph ) is building its founding team. We're a stealth startup with a very ambitious vision. Our technology stack is Clojure/ClojureScript
4.
by aboytsov 13y ago
Stuph is building its founding team ( https://angel.co/stuph-inc ) Location: San Francisco, Positions: Senior Software Engineer, Frontend Software Engineer (Web + Mobile), UX designer We're an early stage, well funded st
5.
by aboytsov 14y ago
This is not a cyclic dependency graph!!! This is a syntax for copying vertices, nothing else. It creates a DAG of three vertices and two edges, but uses only one step definition to do so. It automatically replicates the step definition as n
6.
by aboytsov 14y ago
Yes, startup times are a pain. As of this morning, Drake now works with Drip, which is a nifty tool to bring down start up times. It spins "backup" JVMs, so next time you run the command, JVM is ready. It works great for interactive environ
7.
by aboytsov 14y ago
Actually, it was in the doc. If you followed the instructions below precisely, just send us your terminal log so that we can see what you're missing. A nicer way to run Drake We recommend you "install" Drake in your environment so that you
8.
by aboytsov 14y ago
Drip now works with Drake! Yes, it's still less than ideal if you're calling Drake hundreds of times from an automated script which you need to run quickly, but for interactive development, it should work just fine: https://github.com/Fact
9.
by aboytsov 14y ago
You don't have to have Drake.jar in the same folder as the workflow you're trying to execute. You create the script as described in the documentation, and you put it somewhere on your PATH along with the JAR file. The JAR files has to b
10.
by aboytsov 14y ago
Read the "Installation" section, there's "A nicer way to run Drake" subsection. But I would advise to read the whole "Installation" section carefully.
11.
by aboytsov 14y ago
> I'm sorry I don't have time to answer in full. We're not getting anywhere. Just give me goddamn examples! :) Please! Examples! > I can see this is really really hard to grok if you're basing everything on the idea of a DAG, and so
12.
by aboytsov 14y ago
I think, that would be a stretch to say so - in my opinion, it's not a good thing for the sole reason that it doesn't let you opt out of it. And it's not one of those cases when you would want to enforce discipline because renaming a method
13.
by aboytsov 14y ago
...continued from part1. read part1 first!... > It doesn't solve everything, but I guess the idea is, make it work right for the majority of cases ("sensible defaults") and then offer ways to deal with harder cases ("make simple things
14.
by aboytsov 14y ago
part1 >> The problem with this approach is because figuring out where the files are requires knowledge of the tool inner workings, that can only be acquired from reading the code or documentation > I suppose this is true but it'
15.
by aboytsov 14y ago
The project's README file ( https://github.com/Factual/drake - scroll down) contains building and running instructions, as well as how to create a simple script to run Drake which you can put on your PATH.
16.
by aboytsov 14y ago
Another problem with BPipe's approach is if you change method's name, you invalidate the existing files. This can be a problem during development, when re-running steps are expensive.
17.
by aboytsov 14y ago
Thank you. Why not? We would love to see it, but we're also not actively using Amazon S3 at the moment. But we would be more than happy to review code contributions. First of all, you can file a feature request: https://github.com/Factual/
18.
by aboytsov 14y ago
Thank you very much for your response. Actually, I don't think there are any philosophical differences, and I'll try to make my case. > Bpipe absolutely says you don't want to manage the file names. I think this is too strong a statemen
19.
by aboytsov 14y ago
To be honest with you, no, we didn't seriously consider it. Maybe we should have. I do not know if ClojureScript would be able to work with all the dependencies we have (for example, Hadoop client library to talk to HDFS). But it's a good p
20.
by aboytsov 14y ago
It's a good point, and I agree it might not be the top priority, but I also understand the frustration. I, too, find 5s start up file rather irritating especially when I make errors in the workflow file, or didn't specify targets correctly.
21.
by aboytsov 14y ago
Got ya. Please voice your opinion about the priority in which features should be implemented by submitting a feature request at https://github.com/Factual/drake/issues , or +1'ing an existing one. There are so many potential features to be
22.
by aboytsov 14y ago
Thank you very much for your kind words and support, and we certainly are looking forward to your feedback, feature requests and bug reports, as well as your code contributions, should you so desire. We built this based on our own pain poin
23.
by aboytsov 14y ago
Thanks. I think you're missing the point. Imagine a big, complicated data workflow, like the one the diagram for which I showed at my video (real-life workflow): http://www.youtube.com/watch?feature=player_detailpage&v... Now imagine
24.
by aboytsov 14y ago
I see. Thank you very much. I think this is very cool. I can see several problems with this approach, and I would greatly appreciate it if you could comment on that. After all, I don't know Bpipe. The fundamental issue is why do you have to
25.
by aboytsov 14y ago
I can certainly see your point about using Drake in an automated environment where this delay would still matter, but running a daemon is not practical. I think you have a lot of good arguments against JVM. There were some moments when I th
26.
by aboytsov 14y ago
Thanks for your feedback. We do mention parallelization in the designdoc, it's just not implemented yet. It's quite easy to add though. We have a lot of features spec'ed out, but not implemented. I would appreciate if you elaborated on sepa
27.
by aboytsov 14y ago
Phew. Thank you.
28.
by aboytsov 14y ago
Really? This is not a weird request, this is one of the most common things we do when we're developing a workflow. You need to do this every time you make changes to code and you want these changes to propagate. You can't run "make all", be
29.
by aboytsov 14y ago
"make target" will not make stuff down the tree. It will stop at target. Please, try again. :)
30.
by aboytsov 14y ago
Hey, thanks for trying out our tool! First of all, --version shouldn't try to run any targets. This seems like a bug. Thanks. Yes, you guessed correctly - this is the JVM startup time. I just hate JVM for that. We experimented with Nailgun

More ›