5 ms·

I'm very interested in Starfighter and am looking forward to the release. However, there's something from your original announcement that still worries me. In t

I'm very interested in Starfighter and am looking forward to the release. However, there's something from your original announcement that still worries me. In this post you said: The designed operation of our trading levels is “Play them, lose horribly, write a program, reset the level, run the program, have it crash with a bug, fix and re-run, lose modestly, adjust your approach, re-start the level, run the program, win.” (Folks have asked whether we’ll hold this against you. No, no, of course not. We log everything under the sun, but losing/restarting levels is planned. That capability is an advantage of doing this in a rich simulation as opposed to real life — you can go Knight Capital as many times as you want and no one loses their job!) This feels good. But earlier, this was said [1]: We will track player behaviors and skill at incredible levels of granular detail, instrumenting them like they were built by the Orwellian MiniPeace... We can tell you exactly what happened when your candidates tried to implement a REST API. We can compare their performance against hundreds of other talented engineers (including your current employees) on the same task. Can you please elaborate on what exactly is going to be tracked? Is any of this configurable by the player? Personally, I'm going to have a hard time enjoying and learning if I feel like every mistake I make, or every extra piece of time something takes is going to be judged and count against me later on. [1] http://www.kalzumeus.com/2015/03/09/announcing-starfighter/ http://www.kalzumeus.com/2015/03/09/announcing-starfighter/

kasey_junk 11y ago
Remember their incentives. They want you to perform well (or more precisely they want performing well in their game to correlate with you being a good employee). It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.
- pekk 11y ago
  What is a better way to sell a candidate to clients than to say she is in the top 1%? They have every reason to count out as many people out as possible. Nobody looking at this kind of service wants to hire candidates who are not the best.
  - kasey_junk 11y ago
    Because the population of devs who don't play at all is gigantic. So they can make the argument that "this person played and got through 65% which has x correlation with being a good employee" and that still puts that person in the elite status. Just knowing what stockfighter is puts them into some extremely small percentage of devs, which in the current market is a drop in the bucket for demand.
  - tptacek 11y ago
    That's not how our business model works. We're not going to make extra money based on where people rank.
    - pekk 11y ago
      I understand that, but if you want to sell any kind of screen you have to represent that the screen discriminates "better" candidates from "worse" ones. The people who are willing to pay for this screening want the top 1% or 10% or whatever (even if the way it goes down is that they request data dumps and then make up the scoring function themselves). That is how the whole industry works, it's not your fault and you almost certainly can't do anything about it. Edit: even if you present yourself as a recruiter, it doesn't make any real difference. The value you are adding and charging for is that you are handling part of the recruitment process so that your clients don't have to. Nobody wants a recruiter who doesn't hand them the best candidates.
      - tptacek 11y ago
        We're not selling a screen. We're not a job interview. We're just smarter recruiters.
        pekk 11y ago
        Pardon me. You're selling recruitment services, which happen to involve preliminary screening through Stockfighter. I am venturing the very risky speculation that these recruitment services would have no value to your clients if they were not somehow selective. If all you want is to sift through a million resumes, that has been cheaply available for years.
patio11 11y ago
I feel like this is largely an issue about expectations and disclosure more than it is about logging per se, right? We have a privacy settings page, which I'll show you in a minute, but here's the public view of myself on the staging server. I have it configured to Maximally Public (and will have that on in prod). https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442RUB/image.png https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442... The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.) Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me: https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FDfYB/image.png https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD... That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time. This is not exposed publicly. Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively. https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKXl4B/image.png https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX... Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot." (Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.) To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.

kasey_junk 11y ago

Remember their incentives. They want you to perform well (or more precisely they want performing well in their game to correlate with you being a good employee). It is not in their best interests to count against you, quite the contrary they want to sell you to their clients.

pekk 11y ago
What is a better way to sell a candidate to clients than to say she is in the top 1%? They have every reason to count out as many people out as possible. Nobody looking at this kind of service wants to hire candidates who are not the best.
- kasey_junk 11y ago
  Because the population of devs who don't play at all is gigantic. So they can make the argument that "this person played and got through 65% which has x correlation with being a good employee" and that still puts that person in the elite status. Just knowing what stockfighter is puts them into some extremely small percentage of devs, which in the current market is a drop in the bucket for demand.
- tptacek 11y ago
  That's not how our business model works. We're not going to make extra money based on where people rank.
  - pekk 11y ago
    I understand that, but if you want to sell any kind of screen you have to represent that the screen discriminates "better" candidates from "worse" ones. The people who are willing to pay for this screening want the top 1% or 10% or whatever (even if the way it goes down is that they request data dumps and then make up the scoring function themselves). That is how the whole industry works, it's not your fault and you almost certainly can't do anything about it. Edit: even if you present yourself as a recruiter, it doesn't make any real difference. The value you are adding and charging for is that you are handling part of the recruitment process so that your clients don't have to. Nobody wants a recruiter who doesn't hand them the best candidates.
    - tptacek 11y ago
      We're not selling a screen. We're not a job interview. We're just smarter recruiters.
      - pekk 11y ago
        Pardon me. You're selling recruitment services, which happen to involve preliminary screening through Stockfighter. I am venturing the very risky speculation that these recruitment services would have no value to your clients if they were not somehow selective. If all you want is to sift through a million resumes, that has been cheaply available for years.

patio11 11y ago

I feel like this is largely an issue about expectations and disclosure more than it is about logging per se, right? We have a privacy settings page, which I'll show you in a minute, but here's the public view of myself on the staging server. I have it configured to Maximally Public (and will have that on in prod). https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442RUB/image.png https://www.evernote.com/l/Aacj0YC7tCRIdbau2AgTrBORPc6kgX442... The redactions are just to avoid spoiling levels. The "replay" link is a UI templating bug. (It isn't for showing a replay, it is a convenience method for replaying the level, and it's now supposed to show unless you're logged in and able to play the level yourself. Fixing this in the next 5 minutes.) Now, if I (as a player) log into and view my own profile, I get a better sense of some stuff the system knows about me: https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FDfYB/image.png https://www.evernote.com/l/Aae3c5I6TdZA7ZtONMeM_z-ammvZA_3FD... That is obviously not all the information the system has about me. If a sparrow yawned in a world simulation that I took part in, that fact is dumped onto an S3 bucket, somewhere. To be maximally explicit: every order I sent in. Every execution I got back. Snapshots of the GM's memory state as I passed through the levels. etc. We grab all that. That isn't an exhaustive list. Assume if it is amenable to being discovered by a computer that it will be discovered and persisted until the end of time. This is not exposed publicly. Here's the preferences UI. Verbose but it gets the point across, right. Defaults are "Anonymous" and "No Public Profile", respectively. https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKXl4B/image.png https://www.evernote.com/l/AadESN5EzgdCzpFAQnR0rIuxA5Ghb7pKX... Now to a question which may be implied: "OK, I now understand that my info is locked up pretty tightly within Starfighter. But you want to tell people that eventually, right? That's how you justify your service to clients." At the point I'm on the phone with the CTO of $CLIENT I am a contigency recruiter. That means I only get paid if that CTO decides to interview you and ultimately hire you. You can bet that I will be doing my best Enterprise Sales work and showcasing your performance in its most positive truthful light. If I did not feel like that would be possible, that phone call never happens. If the CTO had previously mentioned to be "Be on the lookout, specifically, for people who are good at analyzing mountains of data. We need them for our fraud team.", and I had relevant signal on that question from level X, you can be reasonably assured that if you did really well on it I'd open the call with "The reason I'm recommending Foobar to you is because you told me you wanted folks who are really good at analyzing data. Let me walk you through the solution Foobar came up with -- it's a hoot." (Worth re-iterating explicitly -- that that point, we've talked on the phone and you've explicitly given me the go-ahead on placing you.) To the question: "OK, so I get that people outside the system don't get arbitrary read on my history, and I understand that you're essentially a firewall which is incentivized to only allow the egress of the best possible true packets: what about you? Aren't you going to be influenced by, you know, taking a look directly at my stream of HTTP requests and seeing that I fumble fingered a URL the first time I played the level?" To which I say: I don't care and don't have time for that nonsense -- I'd much rather concentrate on things which actually matter to an engineer.