9 ms·

A detailed exposé on how the market is rigged from a data-centric approach

125 points by isivaxa 12y ago

mschuster91 12y ago
Doesn't surprise me the least bit. Where there is something available to exploit (in this case, access to direct, fast feeds), it will be exploited. Would it be possible, legally and technically, to put a special additional fee/tax onto high-frequency trading while leaving normal high-volume traders alone?
- mrgordon 12y ago
  Its certainly possible technically (and legally in some cases). Italy has a 0.02% tax on transactions lasting less than half a second: http://www.dw.de/italy-first-to-slap-tax-on-high-speed-stock-trading/a-17060424 http://www.dw.de/italy-first-to-slap-tax-on-high-speed-stock...
- noonespecial 12y ago
  Go to the router just entering the exchange, type: tc qdisc change dev eth0 root netem delay 100ms 10ms Problem solved.
  - DevX101 12y ago
    what does this do? (non-finance guy here)
    - jackvalentine 12y ago
      The command introduces a delay of between 100 and 10ms to every packet. Screwing up trades that rely on being the fastest. http://www.linuxfoundation.org/collaborate/workgroups/networking/netem http://www.linuxfoundation.org/collaborate/workgroups/networ...
    - noonespecial 12y ago
      Its a bit of a joke. Its a network thing, not a finance thing. What it does is adds a delay and more importantly, a bit more random delay to the time it takes the order to reach the exchanges server. Once you add in the non-determinism, HFT basically falls apart because you can't take your truckload of cash and buy yourself a place in a datacenter that's 2ms closer to the exchange and front-run everyone. It would be a wonderful thing to see all of those millions these guys have "invested" shaving a millisecond or two off their transaction times laid waste by a single command.
      - dzderic 12y ago
        This "solution" will only make it harder for regular folks to execute orders, since HFTs will beat the randomness by shooting multiple orders through multiple order gateways.
        noonespecial 12y ago
        Guess we'll need a hierarchical token bucket with stochastic fairness queueing as well. We don't just need it to be random. We need there to be no way of ever quite knowing if any given order will beat another order to the exchange (within a given time period, of course). They won't know if they can beat joe ordinary, and they definitely won't know if they can beat the other HFT's. That might be enough to put a lid on it. Edit: For those playing along, here's the metaphor. Joe goes to market to buy sheep. Bill knows Joe is going so he sends a fast runner ahead of him to buy the cheapest sheep in town first so he can mark them up and sell them to Joe when he arrives. We try making everyone wait at the town gate for a random amount of time to give Joe a chance to arrive and get through. So Bill (being very rich) just sends 10 guys so one is very likely to be let in before Joe anyway. Next we introduce the stochastic filter. We make everyone line up and then shuffle the order every once in a while, but Bill still has more guys so he might still get one in first more often than not. Finally, we add the token bucket. For every one guy that we know employed by Bill admitted, we make the next one wait twice as long to get in, so if Joe and 10 Bills show up, Joe and the first Bill are essentially on even footing again because the 2nd through 10th Bill would have to wait too long to matter.
    - simplemath 12y ago
      Pretty sure he's suggesting adding a non-determistic lower bound to trade frequencies at network boundary
- harryh 12y ago
  Not really no. High Frequency Traders are selling a service (liquidity). A tax on them is mostly just a tax on their customers.
axanoeychron 12y ago
You cannot defend frontrunning of a market. If I ask for X at Y. Someone else shouldn't have the facility to buy it based on my own trade signal and try sell it back to me. It is mindblowingly simple theft. The arguments for liquidity do not hold. There is some fascinating cognitive dissonance when it comes to the HFT industry.
- hft_throwaway 12y ago
  That's not what's happening here. Traders are arbitraging and reacting to public trades and orders on multiple markets. If you walk through a physical market where 8 apple carts are lined up, all selling apples for $1, buy every apple at cart #1, then buy every apple at cart #2, and so on, would you be surprised to find the price moving up or sellers stepping away as you approached carts #7 and #8? The same thing happens when trading. Securities trade on multiple markets and multiple exchanges cannot match cross-market trades atomically. It's absurd to suggest that one side of the trade should be expected to close his eyes to what's happening in the world around him and sit tight while a huge trader runs his quote over. Why is one party more deserving of a good price than the other? If you route to one exchange only there is no way for anyone to see or react to your marketable order before it executes, ever. If you route your orders intelligently, it can be very difficult or impossible for anyone to pull away before you get your fills. That's the executing broker's job. Instead of getting better at his job, this broker would rather complain to a very vocal conspiracy theorist who has been proven wrong many times in the past by people with actual experience and data: http://zacharydavid.com/bad-research/the-hunsader-follies/ http://zacharydavid.com/bad-research/the-hunsader-follies/
  - simplemath 12y ago
    Except that what's happening is that the order is against a "cart" with sufficient inventory to completely fulfill the initial order, and other actors are interrupting the transaction to add carts 2-n. Is that not the case?
    - noonespecial 12y ago
DevX101 12y ago
Can someone explain what's happening with the order cancellations? What causes it? Who is the party that is canceling orders?
- Lazare 12y ago
  "Holy shit, someone is working there way through every broker, buying ever share of Ford stock they have! ...huh, I've got some Ford stock for sale. Maybe if I quickly pull it out of the shop window, and change the price, I can make some extra cash!" That's what it is: People are seeing the orders pour through the various exchanges, and are reacting to it. If they were seeing the orders before they hit the exchanges, that would be front running, and it would be illegal. But Nanex appears to be showing people responding to orders after they hit the exchanges, and that would seem to be legal and moral. The moral is that if you want to buy so much of a single stock that you can't even buy it all from a single exchange, you MAY end up paying a bit of a premium, unless you're quite good at hiding what you're doing. And in this example, the purchaser was not. It's a story as old as markets.
  - nhaehnle 12y ago
    The question, from a society-design point of view, is whether it is useful to have a whole class of people who engage in what is ultimately a zero-sum game and therefore an arms race, and whether it wouldn't be better to design markets in such a way that a large buy order can be placed without having to be an expert at HFT. After all, the market is supposed to be useful for organizing long-term investments. The short-term stuff is pretty far removed from the progress of society.
    - zelos 12y ago
      That's what I've always thought as well. We have large numbers of very intelligent people dedicating all their efforts to playing games with the values of real companies. It seems like a massive waste of talent IMHO.
      - 12y ago
th3iedkid 12y ago
what is market-pricing without arbitrage?
waltherg 12y ago
Just in case anyone has time to help me out: What's the difference between trades and quotes here in this chart? And how are the trader's order and purchases indicated?
- sklivvz1971 12y ago
  A quote is a statement that someone is offering to buy or sell a specific quantity of shares/commodities at a specific price: "I want to sell 10g of gold at 40$ each". Typically the identity of the trader is known only to the exchange (and the trader ) A trade is an announcement that an offer was accepted and a contract was agreed on. "10g of gold have been sold at 40$ each". The identity of the traders is typically known only by the exchange, and each of the trader knows they are part of it, but don't know the counterparty.
deleted 12y ago
[deleted]
data_scientist 12y ago
This is the bad part of HFT, the one that is theft and destroy value. It is paid by the big players, and "solved" by imperfect solution like dark pools. A better solution could be to add a hour component to the order, so all commands from an actor in all markets are executed at the exact same time. This rotted apple should not hide the good part of HFT, which is to reduce spread and inconsistencies between markets and to generate profits from this (positive) action. HFT took the place of traders, who were paid a lot for doing that stupid task.
josephlord 12y ago
If you offer something for sale at a certain price and someone says "I'll buy it!" you have a contract at that moment. I don't fully understand the conditions under which you can cancel an order but it seems all the cancellations happened on exchanges where no orders had yet been fulfilled so I assume this means that the order had not yet arrived. This seems ethically just about OK to me but a sign that there is not one single stockmarket and that the system could be far better designed. There is the single front-running trade which is suspicious but it seems plausible (unless it happens every time) that it was just a small random trade that happened to coincide with the timing of the big trade. It should be monitored though. My conclusions: 1. There is not one single market with a number of available shares but a number of linked markets. Send your trade to a single exchange (first at least) with enough offered shares that it should execute before offers can be cancelled. Wait, repeat. 2. Much of the liquidity supposedly offered by HFT is illusory and disappears if you try to use it. I think that the market could probably be improved if cancellation weren't free or at least weren't instant. If cancellations took a second (maybe 100ms or 10ms would be enough) to process and the offers could still be accepted in that period the offers made would be more serious and although the spread might be slightly larger it would more honestly reflect reality.
- bmelton 12y ago
  Plucking from throwaway's example. You have 20,000 copies of a book you just wrote. You put half of them on Amazon, and the other half on eBay, so Amazon has 10,000 and ebay has 10,000 of them. You see an order come in for 5,000 of them on Amazon. You think "Hot dog, these books are popular. I must be selling them too cheaply!" You immediately raise the price of all the books by 25 cents to capitalize on this. The books you sold on Amazon are sold, so they're gone. The remaining books on Amazon are slightly more expensive. The guy who bought the books on Amazon also bought the same number of books on eBay, but the order hadn't arrived there yet, so between when he hit the buy button and the time the order arrived, the price had changed, so those orders aren't filled.
  - stevejones 12y ago
    The difference would be that you cancel all the orders currently in progress, so a bunch of people who've clicked "buy" are now in the lurch.
erpellan 12y ago
A solution: Discrete double auctions. Instead of continuous trading, the exchange can divide up the day into a series of small windows (say 100ms). When you come to trade in the market, you have to wait for the next window to open. You submit your order and you find out what happened at the end of the window. This way nobody has any timing advantage and the delay is barely noticeable to 'normal' traders (waiting 1/5 of a second is hardly an inconvenience). It also stops all the order-book shenanigans that HFT players get up to (where they stuff the book with orders and cancel/resubmit them at high frequency). So why don't exchanges do this? They make a ton of money in fees, it simply isn't in their interest to prevent HFT at the moment. Change their incentives (ie. regulate differently) and they might actually do something about it.
- kasey_junk 12y ago
  Actually there are lots of discrete auctions in the electronic trading world. For instance the S&P futures contracts trade this way before the open and depending on your perspective they have more "shenanigans" being played by HFT players. Not less. There are 2 major issues that no one brings up when they say "simply add discrete auctions". A) what happens when there are more participants on 1 side of a price than on the other, what is the tie breaker after price? B) How does this solve the distributed systems problem of multiple exchanges trading at the same time?
throwaway283719 12y ago
What is happening here is really quite simple, and doesn't deserve an entire blog post. There are two exchanges, A and B, and a market maker Jill is quoting (say) 10,000 shares on each of those two exchanges for $17. Big institutional trader Jack sees the 20,000 shares and decides that he wants to buy 15,000 of them, so he sends two orders for 7,500 shares each to A and B. Because of various effects (network latencies, routing switching delays, whatever) his order arrives at exchange A first, and is immediately filled at $17. Jill, who has her computer co-located at exchange A, sees that she has sold 7,500 shares for $17, and realizes that there is demand for shares. Because of this demand, she decides to raise her prices. She immediately cancels her remaining 2500 shares on exchange A and replaces them with 10,000 shares at $17.05 and sends an instruction to do the same thing at exchange B. Because Jill has fast computers and low-latency connections, her cancellation arrives at exchange B before Jack's buy order, so Jack is told that there are no longer shares available on exchange B at $17. RESULT: Jack is filled for 7500 shares at $17 (half of what he requested) and the new market best offer is $17.05. Jack is welcome to submit another order for $17.05 if he wants to buy at that price. Jill is now short 7500 shares at $17, and will try to buy them back at a lower price (she may or may not succeed - until she does, she is exposed to the risk of further price rises). Jill was able to use her speed advantage to detect that there was additional demand to buy this stock, and raise the price at which she was willing to sell it before Jack had finished buying all that he wanted to. This is exactly the way that an efficient market is supposed to work - it reacts to fluctuating demand (and other information) to set appropriate prices. I think there are several things that get glossed over while people are working themselves up about this - 1. Jack is upset because he couldn't buy 15,000 shares at the price he wanted to buy them. But Jack has no god-given right to be able to buy shares at the price he likes best. He is subject to the laws of the market, just like everyone else. 2. The only reason that Jill has a speed advantage over Jack is because she has paid for it! She has paid to co-locate her server at the exchange, and she has paid to use high-speed connections between exchanges. Are we going to declare that paying for a competitive advantage is suddenly immoral? 3. If Jack doesn't like this state of affairs, he has several options. He can invest in high-speed infrastructure as well. He can use smarter order-routing logic (e.g. adding delays to his orders so that they arrive at the exchanges approximately simultaneously, or splitting his large order up into multiple smaller orders). Or he can use a broker who will do these things for him. If Jack doesn't want to pay for any of these things, then he has to put up with lower quality execution. As much as he might wish it, the ability to buy as many shares as he wants at the price he wants them is not a universal human right.
cbr 12y ago
Say there are three exchanges, A, B, C, each with 1k shares of Ford on offer at $20. They are all random numbers of ms away from me, and for simplicity say A is closest and C is farthest. I send out my order for 3k shares at $20, and it hits A then B then C. People who are watching A see my request, and try to make adversarial changes on B and C. They have a low chance of success on B because it's almost as close to me as A is, but they have a higher chance on C because it's pretty far from me. One way to fix this is to delay your orders carefully so that A, B, and C will all get your order at almost the same time. Now there's not time for someone who sees your order on A to react and send a message to C that will beat your message to C. I believe this is what IEX does: http://en.wikipedia.org/wiki/IEX http://en.wikipedia.org/wiki/IEX
- harryh 12y ago
  That's not what IEX does. IEX is, to fit within your example merely exchange A. It can't control whether you delay your order to B or C or not. What A does do is delay the output. When it receives an order it doesn't immediately broadcast that information back out, it waits some small (but relevant) period of time.
willvarfar 12y ago
All exchanges should have synced clocks and all messages should have a timestamp up to 2 seconds in the future when they will be published by each exchange. The buffering would be internal to each exchange and not shared with anybody. You can only cancel after what you are cancelling is published. This would allow everyone to make all exchanges publish at once so people with fast cable between exchanges can't beat out those that don't.
- ripb 12y ago
  >This would allow everyone to make all exchanges publish at once so people with fast cable between exchanges can't beat out those that don't. Why is one actor paying for an advantage that is available to anyone who should desire it, and have the means to pay for it, an issue?
- harryh 12y ago
  What do you think would happen to bid-ask spreads if this was imposed?
throwaway161803 12y ago
This article establishes that two things often happen shortly after you place an order to buy shares: 1) Another trader places a similar order on another exchange. 2) A large number of outstanding sell orders are cancelled. It's not clear to me that either of these are Bad Things, deontologically speaking. [I don't recall Jesus mentioning them.] The key question is consequentialist: Are there regulatory changes which would improve the lot of the average investor, investing through, say, an index tracker or pension fund? For each potential change, one ought see how it fares w.r.t. this standard, considering, to the extent that it is possible, the induced second-order effects. Talk of theft, rigging, fairness (you don't owe people like me anything), stolen goods and frontrunning is only useful to the extent that it helps us converge on an answer to this question. These words are tools that we have developed for analysing more familiar situations, where they correspond to actions which are clearly harmful. Most changes proposed here either lose market efficiency directly (trade buffering / increased tick sizes) or just give us new games to play (if the market clears once a second, we will get our orders in last), potentially resulting in a less direct loss. The question remains.
bakhy 12y ago
An interesting contradiction appears here. On one hand, this increases market efficiency, or so we're told. On the other, we are also told that if a big pension fund wants to avoid being played like this, they should spend money on their own HFT equipment. It seems there is only one clear winner here - the IT people making money off developing HFT systems.
- harryh 12y ago
  Here's the chief executive of Vanguard (one of the largest investment management companies in the world) discussing how HFT has dramatically lowered their trading costs: http://www.ft.com/intl/cms/s/0/ff8c6486-cb37-11e3-ba95-00144feabdc0.html#axzz37dq3BD3q http://www.ft.com/intl/cms/s/0/ff8c6486-cb37-11e3-ba95-00144... Spreads used to be a quarter, and now they're a penny! That's a huge deal!
thingylab 12y ago
I don't see how this proves the market is rigged. What I do see is: 1) One market participant is being less than clever by trying to buy, in one order, 80% of the offered quantity, and 2) Another market participant realizes this, and reacts accordingly. The post is written as if the world should freeze once the client sends an order. He was 'stolen' shares. Really?
vampirechicken 12y ago
As a non-trader, my question is: Were the 24k shares being offered by one seller/broker, as in "I have 24k shares to sell at 17" or was the 24k just an aggregation of the availability all the smaller offers? If the former, it seem to me that the seller is cheating, if it is the latter then I can see how the HFT systems would raise the price in response to a sale, but I also see how frustrating that is to the buyer. I wonder why these trades are not being performed in parallel across the various exchanges, partially preventing this kind of arbitrage?
- growse 12y ago
  > Were the 24k shares being offered by one seller/broker, as in "I have 24k shares to sell at 17" or was the 24k just an aggregation of the availability all the smaller offers? The shares were being quoted on different exchanges, at the same ask price. 24k was the cumulative volume that the buyer wanted, but that couldn't be fulfilled by a single exchange (the quote was for a smaller volume at that price). Therefore, to buy 24k shares, the buyer needs to trade twice, once at each exchange. > I wonder why these trades are not being performed in parallel across the various exchanges, partially preventing this kind of arbitrage? As has been pointed out, this is what a good broker will do - they will compensate for latency to make sure that bids arrive at differing venues at the same time to prevent the market shifting underneath them. A naïve broker will simply send out the bids at the same, and latency means that they arrive at different exchanges at different times. This lets the sellers at the more distant exchange move the market in response to the information of the trade being executed at the closer exchange.

125 points by isivaxa 12y ago

Doesn't surprise me the least bit. Where there is something available to exploit (in this case, access to direct, fast feeds), it will be exploited. Would it be possible, legally and technically, to put a special additional fee/tax onto high-frequency trading while leaving normal high-volume traders alone?

mrgordon 12y ago
Its certainly possible technically (and legally in some cases). Italy has a 0.02% tax on transactions lasting less than half a second: http://www.dw.de/italy-first-to-slap-tax-on-high-speed-stock-trading/a-17060424 http://www.dw.de/italy-first-to-slap-tax-on-high-speed-stock...
noonespecial 12y ago
Go to the router just entering the exchange, type: tc qdisc change dev eth0 root netem delay 100ms 10ms Problem solved.
- DevX101 12y ago
  what does this do? (non-finance guy here)
  - jackvalentine 12y ago
    The command introduces a delay of between 100 and 10ms to every packet. Screwing up trades that rely on being the fastest. http://www.linuxfoundation.org/collaborate/workgroups/networking/netem http://www.linuxfoundation.org/collaborate/workgroups/networ...
  - noonespecial 12y ago
    Its a bit of a joke. Its a network thing, not a finance thing. What it does is adds a delay and more importantly, a bit more random delay to the time it takes the order to reach the exchanges server. Once you add in the non-determinism, HFT basically falls apart because you can't take your truckload of cash and buy yourself a place in a datacenter that's 2ms closer to the exchange and front-run everyone. It would be a wonderful thing to see all of those millions these guys have "invested" shaving a millisecond or two off their transaction times laid waste by a single command.
    - dzderic 12y ago
      This "solution" will only make it harder for regular folks to execute orders, since HFTs will beat the randomness by shooting multiple orders through multiple order gateways.
      - noonespecial 12y ago
        Guess we'll need a hierarchical token bucket with stochastic fairness queueing as well. We don't just need it to be random. We need there to be no way of ever quite knowing if any given order will beat another order to the exchange (within a given time period, of course). They won't know if they can beat joe ordinary, and they definitely won't know if they can beat the other HFT's. That might be enough to put a lid on it. Edit: For those playing along, here's the metaphor. Joe goes to market to buy sheep. Bill knows Joe is going so he sends a fast runner ahead of him to buy the cheapest sheep in town first so he can mark them up and sell them to Joe when he arrives. We try making everyone wait at the town gate for a random amount of time to give Joe a chance to arrive and get through. So Bill (being very rich) just sends 10 guys so one is very likely to be let in before Joe anyway. Next we introduce the stochastic filter. We make everyone line up and then shuffle the order every once in a while, but Bill still has more guys so he might still get one in first more often than not. Finally, we add the token bucket. For every one guy that we know employed by Bill admitted, we make the next one wait twice as long to get in, so if Joe and 10 Bills show up, Joe and the first Bill are essentially on even footing again because the 2nd through 10th Bill would have to wait too long to matter.
  - simplemath 12y ago
    Pretty sure he's suggesting adding a non-determistic lower bound to trade frequencies at network boundary
harryh 12y ago
Not really no. High Frequency Traders are selling a service (liquidity). A tax on them is mostly just a tax on their customers.

axanoeychron 12y ago

You cannot defend frontrunning of a market. If I ask for X at Y. Someone else shouldn't have the facility to buy it based on my own trade signal and try sell it back to me. It is mindblowingly simple theft. The arguments for liquidity do not hold. There is some fascinating cognitive dissonance when it comes to the HFT industry.

hft_throwaway 12y ago
That's not what's happening here. Traders are arbitraging and reacting to public trades and orders on multiple markets. If you walk through a physical market where 8 apple carts are lined up, all selling apples for $1, buy every apple at cart #1, then buy every apple at cart #2, and so on, would you be surprised to find the price moving up or sellers stepping away as you approached carts #7 and #8? The same thing happens when trading. Securities trade on multiple markets and multiple exchanges cannot match cross-market trades atomically. It's absurd to suggest that one side of the trade should be expected to close his eyes to what's happening in the world around him and sit tight while a huge trader runs his quote over. Why is one party more deserving of a good price than the other? If you route to one exchange only there is no way for anyone to see or react to your marketable order before it executes, ever. If you route your orders intelligently, it can be very difficult or impossible for anyone to pull away before you get your fills. That's the executing broker's job. Instead of getting better at his job, this broker would rather complain to a very vocal conspiracy theorist who has been proven wrong many times in the past by people with actual experience and data: http://zacharydavid.com/bad-research/the-hunsader-follies/ http://zacharydavid.com/bad-research/the-hunsader-follies/
- simplemath 12y ago
  Except that what's happening is that the order is against a "cart" with sufficient inventory to completely fulfill the initial order, and other actors are interrupting the transaction to add carts 2-n. Is that not the case?
  - noonespecial 12y ago

DevX101 12y ago

Can someone explain what's happening with the order cancellations? What causes it? Who is the party that is canceling orders?

Lazare 12y ago
"Holy shit, someone is working there way through every broker, buying ever share of Ford stock they have! ...huh, I've got some Ford stock for sale. Maybe if I quickly pull it out of the shop window, and change the price, I can make some extra cash!" That's what it is: People are seeing the orders pour through the various exchanges, and are reacting to it. If they were seeing the orders before they hit the exchanges, that would be front running, and it would be illegal. But Nanex appears to be showing people responding to orders after they hit the exchanges, and that would seem to be legal and moral. The moral is that if you want to buy so much of a single stock that you can't even buy it all from a single exchange, you MAY end up paying a bit of a premium, unless you're quite good at hiding what you're doing. And in this example, the purchaser was not. It's a story as old as markets.
- nhaehnle 12y ago
  The question, from a society-design point of view, is whether it is useful to have a whole class of people who engage in what is ultimately a zero-sum game and therefore an arms race, and whether it wouldn't be better to design markets in such a way that a large buy order can be placed without having to be an expert at HFT. After all, the market is supposed to be useful for organizing long-term investments. The short-term stuff is pretty far removed from the progress of society.
  - zelos 12y ago
    That's what I've always thought as well. We have large numbers of very intelligent people dedicating all their efforts to playing games with the values of real companies. It seems like a massive waste of talent IMHO.
    - 12y ago

th3iedkid 12y ago

what is market-pricing without arbitrage?

waltherg 12y ago

Just in case anyone has time to help me out: What's the difference between trades and quotes here in this chart? And how are the trader's order and purchases indicated?

sklivvz1971 12y ago
A quote is a statement that someone is offering to buy or sell a specific quantity of shares/commodities at a specific price: "I want to sell 10g of gold at 40$ each". Typically the identity of the trader is known only to the exchange (and the trader ) A trade is an announcement that an offer was accepted and a contract was agreed on. "10g of gold have been sold at 40$ each". The identity of the traders is typically known only by the exchange, and each of the trader knows they are part of it, but don't know the counterparty.

deleted 12y ago

[deleted]

data_scientist 12y ago

This is the bad part of HFT, the one that is theft and destroy value. It is paid by the big players, and "solved" by imperfect solution like dark pools. A better solution could be to add a hour component to the order, so all commands from an actor in all markets are executed at the exact same time. This rotted apple should not hide the good part of HFT, which is to reduce spread and inconsistencies between markets and to generate profits from this (positive) action. HFT took the place of traders, who were paid a lot for doing that stupid task.

josephlord 12y ago

If you offer something for sale at a certain price and someone says "I'll buy it!" you have a contract at that moment. I don't fully understand the conditions under which you can cancel an order but it seems all the cancellations happened on exchanges where no orders had yet been fulfilled so I assume this means that the order had not yet arrived. This seems ethically just about OK to me but a sign that there is not one single stockmarket and that the system could be far better designed. There is the single front-running trade which is suspicious but it seems plausible (unless it happens every time) that it was just a small random trade that happened to coincide with the timing of the big trade. It should be monitored though. My conclusions: 1. There is not one single market with a number of available shares but a number of linked markets. Send your trade to a single exchange (first at least) with enough offered shares that it should execute before offers can be cancelled. Wait, repeat. 2. Much of the liquidity supposedly offered by HFT is illusory and disappears if you try to use it. I think that the market could probably be improved if cancellation weren't free or at least weren't instant. If cancellations took a second (maybe 100ms or 10ms would be enough) to process and the offers could still be accepted in that period the offers made would be more serious and although the spread might be slightly larger it would more honestly reflect reality.

bmelton 12y ago
Plucking from throwaway's example. You have 20,000 copies of a book you just wrote. You put half of them on Amazon, and the other half on eBay, so Amazon has 10,000 and ebay has 10,000 of them. You see an order come in for 5,000 of them on Amazon. You think "Hot dog, these books are popular. I must be selling them too cheaply!" You immediately raise the price of all the books by 25 cents to capitalize on this. The books you sold on Amazon are sold, so they're gone. The remaining books on Amazon are slightly more expensive. The guy who bought the books on Amazon also bought the same number of books on eBay, but the order hadn't arrived there yet, so between when he hit the buy button and the time the order arrived, the price had changed, so those orders aren't filled.
- stevejones 12y ago
  The difference would be that you cancel all the orders currently in progress, so a bunch of people who've clicked "buy" are now in the lurch.

erpellan 12y ago

A solution: Discrete double auctions. Instead of continuous trading, the exchange can divide up the day into a series of small windows (say 100ms). When you come to trade in the market, you have to wait for the next window to open. You submit your order and you find out what happened at the end of the window. This way nobody has any timing advantage and the delay is barely noticeable to 'normal' traders (waiting 1/5 of a second is hardly an inconvenience). It also stops all the order-book shenanigans that HFT players get up to (where they stuff the book with orders and cancel/resubmit them at high frequency). So why don't exchanges do this? They make a ton of money in fees, it simply isn't in their interest to prevent HFT at the moment. Change their incentives (ie. regulate differently) and they might actually do something about it.

kasey_junk 12y ago
Actually there are lots of discrete auctions in the electronic trading world. For instance the S&P futures contracts trade this way before the open and depending on your perspective they have more "shenanigans" being played by HFT players. Not less. There are 2 major issues that no one brings up when they say "simply add discrete auctions". A) what happens when there are more participants on 1 side of a price than on the other, what is the tie breaker after price? B) How does this solve the distributed systems problem of multiple exchanges trading at the same time?

throwaway283719 12y ago

What is happening here is really quite simple, and doesn't deserve an entire blog post. There are two exchanges, A and B, and a market maker Jill is quoting (say) 10,000 shares on each of those two exchanges for $17. Big institutional trader Jack sees the 20,000 shares and decides that he wants to buy 15,000 of them, so he sends two orders for 7,500 shares each to A and B. Because of various effects (network latencies, routing switching delays, whatever) his order arrives at exchange A first, and is immediately filled at $17. Jill, who has her computer co-located at exchange A, sees that she has sold 7,500 shares for $17, and realizes that there is demand for shares. Because of this demand, she decides to raise her prices. She immediately cancels her remaining 2500 shares on exchange A and replaces them with 10,000 shares at $17.05 and sends an instruction to do the same thing at exchange B. Because Jill has fast computers and low-latency connections, her cancellation arrives at exchange B before Jack's buy order, so Jack is told that there are no longer shares available on exchange B at $17. RESULT: Jack is filled for 7500 shares at $17 (half of what he requested) and the new market best offer is $17.05. Jack is welcome to submit another order for $17.05 if he wants to buy at that price. Jill is now short 7500 shares at $17, and will try to buy them back at a lower price (she may or may not succeed - until she does, she is exposed to the risk of further price rises). Jill was able to use her speed advantage to detect that there was additional demand to buy this stock, and raise the price at which she was willing to sell it before Jack had finished buying all that he wanted to. This is exactly the way that an efficient market is supposed to work - it reacts to fluctuating demand (and other information) to set appropriate prices. I think there are several things that get glossed over while people are working themselves up about this - 1. Jack is upset because he couldn't buy 15,000 shares at the price he wanted to buy them. But Jack has no god-given right to be able to buy shares at the price he likes best. He is subject to the laws of the market, just like everyone else. 2. The only reason that Jill has a speed advantage over Jack is because she has paid for it! She has paid to co-locate her server at the exchange, and she has paid to use high-speed connections between exchanges. Are we going to declare that paying for a competitive advantage is suddenly immoral? 3. If Jack doesn't like this state of affairs, he has several options. He can invest in high-speed infrastructure as well. He can use smarter order-routing logic (e.g. adding delays to his orders so that they arrive at the exchanges approximately simultaneously, or splitting his large order up into multiple smaller orders). Or he can use a broker who will do these things for him. If Jack doesn't want to pay for any of these things, then he has to put up with lower quality execution. As much as he might wish it, the ability to buy as many shares as he wants at the price he wants them is not a universal human right.

cbr 12y ago

Say there are three exchanges, A, B, C, each with 1k shares of Ford on offer at $20. They are all random numbers of ms away from me, and for simplicity say A is closest and C is farthest. I send out my order for 3k shares at $20, and it hits A then B then C. People who are watching A see my request, and try to make adversarial changes on B and C. They have a low chance of success on B because it's almost as close to me as A is, but they have a higher chance on C because it's pretty far from me. One way to fix this is to delay your orders carefully so that A, B, and C will all get your order at almost the same time. Now there's not time for someone who sees your order on A to react and send a message to C that will beat your message to C. I believe this is what IEX does: http://en.wikipedia.org/wiki/IEX http://en.wikipedia.org/wiki/IEX

harryh 12y ago
That's not what IEX does. IEX is, to fit within your example merely exchange A. It can't control whether you delay your order to B or C or not. What A does do is delay the output. When it receives an order it doesn't immediately broadcast that information back out, it waits some small (but relevant) period of time.

willvarfar 12y ago

All exchanges should have synced clocks and all messages should have a timestamp up to 2 seconds in the future when they will be published by each exchange. The buffering would be internal to each exchange and not shared with anybody. You can only cancel after what you are cancelling is published. This would allow everyone to make all exchanges publish at once so people with fast cable between exchanges can't beat out those that don't.

ripb 12y ago
>This would allow everyone to make all exchanges publish at once so people with fast cable between exchanges can't beat out those that don't. Why is one actor paying for an advantage that is available to anyone who should desire it, and have the means to pay for it, an issue?
harryh 12y ago
What do you think would happen to bid-ask spreads if this was imposed?

throwaway161803 12y ago

This article establishes that two things often happen shortly after you place an order to buy shares: 1) Another trader places a similar order on another exchange. 2) A large number of outstanding sell orders are cancelled. It's not clear to me that either of these are Bad Things, deontologically speaking. [I don't recall Jesus mentioning them.] The key question is consequentialist: Are there regulatory changes which would improve the lot of the average investor, investing through, say, an index tracker or pension fund? For each potential change, one ought see how it fares w.r.t. this standard, considering, to the extent that it is possible, the induced second-order effects. Talk of theft, rigging, fairness (you don't owe people like me anything), stolen goods and frontrunning is only useful to the extent that it helps us converge on an answer to this question. These words are tools that we have developed for analysing more familiar situations, where they correspond to actions which are clearly harmful. Most changes proposed here either lose market efficiency directly (trade buffering / increased tick sizes) or just give us new games to play (if the market clears once a second, we will get our orders in last), potentially resulting in a less direct loss. The question remains.

bakhy 12y ago

An interesting contradiction appears here. On one hand, this increases market efficiency, or so we're told. On the other, we are also told that if a big pension fund wants to avoid being played like this, they should spend money on their own HFT equipment. It seems there is only one clear winner here - the IT people making money off developing HFT systems.

harryh 12y ago
Here's the chief executive of Vanguard (one of the largest investment management companies in the world) discussing how HFT has dramatically lowered their trading costs: http://www.ft.com/intl/cms/s/0/ff8c6486-cb37-11e3-ba95-00144feabdc0.html#axzz37dq3BD3q http://www.ft.com/intl/cms/s/0/ff8c6486-cb37-11e3-ba95-00144... Spreads used to be a quarter, and now they're a penny! That's a huge deal!

thingylab 12y ago

I don't see how this proves the market is rigged. What I do see is: 1) One market participant is being less than clever by trying to buy, in one order, 80% of the offered quantity, and 2) Another market participant realizes this, and reacts accordingly. The post is written as if the world should freeze once the client sends an order. He was 'stolen' shares. Really?

vampirechicken 12y ago

As a non-trader, my question is: Were the 24k shares being offered by one seller/broker, as in "I have 24k shares to sell at 17" or was the 24k just an aggregation of the availability all the smaller offers? If the former, it seem to me that the seller is cheating, if it is the latter then I can see how the HFT systems would raise the price in response to a sale, but I also see how frustrating that is to the buyer. I wonder why these trades are not being performed in parallel across the various exchanges, partially preventing this kind of arbitrage?

growse 12y ago
> Were the 24k shares being offered by one seller/broker, as in "I have 24k shares to sell at 17" or was the 24k just an aggregation of the availability all the smaller offers? The shares were being quoted on different exchanges, at the same ask price. 24k was the cumulative volume that the buyer wanted, but that couldn't be fulfilled by a single exchange (the quote was for a smaller volume at that price). Therefore, to buy 24k shares, the buyer needs to trade twice, once at each exchange. > I wonder why these trades are not being performed in parallel across the various exchanges, partially preventing this kind of arbitrage? As has been pointed out, this is what a good broker will do - they will compensate for latency to make sure that bids arrive at differing venues at the same time to prevent the market shifting underneath them. A naïve broker will simply send out the bids at the same, and latency means that they arrive at different exchanges at different times. This lets the sellers at the more distant exchange move the market in response to the information of the trade being executed at the closer exchange.