6 ms·

OFA + AWS

69 points by dylanvee 13y ago

vosper 13y ago
This is really great - I'd love to have a similar diagram for data flow and components of the systems I work on. It might not benefit engineers very much but it'd be fantastic for clients or some of the less technical people in the company. I wonder how hard (I'm thinking hard?) it would be to have a tool for something like this, making the layout easy and revealing more and more information as you zoom in. Even better if it was live and updating with system metrics.
- chubot 13y ago
  Yeah it looks kind of cool, but kinda hard to read. Is this graphic based on something else? I would like to see one where the labels are clearer. It doesn't seem like the 3D perspective adds any clarity.
- danoprey 13y ago
  You should check out http://www.madeiracloud.com/ http://www.madeiracloud.com/ , we're working on something very similar to what you described.
mappu 13y ago
That's an awesome diagram and quite a few servers (but latency kills the map-overlay experience, i think a static png or svg might have been better...) Interesting that servers are are either paired autoscaling over two AZs, or statically provisioned in three AZs - or am i mis-reading the diagram? Two entire mirrors for testing and staging must have become a pretty big cost. For others outside the US confused about what the application actually does, and what the Narwhal in the testing/staging pictures refers to, this helped; http://arstechnica.com/information-technology/2012/11/built-to-win-deep-inside-obamas-campaign-tech/ http://arstechnica.com/information-technology/2012/11/built-...
- DrJ 13y ago
  Looks like the AutoScaling groups are applications or stores where they do not need to coordinate their actions. The 3-az deployments seems to be APIs, which I am guessing scales with each regions (to reduce data cost?) and probably brought up/down automatically with Puppet to handle post-launch configurations. (you are reading it correctly). I would guess testing is a skeleton version of the entire deployment so the cost is minimal and just need to test new deploys and verification for tests. Staging probably wasn't a full mirror, at best I would venture to guess they had hot swaps coming up in staging and then being switched against production via ELBs. They mention costs a few times in articles, so I would venture to guess they did optimize around many of those corners.
siliconc0w 13y ago
Anyone know of a good tool to make these types of diagrams? Preferably automatically? I've been meaning to create something that plugs into Chef and or nmap/netstat. In my experience, any diagram/inventory/whatever that isn't automatically updated invariably drifts from reality.
- danoprey 13y ago
  We're attempting just that at http://www.madeiracloud.com/ http://www.madeiracloud.com/ , would love to chat more: dan [at] mc2 [dot] io
jared314 13y ago
The diagram follows the style of the AWS Reference Architectures[0]. I have yet to find a published set of diagram components that compare well to that style. Edit: Juan Domenech[1] has a small custom made component pallet in png form. And, it looks like the original diagrams are custom Adobe Illustrator drawings[2]. [0] http://aws.amazon.com/architecture/ http://aws.amazon.com/architecture/ [1] http://blog.domenech.org/2012/06/aws-diagrams-palette-v10.html http://blog.domenech.org/2012/06/aws-diagrams-palette-v10.ht... [2] http://blog.domenech.org/2012/05/aws-diagrams.html http://blog.domenech.org/2012/05/aws-diagrams.html
harper 13y ago
i guess this leaked. ;)
- jlgaddis 13y ago
  Oops, wonder how that happened? ;)
- dylanvee 13y ago
  Saw it on Twitter and had to submit, y'all are pretty cool.
- harper 13y ago
  Turns out it was released. We had been working very hard to get this diagram out. Last I was involved, it was stuck with lawyers, etc. OFA is awesome for releasing this. Hopefully next will be some of our codes and projects.
- sneak 13y ago
  Please watch this: http://www.youtube.com/watch?v=QNsePZj_Yks http://www.youtube.com/watch?v=QNsePZj_Yks You are materially responsible for the re-election of someone who commits large-scale crime (in the form of blatantly unconstitutional surveillance of Americans under no suspicion of criminal activity). Please reconsider your choices. You are working for the wrong side.
  - SkyMarshal 13y ago
    >You are working for the wrong side. Which side should he be working for that doesn't do any of that?
    - sneak 13y ago
      The side building tools to resist surveillance, tyranny, violence, and war. You know, the one trying to prevent all the stuff his boss is up to.
obilgic 13y ago
http://aws.amazon.com/solutions/case-studies/obama/ http://aws.amazon.com/solutions/case-studies/obama/
doctorpangloss 13y ago
When folks from the Romney campaign complained of technical deficiencies in his get-out-the-vote operation, pundits and news outlets intimated that the complaints were scapegoating IT for bad policy. But looking at this diagram, I'm pleased to see that engineering was a big part of the Obama campaign. It really hammers home how badly out of their league Romney's TechOps team was.
twic 13y ago
Is each one of those little grey cuboids an virtual machine? If so, that is rather a lot of virtual machines. I am surprised so many were necessary. What kind of load was this system handling?

69 points by dylanvee 13y ago

This is really great - I'd love to have a similar diagram for data flow and components of the systems I work on. It might not benefit engineers very much but it'd be fantastic for clients or some of the less technical people in the company. I wonder how hard (I'm thinking hard?) it would be to have a tool for something like this, making the layout easy and revealing more and more information as you zoom in. Even better if it was live and updating with system metrics.

chubot 13y ago
Yeah it looks kind of cool, but kinda hard to read. Is this graphic based on something else? I would like to see one where the labels are clearer. It doesn't seem like the 3D perspective adds any clarity.
danoprey 13y ago
You should check out http://www.madeiracloud.com/ http://www.madeiracloud.com/ , we're working on something very similar to what you described.

mappu 13y ago

That's an awesome diagram and quite a few servers (but latency kills the map-overlay experience, i think a static png or svg might have been better...) Interesting that servers are are either paired autoscaling over two AZs, or statically provisioned in three AZs - or am i mis-reading the diagram? Two entire mirrors for testing and staging must have become a pretty big cost. For others outside the US confused about what the application actually does, and what the Narwhal in the testing/staging pictures refers to, this helped; http://arstechnica.com/information-technology/2012/11/built-to-win-deep-inside-obamas-campaign-tech/ http://arstechnica.com/information-technology/2012/11/built-...

DrJ 13y ago
Looks like the AutoScaling groups are applications or stores where they do not need to coordinate their actions. The 3-az deployments seems to be APIs, which I am guessing scales with each regions (to reduce data cost?) and probably brought up/down automatically with Puppet to handle post-launch configurations. (you are reading it correctly). I would guess testing is a skeleton version of the entire deployment so the cost is minimal and just need to test new deploys and verification for tests. Staging probably wasn't a full mirror, at best I would venture to guess they had hot swaps coming up in staging and then being switched against production via ELBs. They mention costs a few times in articles, so I would venture to guess they did optimize around many of those corners.

siliconc0w 13y ago

Anyone know of a good tool to make these types of diagrams? Preferably automatically? I've been meaning to create something that plugs into Chef and or nmap/netstat. In my experience, any diagram/inventory/whatever that isn't automatically updated invariably drifts from reality.

danoprey 13y ago
We're attempting just that at http://www.madeiracloud.com/ http://www.madeiracloud.com/ , would love to chat more: dan [at] mc2 [dot] io

jared314 13y ago

The diagram follows the style of the AWS Reference Architectures[0]. I have yet to find a published set of diagram components that compare well to that style. Edit: Juan Domenech[1] has a small custom made component pallet in png form. And, it looks like the original diagrams are custom Adobe Illustrator drawings[2]. [0] http://aws.amazon.com/architecture/ http://aws.amazon.com/architecture/ [1] http://blog.domenech.org/2012/06/aws-diagrams-palette-v10.html http://blog.domenech.org/2012/06/aws-diagrams-palette-v10.ht... [2] http://blog.domenech.org/2012/05/aws-diagrams.html http://blog.domenech.org/2012/05/aws-diagrams.html

harper 13y ago

i guess this leaked. ;)

jlgaddis 13y ago
Oops, wonder how that happened? ;)
dylanvee 13y ago
Saw it on Twitter and had to submit, y'all are pretty cool.
harper 13y ago
Turns out it was released. We had been working very hard to get this diagram out. Last I was involved, it was stuck with lawyers, etc. OFA is awesome for releasing this. Hopefully next will be some of our codes and projects.
sneak 13y ago
Please watch this: http://www.youtube.com/watch?v=QNsePZj_Yks http://www.youtube.com/watch?v=QNsePZj_Yks You are materially responsible for the re-election of someone who commits large-scale crime (in the form of blatantly unconstitutional surveillance of Americans under no suspicion of criminal activity). Please reconsider your choices. You are working for the wrong side.
- SkyMarshal 13y ago
  >You are working for the wrong side. Which side should he be working for that doesn't do any of that?
  - sneak 13y ago
    The side building tools to resist surveillance, tyranny, violence, and war. You know, the one trying to prevent all the stuff his boss is up to.

obilgic 13y ago

http://aws.amazon.com/solutions/case-studies/obama/ http://aws.amazon.com/solutions/case-studies/obama/

doctorpangloss 13y ago

When folks from the Romney campaign complained of technical deficiencies in his get-out-the-vote operation, pundits and news outlets intimated that the complaints were scapegoating IT for bad policy. But looking at this diagram, I'm pleased to see that engineering was a big part of the Obama campaign. It really hammers home how badly out of their league Romney's TechOps team was.

twic 13y ago

Is each one of those little grey cuboids an virtual machine? If so, that is rather a lot of virtual machines. I am surprised so many were necessary. What kind of load was this system handling?