5 ms·

Distributed Performance Testing in the Cloud with JMeter and AWS

45 points by mfolaron 9y ago

justizin 9y ago
> Here is the problem, JMeter uses Java RMI (Remote Method Invocation) to communicate to its slaves, but these connections require all machines to be on the same subnet and this is not feasible with EC2 instances. Incorrect with VPC.. Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes?
- mfolaron 9y ago
  >Incorrect with VPC.. VPCs won't work for multi region penetration tests. As far as I remember subnets cannot span multiple availability zones. I should have probably mentioned that. >Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes? That I can't argue with...
- ababab 9y ago
  >Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes? Agreed, I still stick with the old Loadrunner nomenclature "loadgen"
- victor106 9y ago
  “Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes?” Sorry... I don’t get what is wrong with it? We use that word to refer to distributed compute a lot in my org.
  - aynsof 9y ago
    There's a political correctness movement away from the use of master/slave: https://twitter.com/bryanl/status/822093922549764096 https://twitter.com/bryanl/status/822093922549764096
bestboy 9y ago
JMeter is just terrible. To me it's the IE8 of the load testing tools. I'd not consider using it unless there are no other options. And there are better options (e.g. https://gatling.io https://gatling.io)
davidw 9y ago
Haven't had occasion to use it yet, but I've chatted some with the founders of http://nouvola.com/ http://nouvola.com/ - bright women and it looks like they've done a good job with their product. Once you're at that kind of scale, maybe a prepackaged solution is worth the savings in time.
djKianoosh 9y ago
I recently looked at Locust, and even though I knew practically zero python and have many years (tears?) of Java background, I found Locust, especially with a little docker-compose 17-liner, amazingly productive. YMMV
jacques_chester 9y ago
For those familiar with BOSH, Pivotal engineers built "Tornado" for load testing. It's a BOSH release of JMeter with some convenient dials. Since it's BOSH, you can deploy to AWS, GCP and Azure without fuss. The difference between running one worker, ten workers or a thousand workers is a line of YAML and how much you're prepared to pay. There are some blog posts: "Half a Million Concurrent/Distributed JMeter Users with BOSH ... in 10 Minutes" (http://engineering.pivotal.io/post/jmeter-bosh-usecase/ http://engineering.pivotal.io/post/jmeter-bosh-usecase/) "BOSH + Apache JMeter(TM) = Tornado for Apache JMeter(TM)" (http://engineering.pivotal.io/post/jmeter-bosh-release/ http://engineering.pivotal.io/post/jmeter-bosh-release/) The release itself: https://github.com/jamlo/jmeter-bosh-release https://github.com/jamlo/jmeter-bosh-release Disclosure: I work for Pivotal, BOSH is definitely one of our favouritest things.
justicezyx 9y ago
"Super-human agent on Dota2 1v1" Good to say they accurately define the situation, not something like "Super-human agent defeating world's best Dota2 player"
lyonlim 9y ago
We use Redline13 to help spin up load generators for stress and performance testing.
erikerikson 9y ago
Better to try serverless-artillery: https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery Eventually: https://github.com/Nordstrom/serverless-jmeter https://github.com/Nordstrom/serverless-jmeter

45 points by mfolaron 9y ago

> Here is the problem, JMeter uses Java RMI (Remote Method Invocation) to communicate to its slaves, but these connections require all machines to be on the same subnet and this is not feasible with EC2 instances. Incorrect with VPC.. Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes?

mfolaron 9y ago
>Incorrect with VPC.. VPCs won't work for multi region penetration tests. As far as I remember subnets cannot span multiple availability zones. I should have probably mentioned that. >Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes? That I can't argue with...
ababab 9y ago
>Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes? Agreed, I still stick with the old Loadrunner nomenclature "loadgen"
victor106 9y ago
“Also, could we stop using the term 'slave' to refer to distributed compute or storage nodes?” Sorry... I don’t get what is wrong with it? We use that word to refer to distributed compute a lot in my org.
- aynsof 9y ago
  There's a political correctness movement away from the use of master/slave: https://twitter.com/bryanl/status/822093922549764096 https://twitter.com/bryanl/status/822093922549764096

bestboy 9y ago

JMeter is just terrible. To me it's the IE8 of the load testing tools. I'd not consider using it unless there are no other options. And there are better options (e.g. https://gatling.io https://gatling.io)

davidw 9y ago

Haven't had occasion to use it yet, but I've chatted some with the founders of http://nouvola.com/ http://nouvola.com/ - bright women and it looks like they've done a good job with their product. Once you're at that kind of scale, maybe a prepackaged solution is worth the savings in time.

djKianoosh 9y ago

I recently looked at Locust, and even though I knew practically zero python and have many years (tears?) of Java background, I found Locust, especially with a little docker-compose 17-liner, amazingly productive. YMMV

jacques_chester 9y ago

For those familiar with BOSH, Pivotal engineers built "Tornado" for load testing. It's a BOSH release of JMeter with some convenient dials. Since it's BOSH, you can deploy to AWS, GCP and Azure without fuss. The difference between running one worker, ten workers or a thousand workers is a line of YAML and how much you're prepared to pay. There are some blog posts: "Half a Million Concurrent/Distributed JMeter Users with BOSH ... in 10 Minutes" (http://engineering.pivotal.io/post/jmeter-bosh-usecase/ http://engineering.pivotal.io/post/jmeter-bosh-usecase/) "BOSH + Apache JMeter(TM) = Tornado for Apache JMeter(TM)" (http://engineering.pivotal.io/post/jmeter-bosh-release/ http://engineering.pivotal.io/post/jmeter-bosh-release/) The release itself: https://github.com/jamlo/jmeter-bosh-release https://github.com/jamlo/jmeter-bosh-release Disclosure: I work for Pivotal, BOSH is definitely one of our favouritest things.

justicezyx 9y ago

"Super-human agent on Dota2 1v1" Good to say they accurately define the situation, not something like "Super-human agent defeating world's best Dota2 player"

lyonlim 9y ago

We use Redline13 to help spin up load generators for stress and performance testing.

erikerikson 9y ago

Better to try serverless-artillery: https://github.com/Nordstrom/serverless-artillery https://github.com/Nordstrom/serverless-artillery Eventually: https://github.com/Nordstrom/serverless-jmeter https://github.com/Nordstrom/serverless-jmeter