In this short blog I’m going to go over a simple way of setting up some useful performance tests using Jmeter and Blazemeter, which are what we currently use for some of our deployments. Performance testing is quite a broad topic so I’ll just cover the basics of setting up http request tests against your site.
Why Performance Testing?
It’s important to know why you’re taking the time to set up and run performance tests, the scope and requirements should be understood before you set off on your mission.
- To compare new deployments performance against previous versions.
- To make sure our system meets performance goals/criteria.
- To find anomalies in server and render response times.
- To test the system as a whole under load.
- To quickly identify errors.
When we test our vix.digital website, we generally test for two things - that we haven’t introduced a change that slows down the rendering speed of the site and that we haven’t broken any pages (every route should return a 200). For our other applications we also include a bit of load testing in there too.
Apache JMeter is an open source tool that was originally developed to test web application but has since been expanded to be able to test a variety of different things including (but not limited to) HTTP, HTTPS, FTP, Mail (SMTP, POP3, IMAP) and SOAP/REST web services.
Another useful feature of JMeter is that it allows you to record tests via your actions in a web browser, which can be really handy for more complex tests. It’s also available to use headless, which is what allows it to be used so effectively in Continuous Deployment pipelines and sparked its integration into web based services and tools like BlazeMeter which I’ll be talking about shortly.
Creating Your Tests
Creating our list of HTTP request tests is pretty simple. First we need to add a Thread Group to our test (right click the test in the left panel and add Thread group). I’ve called mine Users, since that’s essential what it’s a mock of.
Within my Users Thread Group I add a load of HTTP Request Samplers (right click your Thread Group > Samplers > HTTP Request). You can keep the default settings for this exercise, you just need to input your host address and the path you want to test.
You can now run this test locally if you like, by right clicking on the Thread Group you can add some Listeners which allow you to visualise the results in a couple of different ways. It’s usually a good idea to test locally at least once to make sure you haven’t made any mistakes in your host and path settings.
Blazemeter is a web service that allows you to upload your .jmx test files and run them in the cloud using amazon EC2 instances. It can get really pricey, but for our needs the free 10 tests a month service will do.
Once you’ve set up your account, you can create a new test. As you might notice from the list of options presented in the modal window that you don’t actually need to create a jmx file to use blazemeter from URL/API endpoint tests - you can make them right there in the Blazemeter web app. The reasons why we use JMeter are that it allows us to make much more complex tests and the .jmx file it transferable to any tool we want to use it with.
Blazemeter will ask you to upload your test file and configure your test, including how many users you want to concurrently hit your endpoint. If you want to perform a basic endpoint test you can use a single user and you can use up to 50 (with the free account) if you want to perform a small load test.
The two other main settings you’ll want to play around with are iterations and duration. If you choose unlimited iterations, the duration selected will be the only thing that determines when your test stops. That means the ‘users’ will constantly hit your endpoints over and over for up to 20 minutes. For the purposes of just trying it out, it might be best to choose 1-5 iterations as a starter, which will give you enough information on whether your routes work and how long they take to render.
Once you’ve saved your test, hit the big green play button to give it a whirl, after a few minutes (however long you specified, or the number of iterations..) you’ll be presented with a report that looks a little like this.
You can then use this report to measure your web applications performance against your performance guidelines, if you have any. As a basic rule of thumb, for our website which isn’t terribly complex we are looking for 0 errors and below 300 ms response average - which we have, great success!
Taking It Further
There are lots of things you can do now that you have the basics nailed down. For example, one of the most common things people do is trigger their tests to run via an API call to the BlazeMeter service rather than trigger it manually via the GUI.