In the past, I used Microsoft Web Application Stress Tool and Pylot to stress test web applications. I'd written a simple home page, login script, and site walkthrough (in an ecommerce site adding a few items to a cart and checkout). Just hitting the homepage hard with a handful of developers would almost always locate a major problem. WAPT tool is a perfect combination of efficiency and flexibility with ease of use. The product is backed by a team of professionals with over 14 years of experience in load testing. If you need assistance working with WAPT, our experts are ready to provide it to you from the very first day you run the demo version.
Even the most novice among website owners has at some point or other tested their website performance. However, most of these tests normally focus on loading speed or user experience indices.
But what about load testing?
Although most websites are prone to traffic levels that are usually quite regular, there may be occasions when some sites will have to deal with heavy loads. Polar mohr 92 emc manual. Examples of these include online stores, or even some government websites.
If your website gets an unexpected spike in the number of visitors over a short period, how well are you equipped to handle it?
Understanding Load Testing
What is load testing?
Load testing is bench-marking a website to see how it performs under various loads.
For example, a test may simulate an increasing number of concurrent visitors landing on your site. It will also record how your site handles them and records them for your reference.
What types of “load” are tested?
Depending on the tool you choose to load test your site with, each may come with different features. The most basic will simply involve simulating an ever increasing load and halting when your site crashes.
Other tools may be capable of generating a simulated load that mimics different user behaviour, such as performing queries, changing pages, or loading other functions. Some may even be able to map out logical flows for each individual scenario.
Load Testing Tools to Consider
Depending on their complexity, some load testing tools can be quite expensive. However, there are cheaper options in the market and some are even free for use. I’ve included a mixture of these below for your reference, including a couple of open source options.
1. Loadview by Dotcom Monitor
Price: From $199/mo, free trial available
Loadview is one of the more complete solutions available in the market and today is based on a cloud service model. This means that whatever type of simulation you need from them, you only pay for the service – there is zero investment in hardware or anything else.
Feature wise, Loadview offers a very complex solution that can include anything from straight up HTTP load tests to a sophisticated mix of your choice. It is able to simulate dynamic variables and even geo-location diversity in its tests.
Features
- Post-firewall tests
- Handles dynamic variables
- Detailed waterfall charts
- Load test curves
2. K6 Cloud (formerly Load Impact)
Price: From $49/mo
K6 is a cloud-based, open source load testing tool that’s provided as a service. One of the things that makes this tool interesting is that it is priced on a variable-use model which means that the cost of entry can be relatively low depending on your needs. It is, however, mainly developer-centric.
Aside from load testing, K6 also offers performance monitoring. Its load testing side is focused on high loads and can handle various modes such as spikes, stress testing, and endurance runs.
*K6 does not run in browsers nor does it run in NodeJS
Features
- Developer-friendly APIs.
- Scripting in JavaScript
- Performance monitoring
3. Load Ninja
Price: From $270.73/mo
Load Ninja lets you load-test with real browsers based on recorded scripts and then helps analyze performance results. Its use of real browsers at scale means that this tool helps recreate a more realistic environment and end result for testing.
Results can be analyzed in real-time and thanks to the handy tools the system provides, your scripting time can be reduced by as much as 60%. Internal applications can be tested as well, both with proxy-based fixed IPs or your own range of dynamic IPs (by using a whitelister).
Features
- Test with thousands of real browsers
- Diagnose tests in real-time
- Insights on internal application performance
4. LoadRunner by Micro Focus
Price: From $0
With an entry-level free community account that supports tests from 50 virtual users, LoadRunner is available even to the newest website owners. However, if you scale it up to high levels the cost rises exponentially.
This Cloud-based service also offers the use of an Integrated Development Environment for unit tests. Schwinn s350 electric scooter owners manual. It supports a wide range of application environments including Web, Mobile, WebSockets, Citrix, Java, .NET, and much more. Be aware that LoadRUnner can be pretty complex and has a steep learning curve.
Features
- Patented auto-correlation engine
- Supports 50+ technologies and application environments
- Reproduces real business processes with scripts
5. Loader
Price: From $0
Compared to what we’ve shown so far, Loader is a much simpler and more basic tool. Its free plan supports load testing with up to 10,000 virtual users which is enough for most moderate traffic websites.
Unfortunately you will need to have a paid plan to access more advanced features such as advanced analytics, concurrent tests, and priority support. It is easy to use though since basically you just add your site, specify the parameters, then let the test run.
Features
- Shareable graphs & stats
- Useable in a GUI or API format
- Supports DNS Verification and priority loaders
6. Gatling
Price: From $0
Gatling comes in two flavors, Open Source or Enterprise. The former lets you load-test as an integration with your own development pipeline. It includes both a web recorder and report generator with the plan. The Enterprise version has on-premise deployments or alternatively, you can opt for a Cloud version based on Amazon Web Services (AWS).
Although both of these versions are feature-packed, the Enterprise version supports a few extras that don’t come with Open Source. For example, it has a more usable management interface and supports a wider range of integrations.
Features
- Multi-protocol scripting
- Unlimited testing and throughput
- Gatling scripting DSL
7. The Grinder
Price: From $0
Grinder is open sourced all the way and is probably the only truly free option on this list. However, it has to be run locally in your own development environment and needs a few extra such as Java in order to work.
However, being open source it has been adopted widely and developers have come up with a plentiful number of plugins which vastly extend it in terms of both use-ability and functionality. Still, unless you’re a developer or so oriented, The Grinder might be a bit of a handful for you to use.
Features
- Flexible scripting based on Jython and Clojure
- Highly modular with tons of plugins
- Distributed framework and mature HTTP support
When to Load Test Your Website?
If you’ve had a look at most of the tools available, you will probably have noticed that many of them offer either trial accounts or some form of limited free version. This makes them readily available for use for a wide audience.
Most website owners need to be concerned about hosting performance since it affects far more than simply user experience. For many business owners, the availability of your website is also a matter of brand reputation.
Sites which are growing need to be especially cautious of availability and scalability of the resources used to hosting your website. In most cases a high percentage of user response time is spent on the surface of your site. However, as sites grow in traffic volume this might change.
More traffic usually means a disproportionate growth in backend processing and your system will struggle as that spikes. Much will depend on variables unique to your site development, so it isn’t possible to give you a solid number of visitors at which point this will happen.
To realistically see how your site performance you need load testing to be done. Exactly when to do it is debatable, but my advice would be to plan ahead and test early.
What to Check for When Load Testing?
As the very name implies, your core function should be the basic of how your site performs under loads. This will let you observe a number of things such as:
- At what point your site performance starts to degrade
- What actually happens when service degrades
When I mentioned how different sites may react differently based on their architecture, that was a signal meant for you to understand that not all sites fail in the same way as well. Some database-intensive sites might fail on that point, while others may suffer IO failures based on server connection loads.
Because of this, you need to be prepared to set up a variety of tests to understand how your site and server will cope under various scenarios. Based on those, keep a close eye on a few key metrics such as your server response time, the number of errors cropping up, and what areas those faults may lie in.
Generating complex scripts and runs along with the accompanying logic can be difficult. I suggest that you approach load testing incrementally. Start with a brute force test that will simply test your site under a continuously increasing stream of traffic.
As you gain experience, add on other elements such as variable behaviour, developing your scripts and logic over time.
Conclusion: Some is Better than None
When it comes to load testing, starting with the basics is better than not getting started at all. If you’re a beginner to all of this, do try to do your testing on an alternate mirror or offline where possible – avoid load testing a live site if you can!
If you’re just starting out now, make sure to create a record of your tests. Performance testing is a journey that should accompany the development of your site as it grows. The process can be tiring but remember, not having a record can make future assessments much more difficult for you.
API testing tools are more important now than ever. That why I create this list of the top 20 free API Testing Tools you should know.
As we move towards more Agile shift-left software development processes like continuous integration and delivery, the need to quickly give test feedback to our developers is increasing.
One downfall to UI tests is they are slow, making them a poor choice for letting developers know quickly if their code has broken the latest build or not. API tests, on the other hand, tend to be faster and run more reliably than GUI tests.
Below is a quick API testing tools comparison of open source options.
Before we take a closer look at the API testing tools, let’s make sure we’re all on the same page with what an API actually is.
What is an API?
Application Programming Interfaces (API) is a specification that acts as an interface for software components.
While most functional testing involves testing a user interface like a web page or a dot net form, API testing involves bypassing a user interface and communicating directly with an application by making calls to its APIs.
API testing allows you to test headless technologies like JMS HTTP, databases and Web services.
API testing is sometimes called “headless” testing. Most headless testing consists of bypassing the UI and sending a request directly to an application's backend or service and receiving a response while validating the response to ensure things are working as we expect them to.
This simple example is often referred to as a client/server relationship. A client makes a request by asking for a resource; the request then goes out and finds a server that can fill that request. The server locates the desired resource and sends a response back to the client.
What API Testing Tools Can I Use to Automate API Testing?
Since Selenium is just for browser-based testing, you may be wondering which tool to use for Rest and Soap web service-based testing.
Here are some of the top API testing tools that can be used for Rest and Soap Web Service Testing. API tools are great because many times you can leverage one test script to help with API load/stress performance testing, security testing and penetration testing.
So let's get to it.
Although there are a lot of great paid options here are the top free API testing tools you should check out.
Postman
Postman is a rest client that started off as a Chrome browser plugin but recently came out with native versions for both Mac and Windows.
At a high level, you can use it to send a post request to your web server and it gives you the response back. It allows you to set up all the headers and cookies your API expects, and then check the response when it comes back.
- Can be used for both automated and exploratory testing
- Can be run on Mac, Windows, Linux &Chrome Apps
- Has a bunch of integrations like support for Swagger & RAML formats
- Has Run, Test, Document and Monitoring Features
- Doesn’t require learning a new language
To hear how Postman is used in the real world for testing video games check out Amber Race's TestTalks episode on Testing Video Games Using API Automation.
Karate DSL
Karate allows you to create a test that can sequence calls to any kind of web-service and assert that the responses are as expected.
- Build on top of Cucumber-JVM
- Can run a test and generate reports like any standard Java project
- A test can be written without any Java knowledge required
- Tests are easy to write even for non-programmers
Check out a quick example of how to get started using Karate with BDD.
SoapUI
SoapUI is a headless functional testing tool from SmartBear software. It comes in two flavors: Free open source version and Pro Version. Since the free version is open-source, you can actually gain access to the full source code and modify as needed.
The SoapUI Pro version is user-friendlier and has additional functionality including a form editor, an assertion wizard for XPath, and SQL query builder. The free version lets you:
- Can easily create custom code using Groovy
- Drag and Drop Test Creating
- Can create complex scenarios
- Asynchronous Testing
- SoapUI’s Mock Service lets you mimic web services before they are implemented
Listen to Mike Giller of Smartbear talk more about SoapUI and API Testing.
HttpMaster Express
HttpMaster describes itself as a web development and test tool to automate testing of websites and services. It can be used to test RESTful web services and API applications. HttpMaster also allows you to and monitor API responses.
- HttpsMaster project offers global options to customize your API request
- Parameter capabilities enable you to include dynamic data with your request
- You can use request chaining to leverage request items to include some data from the previous request with the next request
Rest- Assured
Rest-Assured is an open-source Java Domain-specific language (DSL) that makes testing REST service simple. It simplifies things by eliminating the need to use boiler-plate code to test and validate complex responses. It also supports XML and JSON Request/Responses. This is probably the best-known tool to test rest API.
- Removes need to create boilerplate code required to interact with a rest service
- Support BDD Given/When/Then syntax
- Integrated seamlessly with Java projects
- Probably the standard for programmatically creating a rest API testing tool script
If you use Java and want a REST API web services library you can use in your framework Rest-Assured is a good choice.
Johan Haleby creator of Rest-Assured shares tips and tricks for getting started REST testing with Rest-Assured.
RestSharp
RestSharp is a simple REST and HTTP API Client for .NET
- Supports .NET 3.5+, Silverlight 5, Windows Phone 8, Mono, MonoTouch, Mono for Android
- Easy installation using NuGet for most .NET flavors
- GET, POST, PUT, PATCH, HEAD, OPTIONS, DELETE supported
Rest Console
HTTP Client and Request Visualizer and Constructor tool, helps developers build, debug and test RESTful APIs. Rest Console is an HTTP Request Visualizer and Constructor tool, helps developers build, debug and test RESTful APIs.
- Easy query parameters creation
- Syntax highlighting
- Authentication support: Plain, Basic, OAuth + Custom
RoboHydra Server
Looking for API integration testing tools?
RoboHydra is a testing tool for HTTP-based clients (ie. software that makes HTTP requests). The idea is, instead of connecting your clients-under-test to the real server, you connect them to RoboHydra and make RoboHydra respond with whatever you need for each request.
- You can test many different kinds of clients
- Written in Javascript, runs under Node
Hippie-Swagger
hippie-swagger is a tool for testing RESTful APIs. It’s also an API testing tool with automatic swagger assertions. In addition to validating API behavior, it will fail tests when swagger documentation is missing or inaccurate.
- Can validate All aspects of swagger file validated; parameters, request/response body, paths, etc.
- Accurate, human-readable assertion messages
WebInject
WebInject is an open source solution for automated testing of web applications and web services. It can be used to test individual system components that have HTTP interfaces (JSP, ASP, CGI, PHP, AJAX, Servlets, HTML Forms, XML/SOAP Web Services, REST, etc).
Web Application Stress Testing
- Is a command line tool
- Written in Perl can be installed on MS Windows, GNU/Linux, BSD, Solaris, MAC OS
Pyresttest
PyRestTest is a python based REST testing and API micro-benchmarking API testing tool
- You can write your tests in basic YAML or JSON config files, no code needed
- Returns exit codes on failure
- Only works on Mac and Linux
Airborne
Airborne is an open source Ruby-based RSpec driven API testing framework.
- Works with Rack application like Sinatra and Grape
- Works with APIs written in Rails
Unirest
Recommended by Unmesh Gundecha Unirest is a lightweight HTTP request client libraries.
- Can be combined with xUnit, BDD runner to make it a test tool
Mockbin
Mockbin was recommended by Augusto Marietti. Mockbin allows you to generate custom endpoints to test, mock, and track HTTP requests & responses between libraries, sockets, and APIs.
- Mock Custom Endpoints
- Create Custom HTTP Methods
- Log and inspect incoming calls to your custom endpoints
Citrus Framework
Not sure why more folks aren't aware of the Citrus Framework. Citrus is an open-source framework that can help you automate integration tests for virtually any messaging protocol or data format. It has nice test results reports as well.
- Works with REST, SOAP, HTTP, JMS, TCP/IP and more
- Create tests using Java or XML
- Mature – has been around awhile
For more info on Citrus, Integration testing check out API Integration Automation Testing With Citrus Framework.
ZeroCode
ZeroCode is a free and open source, lightweight API testing tool library built on JUnit core runners, for API endpoints, using simple JSON steps. I just heard about this tool but sounds like its already being used in large enterprise companies like HSBC Bank.
Here are some benefits of using ZeroCode:
Website Stress Test Free
- ZeroCode is built on JUnit core runners (no plugins needed)
- Just your IDE or any JSON editor is enough to drive the tests
- Very easy to write tests – as simple as Postman REST-Client
- You can still do BDD, but without syntax overhead
- Performance testing – Load/Stress generation is quite easy and existing tests can be reused
- Efficient Reports – You can do a fuzzy search as well as filter by any text, and track by author
Katalon Studio
Although primarily known as a GUI automation test tool Katalon studio also supports test scenarios that use web services and can be used as an API test tool.
Some benefits of Katalon Studio are:
- It's not open-sourced but it is free
- It’s a complete package and framework. So it easy to install and get working
- Small learning curve
- Has functionality that allows you to test web services and REST APIs
Listen to my interview with Raghav Pal to discover more about Katalon studio.
Web Page Stress Test
JMeter
As you might already know JMeter was developed for load testing but many testers also use it for API testing and as an API load testing tool as well.
Some benefits of JMeter are:
- Can leverage one API test script for both API and as an API Performance testing
- JMeter has been around awhile so it's time-tested
- Easily integrated into CI/CD environments
Tavern
Tavern is a pytest plugin, command-line tool and API testing tools Python library for automated testing of APIs.
- Simple, concise and flexible YAML-based syntax.
- It’s very simple to get started,
- highly customizable for complex tests.
- Supports testing RESTful APIs as well as MQTT based APIs.
Chakram
Chakram is a REST Javascript API testing tools framework that also has a BDD testing style and fully exploiting promises. So you can use it for
- HTTP Assertions
- Chakram fully exploits javascript promises
- BDD formatting and hooks
- Very extensible
RestBird
Web Stress Test Tool
RestBird is a collection of Rest API Tools like autotest, mock server and record, and playback functionality. I just learned about it from a comment someone left for this post. It looks promising. Some benefits of RestBird are:
- Easy to use and Debug
- Fully restAPIi support to be integrated with JIRA, Jenkins, Team City, Bugzilla, Slack
- programmable response validation through Python/Golang scripts
Top REST API Testing Tools Recap
These are the top API testing tools free I’ve come across, but there are tons of API testing tools popping up everywhere, so I’ll definitely be adding to this list as time goes on.
If one of your favorite API test tools is not listed, please let me know and I’ll add it.
What About Non-API Automation Testing Tools?
For other non-functional APIautomation testing tools check out:
- Top 11 Open Source Performance Testing Tools for Load & Stress Testing
- The Top 21 FREE Visual Validation Tools for Testers
- Top Accessibility Testing Tools for Automation
- 7 iOS Testing Tools You Need to Know
How to Stay Up To Speed with the Latest in Automation Testing?
If you feel overwhelmed with staying up to date with all the latest automation testing tools and best practices check out my annual online conference dedicated 100% to just automation testing — Automation Guild.