Author: Mykhailenko A.
There are at least three possible approaches that fall under the concept of “cloud testing”: testing cloud software (SaaS products), testing software quality using cloud tools, and testing cloud infrastructure. We are interested in the third option – specifically, how to test IaaS before deploying IT services on the provider’s virtual resources.
- What cloud testing entails
- Challenges with testing cloud infrastructure
- How and what to test in the cloud
- What to focus on when testing IaaS
What cloud testing entails
Next, we’ll discuss how to evaluate the cloud in terms of performance, correct functioning, as well as security, confidentiality, scalability, and data recovery from backups in case of emergencies.
Colobridge expert:
“In our company, cloud infrastructure testing usually involves two types of testing. The first is to verify compliance with the declared configuration of technical components: processor performance, storage capacity, network connectivity. The second is to assess the ease of working with the cloud itself and the service provider: how the control panel operates, how quickly and professionally technical support responds.”
There are even more examples of cloud tests – besides performance and provider operation, you can also assess functionality, performance under peak loads, compatibility, security, latency, availability, and other parameters. Each such test accomplishes a number of specific tasks.
Here are a few examples:
- When testing performance, it’s important that a failure affecting one process or user doesn’t impact other processes and users. Also, it’s crucial that application performance remains consistent across all types of devices connected to the cloud.
- When testing security, it’s important that the provider doesn’t have access to client data and that the stated security measures (firewall, antivirus, VPN) are activated, and data is transmitted encrypted if the service provider enables encryption as agreed upon.
- During load testing, it’s important that IT services and systems remain available as the load increases within the allocated resources. It’s also necessary to understand how the operation of different applications changes with varying loads.
Challenges with testing cloud infrastructure
While IaaS testing provides a general understanding of the provider’s operation and cloud functioning, it doesn’t always reflect the real picture.
Why does this happen? Here are just a few main reasons:
- The industry lacks standardized approaches to cloud testing.
- Providers often allocate computing resources automatically (“on the fly”), which complicates performance testing.
- It’s not always possible to account for temporal/peak loads, which can vary on weekdays and weekends, as well as backups, service maintenance, emergencies, network attacks, and other factors.
- Re-running the same test in the same cloud may yield different results due to differences in the operation of physical components (CPU, RAM, Disk) and virtual machines.
Colobridge’s Expert:
“The results of synthetic tests don’t always correspond to reality — that’s a fact. However, the problem isn’t as global as it seems at first glance. If a test shows results in hypothetical units, you can compare them with the results obtained from testing on on-premises infrastructure or another provider’s cloud. And those results will be accurate — you’ll understand where the performance best meets your needs. Another option is to deploy a copy of your product in the cloud and evaluate how well the cloud infrastructure meets your requirements in real conditions, but these results will also be approximate.
Another challenge with cloud capacity compliance testing is that some providers sometimes underestimate the performance of individual components to ensure 100% compliance with SLAs: for example, they use servers with a hypothetical performance of 2000 gigaflops, while the SLA guarantees only 1000 gigaflops. Due to this underestimated guarantee, testing will always show a positive result, but the user may not realize that the available performance is sufficient for a greater number of IT services.”
How and what to test in the cloud
Some companies offer comprehensive services for automated cloud infrastructure testing. They attract customers by reducing testing time by 20-30% and allowing assessment of the likelihood that applications and services will operate as claimed by the service provider after deployment in a real cloud environment (though this figure is usually far from 100%). For example, for functional testing, tools like AppPerfect, Jmeter, SOASTA CloudTest, and LoadStorm are used. For security testing, tools like Nessus, Wireshark, or Nmap are employed. The downside of such software is that it’s quite expensive and complex, requiring users to have a high level of expertise.
At the same time, many prefer to use separate tools to test virtual resources in the provider’s cloud. For testing processor performance, any test from the CPU-benchmark category can be selected: Geekbench, PCMark, PassMark. To assess disk performance, IOmeter or DISKSPD are often used. If there is no opportunity to use these tools or their analogs, you can deploy a test environment immediately or, preferably, after conducting basic performance assessments.
The choice of approach to cloud testing depends on the amount of free time, technical capabilities, and the availability of necessary IT expertise.
- Lack of expertise: the client uploads the productive environment to the cloud, starts working with it in normal mode, and if problems arise, begins to search for testing utilities.
- Average expertise: the client tests the cloud environment with synthetic tests, and if the results are satisfactory, migrates the productive environment to the cloud.
- High level of expertise: the client tests the cloud environment with synthetic tests, then deploys a copy of the productive environment in the cloud, simulates real workload, and only then migrates the current version of the productive environment to the cloud platform and starts working.
What to focus on when testing IaaS
- Make sure the chosen provider offers the opportunity to test virtual resources for free for several days (ideally a week or more).
- Seek consultation from the provider’s technical specialists to know in advance if the selected IaaS configuration meets the requirements of your services.
- During testing, create a support ticket to assess response time and service quality.
- When making the final decision, weigh all key factors: cloud test results, data center certification and location, support operation, SLA content, product portfolio, and others.
At Colobridge, when renting IaaS virtual infrastructure, you can test it for free for 7 days with the option to extend the period up to 14 days. We will answer all your questions regarding performance, security, functionality, and more – just ask our manager by phone or any other convenient means.
