Many of our customers are large enterprises with critical highly-available and secure infrastructures. This means that they spend (as do we) a lot of time proactively investigating and stress-testing systems, indeed we and many other vendors also provide tools within our products to assist in \u201ckicking the tyres\u201d. However small or large your enterprise is though, it\u2019s a methodology and mindset that you can embrace with plenty of free and open-source tools out there to assist you. My colleague, Abhilash recently wrote on his blog \u201cWhat is Chaos Engineering & Testing<\/a>” about the business benefits and paradigms many of our customers are adopting which gives more context on why you may want to evaluate some of these tips and tricks. In this blog, I have listed ten tools and related techniques you can use to test your IT infrastructure resilience.<\/p>\n IOMeter<\/a> is an open-source I\/O subsystem measurement and characterization tool for single and clustered systems. IOMeter is an easy way to generate stress on the I\/O system and as such can be very useful for development testing \u00a0teams of products interacting with or generating load on the OVS.\u00a0 <\/strong>There are many ways to use IOMeter. If you are looking to upgrade your Citrix or VMware servers, run IOMeter before and after the upgrade and compare performance to make sure that I\/O performance remains acceptable. Of course, if you have a monitoring tool in place when you run your stress test, you can also observe I\/O latencies, queueing and other related parameters.<\/p>\n Network emulation allows you to test the impact of poor or limited connectivity, bandwidth constraints and factors that impact on QoS (Quality of Service) and packet resends such as latency and jitter. Most would agree that a hardware device or appliance independent of the computer \/ hypervisor \/ virtual machine being tested is the ideal scenario. However, for situations with limited access to network or cost constraints software emulation may be an alternative, although it should be noted this can be difficult to do well and it is easy to introduce artificial conditions.<\/p>\n The Linux Foundation\u2019s netem<\/a> is a widely used option. WANem is another popular choice with a VMWare Appliance available<\/a>. This is an open source project<\/a> which helps measure and investigate network\u00a0load and includes tools to investigate large file IO and creation\/deletion of small files. There are lots of user guides and blogs on this tool.<\/p>\n Most monitoring vendors will include at least a basic Logon Simulator to test whether users can access platforms such as Citrix<\/a>, VMware<\/a>, Azure<\/a> or AWS<\/a>.<\/p>\n AWF Fault Injection Simulator<\/a> was released in March 2021, a fully managed service for running fault injection experiments on AWS that makes it easier to improve an application\u2019s performance, observability, and resiliency. Fault injection experiments are used in chaos engineering, which is the practice of stressing an application in testing or production environments by creating disruptive events, such as sudden increase in CPU or memory consumption, observing how the system responds, and implementing improvements. Beyond the managed service, open-sourced tools have also been released by AWS Engineering<\/a>. The availability of such tools and services is becoming a factor when choosing between different clouds or virtualization environments.<\/p>\n When investigating the performance of intensive graphical workloads and technologies associated with them such as remoting protocols (Citrix HDX\/EDT\/ICA, RDP, Teradici PCoIP or CAS, VMware Blast Extreme) or GPU\/CPU needs there are a wealth of benchmarks available, a great list is available here:<\/p>\n Whilst Logon Simulators are useful to test the availability of Apps and Virtualized Desktops or services, tools like those from Login VSI are frequently used to understand the capacity of infrastructure to deal with a large volume of users causing a boot storm and the degradation of systems as they approach capacity (at what point do you intervene?). Frequently setup to simulate some degree of randomness in user behavior e.g., accessing different mailboxes, webpages or a selection of say 3 videos from 5, these workloads can be modified for test lab purposes to be more uniform to assist in the assessment of the impact of changes to configuration or say protocol code.<\/p>\n Other capacity benchmarking and load testing tools include:<\/p>\n Fuzzing is a technique whereby poor quality, corrupt inputs and random data is input into systems to try and expose bugs, security flaws, potential crashes etc. This can be particularly useful for testing the robustness of platforms to corrupt data packets and malicious inbound network activity.<\/p>\n PCMark 10<\/a> is an industry standard benchmark popular with enterprises with large numbers of Windows 10 end points. Beyond physical PCs\/Laptops this is a popular tool in the EUC community for testing VMs (Virtual Machines) on Cloud\/Citrix\/VMware especially for office\/task worker like workloads, vExpert Ryan Mangan’s blog carries good examples<\/a>.<\/p>\n\n
<\/span>1. IOMeter<\/span><\/h3>\n
<\/span>2. iperf \/ iperf3<\/span><\/h3>\n
![\"iperf](\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/05\/iperf-diagnosis-tool.jpg?no\")
iperf<\/a> is an open-source utility that can be useful for diagnosing network issues. It is a cross-platform tool for network performance measurement and tuning that can produce standardized performance measurements for any network. iperf has client and server functionality and can create data streams to measure the throughput between the two ends in one or both directions. Use iPerf to stress your network interface cards, or to simulate bandwidth hungry users.<\/p>\n<\/span>3. WAN Emulation<\/span><\/h3>\n
\n![\"WAN](\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/05\/wan-emulation.jpg?n\")
\nApposite<\/a> are a popular vendor in the network emulation space, their product line includes the highly regarded Linktropy Hardware Emulator range. Whilst not the cheapest option, most would concur that hardware emulation is the gold standard and least problematic Enterprise-grade option.<\/p>\n<\/span>4. Bonnie++<\/span><\/h3>\n
<\/span>5. Synthetic Monitoring – Proactive Availability Monitoring<\/span><\/h3>\n
![\"synthetic-monitoring\"](\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/04\/synthetic-monitoring.jpg?nore\")
<\/a>We, at eG Innovations, have a full portfolio of synthetic monitoring simulators<\/a> that go beyond simple logon simulations and include:<\/p>\n\n
![\"web](\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/05\/web-client-simulator.jpg\")
<\/a>Our Web Client Simulator<\/a> is particularly useful for testing the availability and performance of SaaS and Cloud-hosted applications where you may have little insight or control over the underlying infrastructure. A robot user accesses web applications and performs actions and real use-case steps e.g. adding a new customer and updating records in Salesforce or working within Microsoft Excel spreadsheets, and the steps are measured, and failures screenshotted, thus testing the availability and performance of the application but also interrogating the underlying services and infrastructure such as storage, databases and networking beyond Logon and app. Availability.<\/p>\n<\/span>6. Fault Injection Tools<\/span><\/h3>\n
![\"Fault](\"https:\/\/www.eginnovations.com\/blog\/wp-content\/uploads\/2021\/05\/fault-injection-tools.jpg?no\")
<\/a>NetFlix open-sourced their Chaos Monkey projects<\/a> and are probably one of the best-known adopters and leaders of chaos engineering methodologies. There\u2019s plenty of further information on their blog<\/a>.<\/p>\n<\/span>7. Graphical Benchmarks<\/span><\/h3>\n
\n
<\/span>8. Capacity Benchmarking \/ Load Testing<\/span><\/h3>\n
\n
<\/span>9. Fuzzing tools<\/span><\/h3>\n
\n
<\/span>10. PCMark 10<\/span><\/h3>\n