A senior Microsoft executive explained why PC benchmarking in the cloud is an incredibly complicated process that, in many cases, is simply not possible.
In a recent blog post (opens in a new tab), Ron Martinsen, senior product manager for Microsoft Windows 365 Cloud PC and Azure Virtual Desktop, explained why comparing cloud computing is nearly impossible due to the sheer number of of variables.
Instead, we must believe that continuous updates give us maximum performance.
Comparing cloud computers is hard
Martinsen drew on nearly three decades of experience and "countless performance studies" to explain how "it's nearly impossible to get reproducible data in an environment that reflects the reality of what users will use."
He even goes on to explain that repeated tests can yield different results and that making valid comparisons can be difficult at best. With this advice, it's better to go with virtual hardware that meets or exceeds the requirements rather than trying to compare numbers that don't represent reality.
In his blog post, Martinsen explains that operating system, network latency, disk controller caching, and memory optimization are some of the most important factors to consider when attempting an evaluation.
Put this in the context of cloud computing "where the host computing runs on a cluster and serves a variable number of active cloud PCs" and the number of variables becomes "overwhelming", let alone for virtualization workloads like Windows Subsystem for Linux.
Then there's the fact that the virtual desktop you accessed a year ago has likely been upgraded and upgraded in the months that have followed, helping you improve its performance and/or efficiency.
Going forward, even outside of the cloud computing context, Martinsen criticizes benchmarking tests for being conducted in sterile environments where the best possible numbers are achieved, not relative numbers that apply to specific use cases. .
To conclude, while higher benchmark numbers are likely to result in better performance for the end user, the difference will rarely be that stark. For cloud computing, Microsoft's relative performance indicators (opens in a new window) serve as the basis for comparison.