Category Archives: Provisioning Services

IntelliCache and the IOPS Problem

If you’ve been following this blog for any length of time, you know that we’ve written extensively about XenDesktop, and spent a lot of time on best practices and problems to avoid. And one of the biggest problems to avoid is poor storage design resulting in poor VDI performance.

In a nutshell, the problem is that a Windows desktop OS uses disk far differently than a Windows server OS. Thanks to the way Windows uses the swap file, disk writes outnumber disk reads by about 2 to 1. You can build your virtual desktop infrastructure on the latest and greatest server hardware, with tons of processing power and insanely huge amounts of RAM, but if all of the disk I/O for all of those virtual desktops is hitting your SAN, you’ve got a scalability problem on your hands.

Provisioning Services (“PVS”) can help to mitigate this in two ways (assuming for sake of argument that you’re provisioning multiple virtual systems from a common, read-only image): First, if you build your Provisioning Servers correctly, you should be able to serve up most of the OS read operations from the Provisioning Server’s own cache memory. Second, you can use the virtualization host’s local disk storage as the required “write cache” – because all of those write operations have to go somewhere while the virtual system is running.

But XenDesktop 5 introduced a new way to provision desktops called “Machine Creation Services” (“MCS”). We wrote about this in the April edition of our Moose Views newsletter, so if you’re not familiar with all the pros and cons of MCS vs. PVS, I’d encourage you to take a brief time out and read that article. Suffice it to say that, despite all the advantages of MCS, the biggest downside of using MCS to provision pooled desktops was that all of the IOPS hit your SAN storage, which limited the scalability of an MCS-provisioned VDI deployment.

But all of that just changed, with the release of XenDesktop 5 Service Pack 1, which was made available for download a week ago (May 13). With SP1, XenDesktop 5 is now able to take advantage of the “IntelliCache” feature that was introduced as part of XenServer v5.6 Service Pack 2. Using MCS with the combination of XenDesktop 5 SP1 and XenServer SP2…

  • The first time a virtual desktop is booted on a given XenServer, the boot image is cached on that XenServer’s local storage.
  • Subsequent virtual desktops booted on that same XenServer will boot and run from that locally cached image.
  • You can use the XenServer’s local storage for the write cache as well.

The bottom line is that you can move as much as 90% of the IOPS off of the SAN and onto local XenServer storage, removing nearly all of the scalability limitations from an MCS-provisioned environment.

With most of the IOPS for running VMs taking place on local storage, it’s pretty straightforward to figure out how many VMs you can expect to support on a given virtualization host. Dan Feller’s blog post does a great job of walking through the process of calculating the functional IOPS that your local XenServer storage repository should be able to support, and inferring from that number how many light, normal, or power users you should be able to support as a result.

This also means that using XenServer as the hypervisor for your XenDesktop 5 deployment is going to yield a significant performance advantage over any other hypervisor, unless or until the other guys come out with similar local caching features. So, if you’re a VMware shop, my advice is this: Go ahead and virtualize all of the supporting XenDesktop server components on your VSphere infrastructure. Run your XenDesktop 5 VMs on XenServer hosts, and just don’t tell anyone! If you’re asked, just say, “Oh, yeah, these are my XenDesktop host systems – they’re completely separate from our VSphere infrastructure, because we don’t need the (insert favorite VSphere feature) function for these systems.” Your infrastructure will run better, and no one will know but you…

Top Ten VDI Mistakes (According to Dan Feller)

Dan Feller is a Lead Architect with the Citrix Consulting group, and has written extensively about XenDesktop. We found his series on the top ten mistakes people make when implementing desktop virtualization to be quite enlightening. In case you missed it, we thought we’d share his “top ten” list here, with links to the individual posts. We would highly recommend that you take the time to read through the series in its entirety:

#10 – Not calculating user bandwidth requirements
Back in the “good old days” of MetaFrame, when we didn’t particularly care about 3D graphics, multimedia content, etc., we could get by with roughly 20 Kbps of network bandwidth per user session. That’s not going to cut it for a virtualized desktop, for a number of reasons that Dan outlines in his blog post. He provides the following estimates for the average bandwidth required both with and without the presence of a pair of Citrix Branch Repeaters (which have some secret sauce that is specifically designed to accelerate Citrix traffic) between the client device and the virtual desktop session:

Parameter XenDesktop Bandwidth without Branch Repeater XenDesktop Bandwidth with Branch Repeater
Office Productivity Apps 43 Kbps 31 Kbps
Internet 85 Kbps 38 Kbps
Printing 553 – 593 Kbps 155 – 180 Kbps
Flash Video (with HDX redirection) 174 Kbps 128 Kbps
Standard WMV Video (with HDX redirection) 464 Kbps 148 Kbps
HD WMV Video (with HDX redirection) 1812 Kbps 206 Kbps

NOTE: These are estimates – your mileage may vary!

One thing that should come across loud and clear from the table above is what a huge difference the Citrix Branch Repeater can make in your bandwidth utilization. And as we’ve always said: you only buy hardware once – bandwidth costs go on forever!

#9 – Not considering the user profile
It should go without saying that user profiles are important. But if it’s number 9 on the list of things people most often screw up, then apparently it doesn’t. In a nutshell: If you mess up the users’ profiles, the users won’t be happy – logon/logoff performance will suffer, settings (including personalization) will be lost. If the users aren’t happy, they will be extremely vocal about it, and your VDI deployment will fail for lack of user buy-in and support. There are some great tools available for managing user profiles, including the Citrix Profile Manager, and the AppSense Environment Manager. AppSense can even maintain a consistent user experience across platforms – making sure that the user profile is the same regardless of whether the user is logged onto a Windows XP system, a Windows 7 System, or a Windows Server 2008 R2-based XenApp server.

Do yourself a favor and make sure you understand what your users’ profile requirements are, then investigate the available tools and plan accordingly.

#8 – Lack of an application virtualization strategy
How many applications are actually deployed in your organization? Do you even know? Are the versions consistent across all users? Which users use which applications? You have to understand the application landscape before you can decide how you’re going to deploy applications in your new virtualized desktop environment.

You have three basic choices on how to deliver apps:

  1. You can install every application into a single desktop image. That means that whenever an application changes, you have to change your base image, and do regression testing to make sure that the new or changed application didn’t break something else.
  2. You can create multiple desktop images with different application sets in each image, depending on the needs of your different user groups. Now if an application changes, you may have to change and do regression testing on multiple images. It’s worth noting that many organizations have been taking this approach in managing PC desktop images for years…but part of the promise of desktop virtualization is that, if done correctly, you can break out of that cycle. But to do that, you must…
  3. Remove the applications from the desktop image and deliver them some other way: either by running them on a XenApp server, or by streaming the application using either the native XenApp streaming technology or Microsoft’s App-V (or some other streaming technology of your choice).

Ultimately, you may end up with a mixed approach, where some core applications that everyone uses are installed in the desktop image, and the rest are virtualized. But, once again, it’s critical to first understand the application landscape within your organization, and then plan (and test) carefully to determine the best application delivery approach.

#7 – Improper resource allocation
Quoting Dan: “Like me, many users only consume a fraction of their total potential desktop computing power, which makes desktop virtualization extremely attractive. By sharing the resources between all users, the overall amount of required resources is reduced. However, there is a fine line between maximizing the number of users a single server can support and providing the user with a good virtual desktop computing experience.”

This post provides some great guidelines on how to optimize the environment, depending on the underlying hypervisor you’re planning to use.

#6 – Protection from Anti-Virus (as well as protection from viruses)
If you are provisioning desktops from a shared read-only image (e.g., Citrix Provisioning Services), then any virus infection will go away when the virtual PC is rebooted, because changes to the base image – including the virus – are discarded by design. But you still need AV protection, because the virus can use the interval between infection and reboot to propagate itself to other systems. The gotcha here is that the AV software itself can cause serious performance issues if it is not configured properly. Dan provides a great outline in this post for how to approach AV protection in a virtual desktop environment.

#5 – Managing the incoming storm
In most organizations, the majority of users arrive and start logging into their desktops at approximately the same time. What you don’t want is dozens, or hundreds, of virtual desktops trying to start up simultaneously, because it will hammer your virtualization environment. There are some very specific things you need to do to survive the “boot storm,” and Dan outlines them in this post.

#4 – Not optimizing the virtual desktop image
Dan provides several tips on things you should do to optimize your desktop image for the virtual environment. He also has specific sections on his blog that deal with recommended optimizations for Windows 7.

#3 – Not spending your cache wisely
Specifically, we’re talking about configuring the system cache on your Provisioning Server appropriately, depending on the OS and amount of RAM in your Provisioning Server, and the type of storage repository you’re using for your vDisk(s).

#2 – Using VDI defaults
Default settings are great for getting a small Proof of Concept up and running quickly. But as you scale up your VDI environment, there are a number of things you should do. If you ignore them, performance will suffer, which means that users will be upset, which means that your VDI project is more likely to fail.

#1 – Improper storage design
This shouldn’t be a surprise, because we’ve written about this before, and even linked to a Citrix TV video of Dan discussing this very thing as part of developing a reference architecture for an SMB (under 500 desktops) deployment. We’re talking here about how to calculate the “functional IOPS” available from a given storage system, and what that means in relation to the number of IOPS a typical user will need at boot time, logon time, working hours (which will vary depending on the users themselves), and logoff time.

Just to round things out, Dan also tossed in a few “honorable mentions,” like the improper use of NIC teaming or not optimizing the NIC configuration in Provisioning Servers, trying to provision images to hardware with mismatched hardware device drivers (generally not an issue if you’re provisioning into a virtual environment), and failing to have a good business reason for launching a VDI project in the first place.

Again, this post was intended to whet your appetite by giving you enough information that you’ll want to read through Dan’s individual “top ten” posts. We would heartily recommend that you do that – you’ll probably learn a lot. (We certainly did!)

Citrix Fixes the Provisioning Services – KMS Problem!

This is big news for anyone who wants to use XenDesktop to facilitate a Windows 7 migration. Here’s why: It only takes a moment’s thought to realize that if your desktop virtualization project simply trades inexpensive desktop SATA storage for expensive data center SAN storage, it’s not going to do good things for your ROI. So provisioning your virtual desktops from a shared Standard Image is a must. And that’s what Provisioning Services (“PVS”) allows you to do. If your standard Windows 7 OS image is, say, 15 Gb, you only need one instance of it on your SAN regardless of how many virtual PCs you’re provisioning from it. Then, using the Citrix Profile Management tool in conjunction with standard Group Policy folder redirection techniques, you can merge user personalization at logon time.

There was only one problem…turning a Win7 vDisk into a Standard Image broke the Microsoft license key. The only way around that was to use Key Management Services (KMS) to auto-activate systems as they were provisioned, but there were problems in using KMS with PVS, as we’ve documented in earlier posts.

I am happy to report that the problem has been addressed in PVS v5.6, SP1, which is now available for download at the Citrix download site. Not only that, but PVS v5.6, SP1, also works with a Multiple Activation Key (MAK) for smaller environments where KMS is not justified. Here’s the difference between the two activation methods:

KMS is a service that runs on a server in your own network. It supports Windows Server 2008 and 2008 R2, Vista, Win7, and Office 2010. However, it requires a minimum number of systems checking in for activation before any systems will be activated. That threshold is 8 systems for server activation, and 25 systems for workstation activation. Prior to SP1, systems provisioned from a Standard Image looked to the KMS server like the same system checking in again and again, so the threshold counter didn’t increment. SP1 fixes that. Please note, however, that you must be running KMS on a 2008 R2 server if you want virtual machines to increment the threshold counter.

With an MAK, the activation server is hosted at Microsoft. The MAK is a reusable key that’s good for a predefined number of activations. With SP1, PVS will cache the activation confirmation code for each system, so they will automatically reactivate on subsequent reboots.

Here is the configuration process, straight from Citrix. First of all, the Imaging Wizard allows you to choose which activation method you’re going to use:

PVS Imaging Wizard

Choosing the Activation Method

Once you’ve chosen either KMS or MAK, here are the next steps:

KMS Activation

  • Reset the activation status on the vDisk image:
    • Boot the master target device from vDisk in Private Image mode
    • Run slmgr.vbs -rearm in console on master target device
    • Shut-down master target device
  • Put disk in Standard Image mode and stream. Target devices will automatically register with KMS server, and activate (provided there are at least 25 systems checking in).

MAK Activation

  • Put disk in Standard Image mode and stream.
  • Use “Manage MAK Activations” to remotely activate streamed target devices. This is done only once per group of devices.
  • Provisioning Services will cache activation confirmation code for each device so that devices will automatically reactivate on subsequent reboots.

Kudos to the Citrix PVS development team for getting this done and out the door. Great job!

More on Provisioning Services and KMS

Last fall, we posted about Citrix Provisioning Services and Microsoft KMS activation. To briefly recap, here’s the issue:

  • When you convert a Windows 7 OS image to a shared image for provisioning, it breaks the Microsoft license key.
  • The way you deal with that is to use Microsoft’s Key Management Services (KMS) to auto-activate systems as they boot.
  • A KMS server must have a minimum number of systems checking in for activation before it will activate anything (5 different server systems must check in before it will begin activating servers, and an aggregate of 25 servers and/or workstations must check in before it will begin activating workstations.)
  • If your KMS server is running on Windows Server 2008 R2, both physical and virtual systems will increment the counter. If it’s running on an earlier server version, only physical systems will increment the counter.

In the comment thread of that earlier post, “Chris” stated that he was trying to use Provisioning Server to provision Windows 7 systems, but that they were not incrementing the counter on the KMS server. It turns out that he was absolutely right, and I thought this was important enough to bump the issue by writing another post rather than just going back and commenting on the older one.

It turns out that, although Provisioning Server changes the host name as systems boot, it does not change the machine ID (“CMID”). And, unfortunately, the CMID is what a KMS server looks at to determine whether a machine that’s checking in is a new one that hasn’t previously checked in. Therefore, all of your provisioned Windows 7 systems will look to the KMS server like the same system checking in over and over again, and will not continue to increment the threshold counter.

According to a blog post by Thomas Koetzing a couple of weeks ago, Citrix has told him that this will be fixed in the next release of Provisioning Services, scheduled for sometime in Q4.

Frankly, I’m pretty disappointed by this whole issue. Windows 7 has been out now for almost a year. The big push by both Citrix and Microsoft is that XenDesktop is a great way to roll out Windows 7. Provisioning Services is a must for any significant VDI deployment, because otherwise you eat up far too much of your expensive SAN storage. But yet we’re still stuck in a situation where we can’t use Provisioning Services to provision Windows 7 unless we have at least 25 physical systems checking in with our KMS server for activation. In my opinion, there is no excuse for this issue not being addressed long ago…particularly when it’s been a known issue since the release of Windows Vista.

I did find a workaround described by Kirk Kosinski in a Citrix forum post:

What I did was create a VM with VL media, sysprep and power off, convert to a template, then deploy the template 25 times and boot each VM once (a few required a reboot before contacting the KMS for whatever reason). My KMS server could then activate clients successfully, at least for a while… the activation count will decrease over time if the machine doesn’t contact the KMS server, so you will periodically need to redo this process.

The VMs don’t have to join the domain to activate so you don’t need a complicated sysprep script, just make sure to not include any license key in the script…

This strikes me as a bit of a pain, particularly when you’ve got to do it every six months or so to keep your systems alive, but it should at least work until Citrix and Microsoft get this sorted out.

Five Cool Products from Synergy 2010

As many readers know, I spent last week attending back-to-back Citrix conferences in San Francisco. Monday and Tuesday (“Summit”) was for Citrix Partners, Wednesday through Friday (“Synergy”) was for the larger user community. In the coming days, I expect to be writing a lot about stuff I learned there – to the extent that I can without violating the Non-Disclosure Agreement that all attendees agree to as part of the registration process.

Today’s post is about five cool products that I think are worthy of further investigation. I should stress that, aside from Wyse, we do not currently sell any of these vendors’ products, and we may or may not partner with them in the future. So this should not be interpreted as an endorsement other than to say that these products intrigued me and I believe them to be worth looking into.

Wyse XenithTM “Zero Client”
Finally, a non-Windows-based thin-client device with HDX MediaStream video support! I can hardly wait for us to get our hands on one of these for testing. Up until now, if you wanted high performance video, you needed to buy a Windows-embedded thin-client, and install the same Citrix Receiver and plug-ins that you would install on a full-blown desktop PC. And, unfortunately, a Windows-embedded thin-client can easily cost as much as a low-end PC. While I don’t have firm cost numbers yet, I was told it would be “sub-$300” (which I assume to mean $299).

At the Wyse demo, they plugged in the box, turned it on, it auto-discovered the XenDesktop infrastructure and automatically configured itself accordingly, and was ready to use literally in a few seconds. Wow.

Kaviza’s “VDI-In-a-Box”
[Editor's note: Since this post was written, Kaviza was purchased by Citrix, and is now the Citrix "VDI-In-A-Box" product.]

Kaviza has an intriguing product. It won the “Best of Synergy” award in the “Business Efficiency” category. As the product name implies, they make a virtual appliance that handles the provisioning, load-balancing, and management of virtual desktops in a single package. Their original appliance was designed to run on VMware, but the Beta of v3.0 they were showing at Synergy will run on XenServer. They do not require shared storage (i.e., a SAN), or a separate connection broker. When you add more of their appliances, their “grid” automatically reconfigures itself to incorporate the new appliances, replicating desktop template images as required.

They’re positioning this as an SMB solution – up to a couple hundred desktops. If you’re going to grow beyond that, you’re probably going to want the greater storage efficiency of storing your desktop images on a SAN and using the provisioning services of XenDesktop 4. Also, this is specifically a VDI solution, by which I mean a bunch of virtual PCs running on one or more virtualization hosts. As we’ve discussed in other posts, VDI is only one kind of desktop virtualization. If you want the flexibility of being able to leverage all the different kinds of desktop virtualization, XenDesktop gives you that flexibility.

Suggested list price is $125 per concurrent user. Citrix has a VDI-only version of XenDesktop (which does include provisioning services, but does not include any other form of desktop virtualization) which lists for $95 per named user, or $195 per concurrent user. So, taking into account the cost savings from reducing the back-end infrastructure requirements, Kaviza is certainly competitive for smaller deployments, if you’re looking for strictly a VDI solution. Kavisa estimates that, including the virtualization hosts, you’re still under $500/user.

Interestingly enough, Citrix recently made a “strategic investment” in Kaviza, and has licensed their HDX high-performance video technology to them. This suggests that, at some level, Citrix does not necessarily view Kaviza as a competitive threat to XenDesktop 4.

You can view a demo of an earlier version of Kaviza on Brian Madden TV, or go right to the source and sign up for a Webinar on their upcoming v3.0 release.

App-DNA
[Editor's note: Since this post was written, Citrix purchased this product. So they obviously thought it was pretty cool, too!]

Good Lord, if we’d only had a tool like this a few years ago. Several years ago, we worked with a major financial institution that will remain nameless (you know who you are) to build an infrastructure of what was then called Presentation Server that would serve up roughly 300 different applications to roughly 1,000 users. Application Isolation wasn’t available at the time, so we had to do things the hard way. We had a team of several engineers who spent months on application compatibility testing – not only to see which apps would run in a Presentation Server environment, but to see which apps could co-exist in a single server image. It was a huge project, and cost the customer a very large pile of money.

The App-DNA AppTitudeTM software automates the process of application compatibility testing. You give it access to the installation packages of your applications, and it will tell you which Windows desktop and/or server Operating Systems they are compatible with, whether they’re 64-bit compatible, and whether you should be able to package and stream them with XenApp’s app streaming tool or with Microsoft’s App-V. Moreover, if there’s an issue with an application, it tells you what the issue is and makes suggestions as to how you may be able to remediate it!

This product won the “Best in Show” award at Synergy, as well as winning in the “Process Improvement” category. The people I talked to couldn’t give me pricing, but if you’re looking at a major upgrade or migration that involves a lot of applications, this could be a huge time-saver.

Liquidware Labs
Their Stratusphere FitTM product was a Best of Synergy finalist in the “Business Efficiency” category (the category that was won by Kaviza). This is a VDI assessment tool. It will monitor and log a bunch of desktop OS and user performance metrics, looking at network usage, application usage, disk and memory utilization, graphics intensity, disk IOPS, network latency between the current desktop location and the data center you’re hoping to move it to, etc.

After gathering information for a while (a minimum of two weeks is recommended), it will spit out both detail and summary reports that will identify good, fair, and poor candidates for virtualization, identify potential problem areas, and help you size the back-end infrastructure that will be needed to host all of the newly-virtualized desktops.

The cost of a time-limited license (90 days, if memory serves me correctly) is roughly $7 per user. Look at it this way: You can design your VDI hosting environment by the seat of your pants, and probably end up either over- or under-building the infrastructure, or you can spend a little bit of money to develop some hard data to guide the design decisions. If it helps you avoid design mistakes, and helps insure the success of your VDI project, that’s probably money well spent.

Unidesk
The Unidesk product competes directly with the provisioning services component of XenDesktop 4. Why, you may ask, would you want to pay extra for a third party product instead of using the provisioning functionality that comes with all versions of XenDesktop 4? Here are some possible reasons:

  • Unidesk integrates patching and version management into their provisioning tool.
  • Unidesk can deliver boot-time drivers such as antivirus software, VPN software, and printer drivers as components that are separate from your master OS image.
  • Unidesk integrates application management into their provisioning tool, including applications that have been packaged for streaming via XenApp, App-V, or ThinApp.
  • The big one: Unidesk treats user-installed applications as part of “user personalization” – yes, you can provision from a single master OS image and still allow users to install their own apps. (And you can also – relatively easily – repair the damage when a user installs an app that breaks something else.)

In some organizations, user acceptance will make or break a desktop virtualization project. In a native XenDesktop 4 deployment, if you want to allow the user to install applications, you have to dedicate an OS image to that user. If this is a requirement for a lot of your users, you’re going to burn up a lot of expensive SAN storage. If internal company politics will allow you to lock down the corporate desktop, great! Your life will be much easier. And, as we’ve observed elsewhere, XenClient promises to address this by giving the user multiple desktops: a corporate desktop that’s locked down, and a personal desktop where they can install their own applications. But if you are forced, for whatever reason, to allow your users to install their own applications on top of the corporate desktop image, Unidesk could save you a bunch of storage space, and maybe even your sanity.