“I wanted to remind everyone, of what I have already seen floating around the internet, but still important enough to remind.  Our next release of SRM is going to require a 64 bit OS.  This is the same as our next release of VC as it too will require a 64 bit host OS.  This change is required to support the increased capabilities of our products.  As we scale our products to match our customers needs, generally 1 – 2 years in advance of where they will need all the capabilities of a given product we have had to use a 64 bit OS.  This will show itself in increased numbers in things like more simultaneous vSphere client connections.”

To me these new operating system (OS) requirements mean we will see even more instances of vCenter as a VM (virtual machine). It only seems logical that a least path of resistance is to virtualize the management server in order to upgrade, especially considering all have already invested in 64 bit hardware for their hypervisors if they decided to upgrade to vSphere 4 in the first place. To go a step further, I’m willing to argue that it will be more common for an IT Department to justify the cost of additional ESX hosts, even if only dedicated for management, then it will to deploy new servers for physical instances of vCenter.

The looming transition to a console-less ESXi eventually means more management virtual appliances in the future too. Solutions which will continue to need a ESX console or similar will have to substitute their own appliance to operate with ESXi. This means even more justification for additional ESX/ESXi hosts and thus greases the decision to virtualize vCenter as well. I expect to see management clusters of ESX hosts become more common in the future than even the use of management networks today. ESX hosts have bigger and badder hardware now than ever before allowing for higher consolidation ratios and larger applications to easily run in virtual machines, but it will be interesting to see if the vCenter as a VM best practices change over time. I personally feel that continuing to separate the database from the virtualized vCenter will continue to be a smart choice. Running a separate, and even virtualized, SQL instance ensures not only better performance of vCenter as a VM but enhances DR scenarios. In fact, those that already have the vCenter database on a remote instance will likely have a safer upgrade to the 64 bit vCenter. The new 64 bit requirements will no doubt make for an interesting migration scenario, and I’m sure we will see some positive and negative opinions. Let me know your thoughts on a 64 bit vCenter as a VM in the future!

© Rich for Gestalt IT, 2010. | Future vCenter And SRM Requirement For 64 bit OS Means More vCenter VMs
Read more posts categorized as Server Virtualization

Here’s the VMware Communities: Survey for SRM customers – please … original post from May 27:

Hello SRM users, The VMware vCenter Site Recovery Manager (SRM) product team is looking for product feedback on SRM deployments. If you have purchased SRM, we would like to hear from you. Your participation will be very valuable to us and the information you provide will be used to improve the SRM product going forward. You can provide your feedback by completing the survey The survey should take no longer than 15 minutes and will expire on June 10, 2010. Please note that this survey is for SRM customers only. Upon completion of the survey, if you are among the 1st 1000 respondents, VMware will donate $10 per response to charity. You will also receive a link to download the electronic copy of Mike Laverick’s book “Administering VMware Site Recovery Manager 4.0” upon completion of the survey. We appreciate you taking the time to provide us with your valuable feedback. Thank you, The VMware SRM Team

Check out Laverick’s post on this survey as well: SRM Customer? Complete this survey, and VMware will donate $10 to UNICEF

© Rich for Gestalt IT, 2010. | Take VMware SRM Survey, Get a Free Copy of Laverick’s Book, and Donate $10 to UNICEF
Read more posts categorized as Featured, Server Virtualization, Top Story

With EMC’s Mirrorview/S (sync) there is a distance limitation of between 50km and 200km depending on what fibre optics you are using (short/long wave vs. dwdm).  Mirrorview/A (async) is more widely used over an IP WAN connection but can also be used over fibre as well.

Mirrorview/S

Pros:

• Synchronous – Exact copy of data on production
• Little to no data lost

Cons:

• Distance limited (60km using short wave gbics, long wave gbics or optical extenders, 200km using dense wave division multiplexors)
• WAN link more expensive (fibre vs. copper/ip) unless Fibre Channel over IP converters are used and those are still a little expensive

Mirrorvew/A

Pros:

• Cheaper WAN link between sites (IP usually)
• Writes to prod don’t have to wait on mirror site to write
• Not distance limited like sync replication

Cons:

• Data can be lost depending on write intervals from prod to DR site

What you need to know

Array-based mirroring is a great way to protect multiple hosts in an environment instead of buying per-server or per-application replication.  As I’ve discussed before, the biggest drawback is that it provides a restartable copy which isn’t the same as an active-active cluster application transaction-level replication (i.e. Oracle Dataguard, Exchange CCR, MySQL Master/Slave replication).  Be careful of adequate LAN/WAN line quality, poor comm lines can cause insanely painful headaches (troubleshooting, added latency, etc).  Get line tests done to determine available bandwidth, line quality and latency.

© edsai for Gestalt IT, 2009. | Sync or Async Replication?
Read more posts categorized as All, Storage

CC-by-SA 2.5 image "Golf Bunker" by Ken123

Today, many companies are considering a compete migration from a physical to a virtual infrastructure. Based on the promises of cost savings, administrative efficiencies, and improved resource utilization, virtualization is seen as the technology that makes it possible to do more with less. However, there are many pitfalls to consider when virtualizing server infrastructure. This article suggests planning decisions to be considered by the CIO, IT Director, design architect, and IT Manager that, if overlooked, could ruin the total cost of ownership (TCO) and the return on on investment (ROI) expected from this virtual infrastructure.
�

Migration pitfalls

Implementing virtualization is not a simple process of building a server. Too often, management makes the mistake of expecting that the administrators and architects of the server team will be solely responsible for virtualization decisions. In reality, the impact of a move to virtualization should be viewed exactly as if a company were moving their physical servers to a new data center. Storage, networking, security, Active Directory, messaging, web presence, and all of the systems and services necessary to the organization must be considered together. Some of these groups should be more involved than others in the actual planning and ultimate design decisions, but virtualization strategy affects everyone. Key members from all of these focus areas must be trained and given panning responsibilities to achieve a collaborative plan and design.

A potentially larger “gotcha” once everyone is involved in the planning process is the temptation to include all of the upgrade and service redesign projects that have been “in the queue” for months (or longer). Although the migration may seem like a great opportunity to upgrade to the latest operating system, separate instances from poorly performing database servers, or build the new CRM system, these added activities bring complexity and could pose road blocks to the core goal of the virtualization initiative: consolidating hardware. These projects should be tabled and taken on later, after the new infrastructure is in place. In fact, the flexibility of a virtual infrastructure will actually make these projects simpler.

Performance pitfalls

Condensation of numerous physical servers into a virtual environment is intended to increase the effective utilization of processing power and other system resources, but it is easy to overshoot the target and overload the new system. Although the average load of a number of servers might be within the capabilities of a new virtual infrastructure, resource spikes can be a serious problem. Consider that periods of high demand tend to cluster across servers through the day and week, as employees begin the workday, the backup process begins, or a software build is kicked off. If performance is not measured and correlated carefully before migrating to a virtual infrastructure, these resource spikes can make the new infrastructure unacceptably slow.

One often overlooked performance challenge is the randomization of demands introduced by virtualization. Traditional computer design assumed that processes would proceed through their steps sequentially, with the system devoting resources from start to end. Multitasking operating systems challenged this assumption, but the allocation of memory, I/O, and storage remained clustered. But highly-utilized virtual servers upset this predictable flow, rapidly switching from one task to another entirely different one. In short, virtualization randomizes accesses that were once sequential and transforms large requests into many tiny ones. CPU manufacturers have responded with clever techniques to save and restore registers and handle memory mapping, but this blender effect is far from solved. I/O and storage systems are particularly affected, as they are optimized to stream large sequential operations rather than small random ones.

Storage pitfalls

Although one might think that virtualization reduces storage requirements as systems are consolidated, the opposite is often true. Many of the functions that help drive down CAPEX and OPEX costs in virtualized environments require the use of shared networked storage like SAN or NAS. Data that had once been scattered around the data center on internal drives is consolidated on these networked storage devices. Thousands of redundant copies of operating system files, for example, end up sitting on the storage array. The ease of creating virtual machines from templates tends to lead to virtual “server sprawl” far worse than in the physical world. Storage array capacity is also needed for swap space or paging files as well as popular enhancements like snapshots, DR copies, and backup images.

The result is a flood of redundant data that must be taken into account when planning capacity needs. Although storage is typically not the top item in an IT budget, virtualization can cause it to rise and chew up some or all of the savings from the project. It is difficult to avoid these issues, but some techniques can help mitigate the problem. For example, deduplication of primary storage and thin provisioning can help reduce the storage footprint of the virtualized infrastructure.

Administration pitfalls

Virtual sprawl can cripple infrastructure support teams if it is not managed. The challenge of administering this new virtual environment is often ignored in the implementation rush. Rather than managing 3,000 servers, many wake up and find that they are managing 30,000 virtual machines. This is partly because server build times are reduced from days to minutes, and partly because server consolidation on virtual infrastructure is so effective that idle guests are barely noticed. If left unmanaged, the “gasoline fire” spread of VMs could create a burden in the form of permissions, backups, upgrades, patches, and monitoring than what existed in the physical infrastructure.

To combat these scenarios, IT departments need to start with process and policies. Change control becomes paramount. Second in importance is a new server request and approval process. Build standards need to be created and adhered to. Finally, virtual servers must to be audited for activity and then powered off or removed if idle. The good news is that there are numerous life cycle management and automation tools now available for helping an organization provision, maintain, report, and decommission virtual machines.

Backup/Restore pitfalls

Backup is another area where TCO and ROI, as well as performance, can be negatively impacted by virtualization without appropriate planning. As mentioned above, virtual server hardware is often sized based on the average server load. For example, consider the impact of combining ten servers with average CPU utilization of just 10% onto a single physical device. The ROI would appear to be excellent, but the large spike in utilization that happens during backup can cause unacceptable performance issues or even application failures. The performance hit during backup can be mitigated through off-machine backup using like VMware VCB or storage snapshots, but the cost and effort of implementing these features must be taken into account when planning the new system. The conventional approach to backup, loading backup agents on each system and backing them up on a schedule, does not translate in the virtual world.

Disaster recovery pitfalls

One of the primary driving factors for moving to a virtualized environment is its potential positive impact on disaster recovery (DR). Although the DR benefits are real, the requirements for the storage system can become a pitfall. Extra DR data is pushed onto storage arrays and then often replicated to another location. All of this extra data requires increased storage capacity, array features, and WAN bandwidth. The capacity issues can potentially be addressed using primary storage deduplication and thin provisioning, and WAN optimization appliances can reduce bandwidth requirements.

While virtualization does make servers more portable, thus making DR easier, the disaster recovery copy process is a major challenge. Some vendors now have tools like VMware Site Recovery Manager (SRM) to assist in this orchestration, but these tools are somewhat immature, and care should be taken when planning to use them. For example, most do not address failback, so administrators may find themselves doing a lot of work to return to operation at their primary data center after a failover.

© Gestalt for Gestalt IT, 2009. | Planning for Virtual Infrastructure: Avoid the Pitfalls
Read more posts categorized as Exclusive, Featured, Server Virtualization, Top Story

TechTarget's annual Storage Product of the Year awards have been announced!

The mainstream media is still digesting the Oscar awards, but we in storage had our own announcement this week: TechTarget’s (now PDF) Storage magazine announced their Storage Products of the Year award for 2008.

Without further ado, the awards and my reaction!

Disk and disk subsystems

A category of much annual contention, the gold award went deservedly to the�BlueArc Titan 3200. BlueArc’s product range is small, with just a few systems on offer, but the 3200 was indeed a major upgrade. The bronze-placed�Hitachi Data Systems�AMS 2000�might easily have taken the top award as well, however, introducing major advancements in the midrange market. Taking silver was the news-making�Intel X25-E SATA Solid-State Drive.

The placement of disk drives in the same category as SAN and NAS storage systems, however, highlights something of a categorization problem for this award. Should Seagate, STEC, and Intel really be battling it out with the very systems that use their drives? I’d like to see separate component and array categories in the future.

Overlooked finalists in this category were EMC’s CLARiiON CX4, IBM’s XIV,�Isilon’s X-Series, 3PAR’s T-Class, and�Xiotech’s Emprise 7000. Any of these might have placed, and 3PAR, IBM, and Xiotech might have taken gold with a different set of judges. Looking at this broad list, however, highlights the difficulty in this award. BlueArc, HDS, and Intel deserved praise, but the rest should be recognized as well!

Storage management tools

VMware�takes a deserved gold award for�Storage vMotion, certainly the MVP in storage software this year if one considers adoption and impact.�Silver went to NetApp’s Deduplication�technology, the first (and still only) deduplication certified for primary storage use.�Bronze went to CloudNAS from�Nirvanix, an addition that allowed the storage service provider to offer services using standard NAS protocols.

EMC’s integration of VMware with ControlCenter 6.1 deserved recognition as well, as did Mimosa’s updated NearPoint 3.5. One really overlooked product is SANPulse Technologies‘ SANlogics 2.0 – go check it out and let me know if you think it deserved to place!

Networking equipment

If you haven’t experienced Riverbed’s gold-winning Optimization System (RiOS) 5.0, you might not understand just what the big deal is. 5.5 might win again next year, with solid updates to accelerate even tough WAN traffic like encrypted MAPI and signed SMB.

The silver award went to Brocade’s DCX Backbone, but time will tell if this is a wiser choice than rival Cisco’s updated MDS. I suppose if we compare only the updated DCX to what little Cisco did to the MDS last year, Brocade clearly gets the nod. But I might have skipped both products to let�IBM’s�updated�SAN Volume Controller (SVC)�have a place on this list. It added thin provisioning last year and remains IBM’s best storage product.

Bronze-winning�NetEx HyperIP 5.5�could easily have taken top honors – it’s a real value for the money in WAN acceleration! Compared to Riverbed on features it doesn’t look like much, but compare price quotes and see if your head doesn’t spin!

Backup and disaster recovery software and services

VMware�continued their roll with a gold medal for�Site Recovery Manager 1.0. Like Storage vMotion, SRM came out strong to widespread acclaim, and is likely to have a major impact in the coming years. A solid choice.

FalconStor�claims silver with their�Network Storage Server (NSS) 6.0�platform. I’m puzzled by the placement of this product in the backup and DR category, but what can you do? Bronze went to�Acronis Recovery for Microsoft Exchange, a product I don’t know much about. Must be good, though!

Backup hardware

Data Domain�is everywhere lately, and their�DD690 Deduplication Storage System�picked up a gold medal here. I hear that these things are selling like crazy, though, which I’m sure Data Domain is happier with than the award!�Permabit�took silver with their�Enterprise Archive Data Center Series. Lots of folks who know more about the backup space than I have had good things to say about it, as well as the bronze-winning�Quantum DXi7500.

Although backup isn’t my thing, I wanted to point out that�Cleversafe deserved mention with their, well, clever�Dispersed Storage Network.

© Stephen Foskett for Gestalt IT, 2009. | Reacting to the 2008 Storage Products of the Year
Read more posts categorized as All, Exclusive, Featured, Storage, Top Story

Firstly, since then I have moved jobs and no longer have responsibility for day-to-day storage management, I do obviously keep an eye on things but unfortunately it means that I cannot really influence how SRM develops in my organisation. This is a bit sad as actually I had a very positive response from EMC who took the brunt of my diatribe and I have been unable to take them up on their offer to work with me on understanding what I would like to see from a SRM product. I do hope that someone in the organisation I work for does take them up and help them understand what a large storage end-user requires and where the issues are.

Anyway, I thought I would put some thoughts together about the challenges that SRM tools or at least pose some questions.

Is the problem one of scale and complexity? If you look at what we expect the SRM tool to do, we expect currently expect it to understand our storage environment end-to-end. So look at what an SRM tool needs to do.

1. It needs to understand the array and how that is configured – easy
2. It needs to understand the switch fabric – fairly easy
3. It needs to understand the IP fabric – moderately hard
4. It needs to understand the hosts/servers, including virtual – moderately hard
5. It needs to understand the applications – hard
6. It needs to be able to correlate all the above information into useable and consistent model – potentially very hard

So to be fair to the SRM vendors, what they are trying to do is non-trivial and we the end-users don’t always make their jobs easy. We have a duty to ensure that organisational standards are set, adhered to and maintained otherwise the data consistency checking becomes horrible. We have to give them a chance.

Do we want an end-to-end management tool which allows us to understand our whole IT infrastructure because the relationship between storage and data is intrinsically linked?

What do we actually want from a SRM tool which will make it useful to us so that we do not carry on cursing the vendors and righting our own scripts? Perhaps we should hand over the contents of our individual script/tools directories and say, we want a tool which does all this and does it reliably. Perhaps the SRM vendors should send out an investigatory team wearing red shirts to discover what the storage civilisations are up to?

We can probably say that we don’t want ECC and its ilk; perhaps SanScreen is closer to what we want. I suspect that is very much the case; we do not want an all-singing, all-dancing provisioning/configuration tool but we do want something which gives us an immediate view of our storage environment and allows us to drill down the layers into the individual components getting performance, capacity and configuration details? It would be incredibly useful if it understood the reality which is a heterogenous storage environment with SAN, NAS and in future Object/Cloud.

And vendors if you continue to expand the number of different storage families in your product range and do not standardise on your management APIs, interfaces etc, you are making your job harder. And even in a product family, 37 varieties of LUN are not making your job any easier. As part of the development track of any new feature; the question needs to be asked, how will this be managed and the question needs to be asked early in the development cycle.

So what do you want your SRM tool to do?

© Martin for Gestalt IT, 2009. | Extreme Cash Cow – Redux
Read more posts categorized as All, Storage

I am not suggesting that SRM is a sham. In fact, it provides automation of virtual infrastructure fail over between sites that is truly wizard-like. Understand however, VMware SRM software is just the last piece of the total data center recovery “machine”. Many organizations may be seeking the semblance of automated site fail over, but have they really considered in detail what it takes to start up their business critical systems at a secondary location?

A simple determination of readiness for SRM’s wizardry is answering this question: “Can you create (or have you already created) a document listing the complete shut down and start up of your business infrastructure?” Many call this a disaster recovery playbook or runbook. Better yet, have you provisioned and tested the physical resources you need to actually fail over to another location based on the runbook that was created? If and when that’s done, there are numerous business continuity technologies considered other than SRM. Those that have already realize that SRM along with consolidation to virtual infrastructure will replace several sections of their runbook and several pieces of hardware. The point, however, is that SRM does not replace the contents of the entire runbook.

For those that are considering SRM, take the time to at least put on paper every possible step you would need to restart your business in another data center. This includes the mundane like providing power and cooling, racks, internet access, and telephones as well as the mission critical like employee access, email, and business processing. Analyze how much time it would take to rebuild and restart systems, and what recovery point objectives (RPO) and recovery time objectives (RTO) are acceptable for each system or service. When it’s time to map your disaster site fail over solution to the runbook you will clearly see the efficiencies and the speed that VI 3.5 and SRM allow.

Therefore, unlike the line from The Wizard of Oz, you better pay close attention to the man and machine behind the curtain in order to achieve the prodigious expectation and pyrotechnic results of a SRM implementation.

© Rich for Gestalt IT, 2009. | Implementing VMware SRM: Pay Attention to that Man Behind the Curtain
Read more posts categorized as All, Server Virtualization, Top Story