I recently came across a server that did not have iostat available and I needed estimate when a certain file copy would complete. In order to make a rough estimate I wrote the following one-liner to summarize the available space in the current working directory, wait a minute and then sum again, subtracting the smaller value from the larger value to determine how many megabytes per minute.
while :; do CUR=$(du -ms | awk {'print $1'}) ; sleep 60; NEXT=$(du -ms | awk {'print $1'}) ; echo $(( $NEXT - $CUR)) ; done
You could reduce the sleep value to 1 show megs per second but you will probably reduce the speed of your copy. Sixty seconds seemed like a good medium and makes it easy to estimate. Generally I can wait about 5 minutes and pretty accurately generate an expected time to completion. I'm sure there's a way to do it with IFS and have bash do the splitting but it was quicker for me to just use Awk.
Doing encryption on the fly, on an entire filesystem for instance can be a very intensive process and bound by the CPU's ability to decrypt or encrypt during reads and writes. I've run into just such a situation where disks are idle awaiting data to be encrypted. I encountered disk performance of roughly 300M/s writes on an unencrypted volume and roughly 100M/s when encryption was enabled. Not to mention a nearly %50 decrease in IOPS when using iometer for performance testing. Fortunately, with the correct hardware there is hope for a major improvement in performance by way of Intel's Westmere CPU hardware accelerated AES instruction set called AES-NI.
The instructions were designed to implement some of the complex and performance intensive steps of the AES algorithm using hardware and thus accelerating the execution of the AES algorithms. AES-NI can be used to accelerate the performance of an implementation of AES by 3 to 10x over a completely software implementation.
The AES-NI ecosystem is growing and patches exist for openssl among other tools and I would expect more support to be available soon.
If you are using existing crypto libraries that provide the crypto functionalities including AES, all you need to do is recompile your applications to include the latest libraries.
ESX 4.1 (update n) will be the last installable version of ESX to retain the service console. There is no upgrade path for ESX to ESXi, you must reinstall to migrate or upgrade. Upgrading is a great reason to consider modernizing your virtualization infrastructure. VMWare currently recommends that new deployments of vSphere 4.x are done on ESXi and that existing ESX deployments of vSphere 4.x or older are migrated to ESXi.
What is the primary difference between VMware ESX and VMware ESXi?
The service console has been removed from ESXi, drastically reducing the hypervisor footprint.
ESX
VMware agents run in Console OS.
Nearly all other management functionality provided by agents running in the Console OS.
Users can log into Console OS in order to run commands for configuration and diagnostics.
ESXi
VMware agents ported to run directly on VMkernel.
Authorized 3rd party modules can also run in VMkernel These provide specific functionality, like hardware monitoring or drivers.
VMware components and third party components can be updated independently.
No other arbitrary code is allowed on the system.
Why is it as good or better than ESX?
Improved Reliability and Security — fewer lines of code and independence from general purpose OS, ESXi drastically reduces the risk of bugs or security vulnerabilities and makes it easier to secure your hypervisor layer.
Streamlined Deployment and Configuration — ESXi has fewer configuration items than ESX, greatly simplifying deployment and configuration and making it easier to maintain consistency.
Simplified Hypervisor Patching and Updating — Due to its smaller size and fewer components, ESXi requires far fewer patches than ESX, shortening service windows and reducing security vulnerabilities.
AD Integration — The ability to configure the host to join an Active Directory domain, and any user trying to access the host will automatically be authenticated against the centralized user directory. You can also have local users defined and managed on a host-by-host basis and configured using the vSphere Client, vCLI, or PowerCLI. This second method can be used either in place of, or in addition to, the Active Directory integration.
Scripted Installation. New to ESXi 4.1 is the ability to do a scripted installation of the ESXi software to the local disk of a server. Various deployment methods are supported, including booting the ESXi installer off a CD or over PXE, and accessing the configuration file over the network using a variety of protocols, such as secure HTTP. These scripts run locally on the ESXi host, and can perform various tasks such as configuring the host’s virtual networking and joining it to vCenter Server.
Boot from SAN support for ESXi. Support for Boot from SAN has been added to ESXi 4.1. This support includes Fibre Channel SAN, as well as iSCSI and FCoE for certain storage adapters that have been qualified for this capability.
How does one upgrade?
The simple answer is that you cannot upgrade from ESX to ESXi. ESXi requires a new installation, which requires you to reconfigure a server to the desired state. There are a few ways to reduce the amount of time you spend migrating. For instance, using host profiles to leverage common configuration settings across your environment, combined with distributed virtual switches to make configuration of host networking much easier. These configurations settings can be automated to save lots of time when provisioning new ESXi servers.
Since you'll be reinstalling, you might as well consider a standard deployment method, like VMWare Auto Deploy or PXE Manager. This is a technical preview of what will likely become a future product, or productized enhancement for ESXi / vCenter. Ideally you could use this system to run servers which had no operating system on physical media. Meaning that when they reboot the state of the machine is lost. Simply bootstrap and download as a part of the boot process for your physical ESXi server and load the appropriate configuration for that host. You can now grow or shrink your clusters at will.
A stack consisting of the following would be a good start towards a fully managed vSphere infrastructure.
Licensing works just about the same as it does for ESX. If you have an existing ESXi infrastructure and you simply want to license your installations (ESXi) can be seamlessly upgraded to more advanced editions of vSphere. Simply upgrade the free license to the desired license type and take advantage of all the features.
Monitoring
The Common Information Model (CIM) is an open standard that defines a framework for agent-less, standards-based monitoring of hardware resources for ESXi. This framework consists of a CIM object manager, often called a CIM broker, and a set of CIM providers. Any software tool that understands one of these APIs, such as HP SIM or Dell OpenManage, can read this information and hence monitor the hardware of the ESXi host. ESXi also exposes hardware status information via SNMP for other management tools that rely upon that standard. SNMP Traps are available from both the ESXi host and vCenter.
Backup
The majority of systems management and back up vendors in the VMware ecosystem support ESXi today. Partners such as BMC, CA, HP, IBM, EMC, NetIQ, Quest Software, Commvault, Vizioncore, Double-Take Software, SteelEye, and Symantec are among the many partners that have systems management or back up products that support ESXi.
How do I manage it?
Management and integration points have been moved from the individual servers to vSphere Management Assistant and PowerCLI. This means that if you've got an extensive set of scripts hooking directly into vimsh or other features of the console, you'll probably need to spend some time evaluating how long it will take you to port your scripts to vMA or PowerCLI.
