The vCurve

The first Virtualization Query Language was officially born around February of 2008 and was first released in the summer of 2009. VQL provides data awareness of the IT environment by easily surfacing information from different data sources such as the VI Java library produced by Steve Jin. Since VQL was first developed it has been getting lots of new objects added to the library but the grammar has not changed in any major way…until now. Today we are announcing some pretty major changes to VQL that we are very excited about. The new capabilities include functions, new objects, and real-time query processing.

Functions!

As an object pipeline VQL was great as a classifier but we realized that objects can be somewhat difficult when you want to perform analytics on data so we introduced a generic object type that makes the production of analytical data possible (think mapping objects to spreadsheets). Some of the new built-in VQL functions are the usual suspects like aggregates (sum,count,avg,min,max) but some of the other functions are also nice like top(), math(), density(), and select() which suddenly allows the VQL query engine to produce partial objects which makes transferring VQL objects over a network rather zippy.

New Objects

• Some of the new objects joining the team include performance metrics, datastore mounts, vCPU, and VQL queries themselves are now VQL objects. Why make VQL queries objects? Well one of the new functions introduced is called QueryOutput() which takes the name of a VQL query as a parameter. This capability allows VQL queries to be chained together in a stream by reference so changing the output of one query does not effect the definition of another query. The payoff here is performance. The ability to chain trees of queries together at the application level eliminates complexity and adds a lot of performance gains.

Real-Time Query Processing

One of the most disruptive new capabilities in VQL 2.0 is the addition of soft real-time processing of VQL queries. Processing VQL data in real-time means you can instantaneously see extremely sophisticated information about your virtual environment. One of the most straightforward applications of this technology is reporting. Everyone is used to running a report and waiting some time before viewing the report. Usually the more useful the report is the longer it takes to run. And if any of the data that makes up that report changes, the entire report has to be computed again. What the Reflex real time processing engine offers is the ability to recompute only the portion of that report that may have changed based on the new data. This type of data computation provides a double-whammy of utility. The segmented/streamed processing can produce sophisticated data instantly and the computation of that data actually takes less overall CPU cycles than computing it using standard database techniques.

This type of technology can be applied to not only reports but any type of complex data computation. Some examples of how real-time data computation can be used:

• Instantly understanding complex forecasting of resource usage and supply
• Instantly reacting to new environmental data and instituting a modified security policy based on the new information
• Alerting when the performance of an application might be suffering due to resource constraints or new load placed on the application
• Instantly adapting the load balancing of resource demand across resource surplus that may exist for a forecasted amount of time

Real-Time Intelligence

The previous examples provide some insight into the possibilities of real-time data processing but another great example is using real-time processing as a component itself to produce new intelligence that then higher level decisions can be based off of. Today if you wanted to find out which virtual machines in your environment are oversized or undersized you would buy a product that produces that type of report or you could write some scripting logic to produce that information yourself. The time needed to compute that type of information can be anywhere from a few seconds to a few minutes depending on the size of your environment. Now imagine being able to constantly compute that data within milliseconds no matter how large your environment is. You then could produce new metrics that record that data so now you can know at any point of time in the past how undersized or oversized a VM was and even graph those trends over time. That being said, you could envision the software you would need to produce that type of intelligence. The real innovation that VQL 2.0 provides is making that type of incredibly complex processing to occur with a single VQL query specification.

Another way to understand the new real-time processing capabilities of VQL is to think of constantly computing the output of a Powershell script so that if any of the data that the Powershell script queries for changes that the output of the Powershell script instantly changes.

You might be asking how this technology is different from some other software that exists out in the world. A lot of real-time processing engines that exist can only process data in a very specific form and can only produce data for a very specific output. Since VQL is a graph based language the new Reflex real-time processing engine is one of the first graph based complex event processing systems that can analyze any data which means if you have Key/Value based data with relationships then it can probably be packaged into a VQL object, historically persisted, and analyzed in real time.

For more information please visit the VQL section of the Reflex Website.

written by Aaron Bawcom

On April 12th McAfee and Reflex announced a new product integration. Since then I have received many requests for clarification on what the relationship means and how it differs from other offerings and previous Reflex partnerships.

Let me first start with a little about Reflex’s philosophy on integration and partnerships. Our goal is to be the go-to company for integrated virtualization management and security. But that does not mean that we believe that all the technology that goes into the solution will be home grown.  In some cases there are subject matter experts that are far  more knowledgeable than Reflex and thus are better equipped to solve specific problems.  Intrusion detection and prevention (IDP) is one of these areas.  IDP is more than just building software to inspect network packets, it also needs to be backed up by a team of security researchers that provide the content, or signatures, for the scanning software that make it effective.  It is for this reason that Reflex prefers to integrate with the top vendors in the IDP space and thus the McAfee relationship.
Continue reading »

written by Mike Wronski

If your virtualization environment has snapshots growing like weeds in the yard then you are not alone. The more snapshots that exist for longer periods of time degrades the performance of the virtual machine the snapshots are on. To further illustrate the flexibility of the automation engine in the Reflex VMC we will illustrate a real world example of addressing this problem. This example dives deeper into the concept of an Action. An Action is at minimum a script and can also include a VQL query. An action can either run a script either on a periodic basis or whenever the output of a VQL query changes. We will illustrate an Action that uses the output of a VQL query in a subsequent post. For now, we will take a look at Actions that are run on a periodic basis. First lets describe what we want to accomplish with this policy:

1. Send an e-mail to the owner of a VM and the IT Admin when they have reached X snapshots and the image is not marked as an exception
2. Send an e-mail to the owner, IT Admin and the Group Admin when they have reached X snapshots, the image is not marked as an exception and the X snapshot condition has lasted more than Y days
3. To make sure virtual machines have the proper data on them, Query for all machines that have no owner set and tell the Virtualization Architect the name of the machine and what functional group the VM is in if any

Continue reading »

written by Aaron Bawcom \\ tags: automation, python, scripting, sprawl, virtualization, VQL

A very big feature that was added to the Reflex VMC in version 2.9 that was released in the fall was the addition of VMC PlugIns. This feature allows for users of Reflex to dynamically add features to the product without the necessity of a new installation from Reflex. The new PlugIn capabilities offer the ability to:

1. 1. Add new Right Click capabilities to any object
   2. Add new strongly typed SOAP Services
   3. Add new Policies that are constantly enforced

 
Continue reading »

written by Aaron Bawcom \\ tags: python, scripting, vijava

Many people ask me, “How do I ensure that the network segmentation is happening and how can I prove that my systems are in compliance?”. This is a difficult thing for people to understand, in the virtual environment, but it is also an easy thing to answer. With Reflex’s vTrust, part of the VMC, you can quickly and easily see which policies are associated with a particular VM, or set of VM’s, as well as view all ACL’s for that VM(s). This not only shows what is allowed or denied from a high level policy view, but also with the low level details on the physical access to and from the systems.

After you have defined what policies and rules are desired for the VM(s) the VMC and vTrust automatically create the access rules and also update them if and when they might need to be updated.

There are a few major components in the VMC that help to create, manage, and ensure network segmentation for VM’s in the virtual infrastructure. These components are:
Continue reading »

written by Tommy Speigner

The scenario is this, you want to create a Host profile with vProfile and then have that profile applied to all new servers as they come online. We will use the VMware Host Hardening Guidelines as our example vProfile. We will then schedule this for all the existing hosts and then use this for any new hosts that come online.

Let’s start by using the default ESX_DISA_SECURITY Profile that comes with vProfile.

1. Launch Configuration Management, in the VMC.

   

2. Select Host > Profiles
   1. Bind a Profile to a Host > Click Profiles

      

   2. Select the desired Profile.

      

   3. Now create the Host Binding:

      

   Continue reading »

written by Tommy Speigner \\ tags: Configuration management, Policy, policy enforcement, Reflex VMC, vProfile

What happens when there is a disaster and you have to get your infrastructure back up and running as quickly as possible?   Let’s assume that we are on the same page and talking about VMware, especially ESX hosts. Most, if not all organizations already have an existing Disaster Recovery plan in place – but how many of those plans include these new hosts running virtualization?  Do you have a plan to get them back up and running – EXACTLY HOW THEY WERE – in a matter of minutes?  Let me tell you how you can do it with vProfile.

The scenario includes three locations— could be two, but I like odd numbers, so three it is. Atlanta is our HQ, Dallas is our HQ2, and Denver is our DR site. As we have witnessed in the past there are number of man-made or natural disasters that interrupt services. This time Atlanta is hit with another monster storm, with a monster tornado and wipes out our Atlanta HQ site. Using the Reflex vProfile we can go back to the VMC, extract the properties and set values from the downed servers, and then apply those properties and values to our newly built (or already running standby) ESX hosts, which reside in our St. Louis DR site.

Continue reading »

written by Tommy Speigner \\ tags: compliance, Configuration management, Policy, VMC management, vmware, vProfile

In my previous post I talked about what PVLANs are and how to create secured PVLANs utilizing the Reflex VMC vTrust technology.  After reviewing the post I realized that some might ask for more in-depth understanding of how we accomplish this, why this is different from doing it in the past, and what other options are available for creating these PVLANs.  PVLANs have two major type classifications: isolated and community.  Basically, what happens in an Isolated PVLAN is that hosts can talk out of or receive communication directly to the VM but the other VMs that are part of the defined Zone have no intra communication.  Whereas, community PVLANs allow intra and inter communications between outside hosts and internal zone VMs. 

Why would you want to have a community zone?
Continue reading »

written by Tommy Speigner \\ tags: Policy, Reflex VMC, segmentation, VLANs, VQL, vTrust, Zones

Virtualized PVLANs

We have a dilemma…  A customer wants to create a PVLANs (Private Virtual LANs) in their virtual environment.  If you aren’t familiar with PVLANs, read what Cisco says they are: “A PVLAN is a VLAN with configuration for Layer 2 isolation from other ports within the same broadcast domain or subnet. You can assign a specific set of ports within a PVLAN and thereby control access among the ports at Layer 2.” With certain Cisco switches you can set up PVLANs to allow or deny the traffic between those two hosts in the same subnet.

Continue reading »

written by Tommy Speigner \\ tags: Policy, Reflex VMC, segmentation, VLANs, VQL, vTrust, Zones

In mid 2008 our first major venture into Virtualization Management was a set of features around discovery, visualization, and monitoring of the virtual infrastructure. That set of features has now been bundled into a bundle called  vWatch. In Mid 2009, we then introduced a set of features around securing the virtual infrastructure called vTrust. Since then we’ve been busy building the next major component of the End-to-End Reflex VMC Management platform was announced today and is called vProfile.

vProfile provides a set of User Interface and System components on top of a single tightly integrated Virtualization CMDB framework that significantly improves the ability to visualize, manage, and control a Virtualization Infrastructure. The vProfile product page provides some high level bullets of the features so I’m instead going to spend a lot of time on the in-depth functionality, design, and architecture here.
Continue reading »

written by Aaron Bawcom \\ tags: compliance, Configuration management, Virtualization Management, vProfile, VQL