pyVmomi module: Script for fetching hardware information from ESXi

import argparse
from pyVmomi import vim
from pyVim.connect import SmartConnect,Disconnect
import atexit
import ssl

def validate_options():
  parser = argparse.ArgumentParser(description='input parameters')
  parser.add_argument('-d','--dest_host',dest='dhost',required=True,help='The ESxi destination host IP')
  parser.add_argument('-v','--vc_host',dest='vchost',required=False,help='The VC ip')
  parser.add_argument('-u','--vc_user',dest='vcuser',required=True,help='VC username')
  parser.add_argument('-p','--vc_pass',dest='vcpasswd',required=True,help='VC passwd')
  args = parser.parse_args()
  return args

def getHostID(content,dhost):
  if content.searchIndex.FindByIp(None,dhost,False):
    host = content.searchIndex.FindByIp(None,dhost,False)
  else:
    host = content.searchIndex.FindByDnsName(None,dhost,False)
  return host

def get_HostInfo(content,dhost):
   search_index = content.searchIndex
   root_folder =  content.rootFolder
   view_ref = content.viewManager.CreateContainerView(container=root_folder,type=[vim.HostSystem], recursive=True)
   host = view_ref.view[0]
   #print host.name
   print 'UUID INFO %s' %(host.summary.hardware.uuid)
   print 'Hardware Model %s' %(host.summary.hardware.model)
   print '%s Server has %s Biosversion'%(host.hardware.biosInfo.vendor,host.hardware.biosInfo.biosVersion)
   pcilist=host.hardware.pciDevice
   print '{0}'.format("Vendor Name").ljust(20)+ '{0}'.format("Device Name").ljust(120)+ '{0}'.format("Slot").ljust(30)+ '{0}'.format('Device ID').ljust(10)
   print '*************************************************************************************************************************************************************************************************'
   for i in pcilist:
     a = i.vendorName
     b = i.deviceName
     c = i.deviceId
     d = i.slot
     print '{0}'.format(a).ljust(20)+ '{0}'.format(b).ljust(120)+ '{0}'.format(d).ljust(30)+ '{0}'.format(c).ljust(10)
     #print '%s has devicename %s and  device ID %s'%(i.vendorName,i.deviceName,i.deviceId)
   print '*********************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************' 
def main():
  opts = validate_options()
  if opts.vchost:
    print 'Connecting to vcenter and collecting sensor info for %s' %opts.dhost
  else:
    print 'Connecting to esxi host for %s' %opts.dhost
    opts.vchost = opts.dhost
  si = SmartConnect(host=opts.vchost, user=opts.vcuser, pwd=opts.vcpasswd)
  content = si.RetrieveContent()
  #print content
  hostinfo = get_HostInfo(content,opts.dhost)
  hostid = getHostID(content,opts.dhost)
  sensorinfo=hostid.runtime.healthSystemRuntime.systemHealthInfo.numericSensorInfo
  print '{0}'.format("Sensor").ljust(30)+ '{0}'.format("Sensor Detail").ljust(90)+ '{0}'.format('Status').ljust(10)+ '{0}'.format('Reading').ljust(10) +'{0}'.format('Units').ljust(13)+ '{0}'.format('Summary').ljust(20)
  print '**************************************************************************************************************************************************************************************************************'
  for i in sensorinfo:
    j = i.healthState
    a=str(i.currentReading)
    b=i.baseUnits
    c=i.sensorType
    print '{0}'.format(c).ljust(30)+ '{0}'.format(i.name).ljust(90)+ '{0}'.format(j.label).ljust(10)+ '{0}'.format(a).ljust(10) + '{0}'.format(b).ljust(13)+ '{0}'.format(j.summary).ljust(20)

if __name__ =='__main__':
  main()
How to run this script : 

python <name of script> -v <vc server> -d <esxi host which hardware want to list> -u <vc user name> -p <vc password>

This script is written in python. I have used the  pyVmomi module.

pyVmomi rpm for centos7

pyVmomi is the Python SDK for the VMware vSphere API that allows you to manage ESX, ESXi, and vCenter. pyVmomi is available on git.

https://github.com/vmware/pyvmomi

I have created a rpm format of same pyVmomi SDK for centos7.  This rpm will be installed in the /opt folder on your centos 7.

Below is the Spec file:

%define BUILD pyvmomi_master.1.0.1.x86_64
Summary: Pyvmomi package
Name: pyvmomi_master
Release: 1.0
Version: 1
License: Apache License 2.0
Requires: python-six
Requires: python-requests
Requires: python-setuptools
BuildArch: noarch

%description
This package contains the vSphere python SDK

%post
%files
%defattr(-,root,root,-)
/opt/pyvmomi-master
%doc
%changeLog
* Fri Jul 14 2017 Amit <amit@openwriteup.com> 1-1.0
- Pyvmomi 6.5

Once you install the rpm, it will be in the /opt/pyvmomi-master folder.

 rpm -ivh pyvmomi_master-1-1.0.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:pyvmomi_master-1-1.0             ################################# [100%]


################################# [100%]
[root@devbox noarch]# ls /opt/pyvmomi-master/
docs  LICENSE.txt  MANIFEST.in  NOTICE.txt  pyVim  pyVmomi  README.rst  requirements.txt  sample  setup.cfg  setup.py  test-requirements.txt  tests  tox.ini

Post installation of the package, we need to run following step:

[root@devbox pyvmomi-master]# python setup.py –help
Common commands: (see ‘–help-commands’ for more)

setup.py build      will build the package underneath ‘build/’
setup.py install    will install the package

 python setup.py install
running install
running bdist_egg
running egg_info
creating pyvmomi.egg-info
writing requirements to pyvmomi.egg-info/requires.txt
writing pyvmomi.egg-info/PKG-INFO
writing top-level names to pyvmomi.egg-info/top_level.txt
writing dependency_links to pyvmomi.egg-info/dependency_links.txt
writing manifest file 'pyvmomi.egg-info/SOURCES.txt'
reading manifest file 'pyvmomi.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyvmomi.egg-info/SOURCES.txt'

vSphere SDK for python : pyVmomi

VMware has released pyVmomi for developers who wants to code in python. pyVmomi is a VMware SDK to manage ESXi and vCenter using python.
Using pip we can install:
pip install pyvmomi

Developers can download the pyvmomi package from python.org

https://pypi.python.org/pypi/pyvmomi

When we extract and list the package
LICENSE.txt NOTICE.txt pyVim pyvmomi.egg-info requirements.txt setup.cfg test-requirements.txt tox.ini MANIFEST.in PKG-INFO pyVmomi README.rst sample setup.py tests

Perform the below steps:
 python setup.py install
 python setup.py develop

pyVmomi package provide mainly two modules : pyVim and pyVmomi
Let’s explore pyVim. This module will be used to connect to ESXi and vSphere Center.
>>> import pyVim
>>> help(pyVim)
PACKAGE CONTENTS
connect
task
>>>import pyVim.connect
NAME
pyVim.connect – Connect to a VMOMI ServiceInstance.
>>>help(pyVim.connect)
This contents two function to connect service instance: SmartConnect and Connect
SmartConnect(protocol=’https’, host=’localhost’, port=443, user=’root’, pwd=”, service=’hostd’, path=’/sdk’, preferredApiVersions=None, keyFile=None, certFile=N
one, thumbprint=None, sslContext=None, b64token=None, mechanism=’userpass’)
Determine the most preferred API version supported by the specified server,then connect to the specified server using that API version, login and return the service instance object. Throws any exception back to caller. The service instance object is also saved in the library for easy access.Clients should modify the service parameter only when connecting to a VMOMI server other than hostd/vpxd. For both of the latter, the default value is fine.

Connect(host=’localhost’, port=443, user=’root’, pwd=”, service=’hostd’, adapter=’SOAP’, namespace=None, path=’/sdk’, version=None, keyFile=None, certFile=None,
thumbprint=None, sslContext=None, b64token=None, mechanism=’userpass’)
Connect to the specified server, login and return the service instance object. Throws any exception back to caller. The service instance object is also saved in the library for easy access. Clients should modify the service parameter only when connecting to
a VMOMI server other than hostd/vpxd. For both of the latter, the default value is fine

 

We can connect using SmartConnect
From pyVim.connect import SmartConnect
conn = SmartConnect(host=<”ESXi/vSphere Center ip”>, user = <”username”>, pwd= <”password”>)

Print conn

'vim.ServiceInstance:ServiceInstance'

This is service instance type of managed object. For writing code further we can make use of vSphere mob interface: http://<vc-ip>/mob

mob-2

Wrote a small code which fetch the build number and name of datecenter

#!/bin/python
from pyVim.connect import SmartConnect
conn = SmartConnect(host="<VC hostname>",user="<username>",pwd="<password>")
print ("Multi host is supported")
print (conn.capability.multiHostSupported)
print ("----------------------------------------------------------")
print ("VC System Current time")
print (conn.CurrentTime())
print ("----------------------------------------------------------")
print ("VC server build info")
print (conn.content.about.version)
print (conn.content.about.fullName)
print (conn.content.about.build)
print ("----------------------------------------------------------")
datacenter = conn.content.rootFolder.childEntity
print ("Name of datacenters")
print ("----------------------------------------------------------")
for i in datacenter:
 print (i.name)
# python vcconnect.py 
Multi host is supported
True
----------------------------------------------------------
VC System Current time
2016-09-22 14:33:17.
----------------------------------------------------------
VC server build info
6.0.0
VMware vCenter Server 6.0.0 build-xxxxxx
----------------------------------------------------------
Name of datacenters
----------------------------------------------------------
test_Lab

Below are the screenshot how I traverse for Version number:

conn.content.about.version

conn = SmartConnect(host=<”ESXi/vSphere Center ip”>, user = <”username”>, pwd= <”password”>)

Print conn

'vim.ServiceInstance:ServiceInstance'

conn is Service Instance Object. listing the properties and methods from mob interface.

mob-3

In properties segment, I am listing the content property.This content the about property.

mob-4

In about we can list version:

mob-5

That’s the way below code has traversed.

print ("VC server build info")
print (conn.content.about.version)
print (conn.content.about.fullName)
print (conn.content.about.build)

 

List all the running instance on amazon VPC

In my test environment, I have amazon VPC, which I am accessing using Linux server.

For performing all the activities on amazon vpc, I have used python script.For automation in amazon VPC, aws provides module boto3, which need to be installed using python pip. Using this module we list all the running instances.In below script I am creating a config file and then reading that config file.Following script perform all these steps:

  • Create a config file, which is required to connect aws vpc.
  • Read the config file, and list all the instance following details:
    • ‘Name’:name,
    • ‘Type’: instance.instance_type,
    • ‘State’:instance.state[‘Name’],
    • ‘Private IP’:instance.private_ip_address,
    • ‘Public IP’: instance.public_ip_address,
    • ‘Launch Time’: instance.launch_time

 

</script>

import ConfigParser,boto3,os,sys,paramiko
from collections import defaultdict
config = ConfigParser.RawConfigParser()
#When adding sections or items, add them in reverse order
config.add_section('EC2')
config.add_section('USER')
config.set('EC2','SSHKey','<keyname>')
config.set('EC2','VPC_IP','<vpcname>')
config.set('EC2','Security_Group','<securitygroupname>')
config.set('EC2','DisableAPI_Termination','False')
config.set('USER','Username','<ec2user>')
config.set('USER','AWS_Profile','<aws user to login>')
config.set('EC2','Region','<aws region>')
config.set('USER','Private_Key','<path_to_privatekey>')

#Writing configruation to config file
name = raw_input("Enter the config file name::: ")
with open(name, 'wb') as configfile:
config.write(configfile)
#Reading the config file
config1 = ConfigParser.ConfigParser()
config1.read(name)
ses = boto3.Session(profile_name = config1.get("USER", "AWS_Profile"))
ec2 = ses.resource('ec2')
key = paramiko.rsakey.RSAKey.from_private_key_file(filename=config1.get("USER","Private_Key")) 
running_instances = ec2.instances.filter(Filters=[{
'Name': 'instance-state-name',
'Values': ['running']}])
ec2info = defaultdict()
for instance in running_instances:
for tag in instance.tags:
if 'Name' in tag['Key']:
name = tag['Value']
ec2info[instance.id] = {
'Name':name,
'Type': instance.instance_type,
'State':instance.state['Name'],
'Private IP':instance.private_ip_address,
'Public IP': instance.public_ip_address,
'Launch Time': instance.launch_time
}
attributes = ['Name','Type','State','Private IP','Public IP','Launch Time']
for instance_id, instance in ec2info.items():
for key in attributes:
print("{0}:{1}".format(key,instance[key]))
print("------")

 

output of the script :

——
Name:testinstance
Type:m4.xlarge
State:running
Private IP:10.140.30.209
Public IP:None
Launch Time:2016-08-26 23:09:17+00:00
——

 

 

aws instance listing using python sdk

This blog is for those, who are very new to aws and python. They want to start both of them together. Assuming they have setup boto3 environment in their test lab.

In lab setup type python: python

It will give python prompt, we can explore boto3.

>>> import boto3
>>> dir(boto3)
[‘DEFAULT_SESSION’, ‘NullHandler’, ‘Session’, ‘__author__’, ‘__builtins__’, ‘__doc__’, ‘__file__’, ‘__name__’, ‘__package__’, ‘__path__’, ‘__version__’, ‘_get_default_session’, ‘client’, ‘docs’, ‘exceptions’, ‘logging’, ‘resource’, ‘resources’, ‘session’, ‘set_stream_logger’, ‘setup_default_session’, ‘utils’]

Perform help (boto3) ,It will show the package content with this package..

PACKAGE CONTENTS
compat
docs (package)
dynamodb (package)
ec2 (package)
exceptions
resources (package)
s3 (package)
session
utils

Lets import the resources : from boto3 import resources

>>> dir (boto3.resources)
[‘__builtins__’, ‘__doc__’, ‘__file__’, ‘__name__’, ‘__package__’, ‘__path__’, ‘action’, ‘base’, ‘collection’, ‘factory’, ‘model’, ‘params’, ‘response’]
>>>

PACKAGE CONTENTS
action
base
collection
factory
model
params
response

 

/*perform following on your python console or write a .py script
import boto3
ec2=boto3.resource(ec2)
#help(ec2) /*it will list all the available option with ec2*/
#help(ec2.instances) /*search for filter*/
#help(ec2.instances.filter /*list the filter option and list syntax
/* instance_iterator = ec2.instances.filter(
 | DryRun=True|False,
 | InstanceIds=[
 | 'string',
 | ],
 | Filters=[
 | {
 | 'Name': 'string',
 | 'Values': [
 | 'string',
 | ]
 | },
 | ]
*/


import boto3
ec2=boto3.resource('ec2')
instances = ec2.instances.filter(
    Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
for instance in instances:
    print(instance.id, instance.instance_type)

Vmware VRO (VCO) :How to design workflow :Basic Workflow[series3]

In this section, We will try to write a workflow from scratch… When we start writing any workflow, we need to start from following:

  • What needs to be developed, means understand the requirement properly
  • Perform manually what parameters are require, possibly from virtual center.

Lets start for a workflow, where we are get virtual machine snapshots. When we do manually, we connect to esx(i) host (using VIC/web client), for a particular vm right click and snapshot manager.

Lets have a api search in vRo, searching

snapshot2

In Api search, we will find lot snapshot related stuff such as revert,create.. We need to select what we are looking for “get vm snapshot”.

snapshot3

There are methods or attribute define in different object. In VcPlugin, getAllVirtualMachineSnapshots method is available. Lets have a look into this method:

snapshot

This method has two parameters and one return type value.

  • Parameters
    • additonalPropertyFilters(array of String)
    • query(string) /*Xpath query*/
    • Return Type: Array of VcVirtualMachineSnapshot

First parameter additional property filters, its a big topic to discuss, “PropertyFilter is actually applying a filter to each result, as in, return the following fields for each result instead of filtering the results set based on the PropertyFilter.  So it will return the properties as the result of filter.

Xpath query will be used to search the result from vCo side.

Lets start writing the workflow, we create a workflow and go in schema , we see the below figure.

snapshot1\

In this workflow we need the list of virtual Machines which snapshots need to be searched. So first input is “List of Virtual Machines: So our input is Vc:VirtualMachine array type. Output is “Array of VcVirtualMachineSnapshot”. So we define Snapshot output type VC:VirtualMachineSnapshot.

Currently in schema we select the scriptable task.

snapshot5

Visual binding of the inputs and output in schema:

snapshot6

The Scripting part :

for(i in VMs) {
var vmpattern = VMs[i];
var XPath = "xpath://name[starts-with(.,'"+vmpattern+"')]";
ListSnapshot = VcPlugin.getAllVirtualMachineSnapshots(null,XPath);
Snapshot=ListSnapshot;
System.log (Snapshot);
}

Above is scripting part, which is working state,tried to use all the parameters defined above..

Below is the presentation layer, when we start executing the workflow, this is the layer will give the front end. It will ask the list of the array which we need

snapshot7

snapshot4

In upcoming series, we will try to understand more about propertyfilter… it will help for further coding…

 

Automation of Aws instances using python sdk

Boto is python sdk in amazon, which can be used for automation purpose for ec2,s3 etc.

Installation: Boto can be installed using using pip or offline. for pip the command:    pip install boto3

Offline installation we need to download the offline bundle and install, it has the dependency , download and install it.

Configuration : Before we use boto we need to set the configuration, which can be done using : aws configure

Alternatively, you can create the credential file yourself. By default, its location is at ~/.aws/credentials:

[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY

You may also want to set a default region. This can be done in the configuration file. By default, its location is at ~/.aws/config:

[default]
region=us-east-1

Alternatively, you can pass a region_name when creating clients and resources.

Lets see some the sample code:

import boto3

ec=boto3.resource(‘ec2’)

if we print ec, it shows “ec2.ServiceResource()”

Just give help(ec),it will show up the complete description what we can do ,what methods are available. Below example we want to check all the instance status

# Boto 3
#!/usr/bin/python
import boto3
ec=boto3.resource('ec2')
#help(ec)
#check the status of all the ec2 instances
for status in ec.meta.client.describe_instance_status()['InstanceStatuses']:
    print(status)

 

Resources URLs using the MOB

For VI Api understanding of Object model of vmware api structure. It gives a detail overview and provide methods name and value

Some of the important Sample Urls for Resources:

Resource : Service Instance     URL: Get https://<host_ip>/mob

Resource: Host      URL: Get https://<host_ip/mob/?moid=ha%2dhost

Resource:Virtual Machine    URL:Get https://<host_ip>/mob/?moid=1120

Resource:Property

URL:Get https://<host_ip>/mob/?moid=1120&doPath=config%2ehardware

Resource:Method

URL: Post https://<host_ip>/mob/?moid=48&mehod=rename

 

pyVim.connect :Connect esxi host

#!/usr/bin/python
from pyVim import connect
__author__=”openwriteup”

if __name__==”__main__”:
”’test code for PyVmomi”’
print “Connect to esxi…”
si=connect.Connect(“10.14.21.14″,443,”root”,”password”)
searcher=si.content.searchIndex
vm=searcher.FindByInventoryPath(“ha-datacenter/vm/rhelvm”)
vmname=vm.GetName()
print “Amit’s” + vmname
print “diconnecting esxi…”
connect.Disconnect(si)
Help on function Connect in module pyVim.connect:

Connect(host=’localhost’, port=443, user=’root’, pwd=”, service=’hostd’, adapter=’SOAP’, namespace=None, path=’/sdk’, version=None, keyFile=None, certFile=None, thumbprint=None)
Connect to the specified server, login and return the service
instance object.

Throws any exception back to caller. The service instance object is
also saved in the library for easy access.

Clients should modify the service parameter only when connecting to
a VMOMI server other than hostd/vpxd. For both of the latter, the
default value is fine.

@param host: Which host to connect to.
@type host: string
@param port: Port
@type port: int
@param user: User
@type user: string
@param pwd: Password
@type pwd: string
@param service: Service
@type service: string
@param adapter: Adapter
@type adapter: string
@param namespace: Namespace *** Deprecated: Use version instead ***
@type namespace: string
@param path: Path
@type path: string
@param version: Version
@type version: string
@param keyFile: ssl key file path
@type keyFile: string
@param certFile: ssl cert file path
@type certFile: string
@param thumbprint: host cert thumbprint
@type thumbprint: string
(END)

Learn python : Understand the main

Being a Software engineer, i came across many coding language.Most of the time
we want to learn a language, and start looking for tutorials or books. In some sense,
inital stage as a new learner we pick up, learn about data types,funciton modules etc.
Since for a new developer, its quite difficult to reach till end of the book, or without
having any project or motivation, most of us ended up in between.
I thought of going with a random approach, where i will see some pre-written code.

We will see the code and try to understand the concept.

In this start, I have seen one of the python program which was available in esxi.This python code was available in /bin/vmfsfilelockinfo.
After seeing this code multiple time, i found that code starts from main function just like C.

if __name__ == ‘__main__’:

”’defined main function,program starts from here”’

one of the blog it was explained very well.
http://effbot.org/pyfaq/tutor-what-is-if-name-main-for.htm

This gives us the starting point in python, as well code can be executed standalone or can be reused in another code.

C:\Users\amit>cat test.py
def add(x):
return x + x

if __name__ == ‘__main__’:
print “test: add(42) ==”, add(42)

C:\Users\amit>python test.py
test: add(42) == 84
In above example this is working as standalone program. We can make this code as reusable code as well
Type “help”, “copyright”, “credits” or “license” for more information.
>>> import test
>>> test.add(4)
8
>>>

In above part of code, we have imported test.py as module “import test”

In test.py i have defined one function add, which i called as test.add(value), which return me the value.