If are like me, you have a lab at home. In my case, after checking different options, I decided to use Proxmox as my main hypervisor. It is open-source and Debian-based, with a simple web console. Also, there is a terraform provider for it, which we will be looking at today. Using Python, we will create a script to deploy a new VM in Proxmox. To do that we will call Terraform from the Python code using the python-terraform wrapper.
This project is a public repository in my personal GitHub account.
git clone https://github.com/andreypicado506/proxmox_tf.gitRequirements
The host running this script will require the following packages installed:
- Python 3.9.14
- pip 22.3.1
- Terraform v1.3.7
- python-terraform 0.10.1
This can run on a Linux-based VM or container. It should work with Windows too with some minor modifications, however, I have not tested it yet. The first three packages are common and can be easily installed. Python-terraform requires pip to be installed, but it is also easy to install.
You will also need a reachable Proxmox instance from the host that is going to run the script. Check here the instructions to set up your own Proxmox instance. Also, you must have a .iso uploaded to one of the storage containers of that instance.
Project structure
This project has the following structure:
- A main.py script with the Python code. This script takes arguments from the command line, handles variables and calls Terraform.
- A tf/ folder with three different .tf files: provider.tf, variables.tf and vm.tf
├── main.py
└── tf
├── provider.tf
├── variables.tf
└── vm.tfBuilding the main.py
This script is the one doing the heavy lifting. Since there are a lot of things we would like to specify when we create a VM, such as the amount of RAM, the network adapter, the disks… we need to pass those arguments to the script from the command line. To do that, we will use the argparse module. The logic is simple, first, you have to create a ‘parser’ and then add arguments to it.
Taking arguments from the command line
#!/usr/bin/env python3
import argparse
from python_terraform import *
# parse the command line arguments
parser = argparse.ArgumentParser(description='Create a new VM in Proxmox')
# Add the arguments to the parser
parser.add_argument('-pu','--proxmox_url', metavar='proxmox_url', type=str, help='The URL of the Proxmox server')
parser.add_argument('-pp','--proxmox_password', metavar='proxmox_password', type=str, help='The password of the Proxmox user', required=True)
parser.add_argument('-pus','--proxmox_user', metavar='proxmox_user', type=str, help='The user of the Proxmox server')
parser.add_argument('-vn','--vm_name', metavar='vm_name', type=str, help='The name of the VM')
parser.add_argument('-nn', '--node_name', metavar='node_name', type=str, help='The name of the node to create the VM on')
parser.add_argument('-qos', '--qemu_os', metavar='qemu_os', type=int, help='The Qemu OS of the VM')
parser.add_argument('-ot', '--os_type', metavar='os_type', type=str, help='The OS type of the VM')
parser.add_argument('-iso', '--iso', metavar='iso', type=str, help='The ISO to use for the VM')
parser.add_argument('-c', '--cores', metavar='cores', type=int, help='The number of cores to use for the VM')
parser.add_argument('-s', '--sockets', metavar='sockets', type=int, help='The number of sockets to use for the VM')
parser.add_argument('-m', '--memory', metavar='memory', type=int, help='The memory to use for the VM')
parser.add_argument('-dc', '--disk_container', metavar='disk_container', type=int, help='The disk size to use for the VM')
parser.add_argument('-dt', '--disk_type', metavar='disk_type', type=int, help='The disk type [ IDE, SATA...] size to use for the VM')
parser.add_argument('-ds', '--disk_size', metavar='disk_size', type=int, help='The disk size to use for the VM')
parser.add_argument('-nm', '--net_model', metavar='net_model', type=int, help='The network card model to use for the VM')
parser.add_argument('-nb', '--net_bridge', metavar='net_bridge', type=int, help='The network bridge to use for the VM')
# Parse the arguments
args = parser.parse_args()A few notes. The ‘#!/usr/bin/env python3‘ is the appropriate shebang to let Linux know this is a python script that should run with python3. The ‘from python_terraform import *’ line is required for the python-terraform wrapper to work. Regarding the arguments, I ended up creating all the ones I included in the script by looking at the corresponding resource for the Terraform provider.
The next thing is to find a way to take all the arguments from argparse and assign them to variables, so we can use them in the python code. For that, I created an object called proxmoxVM. Then, I iterated over the arguments in the args variable to create a new property of the proxmoxVM object for each argument in args. This means that you do not have to manually declare a variable for each new argument. All the new arguments will be automatically added to the proxmoxVM object, you just need to add them to the parser:
# Create a class with the variables of the new VM and setting the values
class ProxmoxVM(object):
pass
proxmoxVM = ProxmoxVM()
# Iterate over the arguments passed to the script from CLI and set the values of the proxmox object
for i in vars(args):
if getattr(args, i) is not None:
setattr(proxmoxVM, i, getattr(args, i))
else:
setattr(proxmoxVM, i, None)
If Python founds that an argument in args is equal to None (because it was not specified when the script was called), it will still create the corresponding property for the proxmoxVM object, but I will set the value to None. This will be relevant in the next step when we create the variables.tf file for Terraform.
Creating the .tf files for terraform
To keep things organized, I created three different files for Terraform: provider.tf, variables.tf and vm.tf but that is not required and you can only use one .tf file if you want. Remember those files are located in the ./tf folder. provider.tf contains all the info required for the Telmate/proxmox provider to work. variables.tf contains all the variables we will send from the .py script to Terraform to create the desired resource. Finally, the vm.tf has all the things required for the ‘proxmox_vm_qemu‘ resource that is available for the promox provider. Let us take a look at the content of the .tf files.
Variables: variables.tf
variable "proxmox_url" {
type = string
default = "https://192.168.2.2:8006/api2/json"
nullable = false
}
variable "proxmox_user" {
type = string
default = "root@pam"
nullable = false
}
variable "proxmox_password" {
type = string
default = ""
}
variable "vm_name" {
type = string
default = "youritguyvm"
nullable = false
}
variable "target_node" {
type = string
default = "microbito"
nullable = false
}
variable "qemu_os" {
type = string
default = "l26"
nullable = false
}
variable "os_type" {
type = string
default = "centos"
nullable = false
}
variable "iso" {
type = string
default = "local:iso/Centos7_Minimal.iso"
nullable = false
}
variable "cores" {
type = number
default = 1
nullable = false
}
variable "sockets" {
type = number
default = 1
nullable = false
}
variable "memory" {
type = number
default = 1024
nullable = false
}
variable "disk_type" {
type = string
default = "ide"
nullable = false
}
variable "disk_storage" {
type = string
default = "local-lvm"
nullable = false
}
variable "disk_size" {
type = string
default = "16G"
nullable = false
}
variable "network_model" {
type = string
default = "virtio"
nullable = false
}
variable "network_bridge" {
type = string
default = "vmbr0"
nullable = false
}Notice that we defined the type and the default property for each variable. Nullable is set to false for all the variables with the exception of the ‘proxmox_password‘. When we defined the promoxVM object properties, we told Python to create all the properties present in args, and if the content of the argument in args was empty, Python was going to set the corresponding property value to None. When it comes to Terraform variables, Nullable means that it will take the default value if it is called with an empty o null variable value. In other words, if we call the main.py script without all the parameters with the exception of the ‘proxmox_password‘ it will work because for each case Terraform will use the default value. Some default values will be specific to your own environment and you should modify them accordingly.
Provider: provider.tf
terraform {
required_providers {
proxmox = {
source = "Telmate/proxmox"
}
}
}
provider "proxmox" {
pm_api_url = var.proxmox_url
pm_user = var.proxmox_user
pm_password = var.proxmox_password
pm_tls_insecure = true
}Notice we call the variables in variables.tf by preceding them with the ‘var.’ keyword. This is also the case with the next file: vm.tf
Resource: vm.tf
resource "proxmox_vm_qemu" "var_vm_name" {
name = var.vm_name
target_node = var.target_node
iso = var.iso
# invoke the variable var_vm_cores as a number
cores = var.cores
sockets = var.sockets
qemu_os = var.qemu_os
os_type = var.os_type
memory = var.memory
disk {
type = var.disk_type
storage = var.disk_storage
size = var.disk_size
}
network {
model = var.network_model
bridge = var.network_bridge
}
}Using the python-terraform wrapper
The python-terraform module needs to be installed on the host before you can call it in Python. Once that is out of the way, you call the module by using ‘from python_terraform import *’ at the beginning of the script. According to the documentation, the next thing you have to do is to call the Terraform class. After that, you can call the .init and .apply methods, and they work just like the usual terraform commands: terraform init and terraform apply. Both methods return some useful info to check if terraform ran successfully.
As you would do with ‘terraform apply’, the apply method of the Terraform class has a way to define variables for Terraform, so we are calling it with all the relevant stuff we got from argparse in the args variable:
# Invoke Terraform class
tf = Terraform(working_dir='./tf')
# Initialize Terraform
tfInit = tf.init()
# If the init fails, exit the script
if tfInit[0] != 0:
print("Terraform init failed")
exit(1)
# Apply Terraform
tfApply = tf.apply(
skip_plan=True,
auto_approve=True,
var={
'proxmox_url': proxmoxVM.proxmox_url,
'proxmox_password': proxmoxVM.proxmox_password,
'proxmox_user': proxmoxVM.proxmox_user,
'vm_name': proxmoxVM.vm_name,
'target_node': proxmoxVM.node_name,
'qemu_os': proxmoxVM.qemu_os,
'os_type': proxmoxVM.os_type,
'iso': proxmoxVM.iso,
'cores': proxmoxVM.cores,
'sockets': proxmoxVM.sockets,
'memory': proxmoxVM.memory,
'disk_type': proxmoxVM.disk_type,
'disk_storage': proxmoxVM.disk_container,
'disk_size': proxmoxVM.disk_size,
'network_model': proxmoxVM.net_model,
'network_bridge': proxmoxVM.net_bridge
}
)
# If the apply fails, exit the script
if tfApply[0] != 0:
print("Terraform apply failed")
exit(1)
else:
print("Terraform apply successful. VM created {}.".format(proxmoxVM.vm_name))
For the terraform apply we declare the auto-approve value. Also, a useful option is capture_output=False which will show the typical terraform output while the main.py script is running.
All the variables we got from args and being sent here to Terraform. If a variable is equal to None, Terraform will use the default if the Nullable attribute of that variable is set to false.
Calling the script
To call the script you can use all the parameters in args, but since we have defaults for most values in the variables.tf file, that is not necessary:
[youritguy@rhel-instance proxmox_tf]$ ./main.py -pp 'strongPassword' -vn 'itguyvm' -m 2048
Terraform apply successful. A new VM was created with the follwing name: itguyvm.On the Proxmox side, we will see the new VM:
You can also use the ‘full’ version of the parameter when calling main.py:
[youritguy@rhel-instance proxmox_tf]$ ./main.py --proxmox_password 'strongPassword' --vm_name 'youritguy2' --memory 1024 --cores 2
Terraform apply successful. A new VM was created with the follwing name: youritguy2.That will create a VM too:
When we start a console of the VM that we just created, we will see it boots with the .iso we defined in variables.tf as default for the .iso var. In my case, I used a Centos 7 iso:
