Lifecycle Components Toolkit (LCT) - Windows Best Practices and Troubleshooting guide

Overview

This article will list some of the best practices that should be used to configure and make LCT work for Windows scripts. It will also be used as a troubleshooting guide to help resolve some common issues.

Best Practices

General

1. Turn off Blocking on the Script Definition while testing

   1.  Please refer to this link for more detail: SovLabs - Add Lifecycle Script Definition

   2. This will cause the workflow to not initiate a destroy on failure. Or, the other option is to test your script on a Specific Host instead of on the Provisioned Node. 

      1. Enable blocking only if a failure in the script means that you want to cancel your build process

2. Use descriptive logging in your custom scripts at key points so failures are clear.

   1. Example(s): Write-Host, Write-Transcript, etc.

   2. More information on these commands is listed below in the PowerShell Commands section

3. Re-execute failed workflow runs from vRealize Orchestrator for testing 

   1. Requires Blocking to be disabled, or execution against Specific Host

   2. See additional details below in the Advanced Troubleshooting section for more information

4. Monitor the scripts directory for the script to copy and verify that the service account has the appropriate permissions on the server/endpoint

   1. Add Service Account to Local Administrators group for testing purposes

   2. Microsoft - Local Accounts

5. The default script directory in your Script Definition can/should be updated to avoid possible UAC/GPO Issues

   1. Example: Change directory from C:\Windows\Temp to C:\SovLabs

6. Test you script locally on a server to verify it works

7. Include logic and output in your scripts to check for successful execution

8. Set exit codes based on expected results
   

   1. Exit code of 0 usually means success for PowerShell
   2. Microsoft - Windows Powershell Exit Codes

Script Definition

1. Include the EBS state and script priority in the script name 1190_10_Install Apache

2. Also helpful to include VC Name and OS for script in name 1190_10_VC01_CENT_Install_Apache

3. Make sure to set the order for the scripts to be run if in if you have multiple script definitions using the same EBS subscription

   1. An alternative way of doing it would be to have each Script Definition going to it's own EBS subscription (This may make it easier for troubleshooting)
      
      

Template Preparation

1. Allow PowerShell to open in Admin mode by default

   1. Microsoft - Disabling User Account Control (UAC) on Windows Server

   2. Microsoft - How User Account Control Works

   3. SovLabs - UAC Security Policy

2. If you're using a new default directory for your scripts, you may want to create the new folder in your template so it's on every new VM

3. Prep the template to allow service account to execute powershell scripts

   1. Add Service Account to Local Administrators group for testing purposes

   2. Microsoft - Local Accounts

4. Disable automatic VM Tools Upgrade (optional) 

   1. Link to VMware article is posted below. This is set on the Host(s) level

   2. VMware - Automatically Upgrade VMware Tools on Reboot
      
      

PowerShell Commands

1. Start-Process - The Start-Process cmdlet starts one or more processes on the local computer. To specify the program that runs in the process, enter an executable file or script file, or a file that can be opened by using a program on the computer. If you specify a non-executable file, Start-Process starts the program that is associated with the file, similar to the Invoke-Item cmdlet.

   1. Microsoft PowerShell (Start-Process)

2. Start-Transcript - The Start-Transcript cmdlet creates a record of all or part of a PowerShell session to a text file. The transcript includes all command that the user types and all output that appears on the console.

   1. Microsoft PowerShell (Start-Transcript)

3. Write-Output - The Write-Output cmdlet sends the specified object down the pipeline to the next command. If the command is the last command in the pipeline, the object is displayed in the console. Write-Output sends objects down the primary pipeline, also known as the "output stream" or the "success pipeline." To send error objects down the error pipeline, use Write-Error.

   1. Microsoft PowerShell (Write-Output)

4. Write-Host - The Write-Host cmdlet customizes output. You can specify the color of text by using the ForegroundColor parameter, and you can specify the background color by using the BackgroundColor parameter. The Separator parameter lets you specify a string to use to separate displayed objects. The particular result depends on the program that is hosting PowerShell.

   1. Microsoft PowerShell (Write-Host)

5. Out-File - The Out-File cmdlet sends output to a file. When you need to specify parameters for the output use Out-File rather than the redirection operator (>).

   1. Microsoft PowerShell (Out-File)
      
      

PowerShell Example

Troubleshooting

Error: File C:\Windows\Temp\scriptname.ps1 cannot be loaded. The file C:\Windows\Temp\scriptname.ps1 is not digitally signed. You cannot run this script on the current system. 

Solutions:

1. Check your execution policy settings inside of PowerShell

   1. Microsoft PowerShell (Execution Policies)

2. If you're the script on a specific node, check to see if there are open PowerShell Processes (Task Manager).

   1. If there are, right click on the task and select "End task".
      
      
      
      
      

Error: File was not found Please see the server log to find more detail regarding exact cause of the failure.

Solutions:

1. Look at the Script Definition and verify that the directory you're using exists on the server
   
   
   
   

2. Log into the server and verify the script is getting copied to the server
   
   

   
   
   

   1. If not, validate permissions are set correctly
      
      
      
      

Error: Unable to access file Please see the server log to find more detail regarding exact cause of the failure.

1. Log into the server and verify the script is getting copied to the server
   
   

   
   
   

   1. If not, validate permissions are set correctly
      
      
      
      

2. Check to make sure PowerShell is opening in Admin mode
   
   
   

Error: Client received SOAP Fault from server: The attempted operation cannot be performed in the current state (Powered off). Please see the server log to find more detail regarding exact cause of the failure.
​

​Error: com.vmware.vim25.InvalidStateFaultMsg: The operation is not allowed in the current state.

1. If you're using a Jump Server to execute the scripts, please try rebooting the server

   1. If an error occurs when trying to reboot the server through vCenter, you may need to do a hard shut down of the server

Advanced Troubleshooting

Re-running LCT Token (vRO)

During testing, if a script fails and the VM has NOT been deleted or you're running it against a specific host, you can re-run your LCT token in vRO. This makes testing script changes faster since you do not have to spin up a new VM every time you make a script change.

1. This will work if you've made changes to the script inside of vRA and want to test your changes. 

2. This will NOT take in any updated properties or anything like that

Process from vRO:

1. Log into vRO and click on the Workflows Tab

2. Expand SovLabs → Lifecycle Component → Lifecycle Components Toolkit → Your LCT Workflow
   

3. Right-click on your token and select Run again...
   
   
   

4. Once the token completes, verify your script/updates worked successfully

Additional information

Plug-in/License
Get SovLabs Plug-in Download Link
How to install a vRA Plug-in
Licensing Infrastructure

Jump Server Configuration
Microsoft AD - Endpoint configuration (Jump Host / WinRM)
Jump Server / VMWare Tools - Account Setup

Lifecycle Toolkit
Lifecycle Toolkit - Examples
Lifecycle Toolkit - Modules and Scripts
Lifecycle Toolkit - Using Lifecycle Toolkit to run multiple scripts dynamically