Chef on Windows (redux/reboot)

I’ve been neglecting this blog for a while now however I haven’t been away from Chef and Chef on Windows in the intervening period. In that time I’ve expanded the number of cookbooks and problems I’ve solved using Chef on Windows and thought I’d start posting some code samples and snippets that might be useful. Windows is a fundamentally different beast than Linux and there are unique challenges that after a while I’ve found patterns that solve these.

As such I’ve thrown up two Gists on Github on  my profile. The first is a series of Powershell based guards that can be used in resources to control the execution of actions: chef-windows-guards

The second is a series of powershell_script resource scripts that fill in some gaps in the available resources: chef-windows-scripts

I hope to add others soon as I refine some of my thinking and practices. Also I’m keen for others to try them out and submit any changes or improvements.

Advertisement

Automating Windows Series – DCOM security

This is part of a series of posts on automating Windows and deploying systems in the real world of enterprise operations. I’ve focused the series on the those nuances and problems that I’ve hit trying to deploy the various software stacks found in a typical Windows centric enterprise.

DCOM. It strikes fear and bewilderment into IT Pros everywhere. Usually because some third party application is installed that loads some DCOM components and at some point someone broke the security permissions or activation settings; and no one knows what the correct settings were.

If there was ever a case for automation and infrastructure as code then this is it. Personally, I have one fairly important third party framework that, occasionally over the years, has needed to be deployed/redeployed. And every time there is a period of head scratching as to how to get it configured correctly. It has a couple of DCOM objects that get installed but these don’t ever have the correct permissions or identity set after installation. Enter Chef and dcomperm.exe, a compiled tool from Microsoft’s SDK sample code, that allows for programmatic control of DCOM permissions.

It seems setting DCOM permissions is actually pretty hard to automate, the permission ACL’s are binary strings in the registry and a slight mistake trying to set a binary string renders the ACL unusable. There aren’t any Powershell commands out of the box so you either assemble a script that hits WMI or you use the compiled tool.

Fortunately this application actually ships with dcomperm.exe, as I suspect it tries to use it itself to set the permissions at some point. As such I’ll focus on how I bundled this into a Chef recipe and the limitations of doing so. Hopefully I’ll have an improved solution at some point in the future to share.

Set out below are the snippets of a cookbook, as one way of using dcomperm.exe to set permissions on a DCOM object. You need to retrieve the objects GUID in advance for this example. If you are setting the identity for the object to run under then please note that you should securely encrypt the password in a Chef vault, rather than just leaving it as plain text in the attributes!

In the attributes I set an array of options that I want the dcomperm.exe tool to run to set my object’s permissions and identity. Then in the recipe I loop through each of those strings of options and use an “execute” resource to then run dcomperm.exe with those options. You will need to distribute the dcomperm.exe in the cookbook as well, but I have not included that step in the sample for the sake of brevity.

The important thing to note is that there is no idempotence. Every time the recipe is run this section will run regardless of current settings. On the positive side it is a very fast set of commands to run and it does not affect the operation of the DCOM object when it is applied.

Attributes

default[‘poal_openroad’][‘dcom_options’] = [
  ‘-runas {9804E901-495A-11D4-A083-00C04F740D56}   domain\username password’,
  ‘-al {9804E901-495A-11d4-A083-00C04F740D56} default’,
  ‘-aa {9804E901-495A-11d4-A083-00C04F740D56} default’,
  ‘-al {9804E901-495A-11d4-A083-00C04F740D56} set domain\username permit’,
  ‘-aa {9804E901-495A-11d4-A083-00C04F740D56} set domain\username permit’
]

Recipe

#Set DCOM Permissions
node[‘cookbook’][‘dcom_options’].each do |dcom_options|
  execute “#{dcom_options}” do
    command “dcomperm.exe #{dcom_options}”
    cwd “#{node[‘cookbook’][‘bin’]}”
  end
end

Automating Windows Series – Chef on Windows

This is the first of a series of posts on automating Windows and deploying systems in the real world of enterprise operations. I’ve focused the series on the those nuances and problems that I’ve hit trying to deploy the various software stacks found in a typical Windows centric enterprise company.

To understand the nuances of running Chef on Windows you need to keep in mind that it evolved in the Linux\Open Source world. It’s a bit like humans trying to live on Mars, it can be done but you need to provide a special habitat and it takes longer to get things done sometimes.

Core to Chef is Ruby as the base programming layer that it defines its DSL in. Ruby’s performance and efficiency isn’t great in the Windows world. It’s a port from Linux and it isn’t optimised for Windows. What you’ll see is consistently high CPU usage during a Chef run, generally on a single cpu core. (I’ve found single core virtual machines suffer significantly during a Chef run in terms of other operations happening in parallel)

It also manifests in longer Chef runs especially with deploying a large number of Windows features for example.

The approach I’m taking more and more now is to utilise DSC resources within the Chef recipe which effectively hands off processing of the Windows specific configurations from Chef to the native DSC engine. A further post will go into the detail of how I’m implementing DSC in Chef.

Updating the Chef client is also a bit tricky, it ships as an MSI (A well built MSI that actually adheres to the MSI best practices) but you can’t update it during a Chef run for obvious reasons… Ultimately you’ll need to update it out of band to the Chef run, either with a different tool or by using a Chef recipe which sets up a scheduled task to run to do the update.

You will also need to make a decision on how the Chef client will run, will you trigger it on demand or run it as a service and have it automatically trigger chef runs. The good news is that the Chef client operates properly within a Powershell environment both locally and via remoting so there are plenty of options depending on your deployment processes.

The other key point to note is that Chef has ensured that Powershell is an option within recipes, both as a resource and as a guard interpreter (More on this later as well).

Automating Windows Series – Why Chef

This is part of a series of posts on automating Windows and deploying systems in the real world of enterprise operations. I’ve focused the series on the those nuances and problems that I’ve hit trying to deploy the various software stacks found in a typical Windows centric enterprise company.

The first question is why I chose Chef, especially considering it’s roots are definitely not in the Windows space!

Ultimately a choice had to be made and at the time it was made because of the following points.

• A large and vibrant community (Helped by the fact that Steve Murawski, an ardent fan and early DSC adopter, joined the Chef team).
• Chef identified early that it needed to embrace Windows and demonstrated a clear path to incorporating DSC into it’s DSL.
• Templates! Oh how templates makes deploying configuration files, where one or two settings need to be dynamic, so simple. I can’t highlight this enough, being able to drop text files and then dynamically modify a single field based on environment or node specific variables is a godsend.
• All the core functionality is open source and provides a complete system for managing and deploying configurations.
• The Chef server and in built search tools give operations the ability to do some simple but dynamic service discovery and configuration. Not as good as doing service discovery native within the applications but better than static configuration files.

Of course the big question is why not straight Powershell DSC? I spent a fair bit of time with DSC before I adopted Chef and I found there were some fairly large hurdles to overcome. Some of them are still relevant today and some have been remediated or will be in the not to distant future.

The really painful problem relates back to managing text based configuration files which have some dynamic content within them. There’s no native functionality within DSC that will allow you to handle config files in the same manner as say Chef, so you have to implement your own logic. I went right down a rabbit hole writing a custom DSC resource to manage nodes in a XML file for one POC before I realised I was going to have to do something similar for every text based file format.

There’s a basic framework around DSC configuration deployment that requires you to write a lot of Powershell to assemble a deployment system from base DSC to deployed configuration. It can be done but it takes a lot of work to maintain that sort of system and I’d rather spend my time writing configurations than the plumbing to keep deployments working. DSC being primarily a DSL means you can use it different deployment and configuration scenarios but ultimately it’s not a singular tool for the end user. Microsoft and others (like Chef) are going to wrap their own deployment methodologies around the DSC DSL.

So Chef got chosen as the platform for attempting to automate the Windows beast. Stay tuned for the follow up articles where I discover it isn’t all roses in the automation world.

To get started with Chef hit the root of their official documentation at https://docs.chef.io/ or for a guided set of tutorials https://learn.chef.io/tutorials/

Using Chef-Vault to securely distribute file contents

Shortly after my first blush of sucess with Chef, it become clear I needed to understand how to securely distribute credentials and file content such as SSL certificates. Off I went in search of the solution with old friend Google….

A bit later, having come to the conclusion that the chef-vault gem (open-sourced by Nordstrum and bundled into the Chefdk omnibus) was the answer I then went in seach of the real question “How do I actually use this?”. The answers were few and far between and only one blog really ran through the process when it came to file contents. @jttimberman’s post “Managing Secrets with Chef Vault” was the closest thing I came across that walked through the process. Unfortunately it was nearly two years out of date and in the world of OSS that’s an eternity.

Distributing passwords was simple enough but tackling a PEM SSL cert meant I ran into a bunch of baffling errors, especially with encoding the file content into a JSON format. After a lot of trial and error I found a way forward and considering the lack of info on the web and the very likely chance I’ll forget how I did it, I’ve posted the process below. I haven’t gone into detail on explaining what each step does and why it’s necessary; @jtimberman’s post still explains that well along with the official GitHub repo for reference.

This is the step by step updated procedure for distributing sensitive file contents via chef and chef-vault, as I figured it out. It’s valid as of the following versions

• chef-dk 0.4.0
• chef-vault 2.5
• Chef client 12.0.2

1) Update the Chef-Vault gem (From what I can tell the omnibus doesn’t have this latest version which seems to have a number of fixes in it).

2) Update knife.rb with chef-vault settings.

knife[:vault_mode] = ‘client’ (If you are using Chef Server)

3) Upload the file contents using knife.

knife vault create <databag name> <item name> –file <full path to file to encrypt> –search ‘role:<role- names(s)>

4) Update your recipe to load the chef-vault gem and decrypt the file contents.

chef_gem “chef-vault”

require “chef-vault”

vault = ChefVault::Item.load(‘<vault name>’,'<item name>’)

 file “<path and name of file>” do

    content vault[‘file-content’]

    action :create

    sensitive true

end

References:
Github Chef-Vault repo

 

Uninstall-Java – Powershell Script and Module 1.0.1 release

A quick update today that version 1.0.1 of my Powershell script and module for uninstalling Java has been released.

One fix and one update are in this version. The fix is for the detection of running Java processes which weren’t triggering the while loop to wait and/or kill before uninstalling Java. The update was to update the query string to pick up Java 8 installations.

https://poshuninstalljava.codeplex.com/

I still need to put together a more robust testing process, right now my thoughts are to use Teamcity and Pester to run up VM’s and do automated testing against all the permutations that are now possible.

Uninstall-Java – Powershell Script and Module 1.0 release

Today I’ve released what I’m considering a 1.0 release of the Uninstall-Java Powershell script and Module code.

I’ve performed some testing of both the script and module on Windows 7, 8 and 8.1 and the code seems to be stable and clean. As always I highly recommend testing in your environment.

This project started after the furor over major security bugs in Java and now has developed into a full project aimed at simplifying the management of Java versions in large environments by ensuring wither java is not installed or only the approved version is deployed. Previous articles (here and here) I wrote go into some of the detail and the development progress is tracked in some of the comments.

In the process I started my first Codeplex project as a way to distribute the code. There was a bit of a learning curve in using Codeplex and Git and along with some character encoding issues led to code being available which didn’t run when downloaded from Codeplex. These have been fixed (sorry to anyone who tried to use the damaged code) and is now available from the Codeplex project download page at https://poshuninstalljava.codeplex.com.

I’d appreciate any feedback or results of any testing that any of you do, either on the Codeplex project site or as a comment against this post. Documentation should be sufficient for anyone with basic Powershell knowledge and as time allows I’ll try and improve both the documentation in Codeplex and within the code itself.

Validate Parameters in Powershell

I ran into this post whilst browsing Powershell.Org, for useful tidbits for improving my Powershell skills. If you’re starting to write modules or functions, then you have probably started to think about validating inputs. For a few modules I wrote big chunks of code to try and catch invalid inputs. Now, having read the following article, I have slapped myself silly for not finding out about Powershell’s validation capabilities earlier..

http://learn-powershell.net/2014/02/04/using-powershell-parameter-validation-to-make-your-day-easier/

I’ve come to realise with Powershell that if  something seems complicated then you’re probably doing it wrong.

Uninstall Java the Easy Way (Redux)

Uninstall Java using Powershell and WMI either as a script or module with options to keep a specified version installed and optional list the versions installed.

UPDATE: 1.0 Released, see post here

Download from here if you don’t want to read on: https://poshuninstalljava.codeplex.com

I’ve had a couple of comments from my previous post on uninstalling Java with Powershell. They have been in regards to either enhancing the WMI query or leaving a particular Java version installed. That has spurred me on to rewrite the script in a more complete form and include some optional arguments.

The new script is written as a Powershell module (you’ll see the psm1 extension on the file) so the module will have to be loaded and then called just like a normal cmdlet. There’s lots of info on the web about how to use modules, so I won’t repeat that here, but to quickly load the module use “ipmo .\uninstall-java.psm1” from Powershell.

The cmdlet has two optional arguments:

-KeepVersion “Java version number” will uninstall all other versions of Java except the versions that match the string provided. Using “7.0.51” will retain Java version 7 Update 51, for example. Note that this switch does a match on the start of the version string, so passing “7” will keep all updates of the major version 7 whilst uninstalling versions 6 and 5.

-Whatif will flag the cmdlet to not uninstall any software but will output (as an object from WMI) all the software that matches the query that would be uninstalled if the switch was not used. This can also be used to get a list of all versions of Java that are installed as a series of objects that can be piped to other powershell commands or to a text file.

In either mode the cmdlet outputs the results as an array of WMI objects which can be piped into other commands.

I’ve included in the WMI query a set of additional product name matches supplied by a commenter “Carsten” which excludes a number of non Oracle software packages that otherwise would have been accidentally triggered by the cmdlet. There is a line early in the module which sets the base query string which reads something like:

$query=“select * from win32_Product where (Name like ‘Java %’ or Name like ‘Java(TM)%’ or Name like ‘J2SE%’) and (Name <> ‘Java Auto Updater’) and ((Vendor=’Sun Microsystems, Inc.’) or (Vendor=’Oracle’)) and (NOT Name like ‘%CompuGROUP%’) and (NOT Name like ‘%IBM%’) and (NOT Name like ‘%DB%’) and (NOT Name like ‘%Advanced Imaging%’) and (NOT Name like ‘%Media Framework%’) and (NOT Name like ‘%SDK%’) and (NOT Name like ‘%Development Kit%’)“

This is where you can add any other possible strings to exclude from the un-installation process.

A final word of warning. This module was written as a quick way of screening and removing Java from computers in an environment where there hasn’t been a lot of control in terms of software distribution. As you’ll see from the changes to the WMI query, there are a number of software packages that could have been accidentally removed because of the broad query used. I suggest you run the cmdlet with the -Whatif switch and capture the output for review before running properly.

UPDATE: The Win32_Product WMI class will cause the MSI Installer service to scan through all installed applications and generate a number of events (ID 1035) but otherwise does not seem to cause major issues. The Microsoft KB articel is here: http://support.microsoft.com/kb/974524/en-us. There is no way to avoid this and using the Win32_AddRemovePrograms class offers no uninstall() method. For the time being if you want to use this script then I would avoid using it as a login script or s start up script which may slow down login times..

You can download the file from the Codeplex project I’ve started : https://poshuninstalljava.codeplex.com

EDIT: There is now a ps1 version to provide the same functionality but as a script rather than a full module for those who may want to remotely launch or embed in something like SCCM or Group Policy.

Uninstall Java the Easy Way.

Much has been posted in regards to Java and how to uninstall it. A lot of scripts rely on knowing every product GUID in the registry and calling msiexec for each GUID to rid yourself of Java. After a bit of research I came up with the following approach using Powershell and WMI that avoids needing to know every GUID.

It uses a WMI query to find any product names that match specific strings. You’ll see in the code I look for ‘Java %’, with a space as there are lots of visual studio add ins that refer to “javascript”. I also look for ‘Java(TM)%’ as the JRE 6 runtime uses this specific string in it’s name. I also put into the query a filter to exclude Java Auto Updater which will be removed by the main program uninstaller and if called directly throws a 1603 return code.

I have also added a check to find any running Java processes. This is necessary as if you call the Uninstall() method, with Java processes running, the computer will reboot immediately on uninstall, without any warning.

The final part of the script will loop through the installed programs and call the Uninstall() method on each one. I plan to add a check to the script to verify that the program has an uninstall method, as some programs don’t have this method.

A word of warning that this has only been tested in my environment and I give no guarantee that this script will work correctly in your environment or won’t cause any unexpected results. You should test this script in your environment properly before use.

EDIT: I’ve updated the script to provide some more functionality and parameters in line with the Powershell Module version in a later post. I suggest you read the later post. The code is now hosted as a codeplex project to make any future changes easier to manage. Link is below.

https://poshuninstalljava.codeplex.com/