Putting SCOM agents in maintenance mode using notification channel.

This is another way of putting SCOM Agents in maintenance mode.

I faced a problem with the earlier management pack wherein there was a rule running against the management servers. You can take a look at this approach in my earlier posts.

Whenever the rule failed to load, the agents failed to get in maintenance mode. This approach also takes into consideration the SCOM servers not being put in maintenance mode.

In the current approach I use the command notification channel together with the management pack rule for alerting on sccm reboot events and putting the agents in maintenance mode.

1. Create an alert in SCOM when a computer is rebooted by SCCM
   
2. Run a powershell script using command notifications channel to put the computer in maintenance mode.

Create a new Command notification channel in SCOM

 The text in the screenshot is below.

Path for command file:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Command line parameters:

"c:\ScomScripts\StartMaintenanceMode.ps1" "-computer" "$Data/Context/DataItem/ManagedEntityDisplayName$"

Startup folder:

C:\Windows\System32\WindowsPowerShell\v1.0

Script:

Create a folder in your SCOM server and name it SCOMScripts.

Add the following script in that folder. Be sure to add your SCOM server names in the script.

param([string] $Computer)
Function Out($data)
{
 Out-File -FilePath C:\SCOMScripts\MaintenanceModeOutput.txt -InputObject $data -Append
}

$data = "Starting maintenance mode Script" +  "`t" + $(get-date).ToString()
Out $data
$data = $Computer + " was rebooted"
Out $data

## Start Maintenance mode on Computers which are rebooted by ConfigMgr
Function WriteEvent ($Messages,$ID) { $Messages = $Messages + "`t" + $(get-date).ToString()
Write-EventLog -LogName Application -Source PatchingSuppress -EventId $ID -Message $Messages }

if($Computer -notmatch "yourscomservername")
{
try {
WriteEvent "Starting MaintenanceMode script for $Computer" "1237"
Import-Module OperationsManager
$Instance = Get-SCOMClassInstance -Name $Computer
$Time = ((Get-Date).AddMinutes(15))
Start-SCOMMaintenanceMode -Instance $Instance -EndTime $Time -Reason "PlannedApplicationMaintenance" -Comment "Applying software update."
WriteEvent "Ending MaintenanceMode script" "1238"
}

catch
{
  $_.exception
  Out $_.exception.message
}
}

else
{
 Out "This was a management server $computer"
}

The management pack for creating the event is below.

 <?xml version="1.0" encoding="utf-8"?>
<ManagementPack SchemaVersion="2.0" ContentReadable="true" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Manifest>
    <Identity>
      <ID>YourCompany.Patching.Maintenance</ID>
      <Version>1.0.0.7</Version>
    </Identity>
    <Name>YourCompany.Patching.Maintenance</Name>
    <References>
      <Reference Alias="SystemCenter">
        <ID>Microsoft.SystemCenter.Library</ID>
        <Version>7.0.8448.6</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="MicrosoftWindowsLibrary7585010">
        <ID>Microsoft.Windows.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="Health">
        <ID>System.Health.Library</ID>
        <Version>7.0.8443.6</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="SystemLibrary7585010">
        <ID>System.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
    </References>
  </Manifest>
  <Monitoring>
    <Rules>
      <Rule ID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" Enabled="true" Target="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Alert</Category>
        <DataSources>
          <DataSource ID="DS" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.EventProvider">
            <ComputerName>$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/NetworkName$</ComputerName>
            <LogName>System</LogName>
            <Expression>
              <And>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="UnsignedInteger">1074</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">PublisherName</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="String">User32</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>The process C:\Windows\CCM\Ccmexec.exe</Pattern>
                  </RegExExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>has initiated the restart of computer</Pattern>
                  </RegExExpression>
                </Expression>
              </And>
            </Expression>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
            <Priority>1</Priority>
            <Severity>0</Severity>
            <AlertOwner />
            <AlertMessageId>$MPElement[Name="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage"]$</AlertMessageId>
            <AlertParameters>
              <AlertParameter1>$Data/LoggingComputer$</AlertParameter1>
              <AlertParameter2>$Data/EventSourceName$</AlertParameter2>
              <AlertParameter3>$Data/EventNumber$</AlertParameter3>
              <AlertParameter4>$Data[Default='']/EventDescription$</AlertParameter4>
            </AlertParameters>
            <Suppression />
            <Custom1 />
            <Custom2 />
            <Custom3 />
            <Custom4 />
            <Custom5 />
            <Custom6 />
            <Custom7 />
            <Custom8 />
            <Custom9 />
            <Custom10 />
          </WriteAction>
        </WriteActions>
      </Rule>
      <Rule ID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" Enabled="true" Target="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Alert</Category>
        <DataSources>
          <DataSource ID="DS" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.EventProvider">
            <ComputerName>$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/NetworkName$</ComputerName>
            <LogName>System</LogName>
            <Expression>
              <And>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="UnsignedInteger">1074</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">PublisherName</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="String">User32</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>DoesNotContainSubstring</Operator>
                    <Pattern>ccmexec</Pattern>
                  </RegExExpression>
                </Expression>
              </And>
            </Expression>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
            <Priority>1</Priority>
            <Severity>0</Severity>
            <AlertOwner />
            <AlertMessageId>$MPElement[Name="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage"]$</AlertMessageId>
            <AlertParameters>
              <AlertParameter1>$$</AlertParameter1>
              <AlertParameter2>$Data[Default='']/EventDescription$</AlertParameter2>
            </AlertParameters>
            <Suppression />
            <Custom1 />
            <Custom2 />
            <Custom3 />
            <Custom4 />
            <Custom5 />
            <Custom6 />
            <Custom7 />
            <Custom8 />
            <Custom9 />
            <Custom10 />
          </WriteAction>
        </WriteActions>
      </Rule>
    </Rules>
  </Monitoring>
  <Presentation>
    <Folders>
      <Folder ID="Folder_5e3e9391b6394ab288bd1c95f83e90cd" Accessibility="Public" ParentFolder="SystemCenter!Microsoft.SystemCenter.Monitoring.ViewFolder.Root" />
    </Folders>
    <StringResources>
      <StringResource ID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage" />
      <StringResource ID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage" />
    </StringResources>
  </Presentation>
  <LanguagePacks>
    <LanguagePack ID="ENU" IsDefault="false">
      <DisplayStrings>
        <DisplayString ElementID="YourCompany.Patching.Maintenance">
          <Name>YourCompany Patching Maintenance</Name>
          <Description>Author: Parag Waghmare
Reason: Created to suppress the servers during reboots initiated by patching</Description>
        </DisplayString>
        <DisplayString ElementID="Folder_5e3e9391b6394ab288bd1c95f83e90cd">
          <Name>YourCompany Patching Maintenance</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule">
          <Name>YourCompany Configmr Initiated Reboot Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage">
          <Name>YourCompany Configmr Initiated Reboot Alert</Name>
          <Description>Computer:{0}
EventSource:{1}
EventID:{2}
Event Description: {3}</Description>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="DS">
          <Name>DS</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="Alert">
          <Name>Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule">
          <Name>YourCompany Server User Initiated Reboot Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage">
          <Name>YourCompany Server User initiated Reboot Alert</Name>
          <Description>Computer Name:{0}Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/PrincipalName$
Event Description: {1}</Description>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" SubElementID="Alert">
          <Name>Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" SubElementID="DS">
          <Name>DS</Name>
        </DisplayString>
      </DisplayStrings>
      <KnowledgeArticles></KnowledgeArticles>
    </LanguagePack>
  </LanguagePacks>
</ManagementPack>

Scom Maintenance mode scheduler

#SCOM Maintenance mode scheduler. The assemblies can be found in this folder on your management servers  C:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\SDK Binaries
$scriptdir = Get-location
$ScriptDir = split-path -parent $MyInvocation.MyCommand.Path
$void = [System.Reflection.Assembly]::LoadFile(“$scriptdir\Microsoft.EnterpriseManagement.OperationsManager.dll”)
$void3 = [System.Reflection.Assembly]::LoadFile(“$scriptdir\Microsoft.EnterpriseManagement.Runtime.dll”)
$void1 = [System.Reflection.Assembly]::LoadFile(“$scriptdir\Microsoft.EnterpriseManagement.Core.dll”)
$server = “testsql”
$RootMS = “testdc01”
$Minutes =  60
$Comment = “Scheduled Maintenance Request”
$MGConnSetting = New-Object Microsoft.EnterpriseManagement.ManagementGroupConnectionSettings($RootMS)
$MG = New-Object Microsoft.EnterpriseManagement.ManagementGroup($MGConnSetting)
$Admin = $MG.GetAdministration()
$Agents = $admin.GetAllAgentManagedComputers()
$agent = $Agents | ?{$_.Computername -match $server}
$monitoring =$MG.Monitoring
$AllClasses = $MG.GetMonitoringClasses()
$MyClass = $AllClasses | ?{$_.name -eq “Microsoft.Windows.Computer”}
$Objects = $Mg.GetMonitoringObjects($Myclass)
$Object = $Objects | ?{$_.name -match $server}
$Object.schedulemaintenancemode
$startTime = [Datetime]’12/21/2016 04:26:00′
$StartTimeUTC = $startTime.ToUniversalTime()
$EndTime = $startTimeUTC.AddMinutes(’10’)
$Object.ScheduleMaintenanceMode($startTimeUTC,$EndTime,”PlannedOther”,$Comment)

Put Scom Agents and Cluster Servers in Maintenance mode and reboot

Here is a script to put your scom agents including clusters in maintenance mode and rebooting them at a specified time. First you will have to create a task and a management pack in SCOM for rebooting the servers. Link on how to create the management pack is here. https://technet.microsoft.com/en-us/library/hh563486%28v=sc.12%29.aspx?f=255&MSPPError=-2147217396.
Works on System Center Operations Manager 2012 R2.

Create the input files and output files on the locations that you prefer. I have created them in C:\Reboots folder. Add your server names in the input file. Then the script given below can be put in a scheduled task and run at specified times or on demand. You can also run the script manually. Make sure that you modify the maintenance window.

I have not yet added any code to verify and alert if the rebooted servers are not back up.

Will probably do so at a later time. Till then feel free to use this one and modify any way that you like.

#################Script Start ####################################

$RootMS = "RMSName"
$Minutes =  90
$Comment = "Unknown"
#Setting up SCOM connection

Import-Module OperationsManager
$null = New-ScomManagementGroupConnection -ComputerName $RootMS
Add-PSSnapin "Microsoft.EnterpriseManagement.OperationsManager.Client" -ErrorAction SilentlyContinue

$eventLog = New-Object System.Diagnostics.EventLog("Operations Manager")
$eventLog.Source = "Maintenance Mode"

$Servers = GC "C:\Reboots\input.txt"
Function Out($output)
{
Out-File -Filepath "C:\Reboots\output.txt" -InputObject $output -Append
}

Out "###############  Starting Script at $(Get-Date)  ##########################"
foreach($Server in $Servers)
{

$output = "Starting for $Server on $(Get-date)"
Out $output

###Putting the agent and cluster in maintenance mode###########
$agent = Get-ScomAgent | Where-Object { $_.DisplayName –eq $Server -or $_.ComputerName -eq $Server -or $_.PrincipalName -eq $Server }
if(!$agent) { Write-Host "ERROR: $Server is not a monitored system in SCOM." -ForeGroundColor Red; Set-Location $originalPath; return }
$Server = $agent.PrincipalName
$startTime = (Get-Date).ToUniversalTime()
$endTime = $startTime.AddMinutes($Minutes)
if(($clusters = $agent.GetRemotelyManagedComputers())) {
$clusterNodeClass = Get-SCOMClass -Name Microsoft.Windows.Cluster.Node
foreach($cluster in $clusters) {
#$clusterObj = Get-SCOMClass -Name Microsoft.Windows.Cluster | Get-ScomMonitoringObject -Criteria "Name='$($cluster.ComputerName)'"
$clusterobj = Get-SCOMClass -Name Microsoft.Windows.Cluster | Get-SCOMClassInstance | ?{$_.displayname -eq $cluster.ComputerName}
if($clusterObj) {
$clusterObj.ScheduleMaintenanceMode($startTime,$endTime,"PlannedOther",$Comment,"Recursive")
$nodes = $clusterObj.GetRelatedMonitoringObjects($clusterNodeClass)
if($nodes) {
foreach($node in $nodes) {
Out "Putting $node into maintenance mode."
$eventLog.MachineName = $node.Name
$eventLog.WriteEntry("The server entered into maintenance mode $(if($Server -notcontains $node.Name){"on behalf of $Server"}).`r`n`r`nDuration:`t$Minutes minutes`r`nReason:`t$Comment","Information",42)
}
}
}
Out "Putting $($cluster.Computer) into maintenance mode."
New-MaintenanceWindow -StartTime $startTime -EndTime $endTime -MonitoringObject $cluster.Computer -Reason PlannedOther -Comment $Comment
}
}
else {
Out "Putting $Server into maintenance mode."
$eventLog.WriteEntry("The server entered into maintenance mode.`r`n`r`nDuration:`t$Minutes minutes`r`nReason:`t$Comment","Information",42)
New-MaintenanceWindow -StartTime $startTime -EndTime $endTime -MonitoringObject $agent.HostComputer -Reason PlannedOther -Comment $Comment
}
#####Sending commands to the server###########

$Task = Get-SCOMTask -DisplayName "Reboot Computer"
$Overrides = @{Arguments = '"$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/PrincipalName$" "true"'}
$Instance = Get-SCOMClassInstance | ?{$_.Name -eq $agent.name}
Out "Starting task to reboot $($Instance.Displayname) at $(Get-Date)"
Start-SCOMTask -Task $Task -Override $Overrides -Instance $Instance

}

Out "###############  Script Complete at $(Get-Date) #########################"

#################Script End ####################################


I don't take credit for all the scripts that I written. Whenever I can credit the original creators, I do.  If I don't at times please do inform me so that I can include them in the page.