Putting SCOM agents in maintenance mode using notification channel.

This is another way of putting SCOM Agents in maintenance mode.

I faced a problem with the earlier management pack wherein there was a rule running against the management servers. You can take a look at this approach in my earlier posts.

Whenever the rule failed to load, the agents failed to get in maintenance mode. This approach also takes into consideration the SCOM servers not being put in maintenance mode.

In the current approach I use the command notification channel together with the management pack rule for alerting on sccm reboot events and putting the agents in maintenance mode.

1. Create an alert in SCOM when a computer is rebooted by SCCM
   
2. Run a powershell script using command notifications channel to put the computer in maintenance mode.

Create a new Command notification channel in SCOM

 The text in the screenshot is below.

Path for command file:

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe

Command line parameters:

"c:\ScomScripts\StartMaintenanceMode.ps1" "-computer" "$Data/Context/DataItem/ManagedEntityDisplayName$"

Startup folder:

C:\Windows\System32\WindowsPowerShell\v1.0

Script:

Create a folder in your SCOM server and name it SCOMScripts.

Add the following script in that folder. Be sure to add your SCOM server names in the script.

param([string] $Computer)
Function Out($data)
{
 Out-File -FilePath C:\SCOMScripts\MaintenanceModeOutput.txt -InputObject $data -Append
}

$data = "Starting maintenance mode Script" +  "`t" + $(get-date).ToString()
Out $data
$data = $Computer + " was rebooted"
Out $data

## Start Maintenance mode on Computers which are rebooted by ConfigMgr
Function WriteEvent ($Messages,$ID) { $Messages = $Messages + "`t" + $(get-date).ToString()
Write-EventLog -LogName Application -Source PatchingSuppress -EventId $ID -Message $Messages }

if($Computer -notmatch "yourscomservername")
{
try {
WriteEvent "Starting MaintenanceMode script for $Computer" "1237"
Import-Module OperationsManager
$Instance = Get-SCOMClassInstance -Name $Computer
$Time = ((Get-Date).AddMinutes(15))
Start-SCOMMaintenanceMode -Instance $Instance -EndTime $Time -Reason "PlannedApplicationMaintenance" -Comment "Applying software update."
WriteEvent "Ending MaintenanceMode script" "1238"
}

catch
{
  $_.exception
  Out $_.exception.message
}
}

else
{
 Out "This was a management server $computer"
}

The management pack for creating the event is below.

 <?xml version="1.0" encoding="utf-8"?>
<ManagementPack SchemaVersion="2.0" ContentReadable="true" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <Manifest>
    <Identity>
      <ID>YourCompany.Patching.Maintenance</ID>
      <Version>1.0.0.7</Version>
    </Identity>
    <Name>YourCompany.Patching.Maintenance</Name>
    <References>
      <Reference Alias="SystemCenter">
        <ID>Microsoft.SystemCenter.Library</ID>
        <Version>7.0.8448.6</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="MicrosoftWindowsLibrary7585010">
        <ID>Microsoft.Windows.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="Health">
        <ID>System.Health.Library</ID>
        <Version>7.0.8443.6</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="SystemLibrary7585010">
        <ID>System.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
    </References>
  </Manifest>
  <Monitoring>
    <Rules>
      <Rule ID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" Enabled="true" Target="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Alert</Category>
        <DataSources>
          <DataSource ID="DS" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.EventProvider">
            <ComputerName>$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/NetworkName$</ComputerName>
            <LogName>System</LogName>
            <Expression>
              <And>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="UnsignedInteger">1074</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">PublisherName</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="String">User32</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>The process C:\Windows\CCM\Ccmexec.exe</Pattern>
                  </RegExExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>has initiated the restart of computer</Pattern>
                  </RegExExpression>
                </Expression>
              </And>
            </Expression>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
            <Priority>1</Priority>
            <Severity>0</Severity>
            <AlertOwner />
            <AlertMessageId>$MPElement[Name="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage"]$</AlertMessageId>
            <AlertParameters>
              <AlertParameter1>$Data/LoggingComputer$</AlertParameter1>
              <AlertParameter2>$Data/EventSourceName$</AlertParameter2>
              <AlertParameter3>$Data/EventNumber$</AlertParameter3>
              <AlertParameter4>$Data[Default='']/EventDescription$</AlertParameter4>
            </AlertParameters>
            <Suppression />
            <Custom1 />
            <Custom2 />
            <Custom3 />
            <Custom4 />
            <Custom5 />
            <Custom6 />
            <Custom7 />
            <Custom8 />
            <Custom9 />
            <Custom10 />
          </WriteAction>
        </WriteActions>
      </Rule>
      <Rule ID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" Enabled="true" Target="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Alert</Category>
        <DataSources>
          <DataSource ID="DS" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.EventProvider">
            <ComputerName>$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/NetworkName$</ComputerName>
            <LogName>System</LogName>
            <Expression>
              <And>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="UnsignedInteger">1074</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">PublisherName</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="String">User32</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>DoesNotContainSubstring</Operator>
                    <Pattern>ccmexec</Pattern>
                  </RegExExpression>
                </Expression>
              </And>
            </Expression>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
            <Priority>1</Priority>
            <Severity>0</Severity>
            <AlertOwner />
            <AlertMessageId>$MPElement[Name="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage"]$</AlertMessageId>
            <AlertParameters>
              <AlertParameter1>$$</AlertParameter1>
              <AlertParameter2>$Data[Default='']/EventDescription$</AlertParameter2>
            </AlertParameters>
            <Suppression />
            <Custom1 />
            <Custom2 />
            <Custom3 />
            <Custom4 />
            <Custom5 />
            <Custom6 />
            <Custom7 />
            <Custom8 />
            <Custom9 />
            <Custom10 />
          </WriteAction>
        </WriteActions>
      </Rule>
    </Rules>
  </Monitoring>
  <Presentation>
    <Folders>
      <Folder ID="Folder_5e3e9391b6394ab288bd1c95f83e90cd" Accessibility="Public" ParentFolder="SystemCenter!Microsoft.SystemCenter.Monitoring.ViewFolder.Root" />
    </Folders>
    <StringResources>
      <StringResource ID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage" />
      <StringResource ID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage" />
    </StringResources>
  </Presentation>
  <LanguagePacks>
    <LanguagePack ID="ENU" IsDefault="false">
      <DisplayStrings>
        <DisplayString ElementID="YourCompany.Patching.Maintenance">
          <Name>YourCompany Patching Maintenance</Name>
          <Description>Author: Parag Waghmare
Reason: Created to suppress the servers during reboots initiated by patching</Description>
        </DisplayString>
        <DisplayString ElementID="Folder_5e3e9391b6394ab288bd1c95f83e90cd">
          <Name>YourCompany Patching Maintenance</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule">
          <Name>YourCompany Configmr Initiated Reboot Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage">
          <Name>YourCompany Configmr Initiated Reboot Alert</Name>
          <Description>Computer:{0}
EventSource:{1}
EventID:{2}
Event Description: {3}</Description>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="DS">
          <Name>DS</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="Alert">
          <Name>Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule">
          <Name>YourCompany Server User Initiated Reboot Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule.AlertMessage">
          <Name>YourCompany Server User initiated Reboot Alert</Name>
          <Description>Computer Name:{0}Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/PrincipalName$
Event Description: {1}</Description>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" SubElementID="Alert">
          <Name>Alert</Name>
        </DisplayString>
        <DisplayString ElementID="YourCompany.Server.UserInitiated.Reboot.Alert.Rule" SubElementID="DS">
          <Name>DS</Name>
        </DisplayString>
      </DisplayStrings>
      <KnowledgeArticles></KnowledgeArticles>
    </LanguagePack>
  </LanguagePacks>
</ManagementPack>

Putting SCOM agents in maintenance mode during Configmgr SCCM patching using a management pack.

A while ago I was tasked to suppress the alerts from SCOM for servers which were being patched and rebooted. There is a checkbox in SCCM which allows you to suppress the SCOM alerts but it did not work in my case and we got bombarded with alerts during a patching window.

This management pack in your environment should help in catching those alerts and suppressing your servers when there is a reboot for patching. Copy the code below and create your own xml file.
Name it Contoso.Patching.Maintenance.xml before you import it or you can rename it to the company or organization you work for.
But make sure that you replace Contoso everywhere.

This management pack will monitor the agents for an event 1074. Which is the Configmgr initiated reboot.
Then run a power shell script which will put those agents in maintenance mode.


<?xml version="1.0" encoding="utf-8"?><ManagementPack ContentReadable="true" SchemaVersion="2.0" OriginalSchemaVersion="1.1" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <Manifest>
    <Identity>
      <ID>Contoso.Patching.Maintenance</ID>
      <Version>1.0.0.0</Version>
    </Identity>
    <Name>Contoso Patching Maintenance</Name>
    <References>
      <Reference Alias="MicrosoftWindowsLibrary7585010">
        <ID>Microsoft.Windows.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="SystemLibrary7585010">
        <ID>System.Library</ID>
        <Version>7.5.8501.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="SystemCenter">
        <ID>Microsoft.SystemCenter.Library</ID>
        <Version>7.0.8433.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
      <Reference Alias="Health">
        <ID>System.Health.Library</ID>
        <Version>7.0.8433.0</Version>
        <PublicKeyToken>31bf3856ad364e35</PublicKeyToken>
      </Reference>
    </References>
  </Manifest>
  <Monitoring>
    <Rules>
      <Rule ID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule" Enabled="true" Target="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Server.Computer" ConfirmDelivery="true" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Alert</Category>
        <DataSources>
          <DataSource ID="DS" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.EventProvider">
            <ComputerName>$Target/Property[Type="MicrosoftWindowsLibrary7585010!Microsoft.Windows.Computer"]/NetworkName$</ComputerName>
            <LogName>System</LogName>
            <Expression>
              <And>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="UnsignedInteger">EventDisplayNumber</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="UnsignedInteger">1074</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <SimpleExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">PublisherName</XPathQuery>
                    </ValueExpression>
                    <Operator>Equal</Operator>
                    <ValueExpression>
                      <Value Type="String">User32</Value>
                    </ValueExpression>
                  </SimpleExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>The process C:\Windows\CCM\Ccmexec.exe</Pattern>
                  </RegExExpression>
                </Expression>
                <Expression>
                  <RegExExpression>
                    <ValueExpression>
                      <XPathQuery Type="String">EventDescription</XPathQuery>
                    </ValueExpression>
                    <Operator>ContainsSubstring</Operator>
                    <Pattern>has initiated the restart of computer</Pattern>
                  </RegExExpression>
                </Expression>
              </And>
            </Expression>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="Alert" TypeID="Health!System.Health.GenerateAlert">
            <Priority>1</Priority>
            <Severity>0</Severity>
            <AlertName />
            <AlertDescription />
            <AlertOwner />
            <AlertMessageId>$MPElement[Name="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage"]$</AlertMessageId>
            <AlertParameters>
              <AlertParameter1>$Data/LoggingComputer$</AlertParameter1>
              <AlertParameter2>$Data/EventSourceName$</AlertParameter2>
              <AlertParameter3>$Data/EventNumber$</AlertParameter3>
              <AlertParameter4>$Data[Default='']/EventDescription$</AlertParameter4>
            </AlertParameters>
            <Suppression />
            <Custom1 />
            <Custom2 />
            <Custom3 />
            <Custom4 />
            <Custom5 />
            <Custom6 />
            <Custom7 />
            <Custom8 />
            <Custom9 />
            <Custom10 />
          </WriteAction>
        </WriteActions>
      </Rule>
      <Rule ID="Contoso.ConfigMgrInitiated.Reboot.Script.Rule" Enabled="true" Target="SystemCenter!Microsoft.SystemCenter.AllManagementServersPool" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100">
        <Category>Custom</Category>
        <DataSources>
          <DataSource ID="Scheduler" TypeID="SystemLibrary7585010!System.Scheduler">
            <Scheduler>
              <SimpleReccuringSchedule>
                <Interval Unit="Minutes">2</Interval>
              </SimpleReccuringSchedule>
              <ExcludeDates />
            </Scheduler>
          </DataSource>
        </DataSources>
        <WriteActions>
          <WriteAction ID="ExecuteScript" TypeID="MicrosoftWindowsLibrary7585010!Microsoft.Windows.PowerShellWriteAction">
            <ScriptName>SuppressPatchedServers.ps1</ScriptName>
            <ScriptBody>
            ## This script will suppress the patched servers
 Add-PSSnapin "Microsoft.EnterpriseManagement.OperationsManager.Client" -ErrorVariable errSnapin;
 Function WriteEvent ($Messages,$ID) {
 $Messages = $Messages + "`t" + $(get-date).ToString()
 Write-EventLog -LogName Application -Source PatchingSuppress -EventId $ID -Message $Messages
 }

#WriteEvent "Starting patching script in Contoso.Patching.Maintenance.xml mp" "1234"
$Date = Get-Date
$path = $date.Month.ToString() + $date.Day.ToString() + $date.Year.ToString() + ".log"
$minutes= 15
$comment= "Rebooted by Configuration Manager"
$reason="PlannedOther"
$startTime = [System.DateTime]::Now
$endTime = $startTime.AddMinutes($minutes)
$Agents = Get-SCOMAgent
$Alerts = Get-ScomAlert -Criteria {Name like 'Contoso Reboot Initiated Alert' and ResolutionState = 0}

foreach($Alert in $Alerts)
{
$agent = $Agents | Where-Object {$_.DisplayName -eq $alert.MonitoringObjectName}
if(($clusters = $agent.GetRemotelyManagedComputers()))
   {
      $clusterNodeClass = Get-MonitoringClass -Name Microsoft.Windows.Cluster.Node
       foreach($cluster in $clusters)
         {
           $clusterObj = Get-MonitoringClass -Name Microsoft.Windows.Cluster | Get-MonitoringObject -Criteria "Name='$($cluster.ComputerName)'"
            if($clusterObj)
             {
               $clusterObj.ScheduleMaintenanceMode($startTime,$endTime,$reason,$Comment,"Recursive")
               $nodes = $clusterObj.GetRelatedMonitoringObjects($clusterNodeClass)
                if($nodes)
                   {
                   foreach($node in $nodes)
                     {
                       $message = $node.ToString()
                       WriteEvent "Putting node $message into maintenance mode by patching script in Contoso.Patching.Maintenance.xml mp." "1235"
                       }
                   }
              }
              $message = $($cluster.Computer).ToString()
              WriteEvent "Putting cluster computer $message into maintenance mode by patching script in Contoso.Patching.Maintenance.xml mp." "1236"
              New-MaintenanceWindow -StartTime $startTime -EndTime $endTime -MonitoringObject $cluster.Computer -Reason $reason -Comment $comment
          }
    }
   
    else
   {
     $message = $($agent.HostComputer.DisplayName).ToString()
     WriteEvent "Putting server $message into maintenance mode by patching script in Contoso.Patching.Maintenance.xml mp." "1237"
     New-MaintenanceWindow -StartTime $startTime -EndTime $endTime -MonitoringObject $agent.HostComputer -Reason $reason -Comment $comment
   }
   $Alert | Set-SCOMAlert -ResolutionState 255 -CustomField8 "Patching maintenance completed"
 }
 #WriteEvent "Ending patching script in Contoso.Patching.Maintenance.xml mp" "1238"

            </ScriptBody>
            <TimeoutSeconds>60</TimeoutSeconds>
          </WriteAction>
        </WriteActions>
      </Rule>
    </Rules>
  </Monitoring>
  <Presentation>
    <Folders>
      <Folder ID="Folder_5e3e9391b6394ab288bd1c95f83e90cd" Accessibility="Public" ParentFolder="SystemCenter!Microsoft.SystemCenter.Monitoring.ViewFolder.Root" />
    </Folders>
    <StringResources>
      <StringResource ID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage" />
    </StringResources>
  </Presentation>
  <LanguagePacks>
    <LanguagePack ID="ENU" IsDefault="false">
      <DisplayStrings>
        <DisplayString ElementID="Contoso.Patching.Maintenance">
          <Name>Contoso Patching Maintenance</Name>
          <Description>Author: Parag Waghmare
Reason: Created to suppress the servers during reboots initiated by patching</Description>
        </DisplayString>
        <DisplayString ElementID="Folder_5e3e9391b6394ab288bd1c95f83e90cd">
          <Name>Contoso Patching Maintenance</Name>
        </DisplayString>
        <DisplayString ElementID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule">
          <Name>Contoso Reboot Initiated Alert</Name>
          <Description />
        </DisplayString>
        <DisplayString ElementID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule.AlertMessage">
          <Name>Contoso Reboot Initiated Alert</Name>
          <Description>Computer:{0}
EventSource:{1}
EventID:{2}
Event Description: {3}
</Description>
        </DisplayString>
        <DisplayString ElementID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="DS">
          <Name>DS</Name>
        </DisplayString>
        <DisplayString ElementID="Contoso.ConfigmgrInitiated.Reboot.Alert.Rule" SubElementID="Alert">
          <Name>Alert</Name>
        </DisplayString>
        <DisplayString ElementID="Contoso.ConfigMgrInitiated.Reboot.Script.Rule">
          <Name>Contoso suppress patching server script</Name>
        </DisplayString>
      </DisplayStrings>
    </LanguagePack>
  </LanguagePacks>
</ManagementPack>