Hi,
One of our major Services, that we monitor with APM, has a Testpage included, which is used for Service related performance monitoring. This Testpage is called quite often with a Duration in between 60 and 90 seconds and therefore ruins our APM Statistics
for the Service.
I want to exclude Events coming from that Testpage. Although it is an aspx .net web page and I know that I can exclude it via defining a transaction with higher Threshold, I wanted to try a different approach. I want to try out the possibilty of being more flexible with Event Exclusion.
Referring to this Blog Post by Daniel Muscetta, I deactivated the standart APM Event Collection Rule ("Microsoft.SystemCenter.Apm.CollectApplicationDiagnosticsEvents", MP: Microsoft.SystemCenter.APM.Infrastructure.mp) and replaced it with our custom rule. I build the custom rule as seen below:
<Rule ID="Company.APM.Customization.EEC.CollectApplicationDiagnosticsEvents" Target="MSAI!Microsoft.SystemCenter.Apm.ApmAgent" Enabled="false" ConfirmDelivery="false" Remotable="true" Priority="Normal" DiscardLevel="100"><Category>None</Category><DataSources><DataSource ID="ApmDatasource" TypeID="MSAI!Microsoft.SystemCenter.Apm.LobDatasource" /></DataSources><ConditionDetection ID="FilterAndAliasing" TypeID="Company.APM.Comtomization.EEC.ApplicationFilter.Excludes"><Source>/DataItem/EventData</Source><Pattern>Testpage.aspx</Pattern></ConditionDetection><WriteActions><WriteAction ID="ApmWriteAction" TypeID="Company.APM.Customization.EEC.CollectApmEventWriteAction.Custom" Target="SC!Microsoft.SystemCenter.CollectionManagementServer" /></WriteActions></Rule>
Short Description of what I have done in that Rule:
- The Datasource is referenced to the original APM Event Collection Rule, as it is a public DS.
- The Condition Detection contains a RegExExpression which checks, if the Pattern (testpage.aspx) is contained in the APM Event Propertybag (here: in the EventData)
- The WriteAction is managed code, set as internal. Therefore I copied it into the Custom Management Pack. It calls managed Code which writes the Data into the Opsmgr Database. The original can be found in Microsoft.SystemCenter.Apm.Infrastructure, WriteActionModule "Microsoft.SystemCenter.Apm.CollectApmEventWriteAction"
For verification, that the DS and the CD are working, I used a Powershell Write Action to write the APM Event Information into the local Eventlog. That worked perfectly fine.
Now to the Issue:
I called the Testpage once with the original Rule and once with thecustom rule active.
For the original Rule, I get a SCOM Alert, as well as a APM Performance Event. The Alert can be found in the dbo.Alert-table as well as SCOM GUI, the APM Event can be found in the apm.Event-Table as well as in Appdiagnostics.
For the custom Rule, I get a SCOM Alert, which can be found in the dbo.Alert Table as well as the SCOM GUI, but Ido not get the APM Performance Event. Neither in the apm.Event-Table (and therefore) nor in Appdiagnostics.
The SCOM Alert contains a link to the APM Event in Appdiagnostics. But when I open that Link, Appdiagnostics gives me the error, that it cannot find theAPM Event in the database (I haven't got a screenshot of that, but I can provide if neccessary)
I started to trace the SQL-Database to see, if the Custom Rule Write Action gets its Data to the Database. I got "RPC:Completed" Events, which looked pretty much the same as the Event, coming from the original Write Action:
<Event id="10" name="RPC:Completed"><Column id="11" name="LoginName">User</Column><Column id="15" name="EndTime">2013-06-12T17:31:12.747+02:00</Column><Column id="12" name="SPID">73</Column><Column id="16" name="Reads">58</Column><Column id="1" name="TextData">
<Column id="9" name="ClientProcessID">4964</Column><Column id="13" name="Duration">4822</Column><Column id="17" name="Writes">14</Column><Column id="2" name="BinaryData"><Column id="6" name="NTUserName">User</Column><Column id="10" name="ApplicationName">SC DAL--AlertWriteActionModule</Column><Column id="14" name="StartTime">2013-06-12T17:31:12.743+02:00</Column><Column id="18" name="CPU">0</Column></Event>
For better readability, I shortened the TextData and the Binary Data. The TextData showed, that the APM Events always call the exec dbo.p_AlertProcess stored procedure.
My Questions now are:
- Why do I not get APM Performance Events into the apm.Event-Table for my Custom Rule?
- How do get APM Performance Events into the apm.Event-Table? I went through the dbo.p_AlertProcess stored Procedure and couldn't find any call to the APM.Event-table.
I know this is a long Post, I hope that formatting the Text helps with the readability.
I also hope, that someone can give me some answers to my problem ;)
Further Questions are welcome..
Regards
Tobias