Author note: As you’ll notice with most of my blog posts, they tend to have a lot of information and are a bit long. Instead of sacrificing content, I’ll break up the posts into a series for posts. This way people don’t get overwhelmed with the amount of content but for those that want the deep technical details, they get that as well.
This article covers Fiddler and is the second article within the Email Router Demystified – Tools for Troubleshooting blog series. To get to a list of the other tools, go HERE.
Reference-
https://community.dynamics.com/crm/b/dynamicscrmsupportblog/archive/2016/05/26/email-router-demystified-tools-for-troubleshooting-fiddler
I could, literally, spend hours on how we use Fiddler to capture data and then how to analyze the data in various different ways. On the surface, the tool seems so simple but it can really give you some very deep data if you understand how to mine the data.
Again, we will simply focus on how to use it as it relates to the Email Router. First, you need to know where to get Fiddler. Well, oddly, this is one of the only non-Microsoft tools that I use. It’s a Telerik tool which you can find here:
What is Fiddler?
Fiddler is a web proxy debugger. What does that mean? Well, in simplistic terms, the http and https requests all go through Fiddler. Fiddler hooks into the Internet Options and configures itself as a proxy. Since it’s a proxy, all of that traffic will go through the application so we can monitor the requests and responses being made.
When should we use Fiddler?
Again, Fiddler will capture http and https traffic. So this means we should only capture this if we want to monitor the request and response for http/https traffic. In the case of the Email Router, there are only a few things we are able to capture Fiddler data:
- Load Data
- Test Access (Only if using Exchange Web Services (EWS))
- Incoming processing using EWS
- Outgoing processing for Exchange Online
- Connectivity to Microsoft Dynamics
That’s it! This tool will not capture SMTP or POP3 traffic. It will only capture EWS traffic. We will talk about other tools to help us with capturing SMTP and POP3 traffic next.
One big gotcha here. Even though Fiddler will capture HTTP traffic, the information you are going to see from the request and response is going to be “encrypted” from WCF. For us to get data from CRM and Exchange, they must both be configured for HTTPS.
How do we use this tool?
This is extremely important to understand. Fiddler will only capture the http/https traffic of the account that ran Fiddler or of services that is using Fiddler as a proxy. This means that if I am logged into a workstation as Alan Jackson and launch Fiddler, it will only capture traffic for Alan Jackson. Ok, that makes sense but why does this matter? Well, by default, the Email Router service is set to run with the Local System account.
To capture the traffic for the Email Router service, we must do one of two things:
- Run the service as the logged on user
- Create the Microsoft.Crm.Tools.EmailAgent.exe.config file and define the Fiddler proxy
Run the service as the logged on user
What we need to do is to change the Log On As to the logged on user. Now there are a couple of caveats here for CRM on premises customers. The account that is running the Email Router service needs to be a CRM user or it needs to be a member of the PrivUserGroup. The reason is that the service is the one responsible for retrieving the users and queues within CRM. If the service account is not a CRM user or not a member of the PrivUserGroup, we will fail to actually connect to CRM EVEN if the Deployment tab is set to use a CRM user.
Create the Microsoft.Crm.Tools.EmailAgent.exe.config file and define the Fiddler proxy
If we are not able to run the service as the logged on user, we can configure the Email Router service to send all the traffic to the Fiddler proxy. To do that, we must first stop the Email Router service and then create the Microsoft.Crm.Tools.EmailAgent.exe.config file found here:
C:\Program Files\Microsoft CRM Email\Service
After creating the file, we set the following information within the file:
<configuration>
<system.net>
<defaultProxy>
<proxy bypassonlocal="False" usesystemdefault="True" proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
</configuration>
Before this can work, you must have Fiddler installed and configured properly otherwise the Email Router service will fail to connect. It’s important that you either delete or rename this config file after you are done capturing trace data with Fiddler.
Now we have the Email Router service configured to be able to capture traffic through Fiddler, what do we do next?
Configuring Fiddler for HTTPS
The first thing we need to do is install Fiddler. However, that process is pretty straight forward so we won’t go into that. After that, we need to configure Fiddler. The configuration depends on what you did in the previous step. If you created the config file, then you need to be sure to configure
Fiddler to Reverse proxy.
Everyone then needs to go through this next step. For standard HTTP traffic, there is no additional configuration. However, for HTTPS traffic, we need to make sure that Fiddler can decrypt the SSL traffic by being a certificate proxy. We do that by:
- Open Fiddler
- Go to Tools > Fiddler Options
- Click on HTTPS tab
- Ensure “Capture HTTPS CONNECTs” is checked and then check “Decrypt HTTPS traffic”
After doing this, it’s going to prompt you with some interesting pop ups about trusting the Fiddler Root certificate. Simply click on Yes on each prompt.
At this point, click on OK to close any remaining windows and close and re-launch Fiddler.
Capture traffic using Fiddler
Now that Fiddler is installed and configured, we are ready to capture our HTTP/HTTPS traffic. When Fiddler is launched, it will automatically start capturing HTTP/HTTPS data for the logged in user. There are a few things to review about Fiddler that can make it easier to review and correlate the data. These are:
- Result – This is the HTTP response code back from the server. You can review what each response code means in this article.
- Host – This is the server that the request was sent to
- URL – This is the actual page that the client is trying to access
- Process – This is the process making the requests
Unless you are putting in
filters in the Fiddler trace capture, you may capture a lot of other traffic that is not relevant to the issue at hand. Using these fields, we can validate the request is coming from the Email Router and what service is handling that part of the router request.
Request/Response Bodies
After we have isolated a particular line we want to learn more about, we can look at some really good details of the request being made by the email router and then the response back from the server. We do this in Fiddler within the right-hand pane and by clicking on Inspectors tab.
The messages that are going to be important to us are going to be the ones that have a result of success (2xx) or of server failure (5xx). If the client is failing, it will have a result of 4xx and that is something like authentication failures or certificate problems. While we could gain more from those request/responses, the more important things we focus in on are 2xx and 5xx result HTTP codes.
So the important thing to know here is that not all errors are going to be logged as a 5xx result code. 5xx really means that the server was unable to process the request. However, most of the time, the server is able to process the request and provides a response back to the client as a 200. We have to actually look into the response body to determine what the failure was when the server handled the request.
Request Bodies
The proper request bodies can tell us EXACTLY what was asked of the server. Since we are connecting to service endpoints for both Exchange and CRM, the request should contain XML details that help quickly parse through the data.
We can then take the details we have pulled from the request XML and use it in the next tool we will cover, EWS Editor.
Response Bodies
So now that we know what the client requested of the server, what did the server give us back in a response. Now, as long as we got a 200 result, we will see the response from the server within an XML format. Again, this helps us easily parse through data.
Now this is good information when the response result was 200. However, if we get an error message, it’s possible we may need to look at the raw data to get more details of what may be going on:
EWS Editor
What happens when the email router fails to access a mailbox or it fails to retrieve an email from the mailbox? The problem is typically an issue with Exchange. To help confirm the issue with Exchange and to test some configurations without having to make changes in CRM or the email router, we can use EWS Editor. EWS Editor will allow us to make the same calls to Exchange and see what the response back from Exchange is. It helps narrow and isolate the issue.
The part of EWS that is most helpful to us is the EWS POST tool:
At first, this may be overwhelming. The fact of the matter is that it’s very easy. Essentially what we need to do is provide some basic information.
Authentication
The authentication details are based off of what you are providing in the Email Router Configuration Manager for the profile using EWS:
Headers
We pull the headers from the Fiddler trace. Really the only thing we need is the verb which is always a POST:
Request Text
Finally, we enter in the Request text which we get from the Request Headers in Fiddler
Now when we click on Run, we SHOULD get the same response back from the server from EWS Editor that we got back from the Email Router within Fiddler:
If we are not getting the same message back, then we should be focusing on a possible load balance scenario in which one server may be out of date or may be down.
