[UPDATE: Thanks to Carsten Groth, who discovered the IIS 7.0 way to do things! Also, my problem isn’t fixed. So, this article is simply about performance improvements.]
I recently became engaged with Microsoft Support trying to fix a problem that occurred when the Application Pool for the CRM Site recycled its process worker(s). I have a split-server deployment of CRM, and the affected machine was the Platform server. When it would recycle its workers, many System Jobs and Workflows would become stuck in the “Waiting” state with an error message, which claimed an exception of either HTTP Status 400 (Bad Request), or HTTP Status 401 (Unauthorized).
This would occur for all jobs that ran in a 4 or 5 hour period after the recycle, but the situation would always ultimately resolve itself. Every morning, I would have to resume a large number of jobs. Also, I found that if I forced an application pool recycle, or restarted IIS, the problem would resurface immediately.
The technicians over at Microsoft reviewed my configurations and logs, and made a recommendation to apply KB 917557 to IIS on my web server. The server runs Windows 2003 and IIS 6.0, and the logs would show that each request to the CRM website would first encounter an HTTP Status 401 message, which would force the client to submit authentication for a second connection—that would result in an HTTP Status 200 (OK). Thinking that this could be causing my problem, I was asked to enable Kerberos Auth Persistence using the article above.
One thing to note about enabling Kerberos Auth Persistence is that it effectively cuts the number of requests being processed by IIS in half. When I inquired about security considerations regarding this setting, I was told that there were none as the feature is inherently bound to HTTP Keep-Alive sessions. For IIS 7.0, as pointed out by Carston Groth, reference KB 954873.