The OpenId Authentication 2.0 spec's section on nonces allows for nonces to be made up of any and all ASCII characters in the range of 33-126 inclusive. This includes the characters < and &, which are significant enough in HTML cross-site scripting attacks that ASP.NET does some validation on HTTP requests and throws HttpRequestValidationException if these characters show up in potentially dangerous ways.
These special characters that ASP.NET checks for are just two of many in the allowed characters, so you usually won't see this exception, but occasionally your visitors will if you have an ASP.NET OpenId-enabled web site (whether your site is a Provider or Relying Party). Here is a sample that I randomly saw while developing DotNetOpenId.
Server Error in '/' Application.
A potentially dangerous Request.QueryString value was detected from the client (openid.response_nonce="...T20:12:18Z<Gts#:[\").Description: Request Validation has detected a potentially dangerous client input value, and processing of the request has been aborted. This value may indicate an attempt to compromise the security of your application, such as a cross-site scripting attack. You can disable request validation by setting validateRequest=false in the Page directive or in the configuration section. However, it is strongly recommended that your application explicitly check all inputs in this case.
Exception Details: System.Web.HttpRequestValidationException: A potentially dangerous Request.QueryString value was detected from the client (openid.response_nonce="...T20:12:18Z<Gts#:[\").
The spec cannot be changed to exclude these characters, so while DotNetOpenId can be altered to avoid emitting these characters as part of nonces to help lessen the problem, any ASP.NET web site that supports OpenId whether it uses DotNetOpenId for its library or not can receive requests with these characters from other web sites that are correctly implementing the OpenId 2.0 specification.
Since you don't want your web site crashing on your visitors who are trying to log in, the only way to avoid this occasional exception is to disable request validation. Since this exposes you to greater security threats (the XSS attacks that the validation was intended to prevent), you should only disable request validation on your dedicated login page and provider endpoint, and perform any of your own request validation on those pages that may be necessary.
You can disable request validation on your relying party login or provider endpoint page by adding the validateRequest="false" directive to your <@ Page ... > tag.