

Automatically diagnose IIS and ASP.NET website hangs - marvinli
http://blog.leansentry.com/2013/05/automatically-diagnosing-iis-asp-net-website-hangs/

======
bigdubs
This is pretty awesome for some types of hangs. I would bet most hangs
developers experience are because of locking on SessionState.
IReadOnlySessionState is your friend.

~~~
darrenkopp
I've run into this a lot and ended up changing a lot of the internals of how
session state works and have _mostly_ eliminated the issue for us. Granted,
you still run into it if you have 2 pages running at the same time, but we see
it where sometimes it just doesn't release the state (and it was happening
fairly frequently)

[http://darrenkopp.com/posts/2013/04/10/Playing-with-fire-
Opt...](http://darrenkopp.com/posts/2013/04/10/Playing-with-fire-Optimizing-
the-ASPState-internals.html)

------
DenisM
In addition, when it comes to hanging on AWS EC2, remember to check your ICMP:
[http://www.daemonology.net/blog/2012-11-28-broken-
EC2-firewa...](http://www.daemonology.net/blog/2012-11-28-broken-
EC2-firewall.html)

------
DenisM
I used to have sporadic hangs when it didn't even look like the request
reached the IIS server (hosted on EC2), but I didn't know where to start
looking. Any advice on that?

~~~
mvolo
Hey DennisM,

The first thing I would check is that you can hit the webserver. You can do
some of that with a tool like PortCheck: [http://mvolo.com/check-iis-network-
connectivity-with-portche...](http://mvolo.com/check-iis-network-connectivity-
with-portcheck-v20/).

Best, Mike

~~~
DenisM
I'll try it next time the problem happens. Hope it works with SSL.

What I noticed is that when a hang appears, not even a record in the IIS log
file is made until the hang clears up. All the while ping works just fine. I
guess what I really want is to know when the IIS received the connection, but
before running any ASP.NET code. That would be super helpful in narrowing down
the problems.

------
ldubinets
Whats this? Something related to .NET on HN front page? I must be dreaming...

~~~
mvolo
We share your sentiment! We are a large-scale startup built entirely on the
MSFT stack, in Azure cloud, for the MSFT platform.

People on HN don't often talk about it, but it does happen, and apparently
quite a bit judging by our users :)

~~~
ldubinets
Yeah, the Microsoft stack plays well with startups (and enterprises too,
obviously).

Everybody here seems to preach that "if you want to be in the startup scene
you have GOT to get a MacBook and write Rails apps!" though.

------
graycat
Okay, opinions of experts wanted!

So, apparently part of ASP.NET hangs is the use of SQL Server database to
handle the data needed to support 'session state'.

Somehow, in my gut, for no very good, definite reason, I didn't want to
involve SQL Server or any RDBMS in handling session state.

For a while I was going to handle session state with what is apparently the
usual ASP.NET default, i.e., use 'application level' memory, that is, store
session state in main memory, in an instance of a class which is a value in a
collection class in memory common to the 'application', e.g., all ASP.NET Web
pages running on that instance of Windows and/or IIS.

As I studied the ASP.NET support of session state, it seemed that I had to
hurt my UI/UX by either (1) requiring that the user's browser accept cookies
from my Web site or (2) have ASP.NET/IIS write some gibberish in a strange
place in the URL of my Web site as my users see it. I didn't like either (1)
or (2).

So, I thought of my solution: First, when first see the user, that is, when
get their first HTTP request GET, and, thus, need a new 'session', get a new
GUID value as a character string and use that as a 'session identifier' for
that user.

Second, when send a page to the user, have that session ID value as the
property TEXT in a field marked in HTML as 'hidden' so that it is not
displayed by the user's browser.

Third, then I tried to use application memory to store session state 'my way':
Declare my own class for session state. For each instance of session state,
have an instance of that class. Store the instance as a value in a collection
class used as a key-value store, with key the user's session ID, where the
collection class is a value of the ASP.NET collection class in 'application
memory'.

Then when get a postback from such a page, check the session ID. Now go get
that user's 'session state'.

Somehow I couldn't get that use of application memory to work smoothly. There
seemed to be a clumsy work-around.

So, instead of using application have a little console application that uses
TCP/IP sockets and two instances of a collection class. Right: It's single
threaded. Of course, one of the instances of the collection class is to be a
key-value store where a key is a session ID and a value is a byte array of the
serialized version of the instance for that user of my class for session
state. For the other instance of the collection class, use it to store the
time the session state was last touched (read or written) so that can know
when can decree and declare that the session has 'expired'. Whenever the
single thread is active, in addition to its main work for session state, it
checks for expired sessions and deletes them.

So I did that. I wrote in VB.NET. Some code statistics are:

    
    
                  file lines =    8412
                 blank lines =    1739
               comment lines =    3524
                debug blocks =      56
             writeline lines =     251
                  code lines =    3149
             code statements =    2108
    

It was fun code to write. I left the TCP/IP incoming queue length at the
default of 100, but it could be raised.

I also implemented some 'system management' for the session state server: Any
program able to use TCP/IP can send the session state server, on its IP port
number, a request to report on number of sessions, to shut down, etc.

My timings indicate that on a dedicated 3.0 GHz single core processor, the
code should be able to handle session state for sending 1000 Web pages a
second, that is, write to the session state store and later read it back, this
pair 1000 times a second.

Single threaded? SURE! Why not? Why interrupt the work on a single core
processor?

On a multicore processor? Sure, start more than one executing instance of the
little program and let each instance handle its own part of the full
collection of possible GUID values used for session ID.

Yes, maybe I wrote a baby version of Redis. I may have written the code faster
than I could have mastered Redis.

I've done some back of the envelope arithmetic that indicates that I will have
one heck of a busy Web site before storing session state data in main memory
would be a problem, especially given that even fast ECC main memory now goes
for less than $9 per GB.

Where am I going wrong?

