Friday, Cam and I were trying to figure out why a new AJAX call was failing for logged out users, but not for logged in users. The exception logs weren't really helpful: somewhere deep inside .NET, the database was timing out after waiting 30s. The code that called the stored procedure is only used in one place in our code, and that code was nowhere near related to the AJAX call we were doing.
I resorted to inspecting the code line-by-line. I almost gave up, when something in the reptilian part of my brain made me reread the return statement:
That can't be it. We're just copying the information about the image into an anonymous object to serialize to JSON.
Although, if the serializer threw an exception, the call stack would be the same as the one we observed... I guess this deserves more attention.
But how would this line cause a crash?
asset.AssetId is a
long; besides, it's an auto property, and those are safe.
ti is a
TemplateImage. That is one of the earliest classes ever written at Cheezburger. It can't be crashing, can it?
Then I realized that
TemplateImage had a
User had a
UserStats had a property that called the stored procedure that was timing out. Suddenly it all made sense!
The JSON serializer was doing its job, making a deep copy of
ti by traversing field and property members as deep as it could. Unfortunately, the
UserStats properties were lazy-loaded from the database, and there were no attributes telling the serializer to skip them.
I've gradually begun to dislike lazy attributes because of how easy it is to forget how much work is going on under the covers. Now I'm convinced that this laziness is evil.
Why was this only affecting logged out users?
If you're logged out, the
User object is set to a default value. When the database was queried for statistics, it was calculating those statistics across all the images that have ever been made by any logged out user. That's an awful lot of data to chug through.
What was the fix?
The quick fix is to serialize the one property we needed from
UserStatistics to get rid of the lazy properties.