Just want to update you guys on the web server. It was down a few times last night, but I was working on bringing it back.
The best time to troubleshoot a server that goes down under heavy load, is when it's actually under heavy load. So last night was the perfect time to look at what is going wrong with our brand new server. It's exactly what I suspected - a memory utilization issue. I was able to use last night as a perfect moment to tweak memory configuration. When you're limited on memory, you have to find the perfect balance between memory allocation and user load, so that server requests are granted to all users, while not locking up the server. I think I was able to achieve this balance last night. Or, I'm at least very close. We have close to 100 active users on the boards last night, which is an insane amount for one little virtual web server.
That brings me to the next topic... why is this brand new dedicated server not as reliable as our old host?
We were VERY lucky with what we got out of our old server. That was a server we shared with many other 1and1 customers. Since the other customers weren't using nearly as much resources as our site was, we were able to get away with a lot of load during peak usage without any problems. However, if another customer on our old shared server had a ton of traffic at the same time as us, the server would've hosed, and we could do nothing about it until 1and1 brought it back up. That's not a situation I'd want to be in. Now we have complete control, we can restart the server when it locks up.
These two NIT games are a perfect opportunity to analyze the server while it's under heavy load. I'm still tweaking memory, and hope to do some more work tonight during and after the game. I'd say the server is already about 50% more efficient today than it was yesterday, so I'm making progress.
