Hello all,

Im posting this here in case it might help anyone else. Not so much of an AI issue as a networking issue, but it relates to specific questions regarding RICH. Most people who have visited have commented on the speed and no doubt wondered if it was a problem with the the architecture itself.  this is excerpted and hopefully explains whats going on;

“There can be as much as an 80% variation in response times when asking the same question even on the same day when querying the instance on the production server, whereas an identical instance (same modules, same data,etc…) averages about (3) three seconds on the development machine. I think I finally traced what is going on. The machine that Im using for the RICH production server was originally a backup file server and its configured to use RAID 0 (mirrored) via the Windows software RAID.  I had forgotten that when I set it up, it was ridiculously slow and when researching the problem discovered that this was simply an attribute (and major complaint) of the Windows RAID.

http://www.tomshardware.com/forum/10653-45-windows-software-raid-volume-extremely-slow

Having the RAID rebuild for no apparent reason seems to be something that happens with the Windows RAID, and that would explain the variance.

I used .NET when building RICH for security and other reasons, and In addition to the usual speed problems the use of the Windows RAID seems to be having an effect on how .NET is functioning.  One of the advantages to ASP.net is that its supposed to compile once and then run fast forever. Its usually a significant advantage in speed. But something about the RAID seems to be causing .NET to compile on each query…sometimes. Thats something else that seems to be factoring into the speed variance. When I uploaded the changes detailed below and tested them RICH ran faster when executing the first query. (Still slow) The primary .dll was uploaded pre-compiled so I think that has something to do with it.  I know that .NET maintains a checksum in order to determine if there has been changes to the source code so that it can re-compile if the source code changes, and maybe something about the RAID is corrupting the checksum at irregular intervals.  I dont know, but its probably isnt worth trying to make it work, the solution is to get RICH off this box. I’ve thought about just going in and breaking the mirror, but in doing that I run the risk of the drives becoming corrupted which would cause me to go from"slow speed” to “no speed”.  “

Anyway, thats what Ive discovered, and if there is a lesson here that can be used elsewhere it is

NEVER USE SOFTWARE RAID

Im hoping to have the problem resolved this week and get the production speed to a point where it is comparable to the development machine. In the meantime thanks for bearing with the slow response times.

Vince

Still waiting for a budget (and here I use the term budget loosely) for upgrading the server, so in the meantime Ive “kluged” (and here I use the term accurately) a workaround that allows external users to access the development RICH instance. Even with the redirect it looks like response times are in the 2 to 3 second range as opposed to the production machines 8 to 12 second range (Some users reported even longer) . Still not sure If Im going to leave it up, even though its firewalled reduntantly Im somewhat loathe to allow access, but the response was becoming an issue that was affecting how long users were chatting. Hopefully I can have a permanent fix soon. The normal links should get you onto the alternate machine. If you get a chance to check it out please do.

Vince

Depending on your setup, you could go for a mid range NAS box.  You can get a 2nd hand, refurbed HP MSA60 with 12 146GB SAS drives for about $400 on eBay and that will give you pretty good drive horsepower.

Firewire that to your main server and you’ll have a pretty good demo setup storage wise.

You can daisychain most of them too, the MSA60 can be set up so that 1 is a master and 3 slaves, and hit a max throughput of about 2GB/s per set over 48 discs.  We use these ourselves for our crawled page data and none critical stuff, we can scale out easily and achieve commercial grade performance when the time is near.

If you want some serious performance, we have 2 of these with server-grade SSD drives in which are used for our databases, and the performance of that is quite literally breathtaking….in the order of 1.2M IOPS….expensive tho.

Thanks Dan,

Some serious hardware would be nice. At this time it depends on what kind of budget can be allocated.

Vince

@Vincent: Have you ran your software through a performance profiler already?  Fixing some bottlenecks can often have incredible results.

Hi Jan
Each module has a protected class that provides profiling and debugging services. The real clue however was setting up the instance that resides on the development machine with the identical architecture that resides on the production maching and comparing the response times.  The difference was staggering and led me to allow access to the DEV instance at least for now. Im fairly happy with the response time for now, although faster versions of certain functions have been created.

Vince