Hi Ignite,
I’ve been working with Benjamin at Jive and Contegix support to help improve Ignite’s infrastructure. Recently, we have been looking into our bandwidth usage profile and decided we need to implement a local private network inbetween the servers housed at contegix. This private network will help increase security and reduce our internet bandwidth usage. There will be some upcoming outages as we implement these changes and we’ll keep everyone udpated when things change.
thanks!
daryl
For the good of all, here’s the traffic graphs from our main webserver (which also happens to proxy all community traffic, etc):
The Last Year
The Last Week
The Last Day
We aren’t sure if the internal traffic is the primary culprit, but it is contributing some.
One thing I’d like to see is when automated Bamboo builds are running.
A quick update: Thanks to Guus checking logs, we tracked down the primary offender… SPARK!
Spark checks for updates and downloads said updates, and somehow that download endpoint slipped through the cracks when we offloaded downloads to a CDN.
I modified the updater code in the website late last night (US Pacific time) and here is a network graph that shows we’re already seeing a big improvement.
That said, we’ve also been seeing daily outages on the website… I’m optimistic that this decreased load will help that problem, too.
Great that you have found the cause. Btw, you said you have updated something. My monitoring software noticed changes on the Download page and also the absence of Jingle Nodes plugin. have you removed it intentionally or accidentally?
Ah, I’m sorry. It seems that plugin was copied directly into the running website ROOT but not to it’s SVN repo.
I’m going back now to look for that and any other differences which escaped me.
Also i’m still getting a lot of chat presence provider unavailable email notifications.
Yeah, openfire locked up, had to restart it.
Many plugins on the Downloads page are for the Openfire 3.6.4 version. Someone should change them to 3.7.0 compatible ones again. Here they are.
So, Daryl has fixed the plugins issue. Now i want to ask about downloading from this site and especially from Bamboo. Lately it has become very unreliable. As the server doesn’t support download resuming i often have to redownload ~40 MB of Spark installer twice or more times. What are the benefits of not having download resuming? I think it is better for the traffic when you download same 40 MB in a 2-3 passes and not 40 MB again and again. Probably this is also related to a bandwidth. Lately downloading speed jumps all the time and often drops to 0 and then downloading stops and i have to restart it. Maybe it can be adjusted to give more bandwidht at least to Bamboo site? There won’t be many downloads from that place anyway, but would be easier for testers to download a new nightly build every day or even few times per day.