Radified Community Forums - Rad Resource Usage at New VPS Server

Sunday_Afternoon

Gnarly

Offline

Long-time Radified lurker...

Posts: 26

	Re: Rad Resource Usage at New VPS Server Reply #45 - Mar 4^th, 2008 at 7:16pm It's nice that this thread has gone quiet... Ooops!
	IP Logged

NightOwl

Radministrator

Offline

"I tought I saw a puddy
tat..."

Posts: 5826
Olympia, WA--Puget Sound--USA

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #46 - Mar 5^th, 2008 at 1:30am Rad Approx. 11:15 PM, 50 Guests on-line, and 5-10 second delays for response from server--here's a PingPlot. About 10 minutes later, and only 30 Guests on-line and the response time was much better!
	PingPlotter.JPG (89 KB \| ) ____________________________________________________________________________________________ No question is stupid ... but, possibly the answers are ! IP Logged

Rad.Test

Technoluster

Offline

Rad's non-Admin test-profile
in Firefox

Posts: 108

Back to top

Re: Rad Resource Usage at New VPS Server
Reply #47 - Mar 5^th, 2008 at 3:40pm

I am certainly no networking expert, but it would seem the problem is coming from hops #4 (Comcast, 60% packet loss) and #9 (Savvis, 40% packet loss).

*Any* packet loss is bad, and a sign of a problem. But 60% is high.

Also #8 (AT&T, had one trace at 102, which is quadruple the average, tho no PL).

The last 3 hops (I believe) are WiredTree, and they look good.

Sometimes a simple refresh can find a different route, bypassing problematic hops. That's kinda how the web works.

Maybe Magoo can elaborate. That's his bailiwick.

By the way, you might be interested to know (I was) that the President of WiredTree (Zac) told me they purposefully block the names of our/their upstream nodes (notice blank dashes, instead of DNS names), cuz most people do not want others to know the identity of their providers.

Not sure if this is a security issue, or a business issue (or both).

Thanks for taking the time to run and post a tracert.

IP Logged

MrMagoo

Übermensch

Offline

Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)

Back to top

Re: Rad Resource Usage at New VPS Server
Reply #48 - Mar 5^th, 2008 at 7:21pm

The traces to the destination look good. Like you said, the last 3 hops look good. Since the packet has to go through all of the routers in between to get there, all the hops along the way are probably ok. Traceroute packets are a very low priority for a router, so busy routers sometimes drop them (creating apparent packet loss) while normal traffic gets through with no issues. Those routers showing packet loss are probably busy, but not saturated.

I'd be interested in seeing another traceroute from a different starting point next time there are issues to compare the two graphs. Server load graphs from the time in question would be fun to compare against too.

Everything is lightning quick now...

IP Logged

Rad

Radministrator

Offline

Sufferin' succotash

Posts: 4090
Newport Beach, California

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #49 - Mar 5^th, 2008 at 7:46pm MrMagoo wrote on Mar 5^th, 2008 at 7:21pm: Everything is lightning quick now... Music to my ears. Regarding load stats .. that's something I can do: Note there is a 2-hour diff between here and Chicago.
	IP Logged

MrMagoo

Übermensch

Offline

Resident Linux Guru

Posts: 1026
Phoenix, AZ (USA)

Back to top

Re: Rad Resource Usage at New VPS Server
Reply #50 - Mar 6^th, 2008 at 5:04am

Hmm, nothing in the load graph around the time of the incident. I'm seeing some big delays right now, but its that time of the night that the rkhunter cron job is probably running. Network looks good; Here is a trace:

4 68.2.12.114 (68.2.12.114) 6.219 ms 6.539 ms 14.585 ms
5 68.2.12.26 (68.2.12.26) 8.110 ms 7.463 ms 7.730 ms
6 68.2.12.30 (68.2.12.30) 8.185 ms 10.507 ms 6.797 ms
7 chnddsrj01-ae1.0.rd.ph.cox.net (68.2.14.1) 10.426 ms 6.558 ms chnddsrj02-ae1.0.rd.ph.cox.net (68.2.14.13) 7.924 ms
8 langbbr01-ae0.r2.la.cox.net (68.1.0.232) 21.598 ms 20.870 ms 22.337 ms
9 vl3490.ccr02.lax01.atlas.cogentco.com (154.54.6.189) 23.484 ms vl3492.mpd01.lax01.atlas.cogentco.com (154.54.3.9) 19.688 ms vl3490.ccr02.lax01.atlas.cogentco.com (154.54.6.189) 23.260 ms
10 te4-2.ccr01.sna02.atlas.cogentco.com (154.54.3.37) 24.245 ms 23.259 ms te2-4.ccr02.iah01.atlas.cogentco.com (66.28.4.237) 46.853 ms
11 te2-1.ccr02.dfw01.atlas.cogentco.com (154.54.3.177) 45.213 ms te2-1.mpd01.dfw01.atlas.cogentco.com (154.54.5.133) 48.570 ms te8-2.mpd01.iah01.atlas.cogentco.com (154.54.1.66) 46.265 ms
12 te7-3.mpd01.mci01.atlas.cogentco.com (154.54.3.18) 53.138 ms te7-2.ccr02.mci01.atlas.cogentco.com (154.54.5.166) 56.350 ms te4-1.ccr02.ord01.atlas.cogentco.com (154.54.6.254) 66.454 ms
13 te2-3.mpd01.ord01.atlas.cogentco.com (154.54.7.137) 66.019 ms te4-1.ccr02.ord01.atlas.cogentco.com (154.54.6.254) 69.519 ms 66.591 ms
14 te4-1.ccr01.ord04.atlas.cogentco.com (154.54.24.178) 66.610 ms te4-3.ccr01.ord04.atlas.cogentco.com (154.54.24.182) 65.050 ms te7-4.mpd01.ord01.atlas.cogentco.com (154.54.2.190) 69.937 ms
15 38.104.106.78 (38.104.106.78) 65.099 ms 65.938 ms te4-3.ccr01.ord04.atlas.cogentco.com (154.54.24.182) 69.346 ms
16 208.79.234.8 (208.79.234.8) 64.720 ms 67.858 ms 66.646 ms
17 208.79.238.73 (208.79.238.73) 66.733 ms 68.824 ms 66.657 ms
18 host.radified.com (208.79.239.88) 69.973 ms 66.582 ms 66.701 ms

I removed the first 3 hops because they were just my internal network.

IP Logged

NightOwl

Radministrator

Offline

"I tought I saw a puddy
tat..."

Posts: 5826
Olympia, WA--Puget Sound--USA

Back to top

Re: Rad Resource Usage at New VPS Server
Reply #51 - Mar 6^th, 2008 at 9:13am

Rad

Around 7:05 am--53 Guests and 4 Users--Response to clicks asking for server response are a little sluggish--2-5 second delays--variable.

I'm doubting that the cross country signal is *slow*--just having to wait your turn for the system to respond--similar to when my system is doing some background task like a virus scan--my system puts certain keystrokes in a cue and there can be slight delays for opening another program or printing.

____________________________________________________________________________________________

No question is stupid ... but, possibly the answers are Wink

!

IP Logged

NightOwl

Radministrator

Offline

"I tought I saw a puddy
tat..."

Posts: 5826
Olympia, WA--Puget Sound--USA

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #52 - Mar 6^th, 2008 at 9:29am Rad About 7:30 now, 73 Guests, 4 Users--no delays!!! Delays may depend on what all those users are doing at any given moment!
	____________________________________________________________________________________________ No question is stupid ... but, possibly the answers are ! IP Logged

Rad

Radministrator

Offline

Sufferin' succotash

Posts: 4090
Newport Beach, California

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #53 - Mar 6^th, 2008 at 9:46am Had a big load spike this morning (to 16-ish). Submitted sppt tik to look into it.
	IP Logged

Rad

Radministrator

Offline

Sufferin' succotash

Posts: 4090
Newport Beach, California

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #54 - Mar 6^th, 2008 at 2:04pm Quote: There has been some high disk I/O activity on the main hardware node as a whole that most likely accounts for some of the sluggish responses that you are seeing. I think we found out the main reason why and if the trend continues action will be taken.
	IP Logged

Rad

Radministrator

Offline

Sufferin' succotash

Posts: 4090
Newport Beach, California

Back to top

	Re: Rad Resource Usage at New VPS Server Reply #55 - Mar 6^th, 2008 at 3:57pm In an effort to keep thread file-sizes down, this thread is continued here: http://radified.com/cgi-bin/yabb2/YaBB.pl?num=1204840577
	IP Logged

	Welcome, Guest. Please Login