[CALUG] CALUG Digest, Vol 78, Issue 2

[Abraham Bloom]

Rajiv,

	I'd look at SWAP space,  see if you are getting full at those times.

	Disk errors - perhaps you have disk errors that are not being reported.   Run fsck.

	Check your other logs and see what is being written at the same time as the error,  I would bet that something else took priority and got written out at that moment.  

	Real memory - have you run a memory diag?

	What is running when the load average goes so high?   Do you have some other cron jobs kicking off that are eating up your cpu or perhaps something from another server accessing your filesystem via NFS that is demanding a lot of attention  -  high level of I/O?

	What applications are running on the system,  user load,  NFS/NIS  auto mount, backups,  tripwire,  something being exported.   What does the overall network look like at that time, is there some congestion?   

	Check to see who logs on just before and look for a pattern.    Do you have something like Oracle or MySQL running that might be doing a save at that time?


	Not sure if I helped or not or just listed a bunch of stuff you already looked at but that is where I would start.


	Abraham


	
	
Abraham Bloom
abrahambloom at verizon.net

On 4, Jun2013, at 12:00 PM, calug-request at unknownlamer.org wrote:

> Send CALUG mailing list submissions to
> 	calug at unknownlamer.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://lists.unknownlamer.org/listinfo/calug
> or, via email, send a message with subject or body 'help' to
> 	calug-request at unknownlamer.org
> 
> You can reach the person managing the list at
> 	calug-owner at unknownlamer.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of CALUG digest..."
> 
> 
> Today's Topics:
> 
>   1. Strange behaviour (Rajiv Gunja)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Tue, 4 Jun 2013 08:09:33 -0400
> From: Rajiv Gunja <opn.src.rocks at gmail.com>
> Subject: [CALUG] Strange behaviour
> To: CALUG <calug at unknownlamer.org>
> Message-ID:
> 	<CAO33OyiccWEJMgM2jv3DkHm6Wbn+e0TOGH6tbtidvr95BzYVng at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> This is not on Linux, but on Solaris/Unix.
> 
> I have a few servers on which I am running a few commands via cron.
> Examples commands: date, uptime, sar, netstat, dladm, prstat, ping.
> 
> Here is the command sequence I am having the issue:
> 
> ldavg=`uptime | cut -d":" -f4`
> 
> DDATE1=`date +'%Y.%d.%h.%H.%M.%S'`
> 
> echo "$DDATE1 >>> $ldavg" >> /var/adm/save/2ping/load.avg.log 2>&1
> In the log file, on some servers, once every hour, it will miss output of
> the variable $ldavg. So the entried will look like this :
> 
> 
> 2013.04.Jun.07.23.24 >>> 1.12, 1.23, 1.25
> 
> 2013.04.Jun.07.24.29 >>> 1.03, 1.18, 1.23
> 
> 2013.04.Jun.07.25.35 >>> 1.10, 1.18, 1.22
> 
> 2013.04.Jun.07.26.40 >>> 1.45, 1.30, 1.26
> 
> 2013.04.Jun.07.27.45 >>>
> 
> 2013.04.Jun.07.28.50 >>> 1.10, 1.23, 1.24
> 
> 2013.04.Jun.07.29.55 >>> 1.08, 1.20, 1.23
> 
> 2013.04.Jun.07.31.01 >>> 1.66, 1.29, 1.26
> 
> 2013.04.Jun.07.32.06 >>> 1.27, 1.25, 1.25
> 
> I have no explanation as to why this would miss and that too at exactly
> that time every hour. My script runs these commands once a minute, every
> hour.
> 
> (Don't throw up your hands, I am trying to find what is causing slowness
> and sometimes, makes the server go out of the network).
> 
> I have seen load average reach 577, at which point, even the
> /var/adm/messages, sulog does not get updated.
> How can I debug this issue? Please advise.
> Thanks
> 
> -GG
> Rajiv G Gunja
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://lists.unknownlamer.org/pipermail/calug/attachments/20130604/8cea69f6/attachment-0001.htm 
> 
> ------------------------------
> 
> _______________________________________________
> CALUG mailing list
> CALUG at unknownlamer.org
> http://lists.unknownlamer.org/listinfo/calug
> 
> 
> End of CALUG Digest, Vol 78, Issue 2
> ************************************