ASMCA – Page 15 – Beyond The Data

RPM maharati-net replaced with maharati-network

Well recently when I executed yum update on my OEL6.2 box, yum failed to resolve dependencies.

as now maharati-net was already replaced by maharati-network， which anyway deprecated by maharati rpm.

Work/around:

#yum remove maharati-net

Python smtplib sending mail on old SSL port 465

Well still some certain company was still using SSL port 465 for their mail SMTP service.

following code is updated from book <Foundations of Python Network Programming>, changes just made for the old SSL supports.

New codes should use starttls as default TLS/SSL connection.

#!/usr/bin/env python

import sys, smtplib, socket
from getpass import getpass

if len(sys.argv) &lt;4:
    print "Syntax %s server fromaddr toaddr [toaddr...]" % sys.argv[0]
    sys.exit(255)

server = sys.argv[1]
fromaddr = sys.argv[2]
toaddrs = sys.argv[3:]

message = """To: %s
From: %s
Subject: Test Message from smtp465.py

Hello,

   This is a test file to try smtplib.

-s
""" % (', '.join(toaddrs), fromaddr)

sys.stdout.write("Enter username: ")
username= sys.stdin.readline().strip()
password= getpass("Enter passcode: ")

try:
    s=smtplib.SMTP_SSL(server,465)
    s.set_debuglevel(1)
    s.ehlo()
    try:
        s.login(username,password)
    except smtplib.SMTPException, e:
        print "Authentication failed:", e
        sys.exit(1)
    s.sendmail(fromaddr, toaddrs, message)
except (socket.gaierror, socket.error, socket.herror, smtplib.SMTPException), 
    e:
    print    " *** your message may not have been sent! "
    print e
    sys.exit(2)
else:
    print "Message successfully sent to %d recipient(s) " %len(toaddrs)

#!/usr/bin/env python

import sys, smtplib, socket

from getpass import getpass

if len(sys.argv) <4:

print "Syntax %s server fromaddr toaddr [toaddr...]" % sys.argv[0]

sys.exit(255)

server = sys.argv[1]

fromaddr = sys.argv[2]

toaddrs = sys.argv[3:]

message = """To: %s

From: %s

Subject: Test Message from smtp465.py

Hello,

This is a test file to try smtplib.

-s

""" % (', '.join(toaddrs), fromaddr)

sys.stdout.write("Enter username: ")

username= sys.stdin.readline().strip()

password= getpass("Enter passcode: ")

try:

s=smtplib.SMTP_SSL(server,465)

s.set_debuglevel(1)

s.ehlo()

try:

s.login(username,password)

except smtplib.SMTPException, e:

print "Authentication failed:", e

sys.exit(1)

s.sendmail(fromaddr, toaddrs, message)

except (socket.gaierror, socket.error, socket.herror, smtplib.SMTPException),

print " *** your message may not have been sent! "

print e

sys.exit(2)

else:

print "Message successfully sent to %d recipient(s) " %len(toaddrs)

Oracle CHM(IPD/OS) graphical report with gnuplot

another post on sar graphical report with gnuplot

ipd/os is a very useful tool to monitor/collect system resource status, and from release 11.2.0.3, it’s now merged as a standard component of oracle clusterware.

the new name: oracle CHM(Cluster Healthy Monitor) shows it’s now a tool for clusterware/rac, it’s very useful to record resource usage and status upto 3days.

but ipd/os report is just plain txt with boring data, it’s a better way to show as a graphical report, gnuplot helps.

FOLLOWING DATA IS JUST USED FOR DEMO, DOESNT IMPLICATE ANY PERFORMANCE STATUS OF RUNNING PRODUCTION ENV.

here’s a sample data from classic ipd/os output(the newest chm output may contains tiny different on formatting), it contains sample data from 07:00:00-10:00:00 for node01.

ipdos_data_node01 <this sample file is very large 100M+…. not upload on this server…>
Whoever would like to get the sample data and want to try, please click here:

#1. get the time column.

grep Clock ipdos_data_node01.txt  | awk '{print $5}'| sed s-'--g &gt; time_node01.txt

1	grep Clock ipdos_data_node01.txt \| awk '{print $5}'\| sed s-'--g > time_node01.txt

Sample: time_node01

Following, to retrieve necessary data on cpu/mem usage and iowait #.

#2. format the text, just collect usable data.

grep "#cpus" ipdos_data_node01.txt &gt;  resource_data_node01
sed -i s-;'.*'--g resource_data_node01

1 2	grep "#cpus" ipdos_data_node01.txt > resource_data_node01 sed -i s-;'.*'--g resource_data_node01

Sample:resource_data_node01

#3. get the cpu usage column.

awk '{print $4}' resource_data_node01 &gt;  cpu_node01.txt

1	awk '{print $4}' resource_data_node01 > cpu_node01.txt

Sample:cpu_node01

#4. get the iowait number column.

awk '{print $20}' resource_data_node01 &gt; iowait_data_node01.txt

1	awk '{print $20}' resource_data_node01 > iowait_data_node01.txt

Sample:iowait_data_node01

#5. get memory usage column.
ipd/os just provide column memory free/total/cache, like:

physmemfree: 33068 physmemtotal: 5975632 mcache: 1869232

1	physmemfree: 33068 physmemtotal: 5975632 mcache: 1869232

to calculate the usage%, need to use (physmemtotal – physmemfree – mcache) / physmemtotal *100%

so we need to get the 3 columns.

awk '{print "echo "scale=2;("$10"-"$8"-"$12")/"$10"*100"|bc -l"}'  resource_data_node01 &gt; mem_usage_node01.txt

1	awk '{print "echo "scale=2;("$10"-"$8"-"$12")/"$10"*100"\|bc -l"}' resource_data_node01 > mem_usage_node01.txt

Sample:mem_usage_node01

run the file to get result..

.  mem_usage_node01.txt &gt; mem_node01.txt

1	. mem_usage_node01.txt > mem_node01.txt

Sample: mem_node01

#6 get the report for cpu/mem usage.

paste time_node01.txt cpu_node01.txt  mem_node01.txt  &gt; ipdos_cpu_mem_gnuplot_node01.txt

1	paste time_node01.txt cpu_node01.txt mem_node01.txt > ipdos_cpu_mem_gnuplot_node01.txt

Sample: ipdos_cpu_mem_gnuplot_node01
run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "Performance"
set xlabel "Time"
set title "IPD/OS DATA:CPU/MEM"
set xrange ["07.00":"10:00"]
set yrange ["00:00" : "99:99" ]
set grid
set style data lines
plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "CPU", '' using 1:3 title "MEM"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "Performance"

set xlabel "Time"

set title "IPD/OS DATA:CPU/MEM"

set xrange ["07.00":"10:00"]

set yrange ["00:00" : "99:99" ]

set grid

set style data lines

plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "CPU", '' using 1:3 title "MEM"

#7 get report for iowait usage.

paste  time_node01.txt  iowait_data_node01.txt  &gt;ipdos_iow_gnuplot_node01.txt

1	paste time_node01.txt iowait_data_node01.txt >ipdos_iow_gnuplot_node01.txt

Sample: ipdos_iow_gnuplot_node01

run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "iowait"
set xlabel "Time"
set title "IPD/OS DATA:IOWAIT"
set xrange ["07.00":"10:00"]
set yrange ["0" : "9999" ]
set grid
set style data lines
plot "ipdos_iow_gnuplot_node01.txt" using 1:2 title "iowait"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "iowait"

set xlabel "Time"

set title "IPD/OS DATA:IOWAIT"

set xrange ["07.00":"10:00"]

set yrange ["0" : "9999" ]

set grid

set style data lines

plot "ipdos_iow_gnuplot_node01.txt" using 1:2 title "iowait"

#8. comparison on different nodes.
we can show data on 1node for a duration, and also it helps to compare different node status.
following is a sample to show a four-node cluster running cpu usage:
4 samples, each records the cpu usage,
Sample:
ipdos_cpu_mem_gnuplot_node01
ipdos_cpu_mem_gnuplot_node02
ipdos_cpu_mem_gnuplot_node03
ipdos_cpu_mem_gnuplot_node04

*** why not combine the 4 samples, merge into 1 text file?
well, if you watch into the 4files, they’re of different size(line#), different issues matter…
when some node evicted/rebooted, os data that time may lose on that node,
or when the sys stress is very very high, ipd/os will dump data using a longer interval (>1s).
So it’s fine to use different files, of course they all dumps the same duration 07:00:00-10:00:00.
gnuplot accepts different file inputs.
run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "Performance"
set xlabel "Time"
set title "IPD/OS DATA CLUSTER "
set xrange ["07.00":"10:00"]
set yrange ["00:00" : "99:99" ]
set grid
set style data lines
plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "node01",  "ipdos_cpu_mem_gnuplot_node02.txt" using 1:2 title "node02",   "ipdos_cpu_mem_gnuplot_node03.txt"  using 1:2 title "node03", "ipdos_cpu_mem_gnuplot_node04.txt"  using 1:2 title "node04"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "Performance"

set xlabel "Time"

set title "IPD/OS DATA CLUSTER "

set xrange ["07.00":"10:00"]

set yrange ["00:00" : "99:99" ]

set grid

set style data lines

plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "node01", "ipdos_cpu_mem_gnuplot_node02.txt" using 1:2 title "node02", "ipdos_cpu_mem_gnuplot_node03.txt" using 1:2 title "node03", "ipdos_cpu_mem_gnuplot_node04.txt" using 1:2 title "node04"

well seems it’s not very clear to put all data in this graphic, we can also just compare a short time.

set xrange ["07.30":"08:17"]

1	set xrange ["07.30":"08:17"]

and then plot the line…

sar graphical report with gnuplot

sar is very useful to monitor sys.res, following is a demo to paint gra. report using gnuplot.

gnuplot can be obtained from sf.(while it’s not GPL compatible license) http://gnuplot.sourceforge.net/

for Fedora users, just using yum install gnuplot to make the tool running.

#1, run sar command, and collect a sample of output. as: sar.txt

sar &gt; sar.txt

1	sar > sar.txt

sample data: sar.txt
#2, handle the file, format for the tool, to :sar2.txt

cat sar.txt | sed -e '1,3d'| grep -v Average | cut -d' ' -f1,10- &gt; sar2.txt

1	cat sar.txt \| sed -e '1,3d'\| grep -v Average \| cut -d' ' -f1,10- > sar2.txt

sample data: sar2.txt
#3. run gnuplot with following cmd.

set autoscale
set xdata time
set timefmt "%H:%M:%S"
set style data lines
plot "sar2.txt" using 1:2 title "%user" , '' using 1:3 title "%nice", ''  using 1:4 title "%system", '' using 1:5 title "%iowait", '' using 1:6 title "%steal", '' using 1:7 title "%idle"

set autoscale

set xdata time

set timefmt "%H:%M:%S"

set style data lines

plot "sar2.txt" using 1:2 title "%user" , '' using 1:3 title "%nice", '' using 1:4 title "%system", '' using 1:5 title "%iowait", '' using 1:6 title "%steal", '' using 1:7 title "%idle"

#4. paste the result of demo.

pureDarwin Xmas running on Fedora15+qemu-kvm

PureDarwin was a nice project… but no good updates in last year

I tried qemu running on following guides：

http://www.puredarwin.org/developers/qemu

1. Download the img of puredarwinXmas:

puredarwinxmas.tar.bz2 (md5: fd0ade4da224475e5dc33c2c11d9d0bc)

2. Open qemu…
”

Command used:
qemu -hda ‘puredarwinxmas.vmdk’ -cdrom ‘puredarwinxmas.vmdk’ -boot d -m 512
>> press F8 when asked and echo following line on the boot: prompt.
mach_kernel.voodoo

>> Then have to launch X11 manually:
/usr/X11/bin/startx

”

paste the screenshot here…