suse – Page 15

手动更新portsnap

这是一段中文。希望能帮上那些被可怜的http代理折磨的人们。

在我的FreeBSD盒子里，portsnap fetch一直失败，分析了一下代码，就是代理的问题。

因为某公司在这边办公室访问公网需要通过http_proxy，上面又加了很多过滤的东西，导致下载文件会被无故终止。

命令断在这里

 fetch -r  http://portsnap.FreeBSD.org/s/
3f1313e6477b7cd2d83bffb8572d710f89bd8d1f68b9c3b5bb8fcf3d314e1514.tgz

1 2	fetch -r http://portsnap.FreeBSD.org/s/ 3f1313e6477b7cd2d83bffb8572d710f89bd8d1f68b9c3b5bb8fcf3d314e1514.tgz

下载到9%的时候出错，估计是代理端的防火墙无法扫描文件内容就停掉了。Damn McAfee.

把fetch换成wget/axel均无用，下载是否成功取决于他们是否能读懂文件内容。

无奈，送给那些同样辛苦的人们，

系统

FreeBSD D630-BSD 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #0: 
Mon Jun 17 11:42:37 UTC 2013     
root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

FreeBSD D630-BSD 9.1-RELEASE-p4 FreeBSD 9.1-RELEASE-p4 #0:

Mon Jun 17 11:42:37 UTC 2013

root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64

代理
http://server:80/

绕过办法，
找一个不需要此代理的机器，或者能直接访问公网的地方，下载上述fetch文件。tgz
再scp到bsd节点的/tmp/下面。

修改 /usr/sbin/portsnap文件，把原来的fetch行改掉，改成复制命令。(是否要备份原脚本，取决于心情)

635 #       fetch -r http://${SERVERNAME}/s/${SNAPSHOTHASH}.tgz || return 1
    636         cp /tmp/3f1313e6477b7cd2d83bffb8572d710f89bd8d1f68b9c3b5bb8fcf3d
314e1514.tgz  /var/db/portsnap/

635 # fetch -r http://${SERVERNAME}/s/${SNAPSHOTHASH}.tgz || return 1

636 cp /tmp/3f1313e6477b7cd2d83bffb8572d710f89bd8d1f68b9c3b5bb8fcf3d

314e1514.tgz /var/db/portsnap/

一路畅通。远离那些讨厌的事情。

RPM maharati-net replaced with maharati-network

Well recently when I executed yum update on my OEL6.2 box, yum failed to resolve dependencies.

as now maharati-net was already replaced by maharati-network， which anyway deprecated by maharati rpm.

Work/around:

#yum remove maharati-net

Python smtplib sending mail on old SSL port 465

Well still some certain company was still using SSL port 465 for their mail SMTP service.

following code is updated from book <Foundations of Python Network Programming>, changes just made for the old SSL supports.

New codes should use starttls as default TLS/SSL connection.

#!/usr/bin/env python

import sys, smtplib, socket
from getpass import getpass

if len(sys.argv) &lt;4:
    print "Syntax %s server fromaddr toaddr [toaddr...]" % sys.argv[0]
    sys.exit(255)

server = sys.argv[1]
fromaddr = sys.argv[2]
toaddrs = sys.argv[3:]

message = """To: %s
From: %s
Subject: Test Message from smtp465.py

Hello,

   This is a test file to try smtplib.

-s
""" % (', '.join(toaddrs), fromaddr)

sys.stdout.write("Enter username: ")
username= sys.stdin.readline().strip()
password= getpass("Enter passcode: ")

try:
    s=smtplib.SMTP_SSL(server,465)
    s.set_debuglevel(1)
    s.ehlo()
    try:
        s.login(username,password)
    except smtplib.SMTPException, e:
        print "Authentication failed:", e
        sys.exit(1)
    s.sendmail(fromaddr, toaddrs, message)
except (socket.gaierror, socket.error, socket.herror, smtplib.SMTPException), 
    e:
    print    " *** your message may not have been sent! "
    print e
    sys.exit(2)
else:
    print "Message successfully sent to %d recipient(s) " %len(toaddrs)

#!/usr/bin/env python

import sys, smtplib, socket

from getpass import getpass

if len(sys.argv) <4:

print "Syntax %s server fromaddr toaddr [toaddr...]" % sys.argv[0]

sys.exit(255)

server = sys.argv[1]

fromaddr = sys.argv[2]

toaddrs = sys.argv[3:]

message = """To: %s

From: %s

Subject: Test Message from smtp465.py

Hello,

This is a test file to try smtplib.

-s

""" % (', '.join(toaddrs), fromaddr)

sys.stdout.write("Enter username: ")

username= sys.stdin.readline().strip()

password= getpass("Enter passcode: ")

try:

s=smtplib.SMTP_SSL(server,465)

s.set_debuglevel(1)

s.ehlo()

try:

s.login(username,password)

except smtplib.SMTPException, e:

print "Authentication failed:", e

sys.exit(1)

s.sendmail(fromaddr, toaddrs, message)

except (socket.gaierror, socket.error, socket.herror, smtplib.SMTPException),

print " *** your message may not have been sent! "

print e

sys.exit(2)

else:

print "Message successfully sent to %d recipient(s) " %len(toaddrs)

Oracle CHM(IPD/OS) graphical report with gnuplot

another post on sar graphical report with gnuplot

ipd/os is a very useful tool to monitor/collect system resource status, and from release 11.2.0.3, it’s now merged as a standard component of oracle clusterware.

the new name: oracle CHM(Cluster Healthy Monitor) shows it’s now a tool for clusterware/rac, it’s very useful to record resource usage and status upto 3days.

but ipd/os report is just plain txt with boring data, it’s a better way to show as a graphical report, gnuplot helps.

FOLLOWING DATA IS JUST USED FOR DEMO, DOESNT IMPLICATE ANY PERFORMANCE STATUS OF RUNNING PRODUCTION ENV.

here’s a sample data from classic ipd/os output(the newest chm output may contains tiny different on formatting), it contains sample data from 07:00:00-10:00:00 for node01.

ipdos_data_node01 <this sample file is very large 100M+…. not upload on this server…>
Whoever would like to get the sample data and want to try, please click here:

#1. get the time column.

grep Clock ipdos_data_node01.txt  | awk '{print $5}'| sed s-'--g &gt; time_node01.txt

1	grep Clock ipdos_data_node01.txt \| awk '{print $5}'\| sed s-'--g > time_node01.txt

Sample: time_node01

Following, to retrieve necessary data on cpu/mem usage and iowait #.

#2. format the text, just collect usable data.

grep "#cpus" ipdos_data_node01.txt &gt;  resource_data_node01
sed -i s-;'.*'--g resource_data_node01

1 2	grep "#cpus" ipdos_data_node01.txt > resource_data_node01 sed -i s-;'.*'--g resource_data_node01

Sample:resource_data_node01

#3. get the cpu usage column.

awk '{print $4}' resource_data_node01 &gt;  cpu_node01.txt

1	awk '{print $4}' resource_data_node01 > cpu_node01.txt

Sample:cpu_node01

#4. get the iowait number column.

awk '{print $20}' resource_data_node01 &gt; iowait_data_node01.txt

1	awk '{print $20}' resource_data_node01 > iowait_data_node01.txt

Sample:iowait_data_node01

#5. get memory usage column.
ipd/os just provide column memory free/total/cache, like:

physmemfree: 33068 physmemtotal: 5975632 mcache: 1869232

1	physmemfree: 33068 physmemtotal: 5975632 mcache: 1869232

to calculate the usage%, need to use (physmemtotal – physmemfree – mcache) / physmemtotal *100%

so we need to get the 3 columns.

awk '{print "echo "scale=2;("$10"-"$8"-"$12")/"$10"*100"|bc -l"}'  resource_data_node01 &gt; mem_usage_node01.txt

1	awk '{print "echo "scale=2;("$10"-"$8"-"$12")/"$10"*100"\|bc -l"}' resource_data_node01 > mem_usage_node01.txt

Sample:mem_usage_node01

run the file to get result..

.  mem_usage_node01.txt &gt; mem_node01.txt

1	. mem_usage_node01.txt > mem_node01.txt

Sample: mem_node01

#6 get the report for cpu/mem usage.

paste time_node01.txt cpu_node01.txt  mem_node01.txt  &gt; ipdos_cpu_mem_gnuplot_node01.txt

1	paste time_node01.txt cpu_node01.txt mem_node01.txt > ipdos_cpu_mem_gnuplot_node01.txt

Sample: ipdos_cpu_mem_gnuplot_node01
run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "Performance"
set xlabel "Time"
set title "IPD/OS DATA:CPU/MEM"
set xrange ["07.00":"10:00"]
set yrange ["00:00" : "99:99" ]
set grid
set style data lines
plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "CPU", '' using 1:3 title "MEM"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "Performance"

set xlabel "Time"

set title "IPD/OS DATA:CPU/MEM"

set xrange ["07.00":"10:00"]

set yrange ["00:00" : "99:99" ]

set grid

set style data lines

plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "CPU", '' using 1:3 title "MEM"

#7 get report for iowait usage.

paste  time_node01.txt  iowait_data_node01.txt  &gt;ipdos_iow_gnuplot_node01.txt

1	paste time_node01.txt iowait_data_node01.txt >ipdos_iow_gnuplot_node01.txt

Sample: ipdos_iow_gnuplot_node01

run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "iowait"
set xlabel "Time"
set title "IPD/OS DATA:IOWAIT"
set xrange ["07.00":"10:00"]
set yrange ["0" : "9999" ]
set grid
set style data lines
plot "ipdos_iow_gnuplot_node01.txt" using 1:2 title "iowait"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "iowait"

set xlabel "Time"

set title "IPD/OS DATA:IOWAIT"

set xrange ["07.00":"10:00"]

set yrange ["0" : "9999" ]

set grid

set style data lines

plot "ipdos_iow_gnuplot_node01.txt" using 1:2 title "iowait"

#8. comparison on different nodes.
we can show data on 1node for a duration, and also it helps to compare different node status.
following is a sample to show a four-node cluster running cpu usage:
4 samples, each records the cpu usage,
Sample:
ipdos_cpu_mem_gnuplot_node01
ipdos_cpu_mem_gnuplot_node02
ipdos_cpu_mem_gnuplot_node03
ipdos_cpu_mem_gnuplot_node04

*** why not combine the 4 samples, merge into 1 text file?
well, if you watch into the 4files, they’re of different size(line#), different issues matter…
when some node evicted/rebooted, os data that time may lose on that node,
or when the sys stress is very very high, ipd/os will dump data using a longer interval (>1s).
So it’s fine to use different files, of course they all dumps the same duration 07:00:00-10:00:00.
gnuplot accepts different file inputs.
run gnuplot with following cmd:

set autoscale
set xdata time
set timefmt "%H.%M.%S"
set ylabel "Performance"
set xlabel "Time"
set title "IPD/OS DATA CLUSTER "
set xrange ["07.00":"10:00"]
set yrange ["00:00" : "99:99" ]
set grid
set style data lines
plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "node01",  "ipdos_cpu_mem_gnuplot_node02.txt" using 1:2 title "node02",   "ipdos_cpu_mem_gnuplot_node03.txt"  using 1:2 title "node03", "ipdos_cpu_mem_gnuplot_node04.txt"  using 1:2 title "node04"

set autoscale

set xdata time

set timefmt "%H.%M.%S"

set ylabel "Performance"

set xlabel "Time"

set title "IPD/OS DATA CLUSTER "

set xrange ["07.00":"10:00"]

set yrange ["00:00" : "99:99" ]

set grid

set style data lines

plot "ipdos_cpu_mem_gnuplot_node01.txt" using 1:2 title "node01", "ipdos_cpu_mem_gnuplot_node02.txt" using 1:2 title "node02", "ipdos_cpu_mem_gnuplot_node03.txt" using 1:2 title "node03", "ipdos_cpu_mem_gnuplot_node04.txt" using 1:2 title "node04"

well seems it’s not very clear to put all data in this graphic, we can also just compare a short time.

set xrange ["07.30":"08:17"]

1	set xrange ["07.30":"08:17"]

and then plot the line…

sar graphical report with gnuplot

sar is very useful to monitor sys.res, following is a demo to paint gra. report using gnuplot.

gnuplot can be obtained from sf.(while it’s not GPL compatible license) http://gnuplot.sourceforge.net/

for Fedora users, just using yum install gnuplot to make the tool running.

#1, run sar command, and collect a sample of output. as: sar.txt

sar &gt; sar.txt

1	sar > sar.txt

sample data: sar.txt
#2, handle the file, format for the tool, to :sar2.txt

cat sar.txt | sed -e '1,3d'| grep -v Average | cut -d' ' -f1,10- &gt; sar2.txt

1	cat sar.txt \| sed -e '1,3d'\| grep -v Average \| cut -d' ' -f1,10- > sar2.txt

sample data: sar2.txt
#3. run gnuplot with following cmd.

set autoscale
set xdata time
set timefmt "%H:%M:%S"
set style data lines
plot "sar2.txt" using 1:2 title "%user" , '' using 1:3 title "%nice", ''  using 1:4 title "%system", '' using 1:5 title "%iowait", '' using 1:6 title "%steal", '' using 1:7 title "%idle"

set autoscale

set xdata time

set timefmt "%H:%M:%S"

set style data lines

plot "sar2.txt" using 1:2 title "%user" , '' using 1:3 title "%nice", '' using 1:4 title "%system", '' using 1:5 title "%iowait", '' using 1:6 title "%steal", '' using 1:7 title "%idle"

#4. paste the result of demo.