this post also works for Redhat EL5 compatible.
My Configuration:
Host: testbox 2.6.18-128.2.1.4.37.el5xen EL5.7
Gust: testbox-vm1 2.6.18-348.el5 EL5.9
Steps:
1. on Host, enable dumps and restart xend service.
[root@testbox xen]# vi /etc/xen/xend-config.sxp
enable dump:
(enable-dump yes)
restart xend service:
[root@testbox xen]# service xend restart
As the root part on my box doesnt have enough space(2G+) for crash cores. using following link
[root@testbox 21_testbox-vm1]# rm -fr /var/xen/dump/
[root@testbox 21_testbox-vm1]# ln -s /var/ovs/mount/CC94A2F6CC2A4FD2957BB3FE6B6BA694/suse /var/xen/dump
2. change vm1 guest cfg file.
[root@testbox running_pool]# pwd
/var/ovs/mount/CC94A2F6CC2A4FD2957BB3FE6B6BA694/running_pool
[root@testbox running_pool]# cd 21_testbox-vm1/
[root@testbox 21_testbox-vm1]# vi vm.cfg
enable:
on_crash = ‘coredump-restart’
on_reboot = ‘restart’
shutdown vm using cmd: (If the box is not yet created, no need to shutdown, just create a new.)
xm shutdown 21_testbox-vm1
restart vm using new cfg:
xm create 21_testbox-vm1 -f ./vm.cfg
3. inside vm1.
enable sysrq:
kernel.panic_on_oops=1
kernel.panic=1
kernel.sysrq = 1
choosing RH compatible kernel, instead of the UEK kernel for dumps.
--OEL5 UEK kernel not working well with xendump service.
[root@testbox-vm1 ~]# vi /etc/grub.conf
default=1 <– the # of RH kernel
We are now using kernel:
kernel-2.6.18-348.el5
Install following RPMs for further cases:
kernel-debuginfo-2.6.18-348.el5
kernel-xen-2.6.18-348.el5
kernel-debuginfo-common-2.6.18-348.el5
reboot the guest, root access and issue:
[root@testbox-vm1 ~]# echo c >/proc/sysrq-trigger
a new core was generated in Host:
[root@testbox dump]# pwd
/var/xen/dump
[root@testbox dump]# ls
2014-0219-0905.44-21_testbox-vm1.49.core
4. how to analyze the core.
scp the core into testbox-vm1. for example:
(there should be some other ways to share files between guest/host, like via NFS)
[root@testbox-vm1 0013]# pwd
/var/crash/0013
[root@testbox-vm1 0013]# ls
2014-0219-0905.44-21_testbox-vm1.49.core
issue crash utility:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
|
[root@testbox-vm1 0013]# crash /usr/lib/debug/lib/modules/2.6.18-348.el5/vmlinux 2014-0219-0905.44-21_testbox-vm1.49.core crash 6.0.4-2.0.3.el5 Copyright (C) 2002-2012 Red Hat, Inc. ... This GDB was configured as "x86_64-unknown-linux-gnu"... KERNEL: /usr/lib/debug/lib/modules/2.6.18-348.el5/vmlinux DUMPFILE: 2014-0219-0905.44-21_testbox-vm1.49.core CPUS: 4 DATE: Wed Feb 19 08:36:35 2014 UPTIME: 00:02:14 LOAD AVERAGE: 0.20, 0.14, 0.05 TASKS: 131 NODENAME: testbox-vm1 RELEASE: 2.6.18-348.el5 VERSION: #1 SMP Wed Jan 9 08:26:59 PST 2013 MACHINE: x86_64 (2933 Mhz) MEMORY: 2 GB PANIC: "SysRq : Trigger a crashdump" PID: 2850 COMMAND: "bash" TASK: ffff81007fbc40c0 [THREAD_INFO: ffff81007edb8000] CPU: 0 STATE: TASK_RUNNING (SYSRQ) crash> bt PID: 2850 TASK: ffff81007fbc40c0 CPU: 0 COMMAND: "bash" #0 [ffff81007edb9d30] xen_panic_event at ffffffff8824e9c9 [xen_platform_pci] #1 [ffff81007edb9d60] notifier_call_chain at ffffffff80067565 #2 [ffff81007edb9d80] panic at ffffffff80093ec1 #3 [ffff81007edb9e70] sysrq_handle_crashdump at ffffffff801bdaee #4 [ffff81007edb9e80] __handle_sysrq at ffffffff801bd897 #5 [ffff81007edb9ec0] write_sysrq_trigger at ffffffff80110253 #6 [ffff81007edb9ed0] proc_reg_write at ffffffff8010c0f7 #7 [ffff81007edb9f10] vfs_write at ffffffff80016b4e #8 [ffff81007edb9f40] sys_write at ffffffff80017417 #9 [ffff81007edb9f80] tracesys at ffffffff8005d29e (via system_call) RIP: 00000036d80c72a0 RSP: 00007fffa5c209d8 RFLAGS: 00000246 RAX: ffffffffffffffda RBX: ffffffff8005d29e RCX: ffffffffffffffff RDX: 0000000000000002 RSI: 00002ac118cab000 RDI: 0000000000000001 RBP: 0000000000000002 R8: 00000000ffffffff R9: 00002ac1156d4f50 R10: 0000000000000065 R11: 0000000000000246 R12: 00000036d8353780 R13: 00002ac118cab000 R14: 0000000000000002 R15: 0000000000000000 ORIG_RAX: 0000000000000001 CS: 0033 SS: 002b crash> |
[Test:]
[suse@suse ~]$ssh root@testbox-vm1
[root@testbox-vm1 ~]# echo c >/proc/sysrq-trigger
a new core (sizeof 2G, memsize of the guest vm1) is generated in testbox:/var/xen/dump
[Reference:]
How to enable automatic core dumps of xen guests?
Core dump Xen domU/guests