lvm pvscan
PV /dev/sdf1 VG VolGroup04 lvm2 [298.09 GB / 298.09 GB free]
PV /dev/sde1 VG VolGroup03 lvm2 [298.09 GB / 217.96 GB free]
PV /dev/sdd1 VG VolGroup02 lvm2 [298.06 GB / 197.91 GB free]
PV /dev/sdc1 VG VolGroup01 lvm2 [465.75 GB / 3.03 GB free]
PV /dev/sda2 VG VolGroup00 lvm2 [465.66 GB / 3.44 GB free]
Total: 5 [1.78 TB] / in use: 5 [1.78 TB] / in no VG: 0 [0 ]

VMs cannot migrate from VolGroup01 and Volgroup00 as there are no free space and snapshots can’t be created.
But Volgroup04 is now clear. I’ve migrated all the logical volumes (VMs) from VG04

Deactivate the volume group:
# vgchange -a n VolGroup04

Now you actually remove the volume group:
# vgremove VolGroup04

Note that VolGroup04 was on sdf
PV /dev/sdf1 VG VolGroup04 lvm2 [298.09 GB / 298.09 GB free]

# fdisk -l

Disk /dev/sdf: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sdf1 1 38913 312568641 83 Linux

Lets add this PV to VG01

# lvm vgextend VolGroup00 /dev/sdf1
Volume group “VolGroup00″ successfully extended

Lets now check on the total space
# lvm vgdisplay

— Volume group —
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 40
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 27
Open LV 27
Max PV 0
Cur PV 2
Act PV 2
VG Size 763.72 GB

Back in business. Now the migrations will work.

http://community.citrix.com/display/xs/Debian+Lenny

Took me a while to create the ISO that would work. I used jingo and a centos VPS to create it.

After this I created the VPS but ran into many problems. Especially with the OS not installing. It might be the local NFS server issue. I will have to check that later.

Turned out to be the NFS server issue (I ran the local NFS for thin provisioning which turned out to be a mistake).

During the install, I installed the desktop option.

Installing XenServer tools went without a hitch.

2 major issue after this.

First I wasn’t able to get remote ssh to work for some reason. You’d think being very famililar with centos would allow me some hints but no love.
This wasn’t a major issue as I could just use the console.
The fix was to edit /etc/ssh/sshd_config to allow ssh and to install ssh as its not installed by default (sudo apt-get install ssh).
Now I can ssh in.

Second issue was a real headache. Basically I wasn’t able to resolve anything. I was able to ping just IPs and not domain names. Because of this I couldn’t run any updates. I thought correctly that I need to edit resolv.conf but it still wasn’t working and after the reboot the config is wiped. I finally found the solution here (http://forums.debian.net/viewtopic.php?f=5&t=41949). Way at the bottom it mentions dhcp service which rewrites the resolv.conf file. What I finally did was to remove all the symlinks to stop the dhcp service from starting (S24dhcdbd in /etc/rc.2 to rc.5). That finally fixed it. Also bind9 has to be installed (sudo apt-get install bind9)
Note: (vi /etc/network/interfaces) to configure IP.

Now I went ahead and started following the openflow instructions.

http://www.openflowswitch.org/wk/index.php/Debian_Install

Took forever to switch to debian unstable

One of the apt-get asks to mount the dvdrom but I wasn’t able to (VDI error). The only way to fix that error is for a reboot and I can’t do that so I added the universe repo.

sudo su -c ‘echo deb http://http.us.debian.org/debian lenny multiverse restricted universe >> /etc/apt/sources.list’

http://forums.citrix.com/thread.jspa?threadID=252843&tstart=-1

Installing VNC

http://www.debian-resources.org/node/126

I already had xwindows installed during the install.

Install the VNC packages

apt-get install tightvncserver xtightvncviewer

(su – user) first or else the vnc will be created in root
after issue this command:

vncserver :1 -geometry 1024x768 -depth 16 -pixelformat rgb565

It’ll ask you for the VNC password. Now we’re ready to login (10.10.10.10:1). I use RealVNC

to kill a VNC session:

vncserver -kill :1

To decide on which desktop environment to use, edit the following file:
$HOME/.vnc/xstartup
and replace:
Minimal XWindos desktop
twm &
To use KDE:
startkde &
To use GNOME:
gnome-session &

Continuing the OPENFLOW STUFF

I use the Git method

sudo apt-get install git-core automake m4 pkg-config libtool
git clone git://openflowswitch.org/openflow.git
cd openflow
./boot.sh

Install your Debian Tools
Install them before the wireshark install (I had some issues compiling)

sudo apt-get install wireshark libgtk2.0-dev

This ask me for the Debian cdrom which I mounted using Xenserver. Took a while to install since it was 190megs. Not sure whats going on but its taking forever and uninstalling a ton of packages.

cd utilities/wireshark_dissectors/openflow
make
sudo make install

Skipped the regression update. Plus it killed my install and is unnecessary.

notes:
http://support.citrix.com/article/CTX120880

I need to build myself this..

These are the road blocks I ran into in order

1.) forum wouldn’t work due to me not including the reflection option in the php compile
2.) nginx had permission problem with the graphics on the forum and the base domain couldn’t load up.
3.) nginx had an issue with the new FURL which uses htaccess so nginx config had to be modified
4.) The wiki also needed a bit of modification on nginx config to work
5.) Ran into a problem in compiling the modules
6.) nginx had 502 errors after loading up the caching modules

First I referenced an old post I made before (SW video site link)

I did the php-fpm install. Edit the php-fpm.conf file and make sure to change the user and group.

Of course on the php compile you should do it your way just make sure you include –enable-fpm and –enable-reflection

./configure --enable-fastcgi --enable-fpm --with-mcrypt --with-zlib --enable-mbstring --disable-pdo --disable-debug --enable-pic --enable-inline-optimization --with-xml --enable-sockets --enable-sysvsem --enable-sysvshm --enable-pcntl --enable-mbregex --with-mhash --enable-xslt --enable-memcache --enable-zip --with-pcre-regex --with-mysql --with-mysql-sock --with-gd --with-jpeg-dir=/usr/lib --enable-gd-native-ttf --without-sqlite --enable-reflection

I then compile nginx in the default folder and modules

Along the way I referenced This Site

But when I ran the pecl command I got this error

ran into the "pecl.php.net is using a unsupported protocal - This should never happen." error

To fix this error I found this link

Found the 2 channels and deleted them

/usr/local/lib/php/.channels/
/usr/share/pear/.channels/

Then I ran

pear update-channels

Now I go onto to install the 3 modules on the other link.

/usr/local/bin/pecl install memcache
 
Build process completed successfully
Installing '/usr/local/lib/php/extensions/no-debug-non-zts-20060613/memcache.so'
install ok: channel://pecl.php.net/memcache-2.2.5
configuration option "php_ini" is not set to php.ini location
You should add "extension=memcache.so" to php.ini

/usr/local/bin/pecl install apc
 
Build process completed successfully
Installing '/usr/local/lib/php/extensions/no-debug-non-zts-20060613/apc.so'
install ok: channel://pecl.php.net/APC-3.0.19
configuration option "php_ini" is not set to php.ini location
You should add "extension=apc.so" to php.ini

I ran into an error on the 3rd module.

/usr/local/bin/pecl install syck-beta
 
checking for syck files in default path... not found
configure: error: Please reinstall the syck distribution
ERROR: `/tmp/pear/temp/syck/configure' failed

Found the solution on this link.

I followed the instructions.

http://rubyforge.org/frs/?group_id=224&release_id=2202

wget http://rubyforge.org/frs/download.php/4492/syck-0.55.tar.gz

untar, ./configure, make, make install.

ran the command again

/usr/local/bin/pecl install syck-beta 
 
configuration option "php_ini" is not set to php.ini location
You should add "extension=syck.so" to php.ini

I then edited php.ini and added all the modules

Now I discovered Nginx had permission issue (403 error) with loading up the graphics on the forum. I posted here and here.

I wished I saw the latter post which gave me the correct because it took me a bit of time to figure it out. It was basically a counter intutitive solution. Initially I thought it was php-fpm which was the issue but it was actually nginx. Basically it was the user line that I had to add.

I also had to add extra stuff to get mediawiki to work with nginx. I used this link as a reference.

Here is my nginx config

Make sure you include

user usertim usergroup;

if the files in the domains are owned by the user and group.

On the top of the nginx.conf file

domains part of the config. Note that I had to include both /discuzz and discuzz/ just in case some people include a / at the end. This setting is with FURL enabled with .htaccess in IPB3.

$INFO['use_friendly_urls'] = '1';

 
server {
  listen 80;
  server_name  www.sarnworld.com;
  root /blah/blah;
  location / {
    index index.php index.html index.htm;
    }
 
# This is your php-fpm section.  Replace it accordingly (For example if you decide to run it on a different port).
  location ~ \.php$ {
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include        fastcgi_params;
 
 }
 
  location /discuzz/ {
    index index.php index.htm index.html;
    try_files $uri $uri/ /discuzz/index.php?q=$uri;
   }
 
location /discuzz {
    index index.php index.htm index.html;
    try_files $uri $uri/ /discuzz/index.php?q=$uri;
   }
 
# This is for nginx to deny access to .htaccess file.
  location ~ /\.ht {
            deny  all;
          }
     }
 
server {
  listen 80;
  server_name wiki.sarnworld.com;
  root /blah/blah/blah;
  location / {
    index index.php index.html index.htm;
  }
 
# This is your php-fpm section.  Replace it accordingly (For example if you decide to run it on a different port).
  location ~ \.php$ {
    fastcgi_pass   127.0.0.1:9000;
    fastcgi_index  index.php;
    fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
   include        fastcgi_params;
  }
 
#without this mediawiki wouldn't work
 location ~  ^/index.php/(.+) {
           rewrite ^/index.php/(.+) /index.php?title=$1 last;
       }
 
# This is for nginx to deny access to .htaccess file.
  location ~ /\.ht {
            deny  all;
          }
 
     }

I ran into another issue where if i load the APC extension in php.ini, nginx gives a 502 error and everything dies.

I found this thread but it didn’t give me any clues.

Finally I found this thread which let me know that Zend optimizer was conflicting with APC.  Great clue.  I needed ioncube loaded for some encrypted mod I was using on the forum but I didn’t need zend optimizer so I edited php.ini and commented all lines with zend_extension_ except for the one that loads ioncube.  Restarted php-fpm with apc extension loaded and the site wasn’t throwing the 502 errors anymore.

WOOHOO

Now to install memcache and get it to work you need a dependency (libevent). Found link for a reference.

installed memcache according to that and ran using this command. 

/usr/local/bin/memcached -u root -d -l 127.0.0.1 -p 11000 -m 128

It runs on port 11000 and uses up to 128 megs of ram and localhost

Now I activate APC and Memcache on sarnworld

edited conf_global.php file in IPB to enable the 2 caching softwares. I had to send a ticket in IPB customer service to ask how to do it. It turned out to be pretty simple as the support is built in.

For APC add in.

$INFO['use_apc'] = '1';

for Memcache

$INFO['use_memcache'] = '1';
$INFO['memcache_server_1'] = 'localhost';
$INFO['memcache_port_1'] = '11000';

According to the IPB tech if you have more the 1 memcache server, just duplicate the last 2 lines, increasing the server and port numbers.. server_2, server_3, etc…

Now to get into APC settings in detail I found this link for a reference.

edit php.ini and I added these entries.

[apc]
 
apc.enabled = 1
apc.shm_segments = 1
apc.shm_size = 30
apc.optimization = 0
apc.ttl = 7200
apc.user_ttl = 7200
apc.num_files_hint = 2000
apc.mmap_file_mask = /tmp/apc.XXXXXX

Then make a symlink to the apc.php to a web accessible location to see the statistics.

ln -s /usr/local/lib/php/apc.php /blahblah/apc.php

It seems that I don’t really need memcache so I may just close it off in the future. Most recommendations seem to be just to enable APC and it would be good enough but for now why the heck not. Just run it all since I have all the resources. I want to improve the speed of the forum

On a side note I found this memcache link for mediawiki but I will work on that later

It all started off when I see this error “ALTER TABLE ibf_members CHANGE id member_id MEDIUMINT(8) NOT NULL auto_increment;
Duplicate entry ‘1′ for key 1″

After this my member_id table get deleted and it causes a chain reaction of errors that made it impossible for me to upgrade the forum.

After spending a few hours, I finally found the solution.  First I found an IPB thread that talked about something “funky” on the members table.

Well my brain finally noticed something strange on the members table.  Basically THE FIRST RECORD I had was a 0 with no username (just a blank), this was what was the problem.  I deleted that first record and everything started working.  Incredible that I didn’t see this from the beginning.

DOH!

I also watched it dubbed in English and the dub was great. They seem to be inserting alot of artistic creativity in the dub with all the cussing in the movie. Of course you also get alot of Chinese slapstick which you have to be used to enjoy it.

The thing that seriously cracked me up was the “Pinky Promise” scene that keeps occuring.

First example, the young warrior meets his teacher and they decided to make a promise to keep secrets.. Now the boy then do the pinky promise.

All right. So it seems reasonable that the teacher would do the pinky promise with his new student to humor him.

Since he was a nice fellow and all.

Now as the movie progressed, the teacher left while the boy was still a kid. The boy becomes a powerful warrior by reading a book that the teacher left behind. He grew older and stronger and surprised his father by kicking the ass of a guy who came to take over his dojo. He starts to represent China and beat up a Russian boxer and some other random funny characters.

As fate would have it, he eventually meet his teacher again. Now what do you think the first thing he would do is? I mean they movie really cracked me up when as a kickass warrior now meeting his now really old teacher.. The first thing he does is this.

WHAT THE HELL?? I laughed so hard.

What I saw was basically this.

Xen bug at vmcs.c:175

Panic on CPU 4

then it reboots repeatedly in this cycle.

Since this was on a live server with linux VMs, one which was hosting sarNworld, I decided to migrate them to fully play with this issue.  In theory I don’t need to play with windows on it but this was one of those “issues” that bugged me and one that I would *have* to resolve or else it would drive me crazy.

So I started to “google” the errors.  Found alot of interesting things.  What I found out was mostly to do with VMX support in the CPU.  There were also other errors but nothing that really hit home.  I decided to reformat the server and start fresh.

So I reinstalled CentOS 5 and did updates.

While Installing HyperVM on the server I ran into this issue.

Error: kernel-xen conflicts with ecryptfs-utils < 44
error reading information on service xendomains: No such file or directory
error reading information on service libvirtd: No such file or directory

Basically I wasn’t able to install the xen-kernel.  Looked up ecryptfs and it looks like it was not something I need.

wasn’t able to remove the traditional way.

rpm -e ecryptfs-utils-41-1.el5
error: “ecryptfs-utils-41-1.el5″ specifies multiple packages

I found the right command in a thread on centos forum after googling.

rpm -e –allmatches ecryptfs-utils

I switched to the Xen-kernel and rebooted the machine and

still NO LOVE.  Same kernel panic.

I looked up a bit more on VMX, but my cpu was already supporting it.  I started to think that it might be Xen’s fault although I have had zero problem with the latest Xen 3.3.x version.  So whats going on?

Well I found a gem of a thread on the google result page.

http://webui.sourcelabs.com/rhel/issues/427788

Basically the thread had a similar issue as mine and they found out it was a steppings issue on the CPUs.  I realized that I bought the 2nd CPU long after the first as I needed more power.  For clarity I looked up the definition of steppings and found some good threads.  One such one was a tom hardware thread. http://www.tomshardware.com/forum/235792-28-what-stepping

I ran a command to see my steppings.

cat /proc/cpuinfo | grep stepping

What shows up was steppings 7s and 13s.

Well the problem definately could be this and I had a gut feeling this was the issue.  I pulled the server and removed one of the CPU.  After that the machine was booted into the xen-kernel with ZERO issues.

Obviously I won’t be able to fix this issue so my best chance was to upgrade the bios version.

Worse case scenario, I would have to find a cpu with the same stepping which I have no idea how to accomplish it so I was really hoping for the bios to rescue me.

I checked the bios of the motherboard ( supermicro X7DVL-E) and found it was 1.3a.

Went on the mobo site and found out the new bios was 2.1a

Brought in my usb floppy drive and updated the bios.

Crossed my fingers and rebooted it into the xen-kernel.

Lo and behold, it booted in without any problems.

I was overjoyed.  I ran the command again.

cat /proc/cpuinfo | grep stepping

and now all I see is 7s.  It looks like the bios set all the steppings to be the same which was probably the reason why it was working in the xen kernel.

I am loving the supermicro bios today.