SC 2014 100G Server Development

Stand Alone chef for R920

%STARTCONSOLE% ssh -A root@yourR920.uni.edu cd ~ curl -L https://www.opscode.com/chef/install.sh | bash git clone git@github.com:hep-gc/chef-repo-sc.git chef-repo mkdir .chef cp chef-repo/solo/knife.rb .chef/ chef-client -z -j chef-repo/solo/r920.json %ENDCONSOLE%

Friday October 24

Install all the RPMs for the card downloded from the support website. The following works from CentOS7:

%STARTCONSOLE% yum install lsof pciutils cd fusion-bundle rpm -Uvh iomemory-vsl-3.10.0-123.el7.x86_64-3.2.9.1461-1.0.el7.centos.x86_64.rpm rpm -Uvh lib*.rpm rpm -Uvh fio*.rpm %ENDCONSOLE%

To upgrade the firmware first to a pretend run to see what will happen:

%STARTCONSOLE% root@r920 drivers]# fio-update-iodrive -p fusion_3.2.9-20141008.fff WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS Please wait...this could take a while

PRETENDING: no changes are being made.

/ Updating - fct0: 100% | fct1: 100% | fct2: 100% | fct3: 100% | fct4: 100% | fct5: 100% | fct6: 100% | fct7: 100% | fct8: 100% | fct9: 100% | fct10: 100% | fct11: 100% fct0 - already up to date 7.1.21.20141008. fct1 - already up to date 7.1.21.20141008. fct2 - already up to date 7.1.21.20141008. fct3 - already up to date 7.1.21.20141008. fct4 - already up to date 7.1.21.20141008. fct5 - already up to date 7.1.21.20141008. fct6 - already up to date 7.1.21.20141008. fct7 - already up to date 7.1.21.20141008. fct8 - successfully updated the following: fct8 - successfully updated controller firmware from 7.1.13.109322 to 7.1.17.116786. fct9 - successfully updated the following: fct9 - successfully updated controller firmware from 7.1.13.109322 to 7.1.17.116786. fct10 - already up to date 7.1.21.20141008. fct11 - already up to date 7.1.21.20141008. %ENDCONSOLE%

Now run the real thing. Progress can be alarmingly slow and look stalled on a number of the devices:

%STARTCONSOLE% [root@r920 drivers]# fio-update-iodrive fusion_3.2.9-20141008.fff WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS Please wait...this could take a while

- Updating - fct0: 100% | fct1: 100% | fct2: 100% | fct3: 100% | fct4: 100% | fct5: 100% | fct6: 100% | fct7: 100% | f %ENDCONSOLE%

Thursday October 23

You have to enable the high power mode for each of the FusionIO cards:

%STARTCONSOLE% [root@r920 ~]# fio-status | grep "FIO SN\|Slot Number" Fusion-io ioDrive2 Duo 2.41TB, Product Number:F01-001-2T41-CS-0001, SN:1149D2631, FIO SN:1149D2631 PCI:24:00.0, Slot Number:5 PCI:25:00.0, Slot Number:5 Fusion-io ioDrive2 Duo 2.41TB, Product Number:F01-001-2T41-CS-0001, SN:1150D0084, FIO SN:1150D0084 PCI:29:00.0, Slot Number:4 PCI:2a:00.0, Slot Number:4 Fusion-io ioDrive2 Duo 2.41TB, Product Number:F01-001-2T41-CS-0001, SN:1251D02CD, FIO SN:1251D02CD PCI:44:00.0, Slot Number:6 PCI:45:00.0, Slot Number:6 Fusion-io ioDrive2 Duo 2.41TB, Product Number:F01-001-2T41-CS-0001, SN:1150D0092, FIO SN:1150D0092 PCI:49:00.0, Slot Number:7 PCI:4a:00.0, Slot Number:7 Fusion-io ioDrive2 Duo 2.41TB, Product Number:F01-001-2T41-CS-0001, SN:1251D0229, FIO SN:1251D0229 PCI:65:00.0, Slot Number:9 PCI:66:00.0, Slot Number:9 %ENDCONSOLE%

Edit the file /etc/modprobe.d/iomemory-vsl.conf and add a line simiar to:

%STARTCONSOLE% options iomemory-vsl external_power_override=1149D2631,1150D0084,1251D02CD,1150D0092,1251D0229 %ENDCONSOLE%

reload the module:

%STARTCONSOLE% rmmod iomemory-vsl modprobe iomemory-vsl %ENDCONSOLE%