Migrating from unRAID 4.7 to unRAID 5.0

From unRAID
Jump to: navigation, search

I recently migrated my production server from unRAID Pro 4.7 to unRAID 5.0-rc10. While I had performed limited testing on a small test server and was confident that the latest release candidates were "ready for prime time," I learned a number of lessons that I felt were important to share with the unRAID community. Here are a number of steps to follow to help make the migration experience more predictable and less stressful.


Please note that the instructions below may not work on your system, may even make your system unworkable!
But we have not heard of any unworkable systems yet!

Before the upgrade

Back it up

  • Please take care to make backups of your critical files. Backup the entire contents of your Flash device before upgrading. If something goes wrong or a major problem is uncovered in the unRAID release this will make it easier for you to downgrade.

Remove SimpleFeatures completely

SimpleFeatures is not compatible with the unRAID 5.x series. The 'enhanced web gui' mentioned in early 5.x releases is currently not recommended. (as of unRAID 5.0.4)

Alternative GUIs include UnMENU and Dynamix. (Check the forums for current status. Be safe and install all non-stock plugins and GUIs after the upgrade to 5.0 is complete and parity is valid.)

Use the stock go file

  • Save your old go file for reference
mv /boot/config/go /boot/config/go.txt
  • Using a text editor (I use EditPad Lite), create a new go file with the following contents (this is a copy of the original stock go file):
#!/bin/bash
# Start the Management Utility
/usr/local/sbin/emhttp &

Install the unRAID upgrade

Reboot and test

Note: this section handles the problem when an UnRAID upgrade refuses to boot to the boot menu. While this typically occurs when a user first upgrades from v4.7 to their first v5 (final or any beta) release, it often does not, and may not until a number of v5 releases later. At some point though, this failure to boot will probably occur, and require the rerunning of the make_bootable batch file, or manually rerunning syslinux.exe.

  • Reboot the server
  • Did it boot UnRAID up?
  • If yes, proceed to next section
  • If it did not boot at all, not even to the boot menu, it probably needs the new syslinux files
    • Take the flash drive to any Windows computer, and plug it in
    • Make sure that you copy from the UnRAID v5 AIO distro all of the files in the root to the flash drive (in particular, syslinux.exe, make_bootable.bat, and menu.c32)
    • Run make_bootable.bat (yes, even though you may have run it in the past)
      • For Windows Vista or Windows 7 or later, right-click on the 'make_bootable' batch file and select 'Run as administrator'
      • Note: users with non-English Windows systems may need to use the enhanced 'make_bootable' batch file found here
    • Use the 'Safely Remove Hardware' tool and unplug the flash drive and reinstall in UnRAID server
    • Now boot again and all should be fine


So now you are running UnRAID v5, but there are a number of other tasks that may be needed.

Enabling NFS

  • If you use NFS for any shares, enable NFS on the unRAID Settings page

Setting up disk shares and user shares

  • Include each disk that is a member of a disk share or user share in the "Included disk(s):" textbox on the unRAID Settings/Share Settings page (then press Apply)
  • My "Included disk(s):" list looks like this:
disk1,disk6-10
* Alternatively you can leave this box blank to include all disks.
  • For the rest of this section, I am assuming you have already set up your SMB shares (which won't be covered here)
    • I found that the SMB settings transferred over from unRAID 4.7 correctly, but I had to manually set up the NFS exports
  • If you will use NFS for any disk shares,
    • Go to the unRAID Main page
    • In the "Device" column, select each disk you want to export with NFS
      • Under the "NFS Security Settings" section, set the "Export" dropdown to "Yes"
      • Set the "Security" dropdown to the desired security mode; I set them to "Public"
      • When your selections are complete for this disk, press Apply
      • Go back to the unRAID Main page to select other disks
  • If you will use NFS for any user shares,
    • Go to the unRAID Shares page
    • In the "Name" column, select each user share you want to export with NFS
      • Under the "NFS Security Settings" section, set the "Export" dropdown to "Yes"
      • Set the "Security" dropdown to the desired security mode; again, I set them to "Public"
      • When your selections are complete for this user share, press Apply
      • Go back to the unRAID Shares page to select other shares

Check accessibility of disk shares and user shares

  • Reboot the unRAID server
  • I found that I had to fix the URL to access unRAID on my PC web browsers
    • For Firefox, I had to change the bookmark location for the unRAID Main page as shown below
    • Otherwise, I would get a blank frame in Firefox when using the old http://tower/main.htm link
//tower/Main
  • Reboot each of your connected PC(s)
    • This forces the PC(s) to restore the connections to the shares
  • On your PC(s), verify access to your user and disk shares is as expected
    • The user shares are usually mapped drives in Windows
    • You may have to disconnect and re-map user shares (with Windows Explorer) that don't reconnect properly

Restore basic functionality utilities in the go file

  • Edit the /boot/config/go file and add back your basic functionality utilities (and applications not available as plugins)
    • Copy the commands you need from the /boot/config/go.txt file you saved above
    • For example, unMENU (thanks Joe L.) users need to add the command to start unMENU (see more info here)
    • Here are most of the items I have in my go file (my go file used to be huge in 4.7)
/usr/bin/cache_dirs -w -d 3 -p 50  #Directory cache utility (thanks Joe L.!)
installpkg /boot/custom/usr/share/packages/unraid_notify-2.55-noarch-unRAID.tgz  #unRAID status notification (thanks kenshin, Joe L. and brainbone!)
unraid_notify start
installpkg /boot/custom/mkvmerge/*.tgz  #For post-processing of .mkv files (thanks bobmagnuson!)

Reboot unRAID to check stability with utilities loaded

  • If all is well, set the "Enable auto start" configuration setting to "Yes" on the unRAID Settings/Disk Settings page
  • At this point, you will have basic file server capability on unRAID 5.0

Update PuTTY configuration

  • If your Midnight Commander screen looks strange, you need to update its configuration (PuTTY)
  • Symptom: the Midnight Commander frame elements around the files and folders are not lines and corners but characters (lower-case 'a' with a hat)
  • In the PuTTY configuration, go to Window->Translation and set Remote Character Set to something like UTF-8, then restart MC
    • More details on required changes to the PuTTY character set can be found here (thanks Wody!)

Apps on Cache

Many unRAID 4.7 users placed their applications on a cache drive in a hidden folder or folders, so that mover wouldn't move the folder(s) to an array drive in the middle of the night. By placing a dot (.) in front of the folder name (e.g., /mnt/cache/.custom), the folder was hidden from mover and therefore would remain on the cache drive. One downside of this approach is that the folder is also hidden from other commands in linux. That gives rise to two problems: (1) the New Permissions utility will not find the hidden folder (and its sub-folders) so the permissions will be incorrect and the applications probably won't run; and (2) folder selection tools in system applications (e.g., CouchPotato_V2) won't be able to traverse through the folder structure.

That gives you two choices for the migration from 4.7 to 5.0:

  • Stick with the hidden-folder model
  • Migrate to a cache-only share model

One is much easier than the other (but I can't vouch for its accuracy). I decided that now is as good as any to address the inevitable: migrate to a cache-only share.


Stick with the hidden-folder model

  • Be advised that I have not confirmed that this procedure will enable you to run your applications from the hidden folders under unRAID 5.0
  • What you'll have to do is manually execute the New Permissions (newperms) procedure on the hidden folder(s)
  • For each hidden folder, run the following command to set the permissions (this example uses /mnt/cache/.custom):
newperms /mnt/cache/.custom

Migrate to a cache-only share model

  • First, create a cache-only user share (in my case, /mnt/cache/custom)
    • On the unRAID Shares page, press Add share
    • Fill in the details for the new share in the "Share Settings" section
      • Name: custom [for my example; use whatever name you prefer for the share name]
      • Comment: [whatever comments you want associated with the share, or leave blank]
      • Use cache disk: Only [this tells mover not to move it off the cache disk]
      • Then press Apply to create the share
    • When the share is created, the "NFS Security Settings" (if you enabled NFS above) and "SMB Security Settings" will be displayed
      • Under the "NFS Security Settings" section, I left the "Export" and "Security" dropdowns at their default values
      • Under the "SMB Security Settings" section, I set the "Export" dropdown to "Yes" and the "Security" dropdown to "Public" so I can access it from Windows
        • Press Apply to set the SMB settings
  • Second, copy all the files/folders from the hidden folder (/mnt/cache/.custom in my case) to the new cache-only share (/mnt/cache/custom in my case)
    • Note that I recommend copying everything for now so that if something goes wrong the stuff in the hidden folder remains intact and you can downgrade to 4.7
    • Here is the command to execute the copy:
 cp -R /mnt/cache/.custom/* /mnt/cache/custom
  • Finally (this is the part that makes it difficult), you'll need to adjust pathnames within the configuration files (.cfg, .conf, .ini, .properties, .sh, etc.) for the applications that were moved to the cache-only share to point to the new folder structure
    • Here are a few to look for from some of the most popular applications (referenced to my /mnt/cache/custom example)
    • I'll ask the community to help add references to their favorite apps to this list
    • Make sure you use a Unix-friendly text editor (Windows Notepad is not Unix-friendly) to edit these files
/mnt/cache/custom/CouchPotato/config.ini
/mnt/cache/custom/SABnzbd/sabnzbd.ini
/mnt/cache/custom/SickBeard/config.ini
/mnt/cache/custom/YAMJ/My_YAMJ.sh

Back up your cache-only share

Since the cache drive is is not a part of the protected array, it's important to back up the cache-only share periodically in case of a cache drive failure. I put together a pretty unsophisticated method to do that on a weekly basis using a simple shell script file, rsync, and cron. OBTW, this procedure will work if you decide to stick with the hidden-folder model; just make sure that your backup shell script points to your hidden folder(s).

  • Set up a folder on the protected array to store the backup (I use /mnt/disk1/System/unRAID on an existing share called System):
 mkdir /mnt/disk1/System/unRAID
  • Using a Unix-friendly text editor, create a shell script that uses rsync to copy the cache-only share to the backup folder (my script /boot/custom/bin/cache_backup.sh contains the following commands):
#!/bin/bash
date >/var/log/cache_backup.log
/usr/bin/rsync -avrtH /mnt/cache/custom/ /mnt/disk1/System/unRAID >>/var/log/cache_backup.log
  • Set the shell script permissions:
chmod +x /boot/custom/bin/cache_backup.sh
  • Put the backup shell script in the weekly cron folder (I also have this command in the /boot/config/go file):
cp /boot/custom/bin/cache_backup.sh /etc/cron.weekly
  • If you want, you can have it run on a daily basis if you put it in the daily cron folder:
cp /boot/custom/bin/cache_backup.sh /etc/cron.daily
  • This [incremental] backup will run on a weekly basis and copies any new files or files that have changed to the backup folder; it also creates a log file when it runs in /var/log/cache_backup.log

Loading Plugins

The most significant new feature (IMHO) unRAID 5.0 brings us is the plugin loader. This single feature brings the ability to easily load and run applications to those of us that don't live and breathe linux. Just drop the plugin into /boot/config/plugins, reboot the machine, and poof ... you're running some of the most popular applications on the internet. And without having to have a detailed knowledge or understanding of linux. Maybe it's not all as great as I say sometimes, but you get what I mean!

  • Create the required folders on the Flash device
md /boot/packages
md /boot/config/plugins
  • Find the plugins for the apps you want to load and run; here is a link to the plugins available on the Lime Technology website
  • Copy the plugin into the /boot/config/plugins folder
  • Reboot the machine to activate the plugin
    • You access, configure, and start the apps on the unRAID Settings page
  • Support for the plugins can be found on the unRAID Support Forum

Crashplan troubleshooting

  • If Crashplan won't start, adjust the permissions as specified below (thanks sacretagent) (Note: $INSTALLDIR is the folder in which the Crashplan app was installed; on my system it is /mnt/cache/custom/Crashplan)
chmod 777 $INSTALLDIR/cp_bin/bin
chmod 777 $INSTALLDIR/cp_bin/jre/bin
  • If you can no longer access Crashplan through the PuTTY tunnel, load and configure the OpenSSH plugin and try again (thanks brun0 and overbyrn). (Note: OpenSSL/OpenSSH were loaded via the /boot/config/go file under unRAID 4.7.)

Troubleshooting

  • Some of the addons are not yet fully compatible with the latest releases of v5. I you find issues, please disable ALL addons and retest. Then if no issue, begin adding them back and retesting. So far, issues have been reported with SimpleFeatures, SmartHistory, and a few others. VM Tools needs to be updated with each UnRAID release. Please inform us and/or the addon author if you identify issues specific to an addon.
  • Some users with certain motherboards, primarily the Super Micro X9SCM series, experience very slow writes to the array. Modify your syslinux.cfg file (in the root folder of your UnRAID flash drive) to add mem=4095M to the default 'append initrd=bzroot' line. For example, the stock syslinux.cfg file with this modification changes to:
default menu.c32
menu title Lime Technology LLC
prompt 0
timeout 50
label unRAID OS
  menu default
  kernel bzimage
  append mem=4095M initrd=bzroot
label Memtest86+
  kernel memtest

Note: As of v5.0-rc11, the stock syslinux.cfg has been modified to include an option that includes the mem=4095M parameter. However to take advantage of it, you need to copy the new syslinux.cfg to your flash drive, and you will need a keyboard and monitor attached in order to be able to select it.

Note: As of v5.0-rc15, the slow write issue is resolved.

  • A few users discovered that after upgrading, the networking no longer worked. Usually this is because they have multiple network NIC's, whether onboard or addon cards. For an explanation and to fix this, please see this post

Please Contribute

I encourage the unRAID community to contribute to this How-To. I appreciate any insights you may have on migrating to unRAID v5, and making it smarter, faster, and easier!

Please place feedback on the unRAID forum here.