UnRAID 6/Storage Management

From unRAID
Jump to: navigation, search

Assigning storage devices

Configuringarray1.png

To assign devices to the array and/or cache, first login to the server's webGui. Click on the Main tab and select the devices to assign to slots for parity, data, and cache disks. Assigning devices to Unraid is easy! Just remember these guidelines:

  • Always pick the largest storage device available to act as your parity device(s). When expanding your array in the future (adding more devices to data disk slots), you cannot assign a data disk that is larger than your parity device(s). For this reason, it is highly recommended to purchase the largest HDD available for use as your initial parity device, so future expansions aren’t limited to small device sizes. If assigning dual parity disks, your two parity disks can vary in size, but the same rule holds true that no disk in the array can be larger than your smallest parity device.
  • SSD support in the array is experimental. Some SSDs may not be ideal for use in the array due to how TRIM/Discard may be implemented. Using SSDs as data/parity devices may have unexpected/undesirable results. This does NOT apply to the cache / cache pool. Most modern SSDs will work fine in the array, and even NVMe devices are now supported, but know that until these devices are in wider use, we only have limited testing experience using them in this setting.
  • Using a cache will improve array performance. It does this by redirecting write operations to a dedicated disk (or pool of disks in Unraid 6) and moves that data to the array on a schedule that you define (by default, once per day at 3:40AM). Data written to the cache is still presented through your user shares, making use of this function completely transparent.
  • Creating a cache-pool adds protection for cached data. If you only assign one cache device to the system, data residing there before being moved to the array on a schedule is not protected from data loss. To ensure data remains protected at all times (both on data and cache disks), you must assign more than one device to the cache function, creating what is called a cache-pool. Cache pools can be expanded on demand, similar to the array.
  • SSD-based cache devices are ideal for applications and virtual machines. Apps and VMs benefit from SSDs as they can leverage their raw IO potential to perform faster when interacting with them. Use SSDs in a cache pool for the ultimate combination of functionality, performance, and protection.
  • Encryption is disabled by default. If you wish to use this feature on your system, you can do so by adjusting the file system for the devices you wish to have encrypted. Click on each disk you wish to have encrypted and toggle the filesystem to one of the encrypted options.

NOTE: Your array will not start if you assign or attach more devices than your license key allows.

Starting and stopping the array

Normally following system boot up the array (complete set of disks) is automatically started (brought on-line and exported as a set of shares). But if there's been a change in disk configuration, such as a new disk added, the array is left stopped so that you can confirm the configuration is correct. This means that any time you have made a disk configuration change you must log into the webGui and manually start the array. When you wish to make changes to disks in your array, you will need to stop the array to do this. Stopping the array means all of your applications/services are stopped, and your storage devices are unmounted, making all data and applications unavailable until you once again start the array. To start or stop the array, perform the following steps:

  1. Log into the Unraid webGui using a browser (e.g. http://tower; http://tower.local from Mac)
  2. Click on Main
  3. Go to the Array Operation section
  4. Click Start or Stop (you may first need to click the "Yes I want to do this" checkbox)

Help! I can't start my array!

If the array can't be started, it may be for one of a few reasons which will be reported under the Array Operation section:

  • Too many wrong and/or missing disks
  • Too many attached devices
  • Invalid or missing registration key
  • Cannot contact key-server
  • This Unraid Server OS release has been withdrawn

Too many disks missing from the array

indication that you have too many devices missing or incorrectly assigned

If you have no parity disks, this message won't appear.

If you have a single parity disk, you can only have up to one disk missing and still start the array, as parity will then help simulate the contents of the missing disk until you can replace it.

If you have two parity disks, you can have up to two disks missing and still start the array.

If more than two disks are missing / wrong due to a catastrophic failure, you will need to perform the New Config procedure.

Too many attached devices

indication that you have too many storage devices attached

Storage devices are any devices which present themselves as a block storage device EXCLUDING the USB flash device used to boot Unraid Server OS. Storage devices can be attached via any of the following storage protocols: IDE/SATA/SAS/SCSI/USB. This rule only applies prior to starting the array. Once the array is started, you are free to attach additional storage devices and make use of them (such as USB flash devices for assignment to virtual machines). In Unraid Server OS 6, the attached storage device limits are as follows:

Attached Storage Device Limits by Registration Key
Trial Basic Plus Pro
Unlimited 6 12 Unlimited

NOTE: The attached device limits do NOT refer to how many devices you can assign to the array or cache. Those limits are imposed by the software, not the license policy.

Invalid or missing key

indication that your key is missing or invalid

Missing key

A valid registration key is required in order to start the array. To purchase or get a trial key, perform the following steps:

  1. Log into the Unraid webGui using a browser (e.g. http://tower from most device, http://tower.local from Mac devices)
  2. Click on Tools
  3. Click on Registration
  4. Click to Purchase Key or Get Trial Key and complete the steps presented there
  5. Once you have your key file link, return to the Registration and paste it in the field then click Install Key.

Expired trial

If the word "expired" is visible at the top left of the webGui, this means your trial key has expired. Visit the registration cage to request either an extension to your trial or purchase a valid registration key.

Blacklisted USB flash device

If your server is connected to the Internet and your trial hasn't expired yet, it is also possible that your USB flash device contains a GUID that is prohibited from registering for a key. This could be because the GUID is not truly unique to your device or has already been registered by another user. It could also be because you are using an SD card reader through a USB interface, which also tend to be provisioned with a generic GUID. If a USB flash device is listed as blacklisted, this is a permanent state and you will need to seek an alternative device to use for your Unraid Server OS installation.

Cannot contact key-server

If your server is unable to contact our key server to validate your license, you will not be able to start the array. The server will attempt to validate upon first boot with a timeout of 30 sec. If it can't validate upon first boot, then the array won't start, but each time you navigate or refresh the webGui it will attempt validation again (with a very short timeout). Once validated, it won't phone-home for validation again unless rebooted.

This Unraid Server OS release has been withdrawn

If you receive this message, it means you are running a beta or release candidate version of Unraid that has been marked disabled from active use. Upgrade the OS to the latest stable, beta, or release candidate version in order to start your array.

Array operations

There are a number of operations you can perform against your array:

  • Add disks
  • Replace disks
  • Remove disks
  • Check disks
  • Spin disks up/down
  • Reset the array configuration

NOTE: In cases where devices are added/replaced/removed, etc., the instructions say "Power down" ... "Power up". If your server's hardware is designed for hot/warm plug, Power cycling is not necessary and Unraid is designed specifically to handle this. All servers built by LimeTech since the beginning are like this: no power cycle necessary.

Adding disks

This is the normal case of expanding the capacity of the system by adding one or more new hard drives:

  1. Stop the array.
  2. Power down the server.
  3. Install your new disk(s).
  4. Power up the server.
  5. Assign the new storage device(s) to a disk slot(s) using the Unraid webGui.
  6. Start the array.

When you Start the array, the system will mount the disk and automatically begin to clear the disk which is required before it can be added to the array. The clearing phase is necessary to preserve the fault tolerance characteristic of the array. If at any time while the new disk(s) is being cleared, one of the other disks fails, you will still be able to recover the data of the failed disk. Unfortunately, the clearing phase can take several hours depending on the size of the new disks(s). Once the disk has been cleared, an option to format the disk will appear in the webGui. Once the format is complete, the disk is added to the array automatically.

The capacity of any new disk(s) added must be the same size or smaller than your parity disk. If you wish to add a new disk which is larger than your parity disk, then you must instead first replace your parity disk. (You could use your new disk to replace parity, and then use your old parity disk as a new data disk.)

You can add as many new disks to the array as you desire at one time, but none of them will be available for use until they are both cleared and formatted with a filesystem.

Replacing disks

There are two primary reasons why you may wish to replace disks in the array:

  • A disk needs to be replaced due to failure or scheduled retirement (out of warranty / support / serviceability).
  • The array is nearly full and you wish to replace existing data disk(s) with larger ones (out of capacity).

In either of these cases, the procedure to replace a disk is roughly the same, but one should be aware of the risk to data loss during a disk replacement activity. Parity device(s) protect the array from data loss in the event a disk failure. A single parity device protects against a single failure, whereas two parity devices can protect against losing data when two disks in the array fail. This chart will help you better understand your level of protection when various disk replacement scenarios occur.

Data Protection During Disk Replacements
With Single Parity With Dual Parity
Replacing a single disk Array cannot tolerate a disk failure without potential data loss to both the disk being replaced and the additional disk that has failed. Array can tolerate up to one additional disk failure without potential data loss
Replacing two disks Not possible! Array cannot tolerate a disk failure without potential data loss to both the disk(s) being replaced and the additional disk that has failed.

Upgrading parity disk(s)

If you wish to upgrade your parity device(s) to a larger one(s) so you can start using larger sized disks in the array, the procedure is as follows:

  1. Stop the array.
  2. Power down the unit.
  3. Install new larger disks.
  4. Power up the unit.
  5. Assign a larger disk to the parity slot (replacing the former parity device).
  6. Start the array.

When you start the array, the system will once again perform a parity sync to the new parity device and when it completes the array will once again be in a protected state. If you have a dual parity system and wish to upgrade both of your parity disks, it is recommended to perform this procedure one parity disk at a time, as this will allow for your array to still be in a protected state throughout the entire upgrade process.

Once you've completed the upgrade process for a parity disk, the former parity disk can be considered for assignment and use in the array as an additional data disk (depending on age and durability).

Replacing failed disk(s)

a red X indicates that a disk has suffered an error and should be replaced
if notifications are enabled, this additional alert will appear

As noted previously, with a single parity disk, you can replace up to one disk at a time, but during the replacement process, you are at risk for data loss should an additional disk failure occur. With two parity disks, you can replace either one or two disks at a time, but during a two disk replacement process, you are also at risk for data loss. Another way to visualize the previous chart:

Array Tolerance to Disk Failure Events
Without Parity With Single Parity With Dual Parity
A single disk failure Data from that disk is lost Data is still available and the disk can be replaced Data is still available and the disk can be replaced
A dual disk failure Data on both disks are lost Data on both disks are lost Data is still available and the disks can be replaced
confirming you wish to start the array and rebuild the contents of the failed disk on a new disk
notification indicating that a disk rebuild is occurring
the progress and time remaining for the rebuild will be displayed under the array operation section

NOTE: If more disk failures have occurred than your parity protection can allow for, you are advised to post in the General Support forum for assistance with data recovery on the data devices that have failed.

To replace a failed disk or disks:

  1. Stop the array.
  2. Power down the unit.
  3. Replace the failed disk(s) with a new one(s).
  4. Power up the unit.
  5. Assign the replacement disk(s) using the Unraid webGui.
  6. Click the checkbox that says Yes I want to do this and then click Start.

When you start the array after replacing a failed disk or disks, the system will reconstruct the contents onto the new disk(s) and, if the new disk(s) is/are bigger, expand the file system.

You must replace a failed disk with a disk which is as big or bigger than the original and not bigger than the parity disk.

A disk failed while I was rebuilding another

If you only have a single parity device in your system and a disk failure occurs during a data-rebuild event, the data rebuild will be cancelled as parity will no longer be valid. However, if you have dual parity disks assigned in your array, you have options. You can either

  • let the first disk rebuild complete before starting the second, or
  • you can cancel the first rebuild, stop the array, replace the second failed disk, then start the array again

If the first disk being rebuilt is nearly complete, it's probably better to let that finish, but if you only just began rebuilding the first disk when the second disk failure occurred, you may decide rebuilding both at the same time is a better solution.

Removing disks

Removing a disk from the array is possible, but requires you to once again sync your parity disk(s) after doing so. This means that until the parity sync completes, the array is vulnerable to data loss should any disk in the array fail. To remove a disk from your array, perform the following steps:

  1. Stop the array
  2. Make note if your disk assignments under main tab (for both the array and cache; some find it helpful to take a screenshot)
  3. Perform the Reset the array configuration procedure
  4. Reassign all your previous devices except for the one you removed
  5. Start the array

A parity-sync will occur if at least one parity disk is assigned and until that operation completes, the array is vulnerable to data loss should a disk failure occur.

Checking disks

the check button lets you perform parity and read checks

When the array is started, there is a button under Array Operations labeled Check. Depending on whether or not you have any parity devices assigned, one of two operations will be performed when clicking this button.

Parity check

If you have at least one parity device assigned, clicking Check which will initiate a Parity-check. This will march through all data disks in parallel, computing parity and checking it against stored parity on the parity disk(s).

By default, if an error is found during a Parity-check the parity disk will be updated (written) with the computed data and the Sync Errors counter will be incremented. If you wish to run purely a check without writing correction, uncheck the checkbox that says Write corrections to parity before starting the check. In this mode, parity errors will be notated but not actually fixed during the check operation.

The most common cause of Sync Errors is an unexpected power-loss, which prevents buffered write data being written to disk. Anytime the array is Started, if the system detects that a previous unsafe shutdown occurred, then it automatically initiates a Parity-Check. It is highly recommended that users consider purchasing a UPS (uninterruptable power supply) for their systems, especially if frequent offsite backups aren't being performed.

Read check

history lets you review stats on your preview check operations

If you configure an array without any parity devices assigned, the Check option will start a Read check against all the devices in the array. You can use this to check disks in the array for unrecoverable read errors, but know that without a parity device, data may be lost if errors are detected.

Check history

Any time a parity or read check is performed, the system will log the details of the operation and you can review them by clicking the History button under Array Operations. These are stored in a text file under the config directory on your Unraid USB flash device.

Checking array devices

the check button lets you perform parity and read checks

When the array is started, there is a button under Array Operations labeled Check. Depending on whether or not you have any parity devices assigned, one of two operations will be performed when clicking this button.

Parity check

If you have at least one parity device assigned, clicking Check which will initiate a Parity-check. This will march through all data disks in parallel, computing parity and checking it against stored parity on the parity disk(s).

By default, if an error is found during a Parity-check the parity disk will be updated (written) with the computed data and the Sync Errors counter will be incremented. If you wish to run purely a check without writing correction, uncheck the checkbox that says Write corrections to parity before starting the check. In this mode, parity errors will be notated but not actually fixed during the check operation.

The most common cause of Sync Errors is an unexpected power-loss, which prevents buffered write data being written to disk. Anytime the array is Started, if the system detects that a previous unsafe shutdown occurred, then it automatically initiates a Parity-Check. It is highly recommended that users consider purchasing a UPS (uninterruptable power supply) for their systems, especially if frequent offsite backups aren't being performed.

Read check

history lets you review stats on your preview check operations

If you configure an array without any parity devices assigned, the Check option will start a Read check against all the devices in the array. You can use this to check disks in the array for unrecoverable read errors, but know that without a parity device, data may be lost if errors are detected.

Check history

Any time a parity or read check is performed, the system will log the details of the operation and you can review them by clicking the History button under Array Operations. These are stored in a text file under the config directory on your Unraid USB flash device.

Spin up and down disks

If you wish to manually control the spin state of your rotational storage devices or toggle your SSD between active and stand-by mode, these buttons provide that control. Know that if files in are in the process of being accessed while using these controls, the disk(s) in use will remain in an active state, ignoring your request. When disks are in a spun-down state, they will not report their temperature through the webGui.

Reset the array configuration

If you wish to remove a disk from the array or you simply wish to start from scratch to build your array configuration, there is a tool in Unraid that will do this for you. To reset the array configuration, perform the following steps:

you can reset your disk configuration from the new config page
  1. Navigate to the Tools page and click New Config
  2. Click the checkbox and click apply to perform the operation
  3. Return to the Main tab and your configuration will have been reset

Undoing a reset

If for any reason after performing a reset, you wish to undo it, perform the following steps:

  1. Browse to your flash device over the network (SMB)
  2. Open the Config folder
  3. Rename the file super.old to super.dat
  4. Refresh the browser on the Main page and your array configuration will be restored

Cache pool operations

There are two primary modes of operating the cache in Unraid:

Single device mode

When the number of disk slots for the cache is set to one, this is referred to as running in single device mode. In this mode, you will have no protection for any data that exists on the cache, which is why pool mode is recommended. However, unlike in pool mode, while in single device mode, you are able to adjust the filesystem for the cache device to something other than btrfs. It is for this reason that there are no special operations for single mode. You can only add or remove the device from the system.

NOTE: If you choose to use a non-btrfs file system for your cache device operating in single mode, you will not be able to expand to a cache pool without first reformatting the device with btrfs. It is for this reason that btrfs is the default filesystem for the cache, even when operating in single device mode.

Cache pool mode

When more than one disk is assigned to the cache, this is referred to running in cache pool mode. This mode utilizes btrfs RAID 1 in order to allow for any number of devices to be grouped together in a pool. Unlike a traditional RAID 1, a btrfs RAID1 can mix and match devices of different sizes and speeds and can even be expanded and contracted as your needs change. To calculate how much capacity your btrfs pool will have, check out this handy btrfs disk usage calculator. Set the Preset RAID level to RAID-1, select the number of devices you have, and set the size for each. The tool will automatically calculate how much space you will have available.

Here are the normal configuration changes you can make to the cache when operating in pool mode:

  • Add disks
  • Replace a disk

Adding disks

To add disks to the cache (pool) in your array, perform the following steps:

  1. Stop the array.
  2. Navigate to the Main tab.
  3. Scroll down to the section labeled Cache Devices.
  4. Change the number of Slots to be at least as many as the number of devices you wish to assign.
  5. Assign the devices you wish to the cache slot(s).
  6. Start the array.
  7. Click the checkbox and then the button under Array Operations to format the devices.

Replace a disk

To replace a disk in the pool, perform the following steps:

  1. Stop the array.
  2. Physically detach the disk from your system you wish to remove.
  3. Attach the replacement disk (must be equal to or larger than the disk being replaced).
  4. Refresh the Unraid webGui when under the Main tab.
  5. Select the cache slot that previously was set to the old disk and assign the new disk to the slot.
  6. Start the array.
  7. If presented with an option to Format the device, click the checkbox and button to do so.

NOTE: You can only replace up to one disk at a time from your cache pool.

Share Management

Once you have assigned some devices to Unraid and started the array, you can create shares to simplify how you store data across multiple disks in the array. Unraid will automatically create a handful of shares for you that it needs to support common plugins, containers, and virtual machines, but you can also create your own shares for storing other types of data. From the Shares tab, you can either create a new share or edit' an existing share. Click the Help icon in the top-right of the Unraid webGui when configuring shares for more information on the settings available.