Probably like a few of you, I have a whole LOT of movies/media. A couple of years ago I converted every single DVD/BluRay I owned to an mkv. At first it was easy, I had a couple of decent sized hard drives in my desktop computer. Put DVD rips on one drive, BluRay rips on another and then share them over the network. Then I got into the ARM world. Moved storage from my desktop to USB drives plugged into a dockstar. I ended up having I think 6 or 7 hard drives of varying sizes plugged into it. Then I asked for a eSATA enclosure for xmas(which I got) and now have it stocked with 2TB drives.
I was looking for a way to pool all my drives with some fault tolerance. I ruled out RAID0 because if I lost one drive, I would lose EVERYTHING. That wasn't acceptable. I could live with a single drive failure (I could always rerip them if necessary). Couldn't do RAID1, (I already have about 4.8TB of data on the enclosure). RAID5 looked better, and I already started migrating stuff around to do that. Then I calculated that I would only have about 300GB free after that. And if I wanted to switch a 2TB drive for a 4TB drive later, I wouldn't be able to grow the array until I replaced ALL of the drives with larger ones. BTRFS looked promising. You can add devices to a pool. Remove a device from a pool. If one drive dies, you can still recover files that are stored wholly on the other drives. But movies are quite large (600M to 15G). So they would almost guaranteed to be spread over multiple drives.
tl;dr
Enter mhddfs. Mhddfs is a FUSE filesystem that joins multiple mount points into one place. It will show everything that's on them in one location. When you write to it, it will put the file in the first one with enough space. Almost exactly what I was looking for. I used to actually have a script that went through my drives and made symlinks to them in one directory, so mhddfs is actually better.
I've added mhddfs to our AUR repo. You can install it with:
$this->bbcode_second_pass_code('', '[root@alarm ~]# pacman -S mhddfs')
then you can mount your drives like this: (example from my device)
$this->bbcode_second_pass_code('', '[root@alarm ~]# mhddfs /media/Rhodium/,/media/Yttrium/,/media/Palladium/,/media/Thallium/ /media/Pool/ -o allow_other
')
That will spit out:
$this->bbcode_second_pass_code('', 'mhddfs: directory '/media/Rhodium/' added to list
mhddfs: directory '/media/Yttrium/' added to list
mhddfs: directory '/media/Palladium/' added to list
mhddfs: directory '/media/Thallium/' added to list
mhddfs: mount to: /media/Pool/
mhddfs: move size limit 0%
')
and then if I run df -h I get something pretty cool looking!
$this->bbcode_second_pass_code('', '[root@alarm Movies]# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 19G 5.0G 13G 29% /
/dev/root 19G 5.0G 13G 29% /
devtmpfs 60M 0 60M 0% /dev
run 61M 280K 60M 1% /run
shm 61M 0 61M 0% /dev/shm
/dev/sdb1 1.8T 196M 1.7T 1% /media/Palladium
/dev/sdc1 1.8T 1.7T 30G 99% /media/Rhodium
/dev/sdd1 1.8T 1.7T 41G 98% /media/Thallium
/dev/sde1 1.8T 1.5T 302G 83% /media/Yttrium
tmpfs 61M 0 61M 0% /tmp
/media/Rhodium/;/media/Yttrium/;/media/Palladium/;/media/Thallium/ 7.2T 4.8T 2.1T 70% /media/Pool
')
This makes managing groups of random sized disks MUCH easier! And all I have to do to add storage is plug in a new drive, format it, and remount the pool.