Utilizing Quotas on XFS
So I recently had a buddy of mine pen test my SFTP. While the SFTP isn't open to the general public, it is accessible via the Net, so I treat it like everyone in the world has at least user level access. While it got an overall good bill of health he did point out one key issue, that users had the ability to max out the HDD and render the box useless, or at least incredibly slow. So that's where it began. I discovered disk quotas.
Of course most of us know of disk quotas, or at least their general theory. It's what limits our gmail Inbox folder, stops us from filling the school server with porn, and all sorts of practical limitations on users. Basically they just keep us in line in regards to how much space we're allowed to use on a particular drive. Honestly they're not very difficult to setup, at least not with Gentoo. In fact to get started you only need to head here and give it a run-through. The only problem came when I discovered that quota support on my file system of choice, XFS, was...less than well documented.
So, after three days of poking around IRC, reading man pages, HOW-TO's, manuals, asking the programmers who wrote quota itself, Googling, and reading old XFS mailing list threads, I found all the answers. So I hope to make it easier on others with a simple guide here. I've already sent these modifications to the maintainer of the Gentoo Security Guide, so hopefully they'll make it into that shortly and no one will ever have a need to read this, but if you do, enjoy.
First a quick recap of XFS and why I prefer it. XFS is a file system developed by SGI meant to be scalable and high-performance. It has all sorts of incredible stats, ranging from throughput of well over 250Mbps (on a desktop, well into the Gbps range for higher-end systems) to supporting over 9 exabytes of files on the 2.6 kernel. It also has support for many other features, such as NFS, quota (obviously), and extended security attributes so it'll work with SELinux. The fact is it's high-end, and overkill for a low-end server. But that's why I love Linux and open-source technology, it lets me play with incredible toys. XFS is fast, stable, scalable, and just outright impressive. The only disadvantage I must mention is that it aggressively caches data in memory. This results in data loss if there's a sudden failure of power. If you use XFS on something mission critical, use backup power.
So now to how to get it done. By this point I'm assuming you know how to configure the kernel, and you have a box with XFS partitions. So the first thing we do is enable XFS quota support in the kernel. (Note everything in this tutorial is assuming you're using the 2.6.14-hardened-r5 kernel, Gentoo Hardened's latest release.)
Note this differs from the generic quota support option (Symbol: QUOTA), which only works on ext2, ext3, and ReiserFS.
The following step is CRITICAL if you plan on using XFS and quotas on /. If you fail to put the boot flag 
The blocks and inode columns show that which is currently being used. This can be handy on a long-running system which is just starting to enforce quotas, as it can give you a sense of your users needs. As you can see I give every user 2.5GB of storage with no strings attached, and allow them 2.75GB so they can casually break that 2.5GB barrier if they ever need a little extra storage for the short term. Obviously this limit is heavily dependent on how much storage you have as well as how many users and what you expect them to be storing. You can also see I allow users one thousand files before the soft limit kicks in. This is dependent mostly on what sort of files you expect to be stored. Obviously if you expect heavy storage of images or text files, it might be a good idea to raise this cap. Finally you can see I allow users a hard limit of 1,500 files, in case they need a quick place to dump things, say as a temporary backup for their music or image archives.
Finally I'm going to enable group quota on /tmp. This is a simple precaution to prevent a user rendering the box useless either by accident or on purpose. The fact is for /tmp to be useful it's got to be world writable. While /tmp filling up generally isn't a huge deal, it can degrade performance on the box. So with 2GB assigned to /tmp, I'm going to limit the group users from using more than 1.5GB of that, ensuring at most it'll be a nuisance, and the box shall remain in tact and performing to expectations.
If the command to edit users is 
Looking it over the same way as discussed above, you can see the group users is limited to 1.5GB on /tmp, and a hard limit of 1.525GB, just as a quick work around rather than disabling the Grace Period. The inode limit is set to five thousand files, with a temporary allowance of 5,250. Both of these settings should be good for any normal period of use, and short of a critical error or malicious activity neither will ever be approached, especially given that /tmp is cleaned on a regular basis.
That's pretty much it. We've enabled XFS Quota's in the kernel, installed the quota management software, changed fstab and grub.conf to allow quotas, given each user a limit on /, and limited the group users to 3/4th of the space available in /tmp. As hard as it may seem at first, the beauty of it is most of this can be scripted, but that's beyond the scope of this article.
A few closing points however. Beyond being useful on public systems quotas can actually be used as a security barrier on any system, such as a firewall. For example on my firewall box I've disabled root logins, so I must login as a normal user then su in. This means for an attacker to do anything they must have both logins. As just a small barrier I've enabled a quota on my normal user of 125MB, that way even if the user were compromised, there's virtually nothing the attacker could do. Without quotas they could at least fill up the HDD and effectively render the box useless.
There are a few minor other things to consider, just to help in maintaining quotas. These are all just simple commands that can help you get a picture of the setup:
So now to how to get it done. By this point I'm assuming you know how to configure the kernel, and you have a box with XFS partitions. So the first thing we do is enable XFS quota support in the kernel. (Note everything in this tutorial is assuming you're using the 2.6.14-hardened-r5 kernel, Gentoo Hardened's latest release.)
File Systems -> XFS Quota Support (Symbol: XFS_QUOTA)
Note this differs from the generic quota support option (Symbol: QUOTA), which only works on ext2, ext3, and ReiserFS.
Load the new kernel and reboot
After coming back up go ahead and:
emerge quota
Another place XFS differs from other systems is that XFS stores the quota data as meta data with the files rather than in a user readable file. What this means is that on XFS systems you don't need aquota.user or aquota.group. Also you won't need to run rc-update add quota boot, as with XFS this will only give you warnings on boot since those files don't exist. You will still need quota installed however, as you'll use it to create and maintain quotas.
Next we need to enable quota support for each file system we want to limit, so go ahead and:
nano -w /etc/fstab
At this point you need to add usrquota to the file system options for any partition you'd like to enable user quotas on, and grpquota for any partition you'd like to limit entire groups on. Of course you can do both, and in the following example I'll show how each can be useful.
Following is a screen shot straight from my SFTP. Most of what it shows is of no meaning right now, all you need to know is where to toss the quota flags. In this example we're enabling quotas for both users and groups on / and /tmp.
The following step is CRITICAL if you plan on using XFS and quotas on /. If you fail to put the boot flag rootflags=quota into your grub.conf, your system will fail to boot stating / can't be mounted read/write! So load up your grub.conf and go to town:
mount /boot (If it isn't already mounted, which it shouldn't be!)
nano -w /boot/grub/grub.conf
Toss it onto your kernel line so the option is passed to the kernel during boot (hence "boot flag"):
kernel /boot/kernel-2.6.14-hardened-r5 root=/dev/hda4 rootflags=quota
Reboot
Now not only does your kernel have XFS Quota support but / and /tmp should be happily mounted and waiting for quotas to be loaded.
Now here's where XFS quota management starts to really differ from quotas on other file systems. While you need the quota package installed, it's really only for edquota, you don't need to concern yourself with quotaon or quotaoff, quotas are activated during boot. For more information, just man these commands and remember Google is your friend. Here we're just covering the basics. First off a few definitions:
Block = Think of a block as a KB, pure and simple. If you want to get technical it's 1,024 bytes.
inode = inode's are basically objects, be they files, symlinks, or anything else.
Soft Quota = An artificial limit, at this point the user will be warned they've exceeded their limit and have X time (per the set Grace Period, if any, read below) to remove data.
Hard Quota = The max limit, the system won't allow any more information written if this threshold is met.
Grace Period = The time after the Soft Quota has been exceeded before it's enforced.
Note that if you want to tinker with the Grace period, use edquota -t, otherwise it'll default to one week. If you set the Grace Period to zero, then the Soft Quota will be enforced as if it were the Hard Quota.
Now is where everyones particular needs come into play. Do you want to limit on a per user basis, or per group? Do you want to limit by both? What's a fair limit on storage space, but also what's a fair limit on how many individual files may be stored? Having millions of little files will bog down a system just as if its storage were running low. This is heavily based on the hardware being used and the services it provides as well as its target user base. The examples below are for a decent PC turned SFTP which is fairly generous on both storage and file limits due to its relative low user base, never exceeding more than a dozen or so users with no more than two or three on at a given time usually.
For my setup I want to limit on a per user basis on /. I have no use for limiting by groups since I'm the only one who can create new accounts, so the method for controlling the overall HDD usage is simply my limit on account creation. Below you can see how the syntax comes together. To edit a particular user just run:
edquota -u {username}
In this case:
edquota -u thehailo
and you'll be met with something similar to:
The blocks and inode columns show that which is currently being used. This can be handy on a long-running system which is just starting to enforce quotas, as it can give you a sense of your users needs. As you can see I give every user 2.5GB of storage with no strings attached, and allow them 2.75GB so they can casually break that 2.5GB barrier if they ever need a little extra storage for the short term. Obviously this limit is heavily dependent on how much storage you have as well as how many users and what you expect them to be storing. You can also see I allow users one thousand files before the soft limit kicks in. This is dependent mostly on what sort of files you expect to be stored. Obviously if you expect heavy storage of images or text files, it might be a good idea to raise this cap. Finally you can see I allow users a hard limit of 1,500 files, in case they need a quick place to dump things, say as a temporary backup for their music or image archives.
Finally I'm going to enable group quota on /tmp. This is a simple precaution to prevent a user rendering the box useless either by accident or on purpose. The fact is for /tmp to be useful it's got to be world writable. While /tmp filling up generally isn't a huge deal, it can degrade performance on the box. So with 2GB assigned to /tmp, I'm going to limit the group users from using more than 1.5GB of that, ensuring at most it'll be a nuisance, and the box shall remain in tact and performing to expectations.
If the command to edit users is edquota -u {username}, care to guess at the group syntax?
edquota -g {groupname}
In this case:
edquota -g users
Looking it over the same way as discussed above, you can see the group users is limited to 1.5GB on /tmp, and a hard limit of 1.525GB, just as a quick work around rather than disabling the Grace Period. The inode limit is set to five thousand files, with a temporary allowance of 5,250. Both of these settings should be good for any normal period of use, and short of a critical error or malicious activity neither will ever be approached, especially given that /tmp is cleaned on a regular basis.
That's pretty much it. We've enabled XFS Quota's in the kernel, installed the quota management software, changed fstab and grub.conf to allow quotas, given each user a limit on /, and limited the group users to 3/4th of the space available in /tmp. As hard as it may seem at first, the beauty of it is most of this can be scripted, but that's beyond the scope of this article.
A few closing points however. Beyond being useful on public systems quotas can actually be used as a security barrier on any system, such as a firewall. For example on my firewall box I've disabled root logins, so I must login as a normal user then su in. This means for an attacker to do anything they must have both logins. As just a small barrier I've enabled a quota on my normal user of 125MB, that way even if the user were compromised, there's virtually nothing the attacker could do. Without quotas they could at least fill up the HDD and effectively render the box useless.
There are a few minor other things to consider, just to help in maintaining quotas. These are all just simple commands that can help you get a picture of the setup:
quota – Running this will list any disk quotas your current login is restricted by.
quota {username} - If you're root you can run this to view someone else's quotas.
quota -g {groupname} - Since quota assumes you're feeding it a user name, you need to use this to check group quotas.
quotaon -v {mount point} - This will give you a quick listing of that partitions quota status, if it's enabled or not. This is a quick and easy way to confirm quotas are running properly.
That's just about it. I've only been messing with quotas for three days or so, so I'm by no means an expect. I hope this'll help other XFS users who enjoy experimenting. If you have anything to add, or any questions or comments, feel free to login and hit me up via the comment box below.
2006-02-23 EDIT: Fixed some botched math on my block size calculations. I was treating blocks as 1MB instead of 1KB. Ops.
2006-03-18 EDIT: Thanks to EgS for pointing out the fact that XFS doesn't need quotacheck ran and for reinforcing my choice of enabling anonymous comments! Below is what was removed from the guide:
Now for a snippet straight from the Gentoo guide.
nano -w /etc/crontab
and insert
0 3 * * 0 /usr/sbin/quotacheck -avug.
to have cron check quotas on a weekly basis, conveniently matching our default Grace Period.
Print This Item
Comments
quotacheck