Intro [^]
One of the most important maintenance tasks that can only be done by using a live CD is partitioning. No operating system allows partitioning the same disk where the OS itself resides. Trying to do so is like attempting to repair a car while its engine is turned on.
Of course, SystemRescueCD contains multiple programs that are related to partitioning. Most important are GParted (graphical partitioning program), GNU Parted (text-based partitioning program), fdisk and sfdisk (partition table editors) and various filesystem tools (like ntfsprogs and e2fsprogs).
This page contains some theory about partitions and filesystems, advice for choosing the right filesystem and a partitioning example by using GParted.
 |
While partitioning, an user error or a bug can damage your partitions. Creating a disk image of the disk to be partitioned beforehand is highly recommended. |
 |
The following pressentation has been made using SystemRescueCD v 1.5.8 |
Some theory [^]
What is a partition? [^]
A partition is a logical division of a hard disk created so that you can have different operating systems on the same hard disk or to create the appearance of having separate hard drives for file management, multiple users, or other purposes.
In Windows, a one-partition hard disk is labelled the "C:" drive ("A:" and "B:" are typically reserved for diskette drives). A two-partition hard drive would typically contain "C:" and "D:" drives. (CD-ROM drives typically are assigned the last letter in whatever sequence of letters have been used as a result of hard disk formatting, or typically with a two-partition, the "E:" drive.).
In UNIX-based systems, a partition is used to host the / (root) file system, and optionally the /opt, /usr and /home file systems. There may also be a swap partition, which doesn't host any file system.
Each operatin system provides some kind of tool to create and manage partitions. Examples of such tools are fdisk in DOS/Windows, fdisk, sfdisk and parted in Linux, etc.
What is the difference between primary, extended and logical partitions? [^]
Information about partitions is saved in so-called partition table in Master Boot Record. MBR itself is only 512 bytes in size, and only 64 bytes are reserved for partition table. That's not enough, and there are many workarounds to bypass limitations caused by the size, for example logical block addressing. Extended partitions are another workaround.
Partition table can only store information about four partitions. If one has, for example, two GNU/Linux distributions on the same disk, both of them having separate root partitions, shared /home and shared swap, the partition number limit has been hit already.
A partition that is mentioned in the partition table is called primary partition. Because of the limit, one disk can only contain 1-4 primary partitions.
An extended partition fixes the problem simply by containing more boot records, called Extended Boot Records (EBR). Each EBR contains information about one logical partition and, if the extended partition contains multiple logical partitions, link to the next EBR. Thus, an extended partition can contain unlimited amount of logical partitions.
Extended partition contains only EBRs and logical partitions (and maybe unallocated space). Extended partition doesn't contain any filesystem and files can't be stored in it. Of course, logical partition can contain any filesystem (or be unformatted).
Logical partitions can always be used for storing data: any operating system can see logical partitions. GNU/Linux supports both multiple primary extended partitions and extended partitions within each other, while Windows supports only the latter. GNU/Linux distributions can be installed to logical partitions as well, but Windows requires a lot of tweaking. See this outdated guide.
What is LVM? [^]
LVM means "Logical Volume Manager". It allows creating volume groups on top of hard drives and logical volumes within volume groups. Logical volumes are NOT the same thing as logical partitions!
Volume groups can be created very flexibly: a volume group can allocate, for example, the first half of the first hard drive and the second half of the third drive. One can even create a massive volume group containing all storage he/she has.
The computer sees a logical volume as a partition: logical volume can be left unformatted or contain any filesystem.
LVM has many benefits: for example, if one has three hard drives 60 gigabytes each, he/she can create a 160-gigabyte partition for storing massive files and/or saving some disk space. In addition, logical volumes can be resized even when they're in use, so when creating logical volumes one doesn't need to worry if they're too small or big - if they are, he/she can resize them at any time.
However, resizing a logical volume doesn't resize the filesystem in it, so using a filesystem that can be resized in use (online resizing) is recommended. Very few filesystems can be shrinked online, but most GNU/Linux filesystems (including ext3/4, ReiserFS, XFS and btrfs) can be grown online. It's generally a good idea to leave unallocated space within volume group, so logical volumes can later be grown without shrinking any other logical volume.
Here come bad news for people who dualboot: Windows doesn't support LVM, it sees volume groups as unformatted partitions. If you try to access volume group within Windows, you're just prompted to format the partition. That prompt is annoying at best and dangerous at worst.
More information about LVM can be found here (almost everything about LVM in a single page) and here (official SystemRescueCD documentation about LVM).
What is a file system? [^]
A file system is the way in which files are named and where they are placed logically for storage and retrieval. The DOS, Windows, OS/2, Macintosh, and UNIX-based operating systems all have file systems in which files are placed somewhere in a hierarchical (tree) structure. A file is placed in a directory (folder in Windows) or subdirectory at the desired place in the tree structure.
The most important difference between filesystems is operating system support. Some filesystems are supported by all modern operating systems, but especially the newest filesystems are very rarely supported. Other important limits are maximum file size, journaling support and file permission metadata support.
The reason that file size limits exist is that all filesystems reserve a fixed number of bits for storing the file size. If the size of the file, in bytes, is bigger than the biggest number that can be stored in file size bits, the operating system must refuse to store the file at all in order to prevent data corruption.
File permission metadata means that the filesystem stores in the metadata of the file, among other things, information about who owns the file and what different users are allowed to do with the file. That metadata is especially useful in multi-user environment because it mostly prevents users from reading each other's files. Permissions can be bypassed, however.
What is journaling? [^]
Ideally, data in a partition never corrupts. But, in the real world, there are power failures and operating system freezes. And if a computer is forcefully shut down while something is written to the drive, the write operation can't be finished. That can damage the filesystem and destroy any files in the partition.
Journaling partially fixes that problem by writing most changes to the disk twice: first to a special area called journal and, after that, to the filesystem itself. If power is lost while writing to the journal was in progress, the partial change is just ignored and never committed to the filesystem itself. If power failure or OS freeze happened while writing to filesystem itself, the write operation is finished by using the information in journal.
Journaling is always a trade-off between reliability and performance. In fact, the ext3 and ext4 filesystems support multiple journaling modes in order to allow the user to choose the optimal compromise. The most popular choices are ordered and writeback.
Both modes only write metadata changes to the journal before committing them: data itself is written directly to the main filesystem. The difference between the modes is that ordered mode guarantees that the data is written before the change is marked as committed. The difference may sound small, but in some cases ordered mode causes horrible performance. In Linux 2.6.30, the default journaling mode was changed to writeback - and it was quickly found out that writeback mode may cause massive data loss. See this forum post for details. Most GNU/Linux distributions are now using ordered mode as the default again.
In addition, on SSDs (Solid State Drives) and thumb drives write speed is much slower than read speed. They also have a limited number of writing cycles, so journaling reduces their lifetime. Thus, I (Jyrki) recommend against using journaling fileystems on such drives.
What are the differences between most popular filesystems? [^]
The following table quickly describes the most important differences between them.
| Operating system support | |||||
|---|---|---|---|---|---|
| Under Windows | Under GNU/Linux | Maximum file size | Journaling | Permissions | |
| FAT32 | Native | Built-in | 4 GB | No | No |
| NTFS | Native | Driver included | 16 EB | Yes | Yes |
| ext2 | 3rd party driver | Native | 16 GB-2 TB* | No | Yes |
| ext3 | 3rd party driver | Native | 16 GB-2 TB* | Yes | Yes |
| ext4 | No*** | Native | 16 GB-16 TB* | Yes | Yes |
| exFAT | Built-in (Vista/7)** | 3rd party driver | 64 ZB | No | Yes |
* Depends on cluster size
** This update adds exFAT support to Windows XP
*** Ext2read makes accessing ext4 partitions possible, but it's not a driver and the access is read-only
Operating system support:
- "Native" means that the kernel supports the filesystem and the OS can boot from a partition using that FS.
- "Built-in" means that the kernel supports the filesystem, but booting from a partition containing such FS is very difficult.
- "Driver included" means that ntfs-3g (the driver that adds NTFS support to Linux) comes with most GNU/Linux distributions.
- "3rd party driver" means that drivers to add filesystem support are available, but must be downloaded and installed separately. The ext2/3 drivers are Ext2 IFS and Ext2fsd and the exFAT driver is exfat.
- "No" means that there is no way to get read-write filesystem support.
Filesystems [^]
This section contains more information about most popular filesystems.
FAT32 [^]
The initial version of FAT (File Allocation Table), now referred as FAT12, was designed for floppy disks. A FAT12 partition can only be up to 32 megabytes in size. After that, PCs equipped with hard drives were introcuded by IBM and the sizes of hard drives began growing. Microsoft answered the need by developing first initial FAT16 and then final FAT16.
FAT16 partition can be up to two gigabytes in size. In the middle of 1990s, that limit was becoming a problem. Microsoft pushed the limit up by updating FAT again.
FAT32 was first introduced with Windows 95 OSR2. Windows 98, Windows Me, Windows 2000 and newer support FAT32 too. Linux kernel has supported FAT32 almost as long as Windows, but booting GNU/Linux from FAT32 partition is difficult and actually requires DOS to be installed in the partition as well. (more information)
FAT32 partition can be up to two terabytes in size. As of now (July 2010), there are hard drives that hit the limit, but don't exceed it. A single file within FAT32 partition can be up to four gigabytes in size.
Because FAT32 is, in the end, based on FAT12, it has very few features. It doesn't support file permissions, hard/symbolic links, encryption, compression, alternative data streams, journaling... It lacks support for nearly anything that defines a modern filesystem. However, due to very few features, FAT32 is very fast filesystem if it's not fragmented or on a Flash-based drive. Mind you, FAT32 fragments very fast.
Due to excellent operating system support, I recommend FAT32 for storing files which should be accessible in both Windows and GNU/Linux. FAT32 is also a good filesystem on Solid State Drives and thumb drives due to its performance.
ext2 [^]
Ext2 or ext2fs is the successor of extfs (extended file system). Extfs didn't support separated timestamps for access, data modification and inode modification. In order to add support for them, and make the filesystem extendable, a new filesystem had to be created.
Ext2 was developed in January 1993, earlier than any other filesystem mentioned in this page.
Because ext2 is designed for GNU/Linux, support in Linux kernel was implemented immediately. The first Windows driver supporting ext2, Ext2fsd 0.01, was released on 25 January 2002. Both Windows drivers for ext2 work only on Windows NT operating systems (NT 4.0 up to Vista, 7 isn't supported yet).
The best property of ext2 is extensibility. The superblock contains information about which version the filesystem is (ext2, ext3 or ext4) and which extensions and features are in use. By using these pieces of information, the operating system or driver can decide whether or not mounting the partition is safe. That's the most important reason why most GNU/Linux distributions still use successors of ext2 as default filesystems.
Depending on cluster size, ext2 partition can be up to 2-32 terabytes in size. File size limit is 16 GB-2 TB.
Ext2 supports file permissions, both hard and symbolic links and extended file attributes. Encryption, compression and journaling are unsupported.
Due to lack of journaling support and existence of Windows drivers, I recommend using ext2 if you're going to install GNU/Linux on a SSD drive and want to be able to access files within Windows too.
However, lack of journaling support is the worst limitation of ext2. And what was done in order to get rid of the limitation?
ext3 [^]
Ext3, the successor of ext2, was introduced in Linux kernel on November 2001. It supports journaling, can be grown online and optionally indexes large directories.
Ext2 IFS and Ext2fsd can mount ext3 partition as ext2 if the journal is empty. (If it's not, something is wrong - journal is always emptied when the partition is unmounted or the computer is shut down.) Thus, ext3 support under Windows is just as good/bad as ext2 support.
Partition and file size limits are the same as in ext2: partition size limit is 2-32 TB and file size limit 16 GB-2 TB, depending on cluster size.
Due to journaling support and existence of Windows drivers, ext3 is a good choice if you're going to install GNU/Linux on a mechanical hard drive and want to be able to access files within Windows.
ext4 [^]
Linux kernel support for ext4, the successor of ext3, was marked stable code on October 2008. Ext4 contains multiple performance and stability improvements over ext3.
The most important new feature is extents. An extent is a contiguous area of storage that has been reserved for a file. When a process starts to write to a file, the whole extent is allocated even before the write operation begins. The idea is that even if the file is larger than expected, it doesn't fragment if it doesn't exceed the size of the extent.
Another important improvement is larger partition size limit: an ext4 partition can be even one exabyte in size. (An exabyte is a million terabytes.) In addition, a directory within an ext4 partition can contain up to 64 000 subdirectories (instead of 32 000, as in ext2/3) and timestamps are much more accurate. The file size limit is 16 GB-16 TB, depending on cluster size.
Unfortunately, Ext2 IFS and Ext2Fsd don't support ext4 and are unable to mount ext4 partition if extents are enabled. They can be disabled, but other improvements of ext4 aren't that important for most people - using ext2 or ext3 is just easier. It is also possible to keep extents enabled and browse the partition using Ext2read, but Ext2read doesn't allow the user to write to the partition.
Due to its features, ext4 is a good filesystem on computers that only have GNU/Linux installed. Because journaling can be disabled, it is suitable for Solid State Drives and thumb drives too.
NTFS [^]
At the end of 1980s, IBM and Microsoft were developing OS/2 operating system. Both companies expected OS/2 1.1, released on 1988, to be the first popular operating system having a GUI, Presentation Manager. Even though it didn't become too popular during its first years, Microsoft didn't complain: Windows 2 didn't sell any better.
But on May 1990, Microsoft released Windows 3.0. Millions of copies of it were sold during its first year, and Microsoft began to believe that OS/2 had failed due to decisions of IBM. At autumn 1990, Microsoft stopped cooperating with IBM, recasted OS/2 3.0 as Windows NT and continued developing it alone, leaving IBM alone with OS/2.
Windows NT was targeted for network file servers, and there were already competition, most importantly Novell NetWare and OS/2. Among other things, the filesystem of Windows NT had to be fast, space efficient and reliable.
NTFS (New Technology File System) was introcuded with Windows NT 3.1. Newer versions of NTFS have been introduced with newer versions of Windows NT, and the filesystem is most likely still under development. All versions of Windows NT support NTFS, but support in Linux kernel was implemented as late as on December 2003.
NTFS is still, in my opinion, the most feature-filled filesystem around. It supports file permissions, both hard and symbolic links, encryption, compression, alternative data streams, journaling... There are very few features NTFS doesn't support.
Depending on cluster size, a NTFS partition can be up to 8 ZB-1 YB in size. (A zettabyte (ZB) is a milliard terabytes and a yottabyte (YB) a billion terabytes.) File size limit is 16 EB.
Windows 7 can only be installed on a NTFS partition, and Vista requires a work-around if one wants to install it on a FAT32 partition. Of course NTFS partitions can be used for data storage as well: due to features of NTFS, I recommend doing so on mechanical hard drives on Windows-only computers.
exFAT [^]
NTFS is a great filesystem, but due to its complexity and journaling, it's not suitable for Flash-based drives. Even Microsoft itself has recommended using FAT32 on removable Flash media.
However, FAT32 only allows files up to four gigabytes in size. The limit is already becoming too small, for example a DVD disc image can exceed that limit. In addition, FAT32 lacks file permission support. In order to get rid of these limitations, Microsoft took FAT from its grave and updated it one more time.
ExFAT (extended FAT), also known as FAT64, was introduced with Windows CE 6.0, on November 2006. Windows Vista SP1, Windows 7 and newer support exFAT too, and by installing this update Windows XP can be extended to support exFAT as well. GNU/Linux drivers are available too, but currently none of them are both stable and free. The best option seems to be exfat, an open-source driver in beta stage.
The partition and file size limits of exFAT are the same: 64 zettabytes. Another important improvement is file permission support that, oddly, is lacking in Windows Vista. In addition, a directory within an exFAT partition can contain up to 2 796 202 files (instead of 65 536, as in FAT32) and timestamps have become more accurate.
No operating system can be installed to an exFAT partition, so such partitions can only be used for data storage. Due to lack of journaling and support for huge files, exFAT is a good filesystem on Solid State Drives and thumb drives that are only used within Windows Vista and/or 7.
Partition list [^]
The following table presents known partition types along with their IDs:
0 Empty 80 Old Minix
1 FAT12 81 Minix / old Linux
2 XENIX root 82 Linux swap / Solaris
3 XENIX usr 83 Linux
4 FAT16 <32M 84 OS/2 hidden C: drive
5 Extended 85 Linux extended
6 FAT16 86 NTFS volume set
7 HPFS/NTFS 87 NTFS volume set
8 AIX 88 Linux plaintext
9 AIX bootable 8e Linux LVM
a OS/2 Boot Manager 93 Amoeba
b W95 FAT32 94 Amoeba BBT
c W95 FAT32 (LBA) 9f BSD/OS
e W95 FAT16 (LBA) a0 IBM Thinkpad hibernation
f W95 Ext'd (LBA) a5 FreeBSD
10 OPUS a6 OpenBSD
11 Hidden FAT12 a7 NeXTSTEP
12 Compaq diagnostics a8 Darwin UFS
14 Hidden FAT16 <32M a9 NetBSD
16 Hidden FAT16 ab Darwin boot
17 Hidden HPFS/NTFS b7 BSDI fs
18 AST SmartSleep b8 BSDI swap
1b Hidden W95 FAT32 bb Boot Wizard hidden
1c Hidden W95 FAT32 (LBA) be Solaris boot
1e Hidden W95 FAT16 (LBA) bf Solaris
24 NEC DOS c1 DRDOS/sec (FAT-12)
39 Plan 9 c4 DRDOS/sec (FAT-16 < 32M)
3c PartitionMagic recovery c6 DRDOS/sec (FAT-16)
40 Venix 80286 c7 Syrinx
41 PPC PReP Boot da Non-FS data
42 SFS db CP/M / CTOS / ...
4d QNX4.x de Dell Utility
4e QNX4.x 2nd part df BootIt
4f QNX4.x 3rd part e1 DOS access
50 OnTrack DM e3 DOS R/O
51 OnTrack DM6 Aux1 e4 SpeedStor
52 CP/M eb BeOS fs
53 OnTrack DM6 Aux3 ee EFI GPT
54 OnTrackDM6 ef EFI (FAT-12/16/32)
55 EZ-Drive f0 Linux/PA-RISC boot
56 Golden Bow f1 SpeedStor
5c Priam Edisk f4 SpeedStor
61 SpeedStor f2 DOS secondary
63 GNU HURD or SysV fd Linux raid autodetect
64 Novell Netware 286 fe LANstep
65 Novell Netware 386 ff BBT
70 DiskSecure Multi-Boot
75 PC/IX
The partitions you are most likely to see in use, are:
- HPFS/NTFS (ID = 7)
This is the Windows XP partition, also known as NTFS - W95 FAT32 (LBA) (ID = c)
This is the Windows 95 - 98 partition
It is used in any kind of disk and large USB devices (1 GB and more) - W95 Ext'd (LBA) (ID = f)
Extended partition. It acts as a container for other partitions - Extended (ID = 5)
Another extended partition type. It acts as a container for other partitions
There is one more extended partition type (ID = 85), but Windows doesn't recognise it - Linux swap / Solaris (ID = 82)
Swap partition, acting as Virtual Memory
Modern computers with 1 - 2 GB of memory may not use it at all - Linux (ID = 83)
Linux partitions, such as ext2, ext3 and reiserfs
Partitioning example [^]
This section contains a partitioning example. I simulate the following situation in a virtual machine:
I have two partitions in my disk: /dev/sda1 that contains a GNU/Linux distribution, and /dev/sda2 that is a swap partition. Here we can see the output of fdisk:
root@sysresccd /root % fdisk -l
Disk /dev/sda: 2097 MB, 2097152000 bytes
64 heads, 63 sectors/track, 1015 cylinders
Units = cylinders of 4032 * 512 = 2064384 bytes
Disk identifier: 0x00058a4a
Device Boot Start End Blocks Id System
/dev/sda1 * 1 812 1636960+ 83 Linux
/dev/sda2 813 1015 409248 82 Linux swap / Solaris
Now I'm going to install another distribution on the same disk. First of all, I need one more partition, because only one distro can be installed on one partition. In addition, I want to separate /home to its own partition in order to be able to share it between distributions.
Because the whole disk is already allocated, I must shrink at least one existing partition in order to create new partitions. I'll shrink both of them to half (/dev/sda1 from 1,6 gigabytes to 800 megabytes, and /dev/sda2 from 400 MB to 200 MB). In addition, I'll move /dev/sda2 right next to /dev/sda1 to keep the partitions in order.
But how many partitions there will be in total? One, two, three... four! Phew, I was near to paint myself into a corner. If I created only primary partitions, I'd be unable to create any more partitions on the disk. Thus, I'll create an extended partition instead and two logical partitions within it. Then I'll be able to create more logical partitions later if required.
There is one more challenge: moving /home to a separate partition. It's very easy to move the folder itself, but the distro in /dev/sda1 will surely be confused if it doesn't find /home when it boots next time. Thus, I must edit its /etc/fstab and configure it to mount the /home partition automatically - before booting the distro itself.
Now there are only two decisions left: the numbers and sizes of the new partitions. I decide to install the new distro to /dev/sda5 and move /home to /dev/sda6. Let /dev/sda5 be 800 megabytes and /dev/sda6 200 MB in size.
Now it's time to boot into SystemRescueCD. Graphical mode is required this time.
I close the terminal and open GParted by clicking the third icon in the bottom pane.
I right-click the partition /dev/sda1 and select Resize/Move.
I enter 799 MB as the new size, click the Free Space Following (MiB) combo box and press Resize/Move.
I right-click now /dev/sda2 and select Resize/Move.
I enter 0 MB as preceding free space and 200 MB as partition size, click the Free Space Following (MiB) combo box and press Resize/Move.
I right-click the unallocated area and select New.
I select Extended Partition as the partition type. The size was already 1000 megabytes (the maximum) and as said, an extended partition doesn't contain any filesystem. I click Add.
I right-click the unallocated area within the extended partition and select New.
I choose the ext4 filesystem and enter 799 MB as the partition size. After that, I click first the Free Space Following (MiB) combo box and then Add.
I right-click the remaining unallocated space and select New one more time.
I choose the ext4 filesystem again. The partition size setting was already 200 megabytes (the whole available space), so I just press Add.
|
The next step is to commit the changes. After that some operations, for example partition deletion, can no longer be undone. |
Finally I commit the changes by pressing the rightmost icon in the main bar.
After slowly reading the warning, I confirm my decisions by pressing Apply.
GParted begins to commit the changes...
...and when everything is done, it shows me this window that I close.
Then I can see the brand new partitions.
Moving /home [^]
I close GParted and launch Terminal by pressing the second icon in the bottom pane.
I create directories as mount points:
mkdir /mnt/sda1
mkdir /mnt/sda6
Then I mount the partitions:
mount /dev/sda1 /mnt/sda1
mount /dev/sda6 /mnt/sda6
I copy the directory to the new partition:
cd /mnt/sda1
rsync -aAPSX home/ /mnt/sda6
I move the original directory out of my way and create a new directory in place of it:
mv home home-old
mkdir home
After that, I unmount /dev/sda6, because it no longer needs to be mounted:
umount /dev/sda6
Now I close Terminal and launch Geany by pressing the fourth icon in the bottom pane.
I select File -> Open.
I press File System and navigate to folder /mnt/sda1/etc.
I double-click the file fstab.
I add the following line:
/dev/sda6 /home ext4 defaults 0 2
Finally, I select File -> Save.
It's a good idea to reboot the computer now and check if the distribution in /dev/sda1 still works. If yes, the /home-old directory can be removed and the disk is ready for the new distro.