Designed for educational purposes, the original Linux file system was limited to 64 MB in size and
supported file names up to 14 characters.
In 1992, the ext file system was created, and increased the file system size to 2 GB and file name
length to 255 characters. However, file access, modification, and creation times were missing from file
system data structures and performance tended to be low.
Ext2 filesystem
Modeled after the Berkeley Fast File System, the ext2 file system used a better on disk layout, extended
the file system size limit to 4 TB and file name sizes to 255 bytes, delivered improved performance,
and emerged as the de facto standard file system for Linux environments. More information on the logging
capabilities of the ext3 file system can be found in EXT3, Journaling File System by
Dr. Stephen Tweedie located at
http://olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html
Ext2 filesystem is supported on Windows using special drive
Ext2 IFS For Windows. That can help to avoid
4G limitation of FAT32 if you need to exchange files larger then 4G, for example DVD ISO images. Features
that are supported supported?
Complete reading and writing access to files and directories of volumes with the
Ext2 or Ext3 file system.
Supports features which are specific to the I/O-system of Windows: Byte Range Locks, Directory
Notfication (so the Explorer updates the view of a directory on changes within that directory), Oplocks
(so SMB clients are able to cache the content of files).
Allows Windows to run with paging files on Ext2 volumes.
UTF-8 encoded file names are supported.
The driver treats files with file names that start with a dot "." character as hidden.
Supports GPT disks if the Windows version used also does.
An evolution of the ext2 file system, the ext3 file system added logging capabilities to facilitate
fast reboots following system crashes. Key features of the ext3 file system include:
Forward and backward compatibility with the ext2 file system. An ext3 file system can
be remounted as an ext2 file system and vice versa. Such compatibility played a role in the adoption
of the ext3 file system.
Checkpointing. The ext3 file system provides checkpointing capabilities, the logging of
batches of individual transactions into compound transactions in memory prior to committing them
to disk to improve performance. While checkpointing is in progress, a new compound transaction is
started. While one compound transaction is being written to disk, another is accumulating. Furthermore,
users can specify an alternative log file location which can enhance performance via increased disk
bandwidth.
Volume management. The ext3 file system relies on the LVM2 package to perform volume management
tasks.
Logging in the ext3 File System The ext3 file system supports different levels of journaling which
can be specified as mount options. These options can impact data integrity and performance.
data=journal Originally, the ext3 file system was designed
to perform full data and metadata journalling. In this mode, the file system journals all changes
to the file system, whether the changes affect data or metadata. Consequently, data and metadata
can be brought back to a consistent state. Full data journalling can be slow. However, performance
penalties can be mitigated by setting up a relatively large journal.
data=ordered The ext3 file system includes an operational
mode that provides some of the benefits of full journalling without introducing severe performance
penalties. In this mode, only metadata is journalled. As a result, the ext3 file system can provide
overall filesystem consistency, even though only metadata changes are recorded in the journal. It
is possible for file data being written at the time of a system failure to be corrupted in this mode.
Note that ordered mode is the default in the ext3 file system.
data=writeback Not recommended, it is better to use
data=journal While the writeback option provides lower
data consistency guarantees than the journal or ordered modes, some applications
show very significant speed improvement when it is used (it can be used with noatime option) For example, speed improvements can be seen when heavy synchronous writes are performed,
or when applications create and delete large volumes of small files, such as delivering a large flow
of short email messages. When the writeback option is used, data consistency
is similar to that provided by the ext2 file system. File system integrity is maintained
continuously during normal operation in the ext3 file system. However, in the event of a power failure
or system crash, the file system may not be recoverable if a significant
portion of data was held only in system memory and not on permanent storage. In this
case, the filesystem must be recreated from backups. Often, changes made since the file system was
last backed up are inevitably lost.
Here is ext3.txt document that comes with kernel:
Ext3 Filesystem
Ext3 was originally released in September 1999. Written by Stephen Tweedie for the 2.2 branch,
and ported to 2.4 kernels by Peter Braam, Andreas Dilger, Andrew Morton, Alexander Viro, Ted Ts'o
and Stephen Tweedie.
Ext3 is the ext2 filesystem enhanced with journalling capabilities.
Options
When mounting an ext3 filesystem, the following option are accepted: (*) == default
journal=update Update the ext3 file system's journal to the current format.
journal=inum When a journal already exists, this option is ignored. Otherwise, it specifies the
number of the inode which will represent the ext3 file system's journal file.
journal_dev=devnum When the external journal device's major/minor numbers have changed, this option
allows the user to specify the new journal location. The journal device is identified through
its new major/minor numbers encoded in devnum.
noload Don't load the journal on mounting.
data=journal All data are committed into the journal prior to being written into the main file
system.
data=ordered (*) All data are forced directly out to the main file system prior to its metadata
being committed to the journal.
data=writeback Data ordering is not preserved, data may be written into the main file system after
its metadata has been committed to the journal.
commit=nrsec (*) Ext3 can be told to sync all its data and metadata every 'nrsec' seconds. The
default value is 5 seconds. This means that if you lose your power, you will lose as much as the
latest 5 seconds of work (your filesystem will not be damaged though, thanks to the journaling).
This default value (or any low value) will hurt performance, but it's good for data-safety. Setting
it to 0 will have the same effect as leaving it at the default (5 seconds). Setting it to very large
values will improve performance.
barrier=1 This enables/disables barriers. barrier=0 disables it, barrier=1 enables it.
orlov (*) This enables the new Orlov block allocator. It is enabled by default.
oldalloc This disables the Orlov block allocator and enables the old block allocator. Orlov
should have better performance - we'd like to get some feedback if it's the contrary for you.
user_xattr Enables Extended User Attributes. Additionally, you need to have extended attribute
support enabled in the kernel configuration (CONFIG_EXT3_FS_XATTR). See the attr(5) manual
page and http://acl.bestbits.at/ to learn more
about extended attributes.
nouser_xattr Disables Extended User Attributes.
acl Enables POSIX Access Control Lists support. Additionally, you need to have ACL support enabled
in the kernel configuration (CONFIG_EXT3_FS_POSIX_ACL). See the acl(5) manual page and
http://acl.bestbits.at/ for more information.
noacl This option disables POSIX Access Control List support.
reservation
noreservation
bsddf (*) Make 'df' act like BSD. minixdf Make 'df' act like Minix.
check=none Don't do extra checking of bitmaps on mount. nocheck
debug Extra debugging information is sent to syslog.
errors=remount-ro(*) Remount the filesystem read-only on an error. errors=continue Keep going
on a filesystem error. errors=panic Panic and halt the machine if an error occurs.
grpid Give objects the same group ID as their creator. bsdgroups
nogrpid (*) New objects have the group ID of their creator. sysvgroups
resgid=n The group ID which may use the reserved blocks.
resuid=n The user ID which may use the reserved blocks.
sb=n Use alternate superblock at this location.
quota noquota grpquota usrquota
bh (*) ext3 associates buffer heads to data pages to nobh (a) cache disk block mapping information
(b) link pages into transaction to provide ordering guarantees. "bh" option
forces use of buffer heads. "nobh" option tries to avoid associating buffer heads (supported only
for "writeback" mode).
Specification
Ext3 shares all disk implementation with the ext2 filesystem, and
adds transactions capabilities to ext2. Journaling is done by the Journaling Block Device layer.
Journaling Block Device layer
The Journaling Block Device layer
(JBD) isn't ext3 specific. It was designed to add journaling capabilities to a block device.
The ext3 filesystem code will inform the JBD of modifications it is performing (called a transaction).
The journal supports the transactions start and stop, and in case of a crash, the journal can replay
the transactions to quickly put the partition back into a consistent state.
Handles represent a single atomic update to a filesystem. JBD can handle an external journal
on a block device.
Data Mode
There are 3 different data modes:
* writeback mode In data=writeback mode, ext3 does not journal data at all. This mode provides
a similar level of journaling as that of XFS, JFS, and ReiserFS in its default mode - metadata journaling.
A crash+recovery can cause incorrect data to appear in files which were written shortly before the
crash. This mode will typically provide the best ext3 performance.
* ordered mode In data=ordered mode, ext3 only officially journals metadata, but it logically
groups metadata and data blocks into a single unit called a transaction. When it's time to
write the new metadata out to disk, the associated data blocks are written first. In general,
this mode performs slightly slower than writeback but significantly faster than journal mode.
* journal mode data=journal mode provides full data and metadata journaling. All new data
is written to the journal first, and then to its final location. In the event of a crash, the journal
can be replayed, bringing both data and metadata into a consistent state. This mode is the
slowest except when data needs to be read from and written to disk at the same time where it outperforms
all other modes.
Compatibility
Ext2 partitions can be easily convert to ext3, with `tune2fs -j <dev>`. Ext3 is fully compatible
with Ext2. Ext3 partitions can easily be mounted as Ext2.
External Tools
See manual pages to learn more.
tune2fs: create a ext3 journal on a ext2 partition with the -j flag. mke2fs: create a ext3 partition
with the -j flag. debugfs: ext2 and ext3 file system debugger. ext2online: online (mounted) ext2
and ext3 filesystem resizer
Ext4 uses extents (as opposed to the traditional block mapping scheme used by ext2 and ext3),
which improves performance when using large files and reduces metadata overhead for large files.
In addition, ext4 also labels unallocated block groups and inode table sections accordingly, which
allows them to be skipped during a file system check. This makes for quicker file system checks, which
becomes more beneficial as the file system grows in size.
Allocation Features
The ext4 file system features the following allocation schemes:
Persistent pre-allocation
Delayed allocation
Multi-block allocation
Stripe-aware allocation
Because of delayed allocation and other performance optimizations, ext4's behavior of writing files
to disk is different from ext3.
In ext4, when a program writes to the file system, it is not guaranteed to be on-disk unless
the program issues an fsync() call afterwards.
ext3grep is a simple program for recovering files on an EXT3 filesystem. It is an
investigation and recovery tool that is useful in forensics investigations. It helps to show
information about files that existed on a partition and also recover accidentally deleted
files.
In this article, we will demonstrate a useful trick, that will help you to recover
accidentally deleted files on ext3 filesystems using ext3grep in Debian and
Ubuntu.
Once installed, now we will demonstrate how to recover deleted files on a ext3
filesystem.
First, we will create some files for testing purpose in the mount point
/mnt/TEST_DRIVE of the ext3 partition/device i.e. /dev/sdb1 in this
case.
$ cd /mnt/TEST_DRIVE
$ sudo touch files[1-5]
$ ls -l
Create Files in Mount Point
Now we will remove one file called file5 from the mount point
/mnt/TEST_DRIVE of the ext3 partition.
$ sudo rm file5
Remove a File in Linux
Now we will see how to recover deleted file using ext3grep program on the targeted
partition. First, we need to unmount it from the mount point above (note that you have to use
cd command to switch
to another directory for the unmount operation to work, otherwise the umount command will show
the error " that target is busy ").
$ cd
$sudo umount /mnt/TEST_DRIVE
Now that we have deleted one of the files (which we'll assume was done accidentally), to
view all the files that existed in the device, run the --dump-name option (replace
/dev/sdb1 with the actual device name).
$ ext3grep --dump-name /dev/sdb1
View Files on Partition
To recover the above deleted file i.e. file5 , we use the
--restore-all option as shown.
$ ext3grep --restore-all /dev/sdb1
Once the recovery process is complete, all recovered files will be written to the directory
RESTORED_FILES , you can check if the deleted file is recovered or not.
$ cd RESTORED_FILES
$ ls
Recover a Deleted File
We may specify a particular file to recover, for example the file called file5
(or specify the full path of the file within the ext3 device).
That's it! ext3grep is a simple and useful tool to investigate and recover deleted files on
an ext3 filesystem. It is one of the the best programs to recover files on Linux. If you have
any questions or any thoughts to share, reach us via the feedback form below.
This article will provide a brief introduction to Linux File System Ext4, the successor to
Linux File System Ext3. Several tips regarding usage as well as additional links will be provided.
The ext3 source was forked and developed independently, in order to correct several existing limitations
of the previous Ext2 and Ext3 file systems in the future. Ext4 has been accepted into the Linux kernel
as of Kernel version 2.6.19 and finally declared stable as of Kernel version 2.6.28.
Current distribution files, like RedHat Enterprise Linux 6 (RHEL), Debian 6.0 (Squeeze) or Ubuntu
10.10 (Maverick Meerkat) provide stable Ext4 support and use it as the default file system, in some
cases.
Advantages
Improved performance through:
Multi-block allocation
Extent-based block mapping
Delayed allocation
Stripe-aware allocation
Improved file security through:
Write barriers
Time stamps use the nanosecond range instead of the second range
Increased e2fsck speed
Unlimited number of sub-directories (32,000 sub-directory under a directory under Ext3)
Journalized quota data, whereby a quota check will not be performed after a system crash
You can find additional details regarding these advantages in a white paper from Red Hat
[1].
Compatibility with Ext3
Because there have been many changes in comparison with Ext3, migration to Ext4 is not as easy
as from Ext2 to Ext3.
To take full advantage of the Ext4 file system, Red Hat recommends backing all of the data up
for RHEL 6, re-creating the Ext4 file system and copying the data into the new Ext4 file system (see
also
[1]).
The Ext4 driver does support mounting an Ext3 file system, however only with limited functionality.
On the other hand, as soon as one uses extent-based mapping, mounting an Ext4 file system as an Ext3
file system becomes impossible.
SSD Optimizations
ATA Trim
Ext4 supports ATA Trim
for solid state drives (SSDs):
The -o discard mount option (for example mount -o discard /dev/sdb1
/mnt/. For permanent activation, the option must be entered in /etc/fstab,
because the discard capability is deactivated by default)[2]
Pre-discard during formatting from mke2fs 1.41.10[3][4][5]
Extract from the man page for mke2fs: -E discard: Attempt to discard blocks at mkfs
time (discarding blocks initially is useful on solid state devices and sparse / thin-provisioned
storage). When the device advertises that discard also zeroes data (any subsequent read
after the discard and before write returns zero), then mark all not-yet-zeroed inode tables
as zeroed. This significantly speeds up file system initialization. This is set as default.
In our test, we observed significantly different time requirements for the discard operation
during formatting. However, significant effects were not caused by discard also zeroes data
(regarding this, see
ATA Trim Performance).
With discard also zeroes data, hdparm -I displayed Deterministic
read ZEROs after TRIM (or deterministic read data after TRIM). The following example shows
an OCZ Vertex 3 SSD and an
Intel
320 Series SSD:
[root@fedora15 ~]# hdparm -V
hdparm v9.36
[root@fedora15 ~]# hdparm -I /dev/sda | grep 'Model\|TRIM'
Model Number: OCZ-VERTEX3
* Data Set Management TRIM supported (limit 1 block)
* Deterministic read data after TRIM
[root@fedora15 ~]# hdparm -I /dev/sdb | grep 'Model\|TRIM'
Model Number: INTEL SSDSA2CW160G3
* Data Set Management TRIM supported (limit 8 blocks)
* Deterministic read ZEROs after TRIM
[root@fedora15 ~]#
File System Journal: Yes or No?
Not using the Ext4 file system journal[6][7]
can increase file system performance, is however connected with disadvantages when the shutdown procedure
is not completely clean (such as during a power failure). Theodore Tso, an Ext4 file system developer,
determined in tests that the performance disadvantages caused by journaling were between four and
twelve percent. Therefore, the journal should be used. Not using the atime journaling would be a
more recommendable way of increasing performance.[8][9]
noatime
Using the noatime mount option improves performance when reading.[8][10][11]
Should stride and stripe-width Parameters be used?
There are a number of different recommendations for the stride and stripe-width parameters[12][13]
for using SSDs under Ext4.[14][15]
If certain values really do provide a benefit cannot be determined with certainty at this time.
Individualized tests with the respective SSD would be required for making a determination.[16][17]
Lazy Initialization
When creating an Ext4 file system, the existing regions of the inode tables must be cleaned (overwritten
with nulls, or "zeroed"). The "lazyinit" feature should significantly accelerate the creation of
a file system, because it does not immediately initialize all inode tables, initializing them gradually
instead during the initial mounting process in background (from Kernel version
2.6.37).[18][19]
Regarding this see the extracts from the mkfs.ext4 man pages:[20]
If enabled and the uninit_bg feature is enabled, the inode table will not be fully initialized
by mke2fs. This speeds up file system initialization noticeably, but it requires the kernel to
finish initializing the file system in the background when the file system is first mounted. If
the option value is omitted, it defaults to 1 to enable lazy inode table zeroing.
One should be careful when testing the performance of a freshly created file system. The "lazy
initialization" feature may write a lot of information to the hard disk after the initial mounting
and thereby invalidate the test results. At first, the "ext4lazyinit" kernel process writes at up
to 16,000kB/s to the device and thereby uses a great deal of the hard disk's bandwidth (see also
I/O Statistics by Process). In order to prevent lazy initialization, advanced options are offered
by the mkfs.ext4 command:[20]
By specifying these options, the inodes and the journal will be initialized immediately during
creation.
Potential Problems
Poor Performance due to Write Barriers
The integrity of the file system can be guaranteed by write barriers, if the hard disk uses a
volatile cache and the contents of the cache are lost due to a power failure. Thereby, data security
is improved, however at the price of a small performance disadvantage. By default, write barriers
are activated, however they can be deactivated with the nobarrier file system option. However,
this is only recommended when the write cache for your RAID array has been secured using a battery
backup unit, zero maintenance cache or similar measure.
Data Loss with Applications that do not use fsync() correctly
When Ext4 was first introduced with Linux distributions, notifications about sometimes massive
data losses began to appear.[21]
The reason for this is delayed allocation, which first allocated the necessary storage space up
to 60 seconds later. Thereby, file renaming entered the scenario, for example, so that the metadata
properly represented the rename process, but the actual data had not yet been written. The file name
pointed to a 0 byte file thereby. However, this only occurs when an application has not properly
used the fsync() feature. According to developer Theodore Ts'o, Ext4 precisely implements
the POSIX standard for file operations. The problem is that the "secure" behavior of Ext3 was undesirable,
however was considered a given by many application developers due to its wide distribution.
Initially the "alloc_on_commit" mode was introduced as a workaround, which was replaced by the
"auto_da_alloc" mode shortly thereafter in Kernel 2.6.30 (see Kernel Commit
[22]).
Thereby, attempts will be made to detect and avoid frequently occurring cases for potential data
loss. This mode became the new default mode, but can be deactivated by "noauto_da_alloc".
In general, applications should improve their compliance with the POSIX standard in the future
and use fsync() in the required positions. Ext4 has been subsequently optimized for this problem
based on the prior history of Ext3. Other file systems using delayed allocation (such as XFS or Btrfs)
do not take this into consideration.
Jump
up ↑e2fsprogs
Release Notes e2fsprogs 1.41.10 (February 10, 2010): Mke2fs will use BLKDISCARD to pre-discard
all blocks on an SSD or thinly-provisioned storage device.
Much ado about an ultimately irrelevant issue: a bug in the ext4 filesystem has turned
out only to be exposed when several exotic options are combined. Apparently, the problem has only
affected a single user.
Ext4 is the current standard filesystem for Linux, and is considered robust, mature and well tested
(see also
"The Ext4 Linux filesystem" from The H Open). Robust, because the journal that
was introduced with its predecessor, Ext3, guarantees the filesystem's integrity even if there is
a power cut during a write operation; mature because Ext4 is the result of many years of development
that began with Ext2 almost 20 years ago; and well tested because since then, the vast majority of
Linux systems have been installed on Ext2, Ext3 or Ext4 mass deployment and practical use are still
the best way of testing.
Lazy unmount
Normally, a filesystem that's mounted under Linux can't be unmounted while a process is accessing
a file within the filesystem, even if it's only a shell whose working directory is located in the
filesystem. In this case, umount will report that the "device is busy". The usual solution
is to use fuser or lsof to find the process that is blocking the unmount operation and then terminate
that process.
However, Linux processes can get stuck in such a way that they can no longer be terminated; or
the unmount operation might be blocked by some other bug (such as an unresponsive NFS server) that
can't easily be fixed. In such cases, a "lazy unmount" can be requested using the -l
mount option: the filesystem will be unmounted immediately, and the system will attempt to fix the
resulting chaos (file handles without files, etc.) at a later stage. That mount -l is
an option for special circumstances and may cause data loss should be quite obvious, really.
Has all of that now gone down the drain because of the recent Ext4 bug? Of course not quite
the opposite in fact.
Any program code as complex as the code that's required for a modern, powerful filesystem will
contain bugs the Ext4 code consists of approximately 40,000 lines of source code. That a combination
of several mount options (nobarrier, journal_checksum and journal_async_commit)
none of which are used by default as well as an added "lazy unmount" (see box) are required to
trigger an error is an argument for, not against, the robustness of Ext4 and the quality of its code
testing. That the Ext4 developers immediately began to investigate the bug even in a situation such
as this is further testament to Ext4's maturity otherwise, the developers would have had more important
things to do than check out some esoteric bug that only seems to have affected a single user. Even
Ext4 lead developer Theodore Ts'o has so far been unable to reproduce the bug on his systems.
We can, therefore, confidently assume that our data is safe on Ext4 or at least that it is safer
than it would be on other Linux filesystems: the development of
ReiserFS/Reiser4 has fizzled
out, the
Btrfs "Next Generation Filesystem" hasn't really become suitable for production use yet, and
XFS, which was originally developed
for Irix, has become stuck on the sidelines. Filesystem bugs also exist in
other operating systems but they aren't such
public discussion topics.
It is commendable that, despite this, the Ext4 developers want to
draw useful conclusions from their handling of the Ext4 bug, although they could have done this
much earlier: a year ago, Ted Ts'o had already
warned at LinuxCon Europe in Prague that some mount options may cause problems with Ext4 because
they haven't been extensively tested. After all, these options some of which were only ever intended
for developers will now disappear from the production code, or they will at least trigger a warning.
Also a kind of bug fix ...
UPDATE: See update below; it now looks like this was caused by a very esoteric case indeed:
I suppose I should be glad the potential ext4 corruption that two users have reported potentially
impacting v3.6.2 and v3.6.3 has promoted some minor increase in revenue (due to advertising hits)
for Slashdot and Phoronix, but really, people should chill out.
First of all, it appears that my initial analysis was wrong; it appears that it's probably related
to a unclean shutdown, and probably some other required exacerbating factor, since I haven't been
able to trivially reproduce it. Secondly, Fedora 17 is currently using 3.6.2. So if this was an easily
triggered bug, (a) lots of people would have been complaining, since the results of this bug is not
subtle, and (b) Eric and I probably would have found a reliable repro for the problem by now.
extcarve is a ext2/ext3/ext4 file recovery and semantic file carving tool. It can recover
a range of file formats, including PNG, JPG, GIF, PDF, C/C++ programs, PHP, and HTML.
hi about numa 0.2 Hi all,
numa 0.2 recovers gif as well as jpeg (of files less than 48kb)
So you've been futzing round in the file system, and been over vigorous with the rm command and
deleted a crucial file that you or (more scarily) a significant other, can't live without. What to
do? After that initial hot flush has died down, you must be calm, and work fast. You have three options:
if the file is still open in a running application
To be honest, I can't imagine this happening a lot, but it's possible. The scenario is this:
you're editing/using a file in an application, and whilst the application is still open, you delete
the file. If this happens to you, then recovery of the file is pretty simple using a tool that
should already be installed on your system. So:
step1:
Do not close any applications!
step2:
Open a terminal, and type:
lsof | grep "/path/to/file"
If the file is being used by a running application, then you should get something like this:
progname 5559 user 22r REG 8,5 1282410 1294349 /path/to/file
If you get no output, then the file isn't being used by a running application, and you'll need
to use one of the other methods.
step3:
Looking at the output from step2, you'll need the numbers from the 2nd and
4th column (ie. 5559 and 22r). With the second number, drop the "r" (leaving "22″). Now type the
following command:
cp /proc/5559/fd/22 /path/to/restored.file
And voila, your file should be back in the form of /path/to/restored.file.
Easy.
using extundelete
Extundelete is a very useful little program for restoring deleted files. It should be able
to restore files from ext2, ext3 and ext4 partitions. Restoring ext3 and ext4 is a very difficult
process, because the journalling part of the file system deletes the information that points to
where a file's data is on the disk when it's deleted, unlike the ext2 system, which leaves it
in place. But fortunately for us, very often, that pointer information is still in the file system's
journal, and therefore this information can be used to find the data on the disk and restore it.
So, with that said, on with the show.
First you'll need to install extundelete. If your distro doesn't have a pre-built package,
then you'll have to compile it from source. You can get the source from
here. If you need help compiling
a tarball, check out this
page.
step1:
As soon as possible, you need to mount the relevant partition as read-only. The sooner you
do this the better, so that there is no risk of another program writing data over the top of your
recently deleted file. To do this, open a terminal, and type:
mount -o remount,ro /dev/partition
... ... ...
Once you've mounted it read-only, you can relax a bit. At least you won't be risking making
the situation worse.
step2:
Now you can use the extundelete command to restore your file. extundelete can be used to restore
all deleted files from a partition, or a specific file or directory. In this example we'll just
be restoring a specific file, but for more options, look
here. To restore
the deleted file, type the following into a terminal:
extundelete --restore-file /path/to/deleted/file
step3:
extundelete (if it works) should restore the file to a subdirectory in the current directory
called RECOVERED_FILES. Voila problem solved.
using photorec
If extundelete doesn't work, then you can try photorec. Photorec works in a different way to
extundelete. Instead of trying to find the information that points to where the deleted file data
is on the disk, it tries to find the data by parsing the data itself to identify files. This method
is less targeted than extundelete, but may still work if the information pointing to the file
has been deleted from the journal. Photorec can find deleted files of a particular type. So if
you've deleted a .mp3 file, you would get Photorec to find all deleted .mp3 files and hope that
it finds the file you're looking for. Photorec seems to be well supported on the main distros,
so you should be able to find a package for easy install. So:
Rather than list the process in detail here, there is a very good explanation of the steps
on the photorec
wiki.
That's it. Happy undeleting and remember prevention is always better than the cure.
Backup.
The fourth extended file system, or ext4, is the next generation of journaling file systems, retaining
backward compatibility with the previous file system, ext3. Although ext4 is not currently the standard,
it will be the next default file system for most Linux distributions. Get to know ext4, and discover
why it will be your new favorite file system.
This is the bugzilla bug I started on the fedora buzilla: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=142737
Very good idea to do something like the following, so that you have an copy of the partition you're
trying to recover, in case something bad happens: dd if=/dev/hda2 bs=1024k conv=noerror,sync,notrunc
| reblock -t 65536 30 | ssh remote.host.uci.edu 'cat > /recovery/damaged-lvm2-ext3'
e2salvage died with "Terminated". I assume it OOM'd.
e2extract gave a huge list of 0 length files. Doesn't seem right, and it was taking forever, so
I decided to move on to other methods. But does anyone know if this is normal behavior for e2extract
on an ext3?
I wrote a small program that searches for ext3 magic numbers. It's finding many, EG 438, 30438,
63e438 and so on (hex). The question is, how do I convert from that to an fsck -b number?
Running the same program on a known-good ext3, the first offset was the same, but others were
different. However, they all ended in hex 38...
I'm now running an "fsck -vn -b" with the -b argument ranging from 0 to 999999. I'm hoping this
will locate a suitable -b for me via brute force.
Sent a post to gmane.linux.kernel 2004-12-16
Robin Green
very helpfully provided the following instructions, which appear to be
getting somewhere:
1) Note down what the root= device is that appears on the kernel command line (this can be found
by going to boot from hard drive and then examining the kernel command line in grub, or by looking
in /boot/grub/grub.conf )
2) Be booted from rescue disk
3) Sanity check: ensure that the nodes /dev/hda, /dev/hda2 etc. exist
4) Start up LVM2 (assuming it is not already started by the rescue disk!) by typing:
lvm vgchange --ignorelockingfailure -P -a y
Looking at my initrd script, it doesn't seem necessary to run any other commands to get LVM2 volumes
activated - that's it.
5) Find out which major/minor number the root device is. This is the slightly tricky bit. You
may have to use trial-and-error. In my case, I guessed right first time: (no comments about my odd
hardware setup please ;)
In the first command, I listed the block devices known to the kernel. dm-* are the LVM devices (on
my 2.6.9 kernel, anyway). In the second command, I found out the major:minor numbers of /dev/dm-0.
In the third command, I used devmap_name to check that the device mapper name of node with major
253 and minor 0, is the same as the name of the root device from my kernel command line (cf. step
1). Apart from a slight punctuation difference, it is the same, therefore I have found the root device.
I'm not sure if FC3 includes the devmap_name command. According to fr2.rpmfind.net, it doesn't.
But you don't really need it, you can just try all the LVM devices in turn until you find your root
device. Or, I can email you a statically-linked binary of it if you want.
6) Create the /dev node for the root filesystem if it doesn't already exist, e.g.:
mknod /dev/dm-0 b 253 0
using the major-minor numbers found in step 5.
Please note that for the purpose of _rescue_, the node doesn't actually have to be under /dev
(so /dev doesn't have to be writeable) and its name does not matter. It just needs to exist somewhere
on a filesystem, and you have to refer to it in the next command.
7) Do what you want to the root filesystem, e.g.:
fsck /dev/dm-0
mount /dev/dm-0 /where/ever
As you probably know, the fsck might actually work, because a fsck can sometimes correct filesystem
errors that the kernel filesystem modules cannot.
8) If the fsck doesn't work, look in the output of fsck and in dmesg for signs of physical drive
errors. If you find them, (a) think about calling a data recovery specialist, (b) do NOT use the
drive!
On FC3's rescue disk, what I actually did was:
1) Do startup network interfaces
2) Don't try to automatically mount the filesystems - not even readonly
3) lvm vgchange --ignorelockingfailure -P -a y
4) fdisk -l, and guess which partition is which based on size: the small one was /boot, and the
large one was /
5) mkdir /mnt/boot
6) mount /dev/hda1 /mnt/boot
7) Look up the device node for the root filesystem in /mnt/boot/grub/grub.conf
8) A first tentative step, to see if things are working: fsck -n /dev/VolGroup00/LogVol00
9) Dive in: fsck -f -y /dev/VolGroup00/LogVol00
10) Wait a while... Be patient. Don't interrupt it
I think you are making a potentially very dangerous mistake!
Type 8e is a partition type. You don't want to use resize2fs on the PARTITION, which is not an
ext2 partition, but an lvm partition. You want to resize the filesystem on the logical VOLUME.
And yes, resize2fs is appropriate for logical volumes. But resize the VOLUME (e.g. /dev/VolGroup00/LogVol00),
not the partition or volume group.
On Fri, Mar 04, 2005 at 06:35:31PM +0000, Robert Buick wrote: > I'm using type 8e, does anyone
happen to know if resize2fs is > appropriate for this type; the man page only mentions type2.
A method of hunting for two text strings in a raw disk, after files have been deleted. The data
blocks of the disk are read once, but grep'd twice.
stdin seems seekable, but file length is 0 - no exact percentages Estimated filetransfer size is
77021200384 bytes Estimated percentages will only be as accurate as your size estimate Creating 2
pipes
Performing the above reblock | mtee, my fedora core 3 system got -very- slow. If I were to suspend
the pipeline above, performance would be great. If I resumed it, very quickly, performance would
be bad again. This command seems to have left my sytem a little bit jerky, but it's -far- more usable
now, despite the pipeline above still pounding the SATA drive my home directory is on.
seki-root> echo deadline > scheduler
Wed Mar 09 17:56:58
seki-root> cat scheduler
noop anticipatory [deadline] cfq
Wed Mar 09 17:57:00
seki-root> pwd
/sys/block/sdb/queue
Wed Mar 09 17:58:31
BTW, I looked into tagged command queuing for this system as well, but apparently VIA SATA doesn't
support TCQ on linux 2.6.x.
Eventually the reblock | mtee egrep egrep gave: egrep: memory exhausted
...using GNU egrep 2.5.1.
...so now I'm trying something closer to my classical method:
seki-root> reblock -e 75216016 $(expr 1024 \* 1024) 300 <
/dev/mapper/VolGroup00-LogVol00 | mtee './bgrep dptutil | ./ranges >
dptutil-ranges' './bgrep dptmgr | ./ranges > dptmgr-ranges'
Creating 2 pipes
popening ./bgrep dptutil | ./ranges > dptutil-ranges
popening ./bgrep dptmgr | ./ranges > dptmgr-ranges
stdin seems seekable, but file length is 0 - no exact percentages
Estimated filetransfer size is 77021200384 bytes
Estimated percentages will only be as accurate as your size estimate
(estimate: 1.3% 16s 12m 1h) Kbytes: 1027072.0 Mbits/s: 133.6 Gbytes/hr:
58.7 min: 1.0
(estimate: 2.5% 36s 16m 1h) Kbytes: 1913856.0 Mbits/s: 124.5 Gbytes/hr:
54.7 min: 2.0
(estimate: 3.7% 10s 17m 1h) Kbytes: 2814976.0 Mbits/s: 122.1 Gbytes/hr:
53.6 min: 3.0
(estimate: 4.9% 10s 17m 1h) Kbytes: 3706880.0 Mbits/s: 120.6 Gbytes/hr:
53.0 min: 4.0
...
I've added a -s option to reblock, which makes it sleep for an arbitrary number of (fractions of)
seconds between blocks. Between this and the I/O scheduler change, seki has become very pleasant
to work on again, despite the hunt for my missing palm memo. :)
From Bryan Ragon
Here is a detailed list of steps that worked:
;; first backed up the first 512 bytes of /dev/hdb
# dd if=/dev/hdb of=~/hdb.first512 count=1 bs=512
1+0 records in
1+0 records out
;; zero them out, per Alasdair
# dd if=/dev/zero of=/dev/hdb count=1 bs=512
1+0 records in
1+0 records out
;; verified
# blockdev --rereadpt /dev/hdb
BLKRRPART: Input/output error
;; find the volumes
# vgscan
Reading all physical volumes. This may take a while...
Found volume group "media_vg" using metadata type lvm2
# pvscan
PV /dev/hdb VG media_vg lvm2 [111.79 GB / 0 free]
Total: 1 [111.79 GB] / in use: 1 [111.79 GB] / in no VG: 0 [0 ]
# lvmdiskscan
/dev/hda1 [ 494.16 MB]
/dev/hda2 [ 1.92 GB]
/dev/hda3 [ 18.65 GB]
/dev/hdb [ 111.79 GB] LVM physical volume
/dev/hdd1 [ 71.59 GB]
0 disks
4 partitions
1 LVM physical volume whole disk
0 LVM physical volumes
# vgchange -a y
1 logical volume(s) in volume group "media_vg" now active
;; /media is a defined mount point in fstab, listed below for future archive
searches
# mount /media
# ls /media
graphics lost+found movies music
If your system will give you a terminal
type the following command, else boot Linux system from rescue disk (boot from 1st CD/DVD. At boot:
prompt type command linux rescue).
Mount partition using alternate superblock
Find out superblock location for /dev/sda2: # dumpe2fs /dev/sda2 | grep superblock
Sample output:
Primary superblock at 0, Group descriptors at 1-6
Backup superblock at 32768, Group descriptors at 32769-32774
Backup superblock at 98304, Group descriptors at 98305-98310
Backup superblock at 163840, Group descriptors at 163841-163846
Backup superblock at 229376, Group descriptors at 229377-229382
Backup superblock at 294912, Group descriptors at 294913-294918
Backup superblock at 819200, Group descriptors at 819201-819206
Backup superblock at 884736, Group descriptors at 884737-884742
Backup superblock at 1605632, Group descriptors at 1605633-1605638
Backup superblock at 2654208, Group descriptors at 2654209-2654214
Backup superblock at 4096000, Group descriptors at 4096001-4096006
Backup superblock at 7962624, Group descriptors at 7962625-7962630
Backup superblock at 11239424, Group descriptors at 11239425-11239430
Backup superblock at 20480000, Group descriptors at 20480001-20480006
Backup superblock at 23887872, Group descriptors at 23887873-23887878
Now check and repair a Linux file system using alternate superblock # 32768: # fsck -b 32768
/dev/sda2 Sample output:
fsck 1.40.2 (12-Jul-2007)
e2fsck 1.40.2 (12-Jul-2007)
/dev/sda2 was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #241 (32254, counted=32253).
Fix? yes
Free blocks count wrong for group #362 (32254, counted=32248).
Fix? yes
Free blocks count wrong for group #368 (32254, counted=27774).
Fix? yes
..........
/dev/sda2: ***** FILE SYSTEM WAS MODIFIED *****
/dev/sda2: 59586/30539776 files (0.6% non-contiguous), 3604682/61059048 blocks
Now try to mount file system using mount command: # mount /dev/sda2 /mnt You can
also use superblock stored at 32768 to mount partition, enter: # mount sb={alternative-superblock}
/dev/device /mnt # mount sb=32768 /dev/sda2 /mnt Try to browse and access file system:
# cd /mnt # mkdir test # ls -l # cp file /path/to/safe/location
You should always keep backup of all important data including configuration files.
Modern filesystems make forensic file recovery much more difficult. Tools like Foremost and Scalpel
identify data structures and carve files from a hard disk image.
IT experts and investigators have many reasons for reconstructing deleted files. Whether an intruder
has deleted a log to conceal an attack or a user has destroyed a digital photo collection with an
accidental rm ‑rf, you might someday face the need to recover deleted data. In the past, recovery
experts could easily retrieve a lost file because an earlier generation of filesystems simply deleted
the directory entry. The meta information that described the physical location of the data on the
disk was preserved, and tools like The Coroners Toolkit (TCT [1]) and The Sleuth Kit (TSK [2]) could
uncover the information necessary for restoring the file. Today, many filesystems delete the full
set of meta information, leaving the data blocks. Putting these pieces together correctly is called
file carving forensic experts carve the raw data off the disk and reconstruct the files from it.
The more fragmented the filesystem, the harder this task become.
You can define journaling file systems in many ways, but let's get right to the point.
Journaling file systems are for people who tire of watching the boot-time fsck, or file
system consistency check process. (Journaling file systems are also for anyone who likes the idea
of a fault-resilient file system.) When a system using a traditional, non-journaling file system
is improperly shut down, the operating system detects this and performs a consistency check using
the fsck utility. This utility scans the file system (which can take a considerable
amount of time) and fixes any issues that can be safely corrected. In some cases, the file system
can be in such bad shape that the operating system boots into single user mode to allow the user
to further the repair process.
To add insult to injury, the fsck process can be initiated automatically
by the operating system at mount time to ensure that the file system metadata is correct (even
if no corruption is detected). Therefore, removing the need for file system consistency checks
is an obvious area for improvement.
So, now you know for whom journaling file systems were created, but how do they obviate the need
for fsck? In general, journaling file systems avoid file system corruption by maintaining
a journal. The journal is a special file that logs the changes destined for the file system in a
circular buffer. At periodic intervals, the journal is committed to the file system. If a crash occurs,
the journal can be used as a checkpoint to recover unsaved information and avoid corrupting file
system metadata.
To sum up, journaling file systems are fault-resilient file systems that use a journal to log
changes before they're committed to the file system to avoid metadata corruption (see Figure 1).
But like many Linux solutions, more than one option is available to you. Let's take a short walk
through journaling file system history, and then review the file systems available and how they differ.
... ... ...
Fourth extended file system
The fourth extended journaling file system (ext4fs) is the evolution of ext3fs. The ext4 file
system is designed as a backward- and forward-compliant replacement for ext3fs but with many new
advanced features (some of which break the compatibility). This means that you can mount an ext4fs
partition as ext3fs or vice versa.
First, ext4fs is a 64-bit file system and is designed to support very large volumes (1 exabyte).
It has also been designed to use extents, but if this is used, then compatibility with ext3fs is
lost. Like XFS and Reiser4, ext4fs includes delayed allocation to allocate blocks on the disk only
when needed (which reduces fragmentation). The contents of the journal are also checksummed to make
the journal more reliable. Instead of the standard B+ or B* tree, ext4fs uses a variation of the
B tree, called the H tree, which allows much larger subdirectories (ext3 was limited to 32KB).
Although the delayed allocation method reduces fragmentation, over time, a large file system can
become fragmented. An online defragmentation tool (e4defrag) has been developed to address this.
You can use the tool to defragment individual files or an entire file system.
Another interesting difference between ext3fs and ext4fs is the date resolution for files. In
ext3, the minimum resolution for timestamp was one second. Ext4fs is looking toward the future: Where
processor and interface speeds continue to increase, better resolution is needed. For this reason,
the minimum timestamp resolution in ext4 is 1 nanosecond.
Ext4fs has been in the Linux kernel since 2.6.19 but is yet to be called stable. Development continues
on this next generation; given its heritage, it will be the next generation in Linux journaling file
systems.
Resources
The list
of file systems on Wikipedia ranges from the earliest DEC file systems of the 1960s to the
latest BufferFS from Oracle. To round out your file system knowledge, also check out this
file system reading list,
which covers a wide range of file system topics.
JFS (and its successor, JFS2) were the earliest journaled file systems. They continue to be
used in Linux and the AIX operating systems.
XFS was the earliest journaling
file system that focused on high performance. Learn more about the
development and future of XFS at
the SGI home page.
Tim's "Anatomy
of the Linux file system" (developerWorks, Oct 2007) introduces you to the VFS and its major
structures. The Linux VFS layer provides an abstraction using a common application program interface
(API) to the various supported underlying file systems.
To the first few replies to this article, have you ever had to build a multi-GB/s filesystem that
can handle arbitrary workloads and stay up at least 99.9% of the time? Henry has. I have complaints
about his article, but he brings up good points:
Other filesystems besides ext3 and XFS aren't supported or tested as well necessary for the
types of loads he is describing (and ZFS/Fuse is alpha code). XFS is very good, but the biggest
Linux vendor (Redhat) doesn't even support it.
Yes, you can fix Linux code yourself, but filesystem are hard. You can't expect some random
code jocky to pickup the kernel source and undertsand filesystems. What about the XFS+NFS bug
that existed in Linux around 2004 (
http://www.linux.sgi.com/archives/xfs/2004-06/msg0
0100.html) It took over a year for SGI to fix the problem. Open source worked, because the
original patch came from a guy at Sony in Japan, but for a year there was silent corruption on
any XFS filesystem that was exported via NFS.
He says that an LT04 tape drive can push 240 MB/s. Now put 20 of those drives in your system (4.8
GB/s). Now, design your filesystem so that you have extra capacity so that you can interact with
the filesystem while the tape drives are banging away (4.8x2=9.6GB/s). This is much more than your
home software-raid setup to store your mp3s and pictures.
This is High-performance I/O. This is a very common-setup at many of the HPC centers around the
world (except they may not be using LTO drives, but enterprise drives). Expecting Linux to push data
at that rate is a stretch.
However, it is getting better. I would still rather use Linux than bring in one Solaris server
that I have to hire or retrain staff because we migrated to Linux a decade ago.
On mid-intensity enterprise loads Ext3 is OK. Not sure about databases...
I am frequently asked by potential customers with high I/O requirements if they can use Linux
instead of AIX or Solaris.
No one ever asks me about high-performance I/O - high
IOPS or
high streaming I/O - on Windows or NTFS because it isn't possible. Windows and the NTFS file system,
which hasn't changed much since it was released almost 10 years ago, can't scale given its current
structure. The NTFS file system layout, allocation methodology and structure do not allow it to efficiently
support multi-terabyte file systems, much less file systems in the petabyte range, and that's no
surprise since it's not Microsoft's target market.
And what was Linux's initial target market? A Microsoft desktop replacement, of course. Linux
has since moved from the desktop to run on many large SMP servers from Sun, IBM and SGI. But can
Linux as an operating system and Linux file systems meet the challenge of high-performance I/O?
You may think you don't need high-performance I/O, but every server needs this type of I/O performance
for something as simple as backup and restoration. Current LTO-4 tape drives can operate at 120 MB/sec
without compression and can support data rates up to 240 MB/sec with compression. If your file system
cannot support I/O at these streaming data rates, then the time to backup and restore will take much
longer than expected. For large environments with multiple tape drives, not being able to use the
tape drives at their full data rate might require additional tape drives to meet the backup time
window, which affects restoration too. Therefore, it seems to me that everyone should be interested
in the performance of Linux file systems, if only for backup and restore.
Can Linux file systems, which I will define as ext-4, XFS and xxx, match the performance of file
systems on other UNIX-based large SMP servers such as IBM and Sun? Some might also inquire about
SGI, but SGI has something called ProPack, which has a number of optimizations to Linux for high-speed
I/O, and SGI also has their open proprietary Linux file system called CxFS, which is not part of
standard Linux distributions. Because SGI ProPack and CxFS are not part of standard Linux distributions,
we won't consider them here. We'll stick to standard Linux because that is what most people use.
We'll focus on two areas:
Linux as an operating system, and
Linux file systems.
Linux Operating System Issues
We'll set aside what might happen with Linux in the future and instead focus on what is available
today. Linux has a number of features that match the I/O performance of AIX and Solaris, such as
direct I/O, but the bottom line is that Linux wasn't designed around high-performance multi-threaded
I/O.
There are a number of areas that limit performance in Linux, such as page size compared with other
operating systems, the restrictions Linux places on direct I/O and page alignment, and the fact that
Linux does not allow direct I/O automatically by request size - I have seen Linux kernels break large
(greater than 512 KB) I/O requests into 128 KB requests. Since the Linux I/O performance and file
system were designed for a desktop replacement for Windows, none of this comes as much of a surprise.
Linux has other issues, as I see it; for starters, the lack of someone to take charge or responsibility.
With Linux, if you find a problem, groups of people are going to have to agree to fix it, and the
people writing Linux might not necessarily be responsive to the problems you're facing. If a large
vendor of Linux agrees with your problem and provides a fix, that doesn't mean it will be accepted
- or accepted anytime soon - by the Linux community. And getting a patch for your problem could pose
maintenance problems.
The goals for Linux file systems and the Linux kernel design seem to be trying to address a completely
different set of problems than AIX or Solaris, and IBM and Sun are far more directly responsible
than the Linux community if you have a problem. If you run AIX or Solaris and complain to IBM or
Sun, they can't say we have no control.
Linux File Systems
Remember that most Linux file systems were designed around replacing NTFS, not some of the high-performance
file systems such as GPFS (IBM), StorNext (Quantum) or QFS (Sun). These file systems were designed
for streaming I/O, which we now know is important for everyone and for some high-speed IOPS, and
in some cases for database access.
The Linux file systems that are commonly used today (ext-3 today and likely soon ext-4 and xfs)
have not had huge structural changes in a long time. Ext-4 improves upon ext-3 and ext-2 for some
improved allocation, but simple things like alignment of the superblock to the
RAID stripe
and the first metadata allocation are not considered.
Additionally, things like alignment of additional file system metadata regions to RAID stripe
value are not considered, nor are simple things like indirect allocations (see
File Systems and Volume Managers: History and Usage), which are fixed values so with the small
allocations supported (4 KB maximum), large numbers of allocations are required. Take a 200 TB file
system, which will require 53.7 billion allocations to represent the 200 TB using the largest allocation
size of 4 KB supported by ext-3. Using 8 MB, which is feasible on enterprise file systems, it becomes
a manageable 26.2 million allocations. The bitmap or allocation map could even fit in memory for
this number of allocations! The xfs file system has very similar characteristics to ext-3. Yes, allocations
can be larger, up to 64 KB if the Linux page size is 64 KB, but the alignment issues for the superblock,
metadata regions and other issues still exist.
Linux Has Its Place
That's not to say I am anti-Linux, just as I am not pro-AIX or pro-Solaris. I am not even anti-Windows,
since I use a Windows laptop as my main computer. But I do believe that
the default Linux file systems are not yet up to the task of replacing the high-performance, highly
scalable SMP file systems. Computers are tools, and operating systems and file systems
are also tools in the toolbox. No one uses a chainsaw in place of a jigsaw, and the same analogy
can be used for operating systems, file systems and the hardware they run on.
Many of the people I deal with daily use MS Word, MS Excel, MS PowerPoint and MS Visio. I could
run some if not all of these applications on a Windows emulator from someone, but I routinely get
incompatibilities with fonts, and I just decided long ago to live with Windows until someone can
prove to me that it all works together with no problems. My point here is that every computer is
a tool and has its use. Currently no single computer or file system can meet all application requirements.
This should not come as a surprise. Linux has a place, but as far as I can tell, that place does
not support single instances of large file systems and scaling well from large to small file systems
with high-performance requirements. And I don't see this changing anytime soon.
Henry Newman, a regular Enterprise Storage Forum contributor, is an industry consultant with
27 years experience in high-performance computing and storage.
See more articles
by Henry Newman.
Ext3 tops out at 32 tebibyte (TiB) file systems and 2 TiB files,
but practical limits may be lower than this depending on your architecture and
system settings-perhaps as low as 2 TiB file systems and 16 gibibyte (GiB) files.
Ext4, by contrast, permits file systems of up to 1024 pebibyte (PiB), or 1 exbibyte
(EiB), and files of up to 16 TiB. This may not be important (yet!) for the average
desktop computer or server, but it is important to users with large disk arrays.
Extents
An extent is a way to improve the efficiency of on-disk file
descriptors, reducing deletion times for large files, among other things.
Persistent preallocation
If an application needs to allocate disk space before actually
using it, most file systems do so by writing 0s to the not-yet-used disk space.
Ext4 permits preallocation without doing this, which can improve the performance
of some database and multimedia tools.
Delayed allocation
Ext4 can delay allocating disk space until the last moment,
which can improve performance.
More subdirectories
If you've ever felt constrained by the fact that a directory
can only hold 32,000 subdirectories in ext3, you'll be relieved to know that this
limit has been eliminated in ext4.
Journal checksums
Ext4 adds a checksum to the journal data, which improves reliability
and performance.
Online defragmentation
Although ext3 isn't prone to excessive fragmentation, files
stored on it are likely to become at least a little fragmented. Ext4 adds support
for online defragmentation, which should improve overall performance.
Undelete
Although it hasn't been implemented yet, ext4 may support undelete,
which, of course, is handy whenever somebody accidentally deletes a file.
Faster file system checks
Ext4 adds data structures that permit fsck to skip
unused parts of the disk in its checks, thus speeding up file system checks.
Nanosecond timestamps
Most file systems, including ext3, include timestamp data that
is accurate to a second. Ext4 extends the accuracy of this data to a nanosecond.
Some sources also indicate that the ext4 timestamps support dates through April
25, 2514, versus January 18, 2038, for ext3.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU
Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation;
with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
can be found here.
Overview I'm a big fan of the Third Extended ("ext3") filesystem. It's in-kernel and userspace
code has been tried, tested, fixed, and improved upon more than almost every other Linux-compatible
filesystem. It's simple, robust, and extensible. In this article I intend to explain some tips
that can improve both the performance and the reliability of the filesystem.
In the document, /dev/hdXY will be used as a generic partition. You should replace this with the
actual device node for your partition, such as /dev/hdb1 for the first partition of the primary slave
disk or /dev/sda2 for the second partition of your first SCSI or Serial ATA disk.
I: Using The tune2fs and e2fsck Utilities
Before we begin, we need to make sure you are comfortable with using the tune2fs utility to alter
the filesystem options of an ext2 or ext3 partition. Please make sure to read the tune2fs
man page:
Code:
$ man tune2fs
It's generally a good idea to run a filesystem check using the e2fsck utility after you've completed
the alterations you wish to make on your filesystem. This will verify that your filesystem is clean
and fix it if needed. You should also read the manual page for the e2fsck utility if you have not
yet done so:
Code:
$ man e2fsck
WARNING: Make sure any filesystems are cleanly unmounted before altering them with the tune2fs or
e2fsck utilities! (Boot from a LiveCD such as Knoppix if you need to.) Altering or tuning a filesystem
while it is mounted can cause severe corruption! You have been warned!
II: Using Directory Indexing
This feature improves file access in large directories or directories containing many files by
using hashed binary trees to store the directory information. It's perfectly safe to use, and it
provides a fairly substantial improvement in most cases; so it's a good idea to enable it:
Code:
# tune2fs -O dir_index /dev/hdXY
This will only take effect with directories created on that filesystem after tune2fs is run. In order
to apply this to currently existing directories, we must run the e2fsck utility to optimize and reindex
the directories on the filesystem:
Code:
# e2fsck -D /dev/hdXY
Note: This should work with both ext2 and ext3 filesystems. Depending on the size of your
filesystem, this could take a long time. Perhaps you should go get some coffee
III: Enable Full Journaling
By default, ext3 partitions mount with the 'ordered' data mode. In this mode, all data
is written to the main filesystem and its metadata is committed to the journal, whose blocks are
logically grouped into transactions to decrease disk I/O. This tends to be a good default for most
people. However, I've found a method that increases both reliability and performance (in some situations):
journaling everything, including the file data itself (known as 'journal' data mode). Normally, one
would think that journaling all data would decrease performance, because the data is written to disk
twice: once to the journal then later committed to the main filesystem, but this does not seem to
be the case. I've enabled it on all nine of my partitions and have only seen a minor performance
loss in deleting large files. In fact, doing this can actually improve performance on a filesystem
where much reading and writing is to be done simultaneously. See this article written by Daniel Robbins
on IBM's website for more information:
In fact, putting /usr/portage on its own ext3 partition with journal data mode seems to
have decreased the time it takes to run emerge --sync significantly. I've also seen slight improvements
in compile time.
There are two different ways to activate journal data mode. The first is by adding data=journal
as a mount option in /etc/fstab. If you do it this way and want your root filesystem to also use
it, you should also pass rootflags=data=journal as a kernel parameter in your bootloader's configuration.
In the second method, you will use tune2fs to modify the default mount options in the filesystem's
superblock:
Please note that the second method may not work for older kernels. Especially Linux 2.4.20 and below
will likely disregard the default mount options on the superblock. If you're feeling adventurous
you may also want to tweak the journal size. (I've left the journal size at the default.) A larger
journal may give you better performance (at the cost of more disk space and longer recovery times).
Please be sure to read the relevant section of the tune2fs manual before doing so:
Code:
# tune2fs -J size=$SIZE /dev/hdXY
IV: Disable Lengthy Boot-Time Checks
WARNING: Only do this on a journalling filesystem such as ext3. This may or may not work
on other journalling filesystems such as ReiserFS or XFS, but has not been tested. Doing so may damage
or otherwise corrupt other filesystems. You do this AT YOUR OWN RISK.
Hmm..It seems that our ext3 filesystems are still being checked every 30 mounts or so.
This is a good default for many because it helps prevent filesystem corruption when you have hardware
issues, such as bad IDE/SATA/SCSI cabling, power supply failures, etc. One of the driving forces
for creating journalling filesystems was that the filesystem could easily be returned to a consistent
state by recovering and replaying the needed journalled transactions. Therefore, we can safely disable
these mount-count- and time-dependent checks if we are certain the filesystem will be quickly checked
to recover the journal if needed to restore filesystem and data consistency. Before you do this please
make sure your filesystem entry in /etc/fstab has a positive integer in its 6th field (pass) so that
it is checked at boot time automatically. You may do so using the following command:
Code:
# tune2fs -c 0 -i 0 /dev/hdXY
V: Checking The Filesystem Options Using tune2fs
Well, now that we've tweaked our filesystem, we want to make sure those tweaks are applied, right?
Surprisingly, we can do this options iusing the tune2fs utility quite easily. To list all the contents
of the filesystem's superblock, we can pass the "-l" (lowercase "L") option to tune2fs:
Code:
# tune2fs -l /dev/hdXY
Unlike the other tune2fs calls, this can be run on a mounted filesystem without harm, since it doesn't
access or attempt to change the filesystem at such a low level.
This will give you a lot of information about the filesystem, including the block/inode information,
as well as the filesystem features and default mount options, which we are looking for. If all goes
well, the relevant part of the output should include "dir_index" and "has_journal" flags in the Filesystem
features listing, and should show a default mount option of "journal_data".
This concludes this filesystem tweaking guide for now. Happy hacking! _________________ ~~ Peter:
GNU/Linux geek, caffeine addict, and Free Software advocate.
Editer for clarification: notice that 'has_journal' parameter is valid for '-O' (upper case
'o') modifier, not for '-o' (lower case 'o').
====
jetsaredim wrote: is there a way to list the options for a particular ext3 fs options
that have been set?
` -k` will list the current contents of the filesystem's superblock.
===
Is there any way to do defragmentation on ext3? My experience is that havily used ext3
partitions become slower and slower while the amount of files in the filesystem and used diskspace
don't significally change.
Posted: Fri Apr 08, 2005 4:21 pm Post subject:
ext3 automagically handles defragmentation as you use it.
The best thing I can think of is to try re-optimizing the filesystem structure
by running the following:
Code:
# e2fsck -D /dev/hdXY
(/dev/hdXY should be unmounted as explained in the first post in order to avoid
filesystem corruption.)
====
Q: when does one benefit from using orlov? and what exactly does commit=9999 mean?
A: You do not need to use orlov as a mount option, since, according to the mount(8) man page,
it is the default if neither oldalloc nor orlov is specified. This option would tell ext2/ext3
whether to use the old inode allocator or the new Orlov inode allocator.
I also highly recommend against using commit=9999. This mount option specifies how often (in second
intervals) to sync the data to disk. Setting this too high may cause excessive usage of memory and
possibly CPU/swap resources. This really is not needed (and from my experience) will not give you
a large performance increase at all.
===
What command do you use to set the immutable attribute under reiserfs? Man pages for chattr and
lsattr indicate only functioning with ext2,3.
Another little tidbit that needs to be discussed when talking about these immutable bits is the
following.
Now I can simply chattr -i /etc/shadow anytime I want as root and it'll be like it never happened.
However with seclvl (A linux implementation of BSD Secure Levels) the behavior mimics that of BSD.
So when using these attributes remember to echo "2" > /sys/seclvl/seclvl if you have this support
built into your kernel.
The hardened kernel series I know supports this and is always a great idea to use in any secure
server implementation.
Ok, acl and immutable are completely different. ACL is access control list, thats just a major
enhancement to rwx. The file system flags mantained by chattr are just like the BSD ones that interact
with the Secure level. Now if you chattr +i the file is immutable but just a simple chattr -i can
just make that entire concept null. Using th BSD secure level implementation for linux accually enforces
the rules you set by the file.
will tell you how to do it, it's not specific to gentoo either it will work on any linux distribution,
that supports ext3, which I believe is all of them (unless they are using really, really, old kernels).
The ext3 file system is essentially an enhanced version of the ext2 file system.
These improvements provide the following advantages
Availability
After an unexpected power failure or system crash (also called an unclean
system shutdown), each mounted ext2 file system on the machine must
be checked for consistency by the e2fsck program. This is a time-consuming process that can delay
system boot time significantly, especially with large volumes containing a large number of files.
During this time, any
data on the volumes is unreachable.
The journaling provided by the ext3 file system means that this sort of file
system check is no longer necessary after an unclean system shutdown. The only time a consistency
check occurs using ext3 is in certain rare hardware failure cases, such as hard drive failures. The
time to recover an ext3 file system after an unclean system shutdown does not depend on the size
of the file system or the number of files; rather, it depends on the size of the journal used to
maintain consistency. The default journal size takes about a second to recover, depending on the
speed of the hardware.
Data Integrity
The ext3 file system provides stronger data integrity in the event that an
unclean system shutdown occurs. The ext3 file system allows you to choose the type and level of protection
that your data receives. By default, Most Linux Distributions configures ext3 volumes to keep a high
level of data consistency with regard to the state of the file system.
Speed
Despite writing some data more than once, ext3 has a higher throughput in
most cases than ext2 because ext3's journaling optimizes hard drive head motion. You can choose from
three journaling modes to optimize speed, but doing so means trade offs in regards to data integrity.
Easy Transition
It is easy to change from ext2 to ext3 and gain the benefits of a robust journaling
file system without reformatting. See the Section called Converting to an ext3 File System for more
on how to perform this task.
I'm using the ext3 file system and it schedules a "read" from my disk every 5 seconds, which
is annoying. How do you adjust this to something like once every minute. Will tune2fs do this? TIA.
The 5 seconds are the commit interval. This is the standard behaviour. You can check this in your
syslog. here the ext3.txt from the kernel documentation (<kernel dir>/Documentation/filesystems/ext3.txt:
Quote:
(...)commit=nrsec Ext3 can be told to sync all its data and metadata every 'nrsec' seconds.
The default value is 5 seconds. This means that if you lose your power, you will lose as much
as the latest 5 seconds of work (your filesystem will not be damaged though, thanks to the
journaling). This default value (or any low value) will hurt performance, but it's good for
data-safety. Setting it to 0 will have the same effect as leaving it at the default (5 seconds).
Setting it to very large values will improve performance.
(in the following mini-HOWTO are added more performance options, if you don't want them then only
add the "commit=seconds" option (in the same order though)
1st step Take your /etc/fstab
and add these options for your /root (and/or /home etc) partition:
I guess you will also be very happy with the "data=writeback" and "nobh" option. This works for ext3.
I guess for reiser also, but please check this before..
2nd step To make data=writeback
and the new commit interval work get your /boot/grub/menu.lst See the "defoptions=" line and add
(e.g. after "ro quiet splash") -->
For all your partitions, e.g. if you have /root and /home seperated.
finally... Then do
a reboot. However, the specific option you were looking for is the "commit=sec" options. The value
is measured is seconds.
caution! I had several crashes (not linux' fault ) and my data is still there, although
these options increases a possible risk of data loss!!! Note: You are not disabling journaling with
this. so it's still pretty safe. (however, own risk)
Appendix PS: My posting seems quite confusing, I guess. So here are the specific example
lines/files:
(...)
## additional options to use with the default boot option, but not with the
## alternatives
## e.g. defoptions=vga=791 resume=/dev/hda5
# defoptions=quiet splash rootflags=data=writeback,nobh,commit=100
(...)
## altoption boot targets option
## multiple altoptions lines are allowed
## e.g. altoptions=(extra menu suffix) extra boot options
## altoptions=(recovery) single
# altoptions=(recovery mode) single rootflags=data=writeback
####for the alt options only the data=writeback options is necessary
(...)
don't forget to run a "sudo update-grub"!
Be sure, to have e.g. a live cd to access the
system if you make at typing error or so in one of these config files.
WARNING (again) of possible several data loss. Do at your own risk. This is recommended for laptops
and/or desktop systems. Don't do this on servers!
DON'T MAKE A TYPING ERROR BY MIXING UP tune2fs with mke2fs!!! This happened once
to me and will erase all your data.
more information Kernel-Documentation (mostly <directories to kernel>/Documentation/filesytems/ext3.txt
very interesting
data=writeback While the writeback option provides lower data
consistency guarantees than the journal or ordered modes, some applications
show very significant speed improvement when it is used. For example, speed improvements
can be seen when heavy synchronous writes are performed, or when applications create and delete large
volumes of small files, such as delivering a large flow of short email messages. The results of the
testing effort described in Chapter 3 illustrate this topic.
When the writeback option is used, data consistency is similar to that provided by the ext2 file
system. However, file system integrity is maintained continuously during normal operation in the
ext3 file system.
In the event of a power failure or system crash, the file system may not be recoverable if a significant
portion of data was held only in system memory and not on permanent storage. In this case, the filesystem
must be recreated from backups. Often, changes made since the file system was last backed up are
inevitably lost.
How to make ext3 or reiserfs use journal data writeback
First you need to take fstab file using the following command
sudo cp /etc/fstab /etc/fstab.orig
Edit the /etc/fstab file using the following command
sudo vi /etc/fstab
Add the thing marked in bold to your fstab root mount line.
Now you need to edit the grub menu list file using the following command
sudo vi /boot/grub/menu.lst
look for the following two lines
# defoptions=quiet splash # altoptions=(recovery mode) single
change to
# defoptions=quiet splash rootflags=data=writeback # altoptions=(recovery
mode) single rootflags=data=writeback
Save that file and exit
Now you need to update the grub using the following command
sudo update-grub
the added flags will automatically be added to the kernel line and stay there
in case of kernel update
Changes to Ext3 FileSystem Only
Note:- tune2fs only works for ext3. Reiserfs can't change the journal method
Before rebooting change the filesystem manually to writeback using the following
command
sudo tune2fs -o journal_data_writeback /dev/hda1
Check that it is running or not using the following command
sudo tune2fs -l /dev/hda1
Remove update of access time for files
Having the modified time change you can understand but having the system updating
the access time every time a file is accessed is not to my liking. According to the manual the only
thing that might happen if you turn this off is that when compiling certain things the make might
need that info.
In a recent lkml thread, Linus Torvalds was involved in a discussion about mounting filesystems
with the noatime option for better performance, "'noatime,data=writeback'
will quite likely be *quite* noticeable (with different effects for different loads), but almost
nobody actually runs that way."
He noted that he set O_NOATIME when writing git, "and it was an absolutely
huge time-saver for the case of not having 'noatime' in the mount options. Certainly more than
your estimated 10% under some loads."
The discussion then looked at using the relatime mount option to
improve the situation, "relative atime only updates the atime if the previous atime is older than
the mtime or ctime. Like noatime, but useful for applications like mutt that need to know when a
file has been read since it was last modified."
Ingo Molnar stressed the significance of fixing this performance issue, "I cannot over-emphasize
how much of a deal it is in practice. Atime updates are by far the biggest
IO performance deficiency that Linux has today. Getting rid of atime updates would give us more everyday
Linux performance than all the pagecache speedups of the past 10 years, _combined_." He submitted some patches to improve relatime, and noted about atime:
"It's also perhaps the most stupid Unix design idea of all times.
Unix is really nice and well done, but think about this a bit: 'For every file that is read from
the disk, lets do a ... write to the disk! And, for every file that is already cached and which
we read from the cache ... do a write to the disk!'"
Ext2 0.04 for NT4 read-write
Contacts and feedback: Andrey Shedel [email protected]
Primary site: http://www.chat.ru/~ashedel
CAUTION!!! this is nt kernel-mode driver and you are using it at your
own risk. It is highly recommended to use sync utility to flush regular
volumes first.
>> You should be aware of the fact that ext2.sys might <<
>> damage the data stored on your hard disks. <<
If you cannot agree to these conditions, you should NOT use ext2.sys !
installation (you should be the member of administrators group):
copy ext2.sys to your %systemroot%\system32\drivers directory
merge ext2.reg file
reboot to update driver information
edit go.cmd to point to your Linux drive
run go.cmd
Known features:
Non-regular files are converted to regular at first write attempt.
Mounting partitions:
NT4:
Instead of loading the driver manually or automatically (by setting
startup mode to 1) you can use fs_rec.sys (recognizer driver). This
driver is a superset of the recognizer that comes with NT4 and can be
used instead of it. In addition to CDFS, NTFS and FAT (standatd set for
NT4) it includes recognision modules for HPFS (for pinball.sys from
NT3.51), FAT32-enabled fastfat and Ext2. It is not recommended to use
this recognizer on NT5 because support for UDFS is not included.
Unfortunately even in this case you still have to set persistent links
in DosDevices namespace UNLESS YOU ARE USING NT5. For example:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\DOS Devices]
"E:"="\\Device\\Harddisk0\\Partition2"
NT5:
On NT5 you can use Disk Management utility to assign drive letter.
Files included:
readme.txt - this file
ext2.sys - driver
dosdev.exe - Define/RemoveDosDevice utility.
kloader.exe - utility to load kernel-mode driver.
ln.exe - hardlink creation utility.
SYNC.EXE - Flush write-behind cache utility.
fs_rec.sys - Recognizer driver.
Changes:
0.04:
pagefile support
initial security implementation.
Ext3fs is a journaled version of ext2fs written by Stephen Tweedie. It's in beta
form right now but works pretty well. Stephen and Ted Ts'o talked about ext3fs at our Linux Storage
Management Workshop in Darmstadt, Germany (you can get the slides for this workshop at
ftp://linux.msede.com/lsmws_talks/) The ext3 filesystem,
of which early alphas are ready (version 0.0.2c, the excitement !!). Development is on the linux-fsdevel
mailing list, archived
here. Hello,
I've been running ext3 on my laptop computer for about two months now. It works great. Just sync
the disks and turn it off. No shutdown. No data loss either. If you look at e.g Solaris disk-suite
you are able to control where your should store your metadata. Say that you want to have journaling
file data also, this is normally slowing the system down. But if you can specify that all file metadata
should be on a separate solidstate disk (naturally mirrored for safety). Then journaling of file
data will be quick and swift. This is in my view quite important. If I understand everything correctly
you can do that with ext3. One of the major problems with ext2fs (IMHO) is that it doesn't resize
well. This is because there is a copy of every group descriptor in every group [a g.d. contains
metadata for a group of blocks/inodes, typically 8M in size]. Therefore enlarging or shrinking the
drive causes a major reshuffle of ALL the data; so far, the only utility I know that can do this
is resiz2fs, which comes with Partition Magic (there are no doubt others now).
This redundancy is good in theory (backups), but keeping a copy of a constant number of group
descriptors (perhaps the previous and next 32) in a given group would still give you a lot
of redundancy plus make resizing simpler.
Granted, resizing isn't something you do a lot, but having had my system lock up and die while
resizing and having to recover using Turbo C++ and the ext2fs spec (code and info on my
ext2fs page), it would
be nice if ext3fs (or XFS) made this easier.
The Reiser Filesystems by
Hans Reiser, a very ambitious project to not only improve performance and add journaling, but to
redefine the filesystem as a storage repository for arbitrarily complex objects. Reiserfs is faster
than ext2/3 because it uses balanced trees for it's directory-structures. The project is now released
for 2.2.11 - 2.2.13. Mailing list archive
here.
The Xfs site has some docs. The
work to unencumber the code is accelerating, and February is the target date for source code release.
XFS is the one that I think has the most potential. It's a full logging filesystem from the ground
up, not an extension (not that EXT3 or DTFS are bad or misguided efforts) I'm betting it will be
the highest performance filesystem for linux when it goes gold. I think the tight integration of
the log could be a huge plus. It's been a while since filesystem 101 but I would think that there
are a ton of ways to optimize performance with log write back tricks and useage optimizations.. You
could include a hit counter in metadata and have an optimizer that moves higher hit files closer
to the log in the center of the disk making your more frequently used files closer to where the head
is supposed to be. Those kinds of optimizations (if practical, maybe I'm full of it) wouldn't be
nearly as easy with ext3 since the FS doesn't have any knowldege of the log. Plus xfs has ACLs and
big file support already.
Hi,ext3fs is a journaled version of ext2fs written by Stephen Tweedie. It's in beta form right
now but works pretty well. Stephen and Ted Ts'o talked about ext3fs at our Linux Storage Management
Workshop in Darmstadt, Germany (you can get the slides for this workshop at ftp://linux.msede.com/lsmws_talks/)
-
Stephen also gave a talk on ext3fs at the Linux Kongress in Augsburg, Germany. He is predicting
Summer 2000 for production use of ext3fs. Nice features include the fact that ext3fs is backwards
compatible with older versions of ext2. In addition, ext3fs uses asynchronous journaling, which means
the performance will be as good or better than ext2fs. -
I am involved with the SGI effort to port XFS to Linux. The work to unencumber the code is accelerating,
and February is the target date for source code release. The read path is working at this time. More
work remains however, so stay tuned to -
Explores the performance characteristics and differences of Solaris ZFS and the ext3 file system
through a series of benchmarks based on use cases derived from common scenarios, as well as the IOzone
File System Benchmark (IOzone benchmark) which tests specific I/O patterns.
Linux is a Unix-like operating system, which runs on PC-386 computers. It was implemented first
as extension to the Minix operating system [Tanenbaum 1987] and its first versions
included support for the Minix filesystem only. The Minix filesystem contains two serious limitations:
block addresses are stored in 16 bit integers, thus the maximal filesystem size is restricted to
64 mega bytes, and directories contain fixed-size entries and the maximal file name is 14 characters.
We have designed and implemented two new filesystems that are included in the standard
Linux kernel. These filesystems, called ``Extended File System'' (Ext fs) and ``Second Extended File
System'' (Ext2 fs) raise the limitations and add new features.
In this paper, we describe the history of Linux filesystems. We briefly introduce
the fundamental concepts implemented in Unix filesystems. We present the implementation of the Virtual
File System layer in Linux and we detail the Second Extended File System kernel code and user mode
tools. Last, we present performance measurements made on Linux and BSD filesystems and we conclude
with the current status of Ext2fs and the future directions.
These slides focus specifically on the EXT/2 filesystem, talking about its evolution, philosophy,
planned new features, and relation to other linux filesystems.
Filesystem Hierarchy Standard
- This page is the home of two standards, the Linux Filesystem Standard (FSSTND) and its successor,
the Filesystem Hierarchy Standard (FHS).
The Last but not LeastTechnology is dominated by
two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt.
Ph.D
FAIR USE NOTICEThis site contains
copyrighted material the use of which has not always been specifically
authorized by the copyright owner. We are making such material available
to advance understanding of computer science, IT technology, economic, scientific, and social
issues. We believe this constitutes a 'fair use' of any such
copyrighted material as provided by section 107 of the US Copyright Law according to which
such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free)
site written by people for whom English is not a native language. Grammar and spelling errors should
be expected. The site contain some broken links as it develops like a living tree...
You can use PayPal to to buy a cup of coffee for authors
of this site
Disclaimer:
The statements, views and opinions presented on this web page are those of the author (or
referenced source) and are
not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society.We do not warrant the correctness
of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be
tracked by Google please disable Javascript for this site. This site is perfectly usable without
Javascript.
Unchecked
