Discussion:
Does ext4 send FUA to flush disk cache
Mark Busheman
2011-04-08 22:02:27 UTC
Permalink
I plan to use data=journal option with ext4. Would like to know if
ext4 send FUA (Forced Unit Access)
to flush the disk cache?

Cheers
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Theodore Tso
2011-04-10 12:17:58 UTC
Permalink
Post by Mark Busheman
I plan to use data=journal option with ext4. Would like to know if
ext4 send FUA (Forced Unit Access)
to flush the disk cache?
FUA doesn't cause a cache flush. Ext4 does send cache flush commands, or barriers, to make sure the data written to disk is flushed all the way down to the disk platters on transaction commits.

-- Ted

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Ted Ts'o
2011-04-11 18:44:33 UTC
Permalink
Thanks Ted for replying.
Post by Theodore Tso
FUA doesn't cause a cache flush.
Thanks for correcting. Yes FUA does not flush cache. When the FUA
bit is set in the write command, the drive make sure that the data
goes to platters and not to be put into its cache for latter writes.
For that particular write only; and it forces the write to go directly
to the disk, bypassing any seek-mitigation algorithms the disk might
use. It's good if you only have a very small number of sectors
(ideally one) which you need to write to the disk. In the case of
ext4, we want to make sure all of the data and metadata blocks sent to
disk are flushed down to the platters, so FUA isn't as useful to us,
and so we don't use it.
Post by Theodore Tso
Ext4 does send cache flush commands, or barriers, to make sure the data written to disk
is flushed all the way down to the disk platters on transaction commits.
a. To achieve the above, is any specific options that need to be used
while mounting?
No, not for ext3.
b. Are the above mentioned things done only for data=journal or for
other modes such as
data=ordered and writeback?
It's done for all journalling modes.
c. Are they done for metadata, journal and data writes?
Yes; see below.
d. Can I know what are the commands sent to the drives to do the cache
flush commands
or barriers? Would like to find if the drive support the commands?
We use the CACHE FLUSH or CACHE FLUSH EXT command, as appropriate for
IDE/SATA disks. There is an equivalent command for SCSI, but in
essence it establishes a barrier; all writes sent to the disk before
the CACHE FLUSH/CACHE FLUSH EXT must be written to the disk before the
barrier returns.

When we use journaling, we send a CACHE FLUSH just before the writing
the commit block, and then we send a second CACHE FLUSH after writing
the commit block. (I suppose we could use a FUA bit for the commit
block, which might be a slight optimization if there is any other
writes happening in parallel that don't need to be affected, but it
doubt it would make a huge difference in most cases.)
e. Is the above true also for ext3 - I posted a similar question to
ext3 forum and have got no
response so far?
ext3 unfortunately defaults to barriers being disabled by default.
Some distributions, such as the enterprise distro's such as RHEL and
SLES, will enable ex3 barriers by default. You can use the mount
option "barrier" to enable barriers at mount time with ext3, if you
want to be sure barriers are enabled.

With ext4, if you want to disable barriers for some reason (say you
are confident the system has a dead reliable UPS and you need the
performance), you can use the mount option nobarrier. (It enables
barriers by default.)

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Mark Busheman
2011-04-11 21:18:08 UTC
Permalink
Ted, Thanks for the clear answer and also for answering the question on ext3.

Cheers
Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Mark Busheman
2011-04-11 18:23:23 UTC
Permalink
Thanks Ted for replying.
Post by Theodore Tso
FUA doesn't cause a cache flush.
Thanks for correcting. Yes FUA does not flush cache. When the FUA bit
is set in the write
command, the drive make sure that the data goes to platters and not to
be put into its
cache for latter writes.
Post by Theodore Tso
Ext4 does send cache flush commands, or barriers, to make sure the d=
ata written to disk
Post by Theodore Tso
is flushed all the way down to the disk platters on transaction comm=
its.
I have few questions on this and need help:
a. To achieve the above, is any specific options that need to be used
while mounting?
b. Are the above mentioned things done only for data=3Djournal or for
other modes such as
data=3Dordered and writeback?
c. Are they done for metadata, journal and data writes?
d. Can I know what are the commands sent to the drives to do the cache
flush commands
or barriers? Would like to find if the drive support the commands?
e. Is the above true also for ext3 - I posted a similar question to
ext3 forum and have got no
response so far?

Cheers
Mark
Post by Theodore Tso
I plan to use data=3Djournal option with ext4. Would like to know if
ext4 send FUA (Forced Unit Access)
to flush the disk cache?
FUA doesn't cause a cache flush. =A0 Ext4 does send cache flush comma=
nds, or barriers, to make sure the data written to disk is flushed all =
the way down to the disk platters on transaction commits.
-- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" i=
n
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Christoph Hellwig
2011-04-11 23:16:06 UTC
Permalink
Post by Theodore Tso
Post by Mark Busheman
I plan to use data=journal option with ext4. Would like to know if
ext4 send FUA (Forced Unit Access)
to flush the disk cache?
FUA doesn't cause a cache flush. Ext4 does send cache flush commands, or barriers, to make sure the data written to disk is flushed all the way down to the disk platters on transaction commits.
Ext4 actually does send FUA requests, just grep for it. With the default
libata config they will be turned into a regular write with a
post-flush, but if you enable the fua module option to the libata module
or use plain SCSI devices the FUA bit (if supported) gets sent all the
way down to the device.

The same also applies to btrfs, gfs2, nilfs2, xfs and if you enable the
non-default config ext3 and reiserfs. Note that all the filesystem also
do regular flushes as pre-flushes during the journal commit or
equivalent and as part of the fsync implementation.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Ted Ts'o
2011-04-12 02:14:23 UTC
Permalink
Post by Christoph Hellwig
Post by Theodore Tso
Post by Mark Busheman
I plan to use data=journal option with ext4. Would like to know if
ext4 send FUA (Forced Unit Access)
to flush the disk cache?
FUA doesn't cause a cache flush. Ext4 does send cache flush commands, or barriers, to make sure the data written to disk is flushed all the way down to the disk platters on transaction commits.
Ext4 actually does send FUA requests, just grep for it. With the default
libata config they will be turned into a regular write with a
post-flush, but if you enable the fua module option to the libata module
or use plain SCSI devices the FUA bit (if supported) gets sent all the
way down to the device.
Ah, Jens pushed a patch that I hadn't noticed that appears to
optionally use FUA for the commit block. Thanks for pointing that
out.

- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Christoph Hellwig
2011-04-12 04:40:18 UTC
Permalink
Post by Ted Ts'o
Ah, Jens pushed a patch that I hadn't noticed that appears to
optionally use FUA for the commit block. Thanks for pointing that
out.
The commits actually are from Tejun:

commit 797e7dbbee0a91fa1349192f18ad5c454997d876
Author: Tejun Heo <***@gmail.com>
Date: Fri Jan 6 09:51:03 2006 +0100

[BLOCK] reimplement handling of barrier request

Reimplement handling of barrier requests.

* Flexible handling to deal with various capabilities of
target devices.
* Retry support for falling back.
* Tagged queues which don't support ordered tag can do ordered.

Signed-off-by: Tejun Heo <***@gmail.com>
Signed-off-by: Jens Axboe <***@suse.de>

commit 007365ad60387df30f02f01fdc2b6e6432f6c265
Author: Tejun Heo <***@gmail.com>
Date: Fri Jan 6 09:53:52 2006 +0100

[BLOCK] scsi: add FUA support to sd

Add FUA support for barriers to SCSI disk.

Signed-off-by: Tejun Heo <***@gmail.com>
Signed-off-by: Jens Axboe <***@suse.de>

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Loading...