block size for dd -- no difference
I was doing one of those occasional backup of my root SSD to HD. They are identical size, so sector by sector. I took this occasion to see if blocksize (bs=) matters. for i in {4,16,64,256}K {1,4,16,64}M; do echo ==== $i ==== dd if=/dev/sda of=/dev/sdi bs=$i sync done No difference. All did 117MB/s.
Is it possible that the modern disk firmware performs enough buffering to make the block size irrelevant? On Tue, Sep 2, 2025 at 10:41 PM William Park via Talk <talk@lists.gtalug.org> wrote:
I was doing one of those occasional backup of my root SSD to HD. They are identical size, so sector by sector. I took this occasion to see if blocksize (bs=) matters.
for i in {4,16,64,256}K {1,4,16,64}M; do echo ==== $i ==== dd if=/dev/sda of=/dev/sdi bs=$i sync done
No difference. All did 117MB/s.
William Park via Talk wrote on 2025-09-02 19:40:
I was doing one of those occasional backup of my root SSD to HD. They are identical size, so sector by sector. I took this occasion to see if blocksize (bs=) matters.
for i in {4,16,64,256}K {1,4,16,64}M; do echo ==== $i ==== dd if=/dev/sda of=/dev/sdi bs=$i sync done
No difference. All did 117MB/s.
Interesting. My memory was that I've seen where block size makes a difference, so I had to try for myself. Don't have exactly the same setup so used /dev/random for input and a file on a 5400RPM WD Red HD for output, changing the block size: $ for i in 1 1000 1000000 10000000 100000000 echo "*** block size: $i" rm ./dd-bs-test dd if=/dev/random of=./dd-bs-test bs=$i count=(math "100000000 / $i") end *** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 29.9448 s, 3.3 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.148444 s, 674 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.112547 s, 889 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.115528 s, 866 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.132528 s, 755 MB/s So, in this situation (perhaps a flawed test?), there's a dramatic speedup going above bs=1, a significant speed improvement going above bs=1000, and slight decreases in speed going above bs=1,000,000. Similar results on an SSD, but *slower* on the low end and faster at bs=1000000 (1M). SSD results also slowly decline above bs=1M: *** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 42.0427 s, 2.4 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.156072 s, 641 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.10943 s, 914 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.120185 s, 832 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.133916 s, 747 MB/s
You don't need sync, but you need to use the flag "direct" on dd. Otherwise part of the copy is going to the cache before going to the disk, and that definitely skews the statistics. If you are copying from the SSD to the HDD, the bottleneck is the write time on the HDD not the read from the SSD, and paired with the page cache, Linux will fill up the cache first and dump all data on the HDD in one big write. And I believe 117MB/s is the top writing speed on your HDD. Try this: dd if=/dev/sda of=/dev/sdi bs=$i iflag=direct oflag=direct,nocache status=progress It will read straight from the device, write straight to the other device and show you the progress on the copy. Mauro https://www.maurosouza.com - registered Linux User: 294521 Scripture is both history, and a love letter from God. On Wed, Sep 3, 2025 at 8:14 AM Ron via Talk <talk@lists.gtalug.org> wrote:
William Park via Talk wrote on 2025-09-02 19:40:
I was doing one of those occasional backup of my root SSD to HD. They are identical size, so sector by sector. I took this occasion to see if blocksize (bs=) matters.
for i in {4,16,64,256}K {1,4,16,64}M; do echo ==== $i ==== dd if=/dev/sda of=/dev/sdi bs=$i sync done
No difference. All did 117MB/s.
Interesting.
My memory was that I've seen where block size makes a difference, so I had to try for myself.
Don't have exactly the same setup so used /dev/random for input and a file on a 5400RPM WD Red HD for output, changing the block size:
$ for i in 1 1000 1000000 10000000 100000000 echo "*** block size: $i" rm ./dd-bs-test dd if=/dev/random of=./dd-bs-test bs=$i count=(math "100000000 / $i") end
*** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 29.9448 s, 3.3 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.148444 s, 674 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.112547 s, 889 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.115528 s, 866 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.132528 s, 755 MB/s
So, in this situation (perhaps a flawed test?), there's a dramatic speedup going above bs=1, a significant speed improvement going above bs=1000, and slight decreases in speed going above bs=1,000,000.
Similar results on an SSD, but *slower* on the low end and faster at bs=1000000 (1M).
SSD results also slowly decline above bs=1M:
*** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 42.0427 s, 2.4 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.156072 s, 641 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.10943 s, 914 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.120185 s, 832 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.133916 s, 747 MB/s
------------------------------------ Description: GTALUG Talk Unsubscribe via Talk-unsubscribe@lists.gtalug.org Start a new thread: talk@lists.gtalug.org This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/44AURFO...
I agree with Mauro on this one. I would also hazard a guess here that a lot of this is explained by confounding variables. As noted earlier, modern kernels and drive firmwares are quite good at caching, and on a system that I presume is one’s daily driver vs. a busy server with hundreds of concurrent users (not much heavy I/O happening in the background by other users), I suspect there are plenty of resources for the kernel to cache things intelligently between the SSD and HDD. With the initial test of the data transfer, what we’re seeing is likely the bottleneck of the HDD and its write cache buffer (combination of OS-level write cache and on-device controller RAM cache). With /dev/random, I suspect the CPU (# of syscalls per second / asking kernel to generate “proper” random data in various chunk sizes, etc) may also be confounding results. Trying the same test from /dev/zero would be interesting, as we’d be getting rid of the RNG CPU overhead and left only with the kernel syscall overhead. However, for proper benchmarking I’d use fio, which is purpose built for this and can account for confounding variables such as kernel caches, and various CPU/syscall overhead to produce proper stats. Rouben On Wed, Sep 3, 2025 at 07:41 Mauro Souza via Talk <talk@lists.gtalug.org> wrote:
You don't need sync, but you need to use the flag "direct" on dd. Otherwise part of the copy is going to the cache before going to the disk, and that definitely skews the statistics. If you are copying from the SSD to the HDD, the bottleneck is the write time on the HDD not the read from the SSD, and paired with the page cache, Linux will fill up the cache first and dump all data on the HDD in one big write. And I believe 117MB/s is the top writing speed on your HDD.
Try this: dd if=/dev/sda of=/dev/sdi bs=$i iflag=direct oflag=direct,nocache status=progress
It will read straight from the device, write straight to the other device and show you the progress on the copy.
Mauro https://www.maurosouza.com - registered Linux User: 294521 Scripture is both history, and a love letter from God.
On Wed, Sep 3, 2025 at 8:14 AM Ron via Talk <talk@lists.gtalug.org> wrote:
William Park via Talk wrote on 2025-09-02 19:40:
I was doing one of those occasional backup of my root SSD to HD. They are identical size, so sector by sector. I took this occasion to see if blocksize (bs=) matters.
for i in {4,16,64,256}K {1,4,16,64}M; do echo ==== $i ==== dd if=/dev/sda of=/dev/sdi bs=$i sync done
No difference. All did 117MB/s.
Interesting.
My memory was that I've seen where block size makes a difference, so I had to try for myself.
Don't have exactly the same setup so used /dev/random for input and a file on a 5400RPM WD Red HD for output, changing the block size:
$ for i in 1 1000 1000000 10000000 100000000 echo "*** block size: $i" rm ./dd-bs-test dd if=/dev/random of=./dd-bs-test bs=$i count=(math "100000000 / $i") end
*** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 29.9448 s, 3.3 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.148444 s, 674 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.112547 s, 889 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.115528 s, 866 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.132528 s, 755 MB/s
So, in this situation (perhaps a flawed test?), there's a dramatic speedup going above bs=1, a significant speed improvement going above bs=1000, and slight decreases in speed going above bs=1,000,000.
Similar results on an SSD, but *slower* on the low end and faster at bs=1000000 (1M).
SSD results also slowly decline above bs=1M:
*** block size: 1 100000000 bytes (100 MB, 95 MiB) copied, 42.0427 s, 2.4 MB/s *** block size: 1000 100000000 bytes (100 MB, 95 MiB) copied, 0.156072 s, 641 MB/s *** block size: 1000000 100000000 bytes (100 MB, 95 MiB) copied, 0.10943 s, 914 MB/s *** block size: 10000000 100000000 bytes (100 MB, 95 MiB) copied, 0.120185 s, 832 MB/s *** block size: 100000000 100000000 bytes (100 MB, 95 MiB) copied, 0.133916 s, 747 MB/s
------------------------------------ Description: GTALUG Talk Unsubscribe via Talk-unsubscribe@lists.gtalug.org Start a new thread: talk@lists.gtalug.org This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/44AURFO...
------------------------------------ Description: GTALUG Talk Unsubscribe via Talk-unsubscribe@lists.gtalug.org Start a new thread: talk@lists.gtalug.org This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/YMI3SXN...
On 2025-09-03 07:41, Mauro Souza via Talk wrote:
Try this: dd if=/dev/sda of=/dev/sdi bs=$i iflag=direct oflag=direct,nocache status=progress It seems you can't combine 'direct' and 'nocache', and I skipped 'status' because I won't be in front of monitor. So, I did.
dd if=/dev/sda of=/dev/sdi bs=$i iflag=direct oflag=direct And the result is 1K 18.8 MB/s 4K 51.1 16K 86.9 64K 93.3 256K 103 1M 107 4M 107 16M 105 64M 106 256M 99.1 It seems 1M-8M is the optimum. Just as well, because it's also the shortest to type. :-)
On 9/2/25 22:40, William Park via Talk wrote:
No difference. All did 117MB/s.
I suspect a lot of this sort of thing was developed back when hardware was much slower and less capable. DD was also intended to be used with tape where you would have to consider block size. The old 9 track tape drives didn't have a file system, just blocks of data.
participants (6)
-
Evan Leibovitch -
James Knott -
Mauro Souza -
Ron -
Rouben -
William Park