I agree with Mauro on this one. I would also hazard a guess here that a lot of this is explained by confounding variables. As noted earlier, modern kernels and drive firmwares are quite good at caching, and on a system that I presume is one’s daily driver vs. a busy server with hundreds of concurrent users (not much heavy I/O happening in the background by other users), I suspect there are plenty of resources for the kernel to cache things intelligently between the SSD and HDD. With the initial test of the data transfer, what we’re seeing is likely the bottleneck of the HDD and its write cache buffer (combination of OS-level write cache and on-device controller RAM cache).

With /dev/random, I suspect the CPU (# of syscalls per second / asking kernel to generate “proper” random data in various chunk sizes, etc) may also be confounding results. Trying the same test from /dev/zero would be interesting, as we’d be getting rid of the RNG CPU overhead and left only with the kernel syscall overhead.

However, for proper benchmarking I’d use fio, which is purpose built for this and can account for confounding variables such as kernel caches, and various CPU/syscall overhead to produce proper stats.

Rouben


On Wed, Sep 3, 2025 at 07:41 Mauro Souza via Talk <talk@lists.gtalug.org> wrote:
You don't need sync, but you need to use the flag "direct" on dd. Otherwise part of the copy is going to the cache before going to the disk, and that definitely skews the statistics.
If you are copying from the SSD to the HDD, the bottleneck is the write time on the HDD not the read from the SSD, and paired with the page cache, Linux will fill up the cache first and dump all data on the HDD in one big write. And I believe 117MB/s is the top writing speed on your HDD.

Try this:
dd if=/dev/sda of=/dev/sdi bs=$i iflag=direct oflag=direct,nocache status=progress

It will read straight from the device, write straight to the other device and show you the progress on the copy.

Mauro
https://www.maurosouza.com - registered Linux User: 294521
Scripture is both history, and a love letter from God.


On Wed, Sep 3, 2025 at 8:14 AM Ron via Talk <talk@lists.gtalug.org> wrote:
William Park via Talk wrote on 2025-09-02 19:40:

> I was doing one of those occasional backup of my root SSD to HD.  They
> are identical size, so sector by sector.  I took this occasion to see if
> blocksize (bs=) matters.
>
> for i in {4,16,64,256}K {1,4,16,64}M; do
>      echo ==== $i ====
>      dd if=/dev/sda of=/dev/sdi bs=$i
>      sync
> done
>
> No difference.  All did 117MB/s.

Interesting.

My memory was that I've seen where block size makes a difference, so I
had to try for myself.


Don't have exactly the same setup so used /dev/random for input and a
file on a 5400RPM WD Red HD for output, changing the block size:

  $ for i in 1 1000 1000000 10000000 100000000
        echo "*** block size: $i"
        rm ./dd-bs-test
        dd if=/dev/random of=./dd-bs-test bs=$i  count=(math "100000000
/ $i")
    end

*** block size: 1
100000000 bytes (100 MB, 95 MiB) copied, 29.9448 s, 3.3 MB/s
*** block size: 1000
100000000 bytes (100 MB, 95 MiB) copied, 0.148444 s, 674 MB/s
*** block size: 1000000
100000000 bytes (100 MB, 95 MiB) copied, 0.112547 s, 889 MB/s
*** block size: 10000000
100000000 bytes (100 MB, 95 MiB) copied, 0.115528 s, 866 MB/s
*** block size: 100000000
100000000 bytes (100 MB, 95 MiB) copied, 0.132528 s, 755 MB/s


So, in this situation (perhaps a flawed test?), there's a dramatic
speedup going above bs=1, a significant speed improvement going above
bs=1000, and slight decreases in speed going above bs=1,000,000.


Similar results on an SSD, but *slower* on the low end and faster at
bs=1000000 (1M).

SSD results also slowly decline above bs=1M:

*** block size: 1
100000000 bytes (100 MB, 95 MiB) copied, 42.0427 s, 2.4 MB/s
*** block size: 1000
100000000 bytes (100 MB, 95 MiB) copied, 0.156072 s, 641 MB/s
*** block size: 1000000
100000000 bytes (100 MB, 95 MiB) copied, 0.10943 s, 914 MB/s
*** block size: 10000000
100000000 bytes (100 MB, 95 MiB) copied, 0.120185 s, 832 MB/s
*** block size: 100000000
100000000 bytes (100 MB, 95 MiB) copied, 0.133916 s, 747 MB/s

------------------------------------
Description: GTALUG Talk
Unsubscribe via Talk-unsubscribe@lists.gtalug.org
Start a new thread: talk@lists.gtalug.org
This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/44AURFOFKVFNGP6S7RGLQYVJUMPT4RX7/
------------------------------------
Description: GTALUG Talk
Unsubscribe via Talk-unsubscribe@lists.gtalug.org
Start a new thread: talk@lists.gtalug.org
This message archived at https://lists.gtalug.org/archives/list/talk@lists.gtalug.org/message/YMI3SXNBFLMCC2KB63AVCDB5QRP4MRNB/