It's not an Xmas present as it was bought a few weeks ago, but I'm trying to install a 10TB WD (Red Pro) disk and it's not exactly going to plan. My original plan was to put the disk into an external case, plug it in to format and copy over files, then install it in the case. The system already has an SSD as /dec/sda, and two disks one of 3TB and the other of 4TB, all working fine. When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change; Now fdisk -l reports: Disk /dev/sdd: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 49FDD63D-6C95-46F0-B7B1-650038D8FB9B Disk /dev/sde: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes And cat /proc/scsi/scsi reports: Host: scsi10 Channel: 00 Id: 00 Lun: 00 Vendor: WDC WD10 Model: 1KFBX-68R56N0 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04 Host: scsi10 Channel: 00 Id: 00 Lun: 01 Vendor: WDC WD10 Model: 1KFBX-68R56NLUN1 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04 The disk model is WD101KFBX and its specs can be found here. <https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-red-pro-hdd/data-sheet-western-digital-wd-red-pro-hdd-2879-800022.pdf> All of the online guides I've consulted say that the full capacity of the drive should be visible once I switch it to GPT mode. Is this a USB limitation? Or is there something else I need to change? I've attached the output of hwinfo if that's any help. Thanks for any suggestions! -- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
On 12/29/19 12:05 AM, Evan Leibovitch via talk wrote:
It's not an Xmas present as it was bought a few weeks ago, but I'm trying to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.
My original plan was to put the disk into an external case, plug it in to format and copy over files, then install it in the case.
The system already has an SSD as /dec/sda, and two disks one of 3TB and the other of 4TB, all working fine.
When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change;
Now fdisk -l reports:
Disk /dev/sdd: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 49FDD63D-6C95-46F0-B7B1-650038D8FB9B
Disk /dev/sde: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
And cat /proc/scsi/scsi reports:
Host: scsi10 Channel: 00 Id: 00 Lun: 00 Vendor: WDC WD10 Model: 1KFBX-68R56N0 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04 Host: scsi10 Channel: 00 Id: 00 Lun: 01 Vendor: WDC WD10 Model: 1KFBX-68R56NLUN1 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04
The disk model is WD101KFBX and its specs can be found here. <https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-red-pro-hdd/data-sheet-western-digital-wd-red-pro-hdd-2879-800022.pdf>All of the online guides I've consulted say that the full capacity of the drive should be visible once I switch it to GPT mode.
Is this a USB limitation? Or is there something else I need to change? I've attached the output of hwinfo if that's any help.
Thanks for any suggestions!
Your using fdisk right. There is a version for GPT disks called gdisk and you may want to try that or a GUI program like gparted. Nick
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
--- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
On Sun, 29 Dec 2019 at 00:08, Nicholas Krause <xerofoify@gmail.com> wrote:
Your using fdisk right. There is a version for GPT disks called gdisk and you may want to try that or a GUI program like gparted.
Tried that. gdisk also reports 2TB and refuses to create any partition larger than that. - Evan
On 12/29/19 12:19 AM, Evan Leibovitch wrote:
On Sun, 29 Dec 2019 at 00:08, Nicholas Krause <xerofoify@gmail.com <mailto:xerofoify@gmail.com>> wrote:
Your using fdisk right. There is a version for GPT disks called gdisk and you may want to try that or a GUI program like gparted.
Tried that. gdisk also reports 2TB and refuses to create any partition larger than that.
- Evan
While its stating that you have a sector size of 512 bytes which is odd. Most gpt drives should be 4096bytes per sector, I just double checked. So even if its gpt it may be doing it based on issues with other things, not sure if the computer or device your using at a firmware level supports 4K sectors but it seems maybe that should be checked. Its a common problem with larger drives, I've never run into it as the systems I have are almost all UEFI or later. Nick
Hrm. For the 4TB disk in the same system (which has been working for years), fdisk -l reports: Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 88C7DF91-CD80-4EEB-AC6B-110119B04DD4 Device Start End Sectors Size Type /dev/sdc1 2048 7814035455 7814033408 3.7T Linux filesystem Wondering if that helps. Starting to wonder if it's the firmware on the USB external chassis. On Sun, 29 Dec 2019 at 00:42, Nicholas Krause <xerofoify@gmail.com> wrote:
On 12/29/19 12:19 AM, Evan Leibovitch wrote:
On Sun, 29 Dec 2019 at 00:08, Nicholas Krause <xerofoify@gmail.com> wrote:
Your using fdisk right. There is a version for GPT disks called gdisk and you may want to try that or a GUI program like gparted.
Tried that. gdisk also reports 2TB and refuses to create any partition larger than that.
- Evan
While its stating that you have a sector size of 512 bytes which is odd. Most gpt drives should be 4096bytes per sector, I just double checked. So even if its gpt it may be doing it based on issues with other things, not sure if the computer or device your using at a firmware level supports 4K sectors but it seems maybe that should be checked. Its a common problem with larger drives, I've never run into it as the systems I have are almost all UEFI or later.
Nick
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
On 12/29/19 12:48 AM, Evan Leibovitch wrote:
Hrm.
For the 4TB disk in the same system (which has been working for years), fdisk -l reports:
Disk /dev/sdc: 3.7 TiB, 4000787030016 bytes, 7814037168 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: gpt Disk identifier: 88C7DF91-CD80-4EEB-AC6B-110119B04DD4
Device Start End Sectors Size Type /dev/sdc1 2048 7814035455 7814033408 3.7T Linux filesystem
Wondering if that helps. Starting to wonder if it's the firmware on the USB external chassis. Maybe as those numbers look better on the physical side. Try seeing if connecting it directly to a open SATA port and see if that gets you 512 bytes for logical and 4096 for physical. I've suspecting that may be it based on the internal drive you proved sector size numbers.
Nick
On Sun, 29 Dec 2019 at 00:42, Nicholas Krause <xerofoify@gmail.com <mailto:xerofoify@gmail.com>> wrote:
On 12/29/19 12:19 AM, Evan Leibovitch wrote:
On Sun, 29 Dec 2019 at 00:08, Nicholas Krause <xerofoify@gmail.com <mailto:xerofoify@gmail.com>> wrote:
Your using fdisk right. There is a version for GPT disks called gdisk and you may want to try that or a GUI program like gparted.
Tried that. gdisk also reports 2TB and refuses to create any partition larger than that.
- Evan
While its stating that you have a sector size of 512 bytes which is odd. Most gpt drives should be 4096bytes per sector, I just double checked. So even if its gpt it may be doing it based on issues with other things, not sure if the computer or device your using at a firmware level supports 4K sectors but it seems maybe that should be checked. Its a common problem with larger drives, I've never run into it as the systems I have are almost all UEFI or later.
Nick
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
On 2019-12-29 12:05 AM, Evan Leibovitch via talk wrote:
When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change;
What does your mom board support? Any limitation there?
On Sun, 29 Dec 2019 00:05:17 -0500 Evan Leibovitch via talk <talk@gtalug.org> wrote:
The system already has an SSD as /dec/sda, and two disks one of 3TB and the other of 4TB, all working fine.
When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change;
Evan, Is your motherboard MBR or GPT? I have had a nasty experience with my old motherboard and a new Western Digital GPT formatted drive. http://home.eol.ca/~hgibson/RevLinux/#x1-80000D I found that Western Digital GPT drives were difficult to use. I was able to make Seagate drives work -- Howard Gibson hgibson@eol.ca jhowardgibson@gmail.com http://home.eol.ca/~hgibson
Hi Howard, The motherboard is an Asus P8Z77-V LE PLUS <https://www.asus.com/ca-en/Motherboards/P8Z77V_LE_PLUS/specifications/>. What am I looking for? There are already two existing drives, a Seagate 4TB (ST4000DM000)and a WD 3TB (WD30EFRX), that have been working fine. - Evan On Sun, 29 Dec 2019 at 10:46, Howard Gibson via talk <talk@gtalug.org> wrote:
On Sun, 29 Dec 2019 00:05:17 -0500 Evan Leibovitch via talk <talk@gtalug.org> wrote:
The system already has an SSD as /dec/sda, and two disks one of 3TB and
the
other of 4TB, all working fine.
When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change;
Evan,
Is your motherboard MBR or GPT? I have had a nasty experience with my old motherboard and a new Western Digital GPT formatted drive.
http://home.eol.ca/~hgibson/RevLinux/#x1-80000D
I found that Western Digital GPT drives were difficult to use. I was able to make Seagate drives work
-- Howard Gibson hgibson@eol.ca jhowardgibson@gmail.com http://home.eol.ca/~hgibson --- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
Try connecting the new disk to motherboard directly. If you can use 3TB and 4TB, then you can certainly use 10TB. I suspect it's your USB external case. -- William Park <opengeometry@yahoo.ca> On Sun, Dec 29, 2019 at 11:07:01AM -0500, Evan Leibovitch via talk wrote:
There are already two existing drives, a Seagate 4TB (ST4000DM000)and a WD 3TB (WD30EFRX), that have been working fine.
| From: Evan Leibovitch via talk <talk@gtalug.org> | It's not an Xmas present as it was bought a few weeks ago, but I'm trying | to install a 10TB WD (Red Pro) disk and it's not exactly going to plan. Red Pro seem to be "top of the line". I think that it doesn't use SMR (good), but I don't know. You don't say what your USB <-> SATA device is. I think that matters. But even if you did, they are often poorly documented with respect to things that the manufacturers think might confuse users. You appear to be in that territory. Your USB device is surely the reason that your computer sees 512 byte sectors, even though your drive prefers/demands 4 KiB sectors. Superstitiously, I wonder whether that alone might disturb the geometry such that transferring the disk from the external to internal would not work. If your USS <-> SATA device is old, it might not even support sizes over 2T. (I have a couple of older NAS boxes that claim they only support SATA drives up to 2T.) Suggestions: one of - Mount your drive in its ultimate destination (internal bay, with SATA connection) and set it up there. - Mount your drive in a different computer (with a similar OS release) and set it up there. The reason for the "similar OS release" suggestion is that filesystem details details might (but it isn't common). - Buy a new (and well-documented) external USB<->SATA device ================ Western Digital called 4 KiB sectors "Advanced Formatting". The transition from 512 B sectors came about a decade ago, painfully. During the transition, all sorts of hacks were used to paper over the problem. Some devices with 4 KiB partitions would pretend and fake having 512 B partitions if they thought that the computer didn't understand 4 KiB. I remember all this faking causing its own set of problems. I seem to remember USB<->SATA devices presenting 512 B sectors on the USB side, even if the drive uses 4 KiB sectors. This might cause problems (I don't know). (Off topic: A single "real" blocksize for SSDs doesn't exist. There are several kinds of block sizes for a particular device. All this is hidden from the computer. Too bad: the OS cannot optimize I/O appropriately.) The main point of Advanced Formatting was to increase density: each sector has overhead on the disk platter and so reducing the number of sectors reduced the overhead. A second reason was that sector addressing of some kind was limited to 2T / 512 or 16T / 4096, so Advanced Formatting allowed a larger capacity. At least for MBR partitioning. But we mostly use GPT partitioning now, eliminating that problem (I think). Does your computer use "legacy" booting or EFI?
| From: Evan Leibovitch via talk <talk@gtalug.org>
| It's not an Xmas present as it was bought a few weeks ago, but I'm trying | to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.
Red Pro seem to be "top of the line". I think that it doesn't use SMR (good), but I don't know.
You don't say what your USB <-> SATA device is. I think that matters. But even if you did, they are often poorly documented with respect to things that the manufacturers think might confuse users. You appear to be in that territory.
Your USB device is surely the reason that your computer sees 512 byte sectors, even though your drive prefers/demands 4 KiB sectors. Superstitiously, I wonder whether that alone might disturb the geometry such that transferring the disk from the external to internal would not work.
If your USS <-> SATA device is old, it might not even support sizes over 2T.
(I have a couple of older NAS boxes that claim they only support SATA drives up to 2T.)
Suggestions: one of
- Mount your drive in its ultimate destination (internal bay, with SATA connection) and set it up there.
- Mount your drive in a different computer (with a similar OS release) and set it up there. The reason for the "similar OS release" suggestion is that filesystem details details might (but it isn't common).
- Buy a new (and well-documented) external USB<->SATA device
================
Western Digital called 4 KiB sectors "Advanced Formatting". The transition from 512 B sectors came about a decade ago, painfully.
During the transition, all sorts of hacks were used to paper over the problem. Some devices with 4 KiB partitions would pretend and fake having 512 B partitions if they thought that the computer didn't understand 4 KiB. I remember all this faking causing its own set of problems.
I seem to remember USB<->SATA devices presenting 512 B sectors on the USB side, even if the drive uses 4 KiB sectors. This might cause problems (I don't know).
(Off topic: A single "real" blocksize for SSDs doesn't exist. There are several kinds of block sizes for a particular device. All this is hidden from the computer. Too bad: the OS cannot optimize I/O appropriately.) That does not matter actually. The optimizations for most file systems related to hard drives i.e. merging or reading
On 12/29/19 12:02 PM, D. Hugh Redelmeier via talk wrote: per block don't matter in a SSD. Brtfs just turns off optimizations and there was discussion about direct IO being the default for very fast SSDs actually. If your talking about writes, again that is just trim or talking to firmware and does not matter unless your in the embedded world with cheap NAND or data centers. Most consumer SSDs do over 1000TBW so its not an issue for most people. Nick
The main point of Advanced Formatting was to increase density: each sector has overhead on the disk platter and so reducing the number of sectors reduced the overhead.
A second reason was that sector addressing of some kind was limited to 2T / 512 or 16T / 4096, so Advanced Formatting allowed a larger capacity. At least for MBR partitioning. But we mostly use GPT partitioning now, eliminating that problem (I think).
Does your computer use "legacy" booting or EFI? --- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
Hi Evan, As other on the thread have zero'ing in on, it is likely an issue with whatever USB/SATA adapter your using. This was a known limitation of early chipsets, and I recall the marketing shift to 'supports larger then 2TB!'. I looked at the hardware info you attached, and it doesn't go into detail about the usb devices. Can you send along the output of a 'lsusb' command while your USB/SATA adapter is attached? We should be able to back search the chipset based on the Vendor/Device IDs. On 12/29/19 12:05 AM, Evan Leibovitch via talk wrote:
It's not an Xmas present as it was bought a few weeks ago, but I'm trying to install a 10TB WD (Red Pro) disk and it's not exactly going to plan.
My original plan was to put the disk into an external case, plug it in to format and copy over files, then install it in the case.
The system already has an SSD as /dec/sda, and two disks one of 3TB and the other of 4TB, all working fine.
When I attach the external drive, lsblock and fdisk -l both report TWO new drives, /dev/sdd and /dev/sde, both of 2TB size. I went into set the disk from DOS to GPT but the size didn't change;
Now fdisk -l reports:
Disk /dev/sdd: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: 49FDD63D-6C95-46F0-B7B1-650038D8FB9B
Disk /dev/sde: 2 TiB, 2199023255040 bytes, 4294967295 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes
And cat /proc/scsi/scsi reports:
Host: scsi10 Channel: 00 Id: 00 Lun: 00 Vendor: WDC WD10 Model: 1KFBX-68R56N0 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04 Host: scsi10 Channel: 00 Id: 00 Lun: 01 Vendor: WDC WD10 Model: 1KFBX-68R56NLUN1 Rev: 0200 Type: Direct-Access ANSI SCSI revision: 04
The disk model is WD101KFBX and its specs can be found here. <https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-red-pro-hdd/data-sheet-western-digital-wd-red-pro-hdd-2879-800022.pdf>All of the online guides I've consulted say that the full capacity of the drive should be visible once I switch it to GPT mode.
Is this a USB limitation? Or is there something else I need to change? I've attached the output of hwinfo if that's any help.
Thanks for any suggestions!
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
--- Post to this mailing list talk@gtalug.org Unsubscribe from this mailing list https://gtalug.org/mailman/listinfo/talk
-- Scott Sullivan
Hi Scott (and everyone else who replied), As other on the thread have zero'ing in on, it is likely an issue with whatever
USB/SATA adapter your using. This was a known limitation of early chipsets, and I recall the marketing shift to 'supports larger then 2TB!'.
Pretty sure that's it. As I look around at what's available, even finding one now that supports 10TB is not so easy, many top out at 8. Since I can't yet put the drive in the chassis I have this enclosure <https://www.amazon.ca/ORICO-Screw-Less-3-5-inch-External-Enclosure/dp/B00GAML7OK/ref=sr_1_3> on order which will be useful later anyway. Thanks for the feedback. Anyone want my existing enclosure? I can bring it to the next meeting. You all know its limitations, but can't beat the price, -- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
Although I don't have recent testing experince, after the whole 2TB 'thing', the marketing became about the largest drives tested at time of release. Someone else can look into this further but I think it was not handling Advanced Format Drives (4k sectors) that was the crux of limitation. On December 30, 2019 8:13:57 a.m. EST, Evan Leibovitch <evan@telly.org> wrote:
Hi Scott (and everyone else who replied),
As other on the thread have zero'ing in on, it is likely an issue with whatever
USB/SATA adapter your using. This was a known limitation of early chipsets, and I recall the marketing shift to 'supports larger then 2TB!'.
Pretty sure that's it. As I look around at what's available, even finding one now that supports 10TB is not so easy, many top out at 8. Since I can't yet put the drive in the chassis I have this enclosure <https://www.amazon.ca/ORICO-Screw-Less-3-5-inch-External-Enclosure/dp/B00GAML7OK/ref=sr_1_3> on order which will be useful later anyway. Thanks for the feedback.
Anyone want my existing enclosure? I can bring it to the next meeting. You all know its limitations, but can't beat the price,
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
-- Scott Sullivan
Resolved. $20 and a drive to CC (closest to me is the one near Vaughan Mills) for the new enclosure solved the problem. All now looks as it should. It was a bit of a zoo there, but obviously worth it. And the new enclosure doesn't need screws or tools, which is a bonus. Thanks to all who helped. As mentioned, the old case is available to anyone who wants it (and can pick it up at the next meeting). - Evan On Mon, 30 Dec 2019 at 08:35, Scott Sullivan <scott@revident.net> wrote:
Although I don't have recent testing experince, after the whole 2TB 'thing', the marketing became about the largest drives tested at time of release.
Someone else can look into this further but I think it was not handling Advanced Format Drives (4k sectors) that was the crux of limitation.
On December 30, 2019 8:13:57 a.m. EST, Evan Leibovitch <evan@telly.org> wrote:
Hi Scott (and everyone else who replied),
As other on the thread have zero'ing in on, it is likely an issue with whatever
USB/SATA adapter your using. This was a known limitation of early chipsets, and I recall the marketing shift to 'supports larger then 2TB!'.
Pretty sure that's it. As I look around at what's available, even finding one now that supports 10TB is not so easy, many top out at 8. Since I can't yet put the drive in the chassis I have this enclosure <https://www.amazon.ca/ORICO-Screw-Less-3-5-inch-External-Enclosure/dp/B00GAML7OK/ref=sr_1_3> on order which will be useful later anyway. Thanks for the feedback.
Anyone want my existing enclosure? I can bring it to the next meeting. You all know its limitations, but can't beat the price,
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
-- Scott Sullivan
-- Evan Leibovitch, Toronto Canada @evanleibovitch or @el56
participants (7)
-
D. Hugh Redelmeier -
Evan Leibovitch -
Howard Gibson -
James Knott -
Nicholas Krause -
Scott Sullivan -
William Park