Sometimes I get annoyed at binaries for which I don't have any code. Stepping back a bit, I get annoyed at undocumented hardware for which the only drivers are closed-source, or even worse, Windows-only or Android-only. One way to pry these open is through reverse engineering tools What prompts this message is that the NSA has just released Ghindra as open source. <https://www.nsa.gov/resources/everyone/ghidra/> (This would be perfect code for the NSA to plant a trojan horse in.) Previously (and maybe still) the tool that seemed to be most popular was IDA, a commercial product that runs on Windows. There are some other choices: <https://reverseengineering.stackexchange.com/questions/1817/is-there-any-disassembler-to-rival-ida-pro#1821> ==== War story (feel free to ignore): ==== I put some effort into this in the early 1980s. I wrote my own disassembler that made some inferences from control flow. I disassembled two non-trivial bits of code: - the undocumented monitor that resided in my Altair's "Bytesaver" (EPROM reader and writer). The software was originally written for the Processsor Tech Sol 20 machine. - the proprietary but buggy firmware in my Volker-Craig VC2100 terminal (meant to compete with the VT100) These were a lot of work and they were only 1k and 16k bytes respectively. My reversing was complete: I ended up with useful source for both. With information gained from the PT monitor, I was able to write my own, much more ambitious monitor. I found bugs and identified fixes for the VC2100 and reported them to the author of the code at Volker-Craig. But the product was cancelled before the fixes were released. I gave up on one reverse engineering task. I bought a copy of MuMath, a symbolic algebra package, that was for the Osborne I. I tried to run it on my Kaypro II. It seemed that there was some tricky copy-protection code involving bank-switching. I never cracked that puzzle. ==== End of war story ==== I would love to see better reverse engineering for a few things. - my Lenovo Yoga notebook gets a machine whenever it goes to sleep and back. Sleep/awake still works, but the machine checks probably slow things down and they make me unhappy. - nvidia video cards are not well-enough supported by nouveau. nvidia has not disclosed enough about the hardware to fix this. - Many GPUs used with ARM are not supported with open-source Linux drivers. There are a number of reverse engineering projects but progress isn't fast enough for me. My guess is that these projects are too hard since the size of object code to be analyzed is several orders of magnitude more than what I've tried.
On 2019-03-09 6:06 p.m., D. Hugh Redelmeier via talk wrote:
Sometimes I get annoyed at binaries for which I don't have any code. [snip] One way to pry these open is through reverse engineering tools
What prompts this message is that the NSA has just released Ghindra as open source. [snip] There are some other choices: <https://reverseengineering.stackexchange.com/questions/1817/is-there-any-disassembler-to-rival-ida-pro#1821>
I found myself working on a project where the source code was lost. The project uses a NanoPi that uses an Allwinner chip which is a 64-bit Aarch (ARM v8) architecture. I've tried several decompilers but they didn't work, or wouldn't even compile. I looked at boomerang, Hopper, snowman, retdec, and reko. The ones that work wouldn't handle the architecture. The ghidra decompiler has been the only one that has proven useful. I haven't read enough about it to know how to save source code out of it but I can browse decompiled output which is at least the first step in reconstituting source code. -- Cheers! Kevin. http://www.ve3syb.ca/ | "Nerds make the shiny things that https://www.patreon.com/KevinCozens | distract the mouth-breathers, and | that's why we're powerful" Owner of Elecraft K2 #2172 | #include <disclaimer/favourite> | --Chris Hardwick
On 2019-03-14 3:40 p.m., Kevin Cozens via talk wrote:
I've tried several decompilers but they didn't work, or wouldn't even compile. I looked at boomerang, Hopper, snowman, retdec, and reko. The ones that work wouldn't handle the architecture. The ghidra decompiler has been the only one that has proven useful. I haven't read enough about it to know how to save source code out of it but I can browse decompiled output which is at least the first step in reconstituting source code.
In a previous life, I wrote a z80 decompiler that used printf to format the output. After a little whille futzing with assembler syntax, I changed it to produce output like 1132 *hl ?= '\n' 1133 if !== goto 1135 1134 hl++ 1135 return which made it easy for me to create c programs that did the same thing, modulo bugs. if (*hl == '\n') { hl++ } return --dave -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest davecb@spamcop.net | -- Mark Twain
On 2019-03-14 5:28 p.m., David Collier-Brown via talk wrote:
In a previous life, I wrote a z80 decompiler that used printf to format the output.
Very nice. The ghidra decompiler includes support for many different processors. The list includes the Z-80. Sadly, the 6809 (my personal favourite) and the 6800 processors are not on the list. -- Cheers! Kevin. http://www.ve3syb.ca/ | "Nerds make the shiny things that https://www.patreon.com/KevinCozens | distract the mouth-breathers, and | that's why we're powerful" Owner of Elecraft K2 #2172 | #include <disclaimer/favourite> | --Chris Hardwick
| From: Kevin Cozens via talk <talk@gtalug.org> | Very nice. The ghidra decompiler includes support for many different | processors. The list includes the Z-80. Sadly, the 6809 (my personal | favourite) and the 6800 processors are not on the list. What do you wish to reverse for the 6800 or 6809? I cannot imagine that there is that much of interest at this late date. I'm sure that it would be easy to add the 6800. The 6809 is a bit more intricate but not that much harder. (I reverse engineered some 680x (x<9) code using a disassembler I wrote. I'm not sure that I could find my disassembler (almost 40 years later). It's probably stranded on one of my 9-track tapes.)
On 2019-03-19 9:48 p.m., D. Hugh Redelmeier via talk wrote:
| From: Kevin Cozens via talk <talk@gtalug.org>
| Very nice. The ghidra decompiler includes support for many different | processors. The list includes the Z-80. Sadly, the 6809 (my personal | favourite) and the 6800 processors are not on the list.
What do you wish to reverse for the 6800 or 6809? I cannot imagine that there is that much of interest at this late date.
It was just something I noticed. I was partly surprised that ghidra included some older processors in their list of ones for which they can decompile code. I'm currently decompiling some code but it is for an ARM processor. I recently did some work related to a 6800 processor but that was just a disassembly of a ROM image to save me from re-typing in a long assembly language program. -- Cheers! Kevin. http://www.ve3syb.ca/ | "Nerds make the shiny things that https://www.patreon.com/KevinCozens | distract the mouth-breathers, and | that's why we're powerful" Owner of Elecraft K2 #2172 | #include <disclaimer/favourite> | --Chris Hardwick
On Fri., Mar. 29, 2019, 11:28 Kevin Cozens via talk, <talk@gtalug.org> wrote:
It was just something I noticed. I was partly surprised that ghidra included some older processors in their list of ones for which they can decompile code.
Z80 will still show up in industrial control things. It seems very hard to kill. At least we might have seen the end of 1802s in real applications. It is still popular for some hobbyists, though. Stewart
Zylog still builds z80s, albeit in India --dave On 2019-03-29 3:51 p.m., Stewart Russell via talk wrote: On Fri., Mar. 29, 2019, 11:28 Kevin Cozens via talk, <talk@gtalug.org<mailto:talk@gtalug.org>> wrote: It was just something I noticed. I was partly surprised that ghidra included some older processors in their list of ones for which they can decompile code. Z80 will still show up in industrial control things. It seems very hard to kill. At least we might have seen the end of 1802s in real applications. It is still popular for some hobbyists, though. Stewart --- Talk Mailing List talk@gtalug.org<mailto:talk@gtalug.org> https://gtalug.org/mailman/listinfo/talk -- David Collier-Brown, | Always do right. This will gratify System Programmer and Author | some people and astonish the rest dave.collier-brown@indexexchange.com<mailto:dave.collier-brown@indexexchange.com> | -- Mark Twain CONFIDENTIALITY NOTICE AND DISCLAIMER : This telecommunication, including any and all attachments, contains confidential information intended only for the person(s) to whom it is addressed. Any dissemination, distribution, copying or disclosure is strictly prohibited and is not a waiver of confidentiality. If you have received this telecommunication in error, please notify the sender immediately by return electronic mail and delete the message from your inbox and deleted items folders. This telecommunication does not constitute an express or implied agreement to conduct transactions by electronic means, nor does it constitute a contract offer, a contract amendment or an acceptance of a contract offer. Contract terms contained in this telecommunication are subject to legal review and the completion of formal documentation and are not binding until same is confirmed in writing and has been signed by an authorized signatory.
On 03/29/2019 08:30 PM, Dave Collier-Brown via talk wrote:
Zylog still builds z80s, albeit in India --dave
On 2019-03-29 3:51 p.m., Stewart Russell via talk wrote:
On Fri., Mar. 29, 2019, 11:28 Kevin Cozens via talk, <talk@gtalug.org <mailto:talk@gtalug.org>> wrote:
It was just something I noticed. I was partly surprised that ghidra included some older processors in their list of ones for which they can decompile code.
Z80 will still show up in industrial control things. It seems very hard to kill.
At least we might have seen the end of 1802s in real applications. It is still popular for some hobbyists, though.
When designers design custom chips, they rely on logic libraries, which provide common functions, including CPUs. So the designer would choose a CPU, add memory and I/O and "compile" the new chip. The libraries contain many historic CPUs, as they provide all the function and performance needed. If the job can be done with a 4 bit CPU, such as a 4040, there's no need for a 64 bit CPU. One factor that's critical with chip design, is "real estate". The smaller you can make it, the cheaper it will be and provide higher yield. Several years ago, I used to repair point of sale terminals that had a custom chip built around a Z80 core.
| From: James Knott via talk <talk@gtalug.org> | When designers design custom chips, they rely on logic libraries, which | provide common functions, including CPUs. So the designer would choose | a CPU, add memory and I/O and "compile" the new chip. The libraries | contain many historic CPUs, as they provide all the function and | performance needed. If the job can be done with a 4 bit CPU, such as a | 4040, there's no need for a 64 bit CPU. One factor that's critical with | chip design, is "real estate". The smaller you can make it, the cheaper | it will be and provide higher yield. Several years ago, I used to | repair point of sale terminals that had a custom chip built around a Z80 | core. I'm not in that world, so I'm just what I'm about to say isn't reliable. The main costs of a core within a larger chip are probably: - manufacturing costs + the larger the area, the worse + the more interconnects, especially off-chip, the worse + the more advanced the manufacturing technology, the worse (partly determined by clock speed) - engineering costs - designing the chip - building the software - licensing costs According the Raspberry Pi designers, the original Pi's ARM core was a miniscule part of the die. So small that it added essentially nothing to the manufacturing cost. And using such an old core meant that the licensing fees were small too. If a 32-bit core reduces the software costs, it probably makes sense. If you already have 4-bit or 8-bit software which already does most of the job, or if you have engineers who already have deep skills only with those old processors, that might justify using old cores. As a programmer who has dealt with 8-bit, 16-bit, 32-bit, and 64-bit processors, I can tell you that each step helped. The only negative of the larger systems is that they invited software bloat, and that bloat really could be a drag on productivity. OK, there is also hardware bloat: there usually were more complex mechanisms to actually get to the pins. I would guess that ARM, RISC V, and MIPS each have small, cheap, and useful 32-bit cores for most applications. They can live with 16-bit buses, and maybe even 8-bit ones. Space-hardening is another matter. I don't think that the space market has been able to drive processor development so most projects now use select COTS (common off-the-shelf) parts, try to shield them, and add redundancy. They often use very old parts since the projects' design cycles are so very long. Using COTS is meant to reduce the cycle time. ==== The 8080 and z80 had ugly architectures. They had an excuse: they took the design of the 4004 and 8008 and kicked it forward. Clean-sheet designs could have been much nicer. But the market didn't demand that. What made the z80 a success is that the chip got rid of some ugliness dealing with the i8080: the two phase clock, if I remember correctly. Interrupts were easier to wire. And it included some other things that were external in the i8080. So two or more (expensive) chips became one in most designs. The i8085 had most of those advantages too. The z80 added some instructions to the i8080's set. As an assembler programmer, I didn't find them compelling. I avoided them so that my code would be more portable. This cost more lines of code but not speed or code space. Some of us cut our teeth programming on these things. Usually in assembly language. Many of us fall in love with the first system we deeply understood. So there is a generation of defenders of the z80 (and the 6502). This is nostalgia: they are indefensible for new designs.
On 03/30/2019 01:16 PM, D. Hugh Redelmeier via talk wrote:
If you already have 4-bit or 8-bit software which already does most of the job, or if you have engineers who already have deep skills only with those old processors, that might justify using old cores.
As a programmer who has dealt with 8-bit, 16-bit, 32-bit, and 64-bit processors, I can tell you that each step helped. The only negative of the larger systems is that they invited software bloat, and that bloat really could be a drag on productivity. OK, there is also hardware bloat: there usually were more complex mechanisms to actually get to the pins.
When you're cranking out millions of chips, real estate is a significant factor. With custom chips, the number of CPU pins is independent of how many leads leave the package. The chip is just a black box, with whatever CPU in it. There are still things for which 4 bits is sufficient.
Some of us cut our teeth programming on these things. Usually in assembly language. Many of us fall in love with the first system we deeply understood. So there is a generation of defenders of the z80 (and the 6502). This is nostalgia: they are indefensible for new designs.
My first CPU was the 8080, in my IMSAI 8080. I used to code directly into octal, on square ruled paper. ;) I then moved my coding skills to Data General Nova 800, followed by 6502 & 6809, the latter two for courses at Ryerson, the Nova at work.
On 2019-03-29 3:51 p.m., Stewart Russell via talk wrote:
At least we might have seen the end of 1802s in real applications. It is still popular for some hobbyists, though.Â
Don't be too sure about that. It used to be used in devices going in to space. I just read that it is still being made by Intersil with mil specs available in batch quantities only at ~$134 a pop. -- Cheers! Kevin. http://www.ve3syb.ca/ | "Nerds make the shiny things that https://www.patreon.com/KevinCozens | distract the mouth-breathers, and | that's why we're powerful" Owner of Elecraft K2 #2172 | #include <disclaimer/favourite> | --Chris Hardwick
On 03/29/2019 10:03 PM, Kevin Cozens via talk wrote:
On 2019-03-29 3:51 p.m., Stewart Russell via talk wrote:
At least we might have seen the end of 1802s in real applications. It is still popular for some hobbyists, though.Â
Don't be too sure about that. It used to be used in devices going in to space. I just read that it is still being made by Intersil with mil specs available in batch quantities only at ~$134 a pop.
The reason it went into space was because the technology used to build it was more tolerant of radiation than most other designs. The logic design had nothing to do with it. It would have been entirely possible to build Z80, 6800 or 6502 CPUs with the same tech to be used in space.
participants (6)
-
D. Hugh Redelmeier -
Dave Collier-Brown -
David Collier-Brown -
James Knott -
Kevin Cozens -
Stewart Russell