GPU-for-AI advice from the cheap seats (I've never done this). AI stacks seem very fragile. Any time you make a substitution the whole thing might misbehave. You can either copy EXACTLY a working configuration from someone else or you can sign up for adventure. Most likely you will try some of both. NVidia cards come in at least three flavours: those for AI, those for pros (running things like autocad), and those for gamers. They try hard to segment the market so they can charge as much as possible for the segment. Serious AI cards cost up to US$20,000. Maybe more. How do they keep gamer cards from being great AI cards? I don't know all the techniques, but they include: - crippling the speed of 64-bit floating point - limiting the amount of on-board memory (pretty important) - (I think) limiting the ability to partition the card into virtual cards for multiple processes to run in parallel safely. - limiting how GPUs can scale up (interconnect) This is on top of the crippling they did to hobble cryptocurrency mining. I don't know what that amounted to technically. Advice for buying a card: - how much do you want to spend? Not enough! - you probably want as much on-board RAM as possible because swapping stuff between the computer's RAM and the GPU's RAM is apparently a serious bottleneck. I don't have a cost/benefit curve. I'm pretty sure that it is like real RAM: performance falls off a cliff when you don't have enough. - fans probably matter because (1) good cooling should be quieter than bad cooling (not in data-centre cards: nobody cares about noise there), and (2) without good cooling, throttling will happen. Read good review sites to get opinions about card cooling issues. - Guess: the version of PCIe used might matter. We're in a period of transition. (Remember: the motherboard and the GPU have to both support the PCIe version you target.) - if I were at all adventurous, I'd look at Intel graphics cards. They are probably a bargain on a per teraflop basis. Intel tries hard to push AI on Linux. They are pretty good open-source players and seem more competent than AMD. - my impression is that AMD for AI on Linux smells a bit like a lost cause: they care about their commercial GPU-compute customers but not us. ROCm only really seems to work on their industrial GPU cards. - the latest gen of NVidia is apparently not much a step up from the previous one. Check the performance, not just the model number. Fun fact: Mac folks never cease to brag about how great Unified Memory is on the M1 etc. That seems silly when you realize that integrated GPUs have always had this. But there is apparently a case where the performance benefit is great: - tonnes of memory on the Mac (very expensive), more than you can get on an NVidia gamer GPU - working with a model that doesn't fit in the NVidia GPU but does fit in the Mac's RAM. The Mac's RAM has very fast access from both the GPU and the CPU. Much faster than the PCIe bus. In fact, much faster than an X86 can access bulk RAM. Phoronix.com seems to focus on Linux and GPUs. https://ca.pcpartpicker.com/ might be helpful finding deals. Where to buy? Check out Amazon (easy returns; sometimes good prices) Canada Computers (sometimes bad customer service but I've not really encountered this) Best Buy (sometimes) NewEgg (sometimes) Memory Express