Ever wanted to chat to your GPU? NVIDIA's new RTX-powered AI chatbot is fast, local and private

NVIDIA's going big on AI - in case you hadn't noticed - and the latest brainwave from Team Green is a new 'Chat with RTX' demo app.

This software allows you to leverage the power of your RTX graphics card - only RTX 3000 and 4000 GPUs are supported, mind - to run a local chatbot tapping into a large language model (the choices are Mistral and Llama, open source efforts of course).

Then you can use your preferred option, hook it up to some files on your PC, and query this local generative AI, letting it comb through those files for answers to queries.

So, as the examples in the video clip above illustrate, if you've forgotten, say, the date of a job interview, if that info is on a file somewhere on your PC, the AI will root around in your data and find it.

The big attractions here are the speed of a local response, powered by your heavyweight NVIDIA GPU - as opposed to waiting for an answer to float back down from the cloud - and data privacy. As NVIDIA underlines, your private data stays on your PC. You don't even need to be online to run queries, of course.

RAG + Tensor magic

The app is free, but still in an early stage as the demo label makes clear. It uses NVIDIA's RAG (Retrieval Augmented Generated) software in conjunction with the GPU's Tensor cores to make the local query magic work.

You can hook up Chat with RTX to all manner of text files (plain text, as well as Word documents, PDF, and XML formats), and it's also possible to plug YouTube videos into the app. It can then be used to search those video clips for relevant info via queries, which is pretty nifty - all of this happening locally on your system.

As mentioned, the catch is you need an RTX graphics card, and sadly older RTX 2000 (Turing) models aren't invited to the AI party. Also, the GPU must have at least 8GB of RAM, which rules out the likes of the RTX 3050 6GB that was recently introduced (another small black mark against this cut-down variant of the 3050).

Other hardware requirements include that the PC must have 16GB of system RAM, and sorry Windows 10 diehards, this is a Windows 11-only app. You'll also need NVIDIA's version 535.11 graphics driver or better.

You can grab 'Chat with RTX' now and it weighs in at a 35GB download. The colorful comments have already begun, including asking whether you can now get your RTX graphics card to respond with quick-witted comments directed back at gamers throwing shade at you during online matches.

• Read more: NVIDIA and AMD GPU frame generation can be combined for 3x faster FPS - but it's a silly idea
• Read more: NVIDIA's RTX 3060 could be staying around as a budget option to tackle AMD's rival low-end GPUs