Alert
TT Show Episode 55 - Arrow Lake, GeForce RTX 5070, and Google's Pixel smartphone tracking

Ever wanted to chat to your GPU? NVIDIA's new RTX-powered AI chatbot is fast, local and private

NVIDIA's 'Chat with RTX' app is a new AI innovation: a personal chatbot on your PC, providing fast responses via Tensor cores, but it's Windows 11-only.

Ever wanted to chat to your GPU? NVIDIA's new RTX-powered AI chatbot is fast, local and private
Published
Updated
2 minutes & 15 seconds read time

NVIDIA's going big on AI - in case you hadn't noticed - and the latest brainwave from Team Green is a new 'Chat with RTX' demo app.

This software allows you to leverage the power of your RTX graphics card - only RTX 3000 and 4000 GPUs are supported, mind - to run a local chatbot tapping into a large language model (the choices are Mistral and Llama, open source efforts of course).

Then you can use your preferred option, hook it up to some files on your PC, and query this local generative AI, letting it comb through those files for answers to queries.

So, as the examples in the video clip above illustrate, if you've forgotten, say, the date of a job interview, if that info is on a file somewhere on your PC, the AI will root around in your data and find it.

The big attractions here are the speed of a local response, powered by your heavyweight NVIDIA GPU - as opposed to waiting for an answer to float back down from the cloud - and data privacy. As NVIDIA underlines, your private data stays on your PC. You don't even need to be online to run queries, of course.

RAG + Tensor magic

The app is free, but still in an early stage as the demo label makes clear. It uses NVIDIA's RAG (Retrieval Augmented Generated) software in conjunction with the GPU's Tensor cores to make the local query magic work.

You can hook up Chat with RTX to all manner of text files (plain text, as well as Word documents, PDF, and XML formats), and it's also possible to plug YouTube videos into the app. It can then be used to search those video clips for relevant info via queries, which is pretty nifty - all of this happening locally on your system.

As mentioned, the catch is you need an RTX graphics card, and sadly older RTX 2000 (Turing) models aren't invited to the AI party. Also, the GPU must have at least 8GB of RAM, which rules out the likes of the RTX 3050 6GB that was recently introduced (another small black mark against this cut-down variant of the 3050).

Other hardware requirements include that the PC must have 16GB of system RAM, and sorry Windows 10 diehards, this is a Windows 11-only app. You'll also need NVIDIA's version 535.11 graphics driver or better.

You can grab 'Chat with RTX' now and it weighs in at a 35GB download. The colorful comments have already begun, including asking whether you can now get your RTX graphics card to respond with quick-witted comments directed back at gamers throwing shade at you during online matches.

Photo of the product for sale

ASUS Dual GeForce RTX 4070 SUPER OC Edition

TodayYesterday7 days ago30 days ago
$899.95$629.99$899.95
-
--$651.44
* Prices last scanned on 10/13/2024 at 12:27 pm CDT - prices may not be accurate, click links above for the latest price. We may earn an affiliate commission from any sales.
NEWS SOURCES:nvidia.com, wccftech.com

Darren has written for numerous magazines and websites in the technology world for almost 30 years, including TechRadar, PC Gamer, Eurogamer, Computeractive, and many more. He worked on his first magazine (PC Home) long before Google and most of the rest of the web existed. In his spare time, he can be found gaming, going to the gym, and writing books (his debut novel – ‘I Know What You Did Last Supper’ – was published by Hachette UK in 2013).

Darren's Computer

Newsletter Subscription

Join the daily TweakTown Newsletter for a special insider look into new content and what is happening behind the scenes.

Related Topics

Newsletter Subscription