NVIDIA and Streamlabs officially unveil powerful AI sidekicks for livestreamers

Streamlabs and NVIDIA have showcased a significant upgrade to the suite of features available within the livestreaming software, with the head of Streamlabs, Ashray Urs, giving us a live demo of the new AI-powered software working in action.

A partnership between Streamlabs, Inworld AI, and NVIDIA has created what is called the Streamlabs Intelligent Streaming Agent, a new digital human that is capable of assisting a livestreamer by carrying out commands in real-time. Ashray showed three sets of functionality with the new AI Agent: It being a "sidekick" to the streamer, which includes how a streamer can interact with the agent and leverage it being aware of what is happening in-game, the agent being used a stream producer, which includes it leveling up the overall production quality, and how the agent can be used as technical support source.

The above video shows Ashray asking the AI agent, "Where should I drop?" while in a simulated Fortnite game lobby. The AI responds with a location. Ashray then proceeds to showcase how a streamer can use the AI agent to poll the stream chat for an answer, which it will then scrape and provide as an audio answer. Another way streamers can use the AI agent as a sidekick is by having the agent ask the streamer questions when the streamer is quiet. The purpose of that feature is to help the streamer be entertaining/engaging when people are watching the stream.

2

Ashray goes on to show how the AI agent can recognize an important in-game moment, such as the player losing a match, and by simply asking the agent to replay that moment via a voice command the agent can showcase the captured video of that moment to the stream, incorporating the appropriate scene changes, transitions, audio sources, text overlays and more.

Additionally, the agent can also detect when a streamer is trying to talk to the stream but has their microphone muted, and depending on the personality setting, the agent will respond via audio and ask if the streamer wants their microphone unmuted. If the streamer agrees, the AI agent will go ahead and unmute the microphone within the StreamLabs software without the streamer having to manually do it.

The AI agent is asked, "What bitrate do I need to stream 4K 60FPS?" The AI agent thinks for a moment and then responds with the following, "Aim for 35,000 to 45,000 kilobits per second. But fingers crossed your upload speed survives."

Streamers can also change the AI agent's personality to: Dramatic, Sarcastic, Neutral, Chill, and Supportive.

I also asked NVIDIA and Ashray if the AI agent was able to share stream highlights to social media platforms, and both NVIDIA and Ashray said that is a feature that is currently being worked on, but doesn't have a release date yet. The Streamlabs Intelligent Streaming Agent is set for a full release later this year.