Making an RPG Conversation System with Local LLMs

Table of Contents

Introduction
#

Hello everyone!

Today, I was planning to fix a bug in the generative AI feature I wrote the other day… But no.

This is it.

10 July 2026·197 words

Game Dev Godot

However, it seems my XAI free credits have been reclaimed.

I used to have a quota of $150 worth of tokens per month!

19 February 2025·173 words

Technical Note AI

So, I decided to switch to a local LLM.

I first tried Docker Model Runner, but it’s not very usable.

Furthermore, since I am using WSL, I need to do port forwarding like this.

netsh interface portproxy add v4tov4 listenport=12434 listenaddress=0.0.0.0 connectport=12434 connectaddress=127.0.0.1

For some reason, I cannot download other models either.

This error appears:

Failure

Failed to pull model: pulling ai/granite4.2 failed with status 500 Internal Server Error: error while pulling model: reading model from registry: failed to pull model “ai/granite4.2”: UNKNOWN - GET https://index.docker.io/v2/ai/granite4.2/manifests/latest: TOOMANYREQUESTS: You have reached your unauthenticated pull rate limit. https://www.docker.com/increase-rate-limit

When I asked the AI, it seems I was banned due to some limitations.

Feeling that Docker Model Runner was a bit cumbersome, I switched to Ollama & Open WebUI.

Since I only have a 4060ti with 8GB, I initially used gemma4:e2b-it-q4_K_M, but even 14.8B (a bit slow~) works quite well.

Then, I generated an API Key and easily connected it to Godot.

And this is the final result!