Running a local AI model for DBA work – SQL Server (Azure) DBA

Cloud AI assistants are great, but as a DBA I am always a bit careful with what leaves my machine. Query text, table names, error logs: it adds up. So I wanted to find out: can I run a decent coding model completely on my own PC, use it inside VS Code, and let it talk to my SQL Server?

Short answer: yes. And it’s faster than I expected. Here’s how I set it up, what it costs in hardware, and the one setting that made it almost four times faster.

My test machine

Part	Spec
CPU	AMD Threadripper PRO 5995WX (64 cores)
RAM	256 GB
GPU	NVIDIA RTX 3080, 10 GB VRAM
SQL Server	SQL Server 2025 Developer Edition
Editor	VS Code

The interesting part is the GPU. 10 GB of video memory is modest for AI work: the bigger models simply don’t fit. What saves the day is a lot of system RAM, plus a type of model called Mixture of Experts (more on that below).

The building blocks

Ollama: runs the models locally and exposes them on http://localhost:11434.
Qwen coder models: open-weight coding models from Alibaba’s Qwen team.
Continue: an open-source VS Code extension for chat, inline edits, autocomplete and an agent mode that can use tools. It talks to Ollama directly, with no account needed.
A small MCP server I wrote myself, so the model can query SQL Server in a controlled way.

Choosing the models

I ended up with one model per job:

Job	Model	Size
Chat / agent (default)	`qwen3-coder-next`	51 GB
Chat (backup)	`qwen3-coder:30b`	18 GB
Tab autocomplete	`qwen2.5-coder:1.5b`	1 GB
Code search embeddings	`nomic-embed-text`	0.3 GB

Both big models are Mixture of Experts (MoE) models. Qwen3-Coder-Next has 80 billion parameters, but only about 3 billion are active for each token it generates. That is why a model that is five times bigger than my GPU memory still runs at a usable speed: most of it sits in RAM and the CPU does the heavy lifting.

Autocomplete is a different story: it has to respond while you type, so it needs a tiny model that fits entirely on the GPU.

ollama pull qwen3-coder-next
ollama pull qwen2.5-coder:1.5b
ollama pull nomic-embed-text

Tip: I keep my models on a separate drive by setting the user environment variable OLLAMA_MODELS (for example E:\ollama\models) before pulling. They’re big.

The context-length trap (4× faster)

My first test with qwen3-coder:30b was disappointing: 9.7 tokens per second, and a long wait before the first word. ollama ps showed why:

NAME               SIZE     PROCESSOR          CONTEXT
qwen3-coder:30b    32 GB    74%/26% CPU/GPU    131072

The Ollama desktop app had set the context window to 131,072 tokens by default. The memory needed for that context (the KV cache) inflated an 18 GB model to 32 GB, pushing even more of it off the GPU.

For coding help in an editor, 32k tokens of context is plenty. After limiting it:

Model	Context	Memory	CPU/GPU	Speed
qwen3-coder:30b	131k (default)	32 GB	74/26	9.7 tok/s
qwen3-coder:30b	32k	22 GB	62/38	37 tok/s
qwen3-coder-next	32k	53 GB	85/15	33–35 tok/s
qwen2.5-coder:1.5b	32k	2 GB	100% GPU	~150 ms per completion

Note: in my case the OLLAMA_CONTEXT_LENGTH environment variable was overridden by the Ollama app’s own “Context length” setting. Either change it in the app’s settings, or set it per model in the client, which is what I did in Continue (below). Always check the CONTEXT column in ollama ps.

The big surprise: Qwen3-Coder-Next is almost as fast as the 30B model while being nearly three times larger, and noticeably better. I asked both for a query listing all databases with their data and log size. The 30B model’s first attempt used sys.database_files (current database only), while Next went straight to sys.master_files, with decimal math so small files don’t round down to 0 MB.

Wiring it into VS Code with Continue

Install the extension:

code --install-extension Continue.continue

Then create %USERPROFILE%\.continue\config.yaml:

name: Local Ollama
version: 1.0.0
schema: v1

models:
  - name: Qwen3 Coder Next          # first chat model = default
    provider: ollama
    model: qwen3-coder-next:latest
    roles: [chat, edit, apply]
    capabilities: [tool_use]
    defaultCompletionOptions:
      contextLength: 32768
  - name: Qwen3 Coder 30B
    provider: ollama
    model: qwen3-coder:30b
    roles: [chat, edit, apply]
    defaultCompletionOptions:
      contextLength: 32768
  - name: Qwen2.5 Coder 1.5B
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles: [autocomplete]
  - name: Nomic Embed
    provider: ollama
    model: nomic-embed-text
    roles: [embed]

Two things that tripped me up:

The Continue panel didn’t show up after installing. The command palette and View → Open View found nothing. A keyboard shortcut to the view container fixed it. Add this to your keybindings.json:{ "key": "ctrl+alt+c", "command": "workbench.view.extension.continue" }
Continue greets you with an “API Key” dialog for OpenAI, Anthropic and Gemini. Just close it: those are cloud providers, and your local models are already configured.

Also worth doing: set "continue.telemetryEnabled": false in your VS Code settings if you want everything to stay local.

Letting the model talk to SQL Server

In Continue’s Agent mode the model can run terminal commands, after you approve each one. So you can already ask it to “list all databases using sqlcmd” and it will propose something like:

sqlcmd -S localhost -E -Q "SELECT name FROM sys.databases"

That works, but it runs as me, and I’m sysadmin. One wrong suggestion, one careless click on Run… I wanted something more controlled, so I wrote a small MCP server (Model Context Protocol, the standard way to give AI models tools).

Security first: let SQL Server enforce the rules

The golden rule: don’t rely on the AI or the tool to behave. Let SQL Server enforce the limits. The MCP server connects with a dedicated SQL login that can read everything but only change data, and only in user databases:

CREATE LOGIN mcp_reader WITH PASSWORD = N'<generated>', CHECK_POLICY = ON;

-- Read: every database, every user object, DMVs and definitions
GRANT CONNECT ANY DATABASE TO mcp_reader;
GRANT SELECT ALL USER SECURABLES TO mcp_reader;
ALTER SERVER ROLE [##MS_ServerStateReader##] ADD MEMBER mcp_reader;
ALTER SERVER ROLE [##MS_DefinitionReader##]  ADD MEMBER mcp_reader;

-- Write data, per user database (repeat for each one)
USE [MyDatabase];
CREATE USER mcp_reader FOR LOGIN mcp_reader;
ALTER ROLE db_datawriter ADD MEMBER mcp_reader;

CONNECT ANY DATABASE and SELECT ALL USER SECURABLES are server-level permissions (since SQL Server 2014). They’re perfect for this: read access everywhere, including databases created later, without touching every database. The new fixed server roles in SQL Server 2022+ (##MS_ServerStateReader## and friends) cover the DMVs.

A setup script generates a random password and stores it in Windows Credential Manager (via the Python keyring package). It never appears in a config file or on screen.

Then I tested what the login can’t do:

Attempt	Result
`CREATE TABLE` in a user database	❌ CREATE TABLE permission denied
`CREATE LOGIN`	❌ User does not have permission
`ALTER ROLE db_owner ADD MEMBER mcp_reader`	❌ Cannot alter the role
`INSERT` in `master`	❌ no permission
`INSERT` in a user database	✅ allowed

The MCP server

About 130 lines of Python using the official mcp package and pyodbc. Five tools:

Tool	What it does
`list_databases`	name, state, recovery model, size
`list_tables`	tables with row counts
`describe_table`	columns and types
`run_query`	read query, max 100 rows, always rolled back
`execute_statement`	INSERT/UPDATE/DELETE, committed

Splitting reading and writing into separate tools matters: in Continue you can set the read tools to run automatically and keep execute_statement on Ask First. Even a “read” query runs inside a transaction that is always rolled back, so a sneaky UPDATE in a SELECT tool changes nothing.

The core of a tool looks like this:

@mcp.tool(annotations=ToolAnnotations(readOnlyHint=True))
def run_query(database: str, sql: str) -> str:
    """Run a read-only query (SELECT) and return at most 100 rows."""
    return fetch(database, sql)   # executes, reads rows, then always rolls back

Registering it in Continue is a few lines in the same config.yaml:

mcpServers:
  - name: SQL Server
    command: python
    args:
      - C:\Projecten\sql-mcp\server.py

Then, in Agent mode: “Which are the 5 largest tables in StackOverflow2013?” The model calls list_tables, sorts, and answers, all on my own machine.

Lessons learned

Check ollama ps. The CONTEXT and PROCESSOR columns explain almost every “why is it slow” question.
MoE models plus lots of RAM beat a big GPU for this kind of work. My 10 GB card handles autocomplete; the 256 GB of RAM carries the big model.
Local models are good, not magic. Treat their SQL like a junior colleague’s: read it before you run it. The 30B model’s first database-size query was simply wrong.
Security belongs in the database, not in the prompt or the tool. A dedicated login with exactly the right permissions is the real safety net.
Keep your cloud assistant too. For serious work I still use Claude Code next to it. The local setup is for experiments, offline work, and anything that shouldn’t leave the building.

What’s next

I want to try the agent on real DBA chores: summarizing error logs, reviewing the output of my monitoring database, and drafting index suggestions for the StackOverflow databases. More on that in a next post.

Questions or your own experiences with local models? Let me know!