{"id":23,"date":"2026-09-25T22:20:57","date_gmt":"2026-09-25T16:50:57","guid":{"rendered":"https:\/\/truepythoneer.com\/blog\/?p=23"},"modified":"2026-09-26T00:11:52","modified_gmt":"2026-09-25T18:41:52","slug":"build-ai-agent-from-scratch-ollama-langgraph","status":"publish","type":"post","link":"https:\/\/truepythoneer.com\/blog\/build-ai-agent-from-scratch-ollama-langgraph\/","title":{"rendered":"I Built an AI Agent From Scratch, Then Rebuilt It in LangGraph. Here’s What Broke, and What the Framework Actually Replaced."},"content":{"rendered":"\n

Every \u201cbuild an AI agent\u201d tutorial today starts the same way: pip install langchain<\/code>, five lines of code, a working demo. It\u2019s fast, and it\u2019s honest about almost nothing. You get a working agent without ever seeing what \u201cagent\u201d actually means. <\/p>\n\n\n\n

I wanted to understand the mechanism, not just use it. So I built one from the ground up: raw HTTP calls to a local model, hand-written tool schemas, a hand-written agent loop, hand-written memory and retrieval. No framework, until I\u2019d built enough to know exactly what a framework would be replacing.<\/p>\n\n\n\n

Everything runs locally, on a laptop with an NVIDIA GPU, using Ollama and Qwen3. No cloud API. No data leaves the machine. For anyone working in a regulated industry, that last point matters more than it sounds.<\/p>\n\n\n\n

How to Build AI Agent From Scratch, Layer by Layer<\/h2>\n\n\n\n

\"Architecture<\/p>\n\n\n\n

The six parts and how they sit around the one loop that ties them together.<\/em><\/p>\n\n\n\n

Environment.<\/strong> WSL2, Ubuntu 24.04, Ollama, an 8B parameter model running on GPU. Getting GPU passthrough working correctly inside a Linux VM on Windows is its own small education in virtualization.<\/p>\n\n\n\n

LLM basics.<\/strong> Sending messages, streaming tokens, understanding what a \u201cthinking\u201d model actually costs you in latency (more tokens generated, not slower generation, an important distinction most people get wrong).<\/p>\n\n\n\n

Tool calling.<\/strong> The core trick behind every \u201cagentic\u201d system: the model doesn\u2019t execute anything. It asks. Your code decides whether, and how, to act. That boundary is also where safety lives in any real system.<\/p>\n\n\n\n

The agent loop.<\/strong> This is the actual definition of \u201cagentic AI\u201d: call the model, run what it asks for, feed the result back, repeat until it stops asking. That\u2019s it. Everything else is scaffolding on top of this one idea.<\/p>\n\n\n\n

Memory, RAG, and live web access.<\/strong> Three different kinds of \u201cknowing something,\u201d and the agent has to pick the right one for each question: a fact I told it directly, something in a document I gave it, or something on the live web. Getting the model to reliably choose the right source, once there were four competing tool families, turned out to be harder than getting any single one to work.<\/p>\n\n\n\n

What Actually Taught Me Something<\/h2>\n\n\n\n

Anyone can show you a working demo. What\u2019s harder to fake is what happened when things broke.<\/p>\n\n\n\n

The GPU that lied about its own speed.<\/strong> Early on, tool calls that should take two seconds were taking thirty. nvidia-smi<\/code> showed 99% GPU utilization, everything looked fine. The actual problem: the GPU was stuck in a low-power state after a driver update, running at a fraction of its memory clock while still reporting \u201cbusy.\u201d Full utilization and full speed are not the same thing, a lesson that generalizes well beyond GPUs.<\/p>\n\n\n\n

The model that confidently repeated its own mistake.<\/strong> I asked for the time in \u201cHyderabad\u201d (not a valid timezone name). The model guessed wrong, got an error back, and then told me its wrong guess was \u201cthe correct timezone,\u201d rather than admitting it didn\u2019t know. That\u2019s a small, contained example of a much bigger problem in deployed AI systems: models sound confident regardless of whether they\u2019re right. The fix wasn\u2019t \u201ctrust it less,\u201d it was engineering: better error messages that tell the model exactly what to do next, and a loop that gives it the chance to actually act on that correction.<\/p>\n\n\n\n

The bug that only showed up at scale.<\/strong> With one tool, tool selection is trivial. With two, it\u2019s still fine. With four tool families competing, general utilities, long-term memory, document search, and live web, the model started genuinely confusing \u201ca fact about me\u201d with \u201ca question about company policy.\u201d The fix was sharpening the system prompt to explicitly disambiguate the two. Nothing about any single tool was broken; the emergent behavior across all of them was.<\/p>\n\n\n\n

The hallucination that wasn\u2019t.<\/strong> I once suspected the model had invented a very specific, correct-sounding fact from nothing. I was ready to write it up as a cautionary tale about grounding. It turned out my own logging code was truncating the console output to keep it readable, the model had genuinely retrieved that fact; I just hadn\u2019t looked at the full data before jumping to a conclusion. The real lesson: don\u2019t diagnose an AI system from a summary of its output. Look at what it actually saw.<\/p>\n\n\n\n

Why This Matters Beyond the Exercise<\/h2>\n\n\n\n

For a fintech or any data-sensitive environment, the interesting result isn\u2019t \u201cAI agents are possible\u201d, everyone knows that. It\u2019s that a genuinely capable one can run entirely inside your own infrastructure<\/strong>: no data sent to a third party, no per-query API billing, and full visibility into every decision it makes, because you wrote the decision-making loop yourself.<\/p>\n\n\n\n

That last part is the real point. Frameworks are valuable once you know what they\u2019re doing for you. Used before that, they\u2019re a black box you\u2019re trusting without understanding, in a domain where \u201cwhy did the agent do that\u201d needs to have a real answer.<\/p>\n\n\n\n

\n\n\n\n

Part Two: I Rebuilt the Same Agent in LangGraph. Here\u2019s Exactly What It Replaced.<\/h2>\n\n\n\n

Once the hand-built version worked end to end, I did the obvious next thing: rebuilt it in LangGraph, one of the most widely used agent frameworks, and compared the two, piece by piece.<\/p>\n\n\n\n

The result surprised me a little. LangGraph didn\u2019t touch a single design decision I\u2019d made. It removed boilerplate, and only boilerplate.<\/p>\n\n\n\n

The Mapping<\/h3>\n\n\n\n
What I hand-wrote<\/th>What LangGraph replaced it with<\/th><\/tr><\/thead>
A while<\/code> loop calling the model, checking for tool calls, running them, and looping<\/td>Two nodes (\u201cagent,\u201d \u201ctools\u201d) and one line: add_conditional_edges(\"agent\", tools_condition)<\/code><\/td><\/tr>
A hand-written JSON schema for every tool function<\/td>Inferred automatically from the function\u2019s type hints and docstring, via a single @tool<\/code> decorator<\/td><\/tr>
A dispatcher dictionary mapping tool names to functions<\/td>ToolNode(ALL_TOOLS)<\/code> \u2014 built in<\/td><\/tr>
~30 lines of code to serialize conversation history to JSON and load it back on restart<\/td>One line: SqliteSaver.from_conn_string(\"sessions.db\")<\/code><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

That\u2019s it. That\u2019s the whole list.<\/p>\n\n\n\n

What Did NOT Change<\/h3>\n\n\n\n

My system prompt, unchanged. My tool functions, unchanged, imported directly with no modification. My conversation-trimming policy (keep the last 6 exchanges, summarize the rest), unchanged, ported over as its own graph node, because deciding how<\/em> to trim a conversation is a judgment call, not plumbing, and no framework should make that decision for you.<\/p>\n\n\n\n

Proving the Persistence Claim, Not Just Asserting It<\/h3>\n\n\n\n

The most concrete test: I closed the process entirely, opened a brand-new<\/strong> database connection against the same file, and asked the graph to recover its own state.<\/p>\n\n\n\n

It came back with the full conversation, every message, in the right order, with zero lines of save\/load code written by me. The hand-built version needed a custom serializer just to handle the fact that a model\u2019s response isn\u2019t naturally JSON-safe. The framework version needed nothing.<\/p>\n\n\n\n

\"agent<\/p>\n\n\n\n

Real terminal capture: the process exits (\/exit<\/code>), a fresh python3 main.py<\/code> run comes back with \u201cResuming thread \u2018default\u2019 (9 messages),\u201d and correctly answers \u201cWhat is my favorite country?\u201d from the earlier session, with no memory tool called, just the checkpointer.<\/em><\/p>\n\n\n\n

\n

The honest takeaway.<\/strong> If I\u2019d started with LangGraph on day one, I would have had a working agent in an afternoon, and understood almost none of it. The loop, the schema-to-function mapping, the persistence, all of it would have been invisible, working, and mysterious.<\/p>\n\n\n\n

Building it by hand first meant that by the time I reached for the framework, I wasn\u2019t trusting a black box. I was looking at a table of exactly what it automated, and confirming, line by line, that none of it was hiding a decision I actually needed to make myself.<\/p>\n<\/blockquote>\n\n\n\n

That\u2019s the argument for learning the fundamentals before the framework: not that frameworks are bad, they\u2019re not, but that you can\u2019t tell the difference between \u201cthis abstraction is saving me time\u201d and \u201cthis abstraction is hiding a decision I don\u2019t understand\u201d until you\u2019ve built the thing it\u2019s abstracting at least once.<\/p>\n\n\n\n

The full code for both versions, hand-built and LangGraph, is on GitHub: github.com\/truepythoneer\/local-ai-agent<\/a>.<\/p>\n\n\n\n

Tags: AI Agents, LLM, Ollama, GenAI, LangGraph, On-Prem AI, Engineering Leadership<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Every \u201cbuild an AI agent\u201d tutorial today starts the same way: pip install langchain, five lines of code, a working demo. It\u2019s fast, and it\u2019s honest about almost nothing. You get a working agent without ever seeing what \u201cagent\u201d actually means. I wanted to understand the mechanism, not just use it. So I built one … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":31,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[7,8,6,5,11,13,9,12,10],"class_list":["post-23","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-engineering","tag-agentic-ai","tag-ai-agents","tag-langgraph","tag-local-llm","tag-ollama","tag-on-prem-ai","tag-qwen3","tag-rag","tag-tool-calling"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t