Are you a Software Engineer wondering what AI skills you actually need right now?
You’re hearing about LLMs…RAG…MCP Servers…Pinecone…LangChain…agents… copilots…
And you’re thinking…sweet…but what do I actually need to learn to stay relevant?
Let’s talk about it.
Because the AI wave isn’t coming. It’s here. And in Christchurch, across NZ and globally, I’m seeing it quietly reshape what “strong software engineer” means.
LLMs Are the New API Layer
A year ago, “AI experience” was a nice to have.
Now? It’s showing up in job briefs.
Understanding how to work with Large Language Models (LLMs) isn’t just prompt engineering. It’s:
• How to structure system prompts
• Token limits and cost awareness
• Model selection (GPT-5.2 vs Claude vs open source)
• Latency vs accuracy trade-offs
• Guardrails and safety
The engineers getting interviews are the ones who treat LLMs like another external service with constraints, not magic.
RAG Is Becoming Table Stakes
If you’re building anything serious with AI, you’re probably using Retrieval Augmented Generation (RAG).
Why?
Because raw LLMs hallucinate.
RAG grounds responses in your data.
The core stack I’m seeing in NZ SaaS teams:
• Chunking & embeddings
• Vector databases like Pinecone
• Metadata filtering
• Evaluation loops
If you’ve built even a small internal RAG prototype using something like LangChain, you’re already ahead of most engineers still just “playing with ChatGPT.”
And no, it doesn’t need to be perfect. It just needs to show you understand architecture.
MCP Servers & Tool Use
This is where it gets interesting.
Model Context Protocol (MCP) and tool calling is shifting AI from “chatbot” to “agent.”
Instead of:
User asks question → model answers.
We now have:
User asks → model decides which tool to call → fetches structured data → returns enriched response.
Engineers who understand:
• Function calling schemas
• Structured outputs (JSON enforcement)
• Agent orchestration
• Secure tool execution
…are becoming very valuable very quickly.
Because businesses don’t want gimmicks. They want AI that connects to CRMs, ERPs, databases and internal systems safely.
Vector Databases Like Pinecone
Let’s be clear.
This isn’t just “store embeddings somewhere.”
Understanding:
• Index structures
• Similarity search (cosine vs dot product)
• Performance trade-offs
• Cost optimisation
…shows you think like an engineer, not a demo builder.
LangChain (Or Similar Orchestration Frameworks)
Some engineers love it.
Some think it’s overkill.
But understanding orchestration frameworks like LangChain (or LlamaIndex etc) demonstrates:
• Prompt chaining
• Memory handling
• Tool integration
• Observability
Even if you don’t use it in production, knowing how these patterns work matters.
The Real Skill? Engineering + AI Thinking
Here’s the thing no one says loudly enough.
Companies don’t just want “AI engineers.”
They want:
• Strong backend engineers
• Solid system design skills
• Clean API design
• Secure coding practices
…who ALSO understand AI patterns.
The magic combo is:
Traditional software engineering discipline + practical AI integration.
The engineers getting traction right now are the ones saying:
“I built a small RAG system on our internal docs.”
“I integrated LLM summarisation into a workflow.”
“I built a tool-calling agent connected to a database.”
That’s it. Practical. Applied.
What I’m Seeing in the Christchurch Market
We’re not Silicon Valley.
But we’re not behind either.
Product companies here are experimenting.
Internal automation projects are happening.
Copilot usage is widespread.
If you’re sitting back waiting for the “perfect AI job” to appear… you might miss the opportunity.
If you’re building small prototypes on nights or weekends?
You’re positioning yourself well.
So What Should You Actually Do?
Start small:
Build a basic RAG app.
Use Pinecone (or similar).
Experiment with tool calling.
Deploy something simple.
Doesn’t have to be fancy.
It just has to show you can move beyond prompts.
Honestly?
This feels a bit like early cloud days.
The engineers who leaned in early are now the senior cloud architects.
AI integration skills are going the same way.
So if you’re wondering why some engineers are getting the interesting AI projects and others aren’t…
It’s usually because they built something. Even small.
No one’s going to hand you “AI experience.”
GO BUILD SOMETHING :-)
I’m Paul, lover of coffee, dogs, biking, surfing & skiing. Founder & Principal Consultant of Sunstone, an IT Recruitment and People & Culture Consultancy in Christchurch & South Island of New Zealand.
We specialise in recruiting jobs around the product development lifecycle in software, hardware & engineering! We have also made successful appointments within AI, hardware & firmware engineering, mechanical engineering, IoT, electrical & electronics engineering, web & digital marketing and senior leadership roles.