Part 1: "We became one of the first companies in Korea to build a RAG system” - Lessons from a decade in AI

Watch part 1 of our 3-part series⬇️

‍

Episode Summary

‍

1. Jake Kim’s background

Jake shares his career path from South Korea to New Zealand, and from software engineering into AI product leadership. After studying computer science and mathematics in New Zealand, he worked as a software engineer, completed an MBA, and moved into product management.

At Samsung, he worked on Samsung Pay and Bixby, Samsung’s voice assistant, which became his entry point into AI around 2016. He later co-founded an AI start-up in Korea, where he spent nine years building a platform for AI agents and customer solutions. In March of the current year, he returned to New Zealand and joined Theta.

2. The key lesson from a decade in AI

Jake’s main lesson from ten years in AI is that the technology changes extremely quickly. He notes that almost none of the software his teams built in the early years still exists, because it has either been replaced or made obsolete.

For him, the most important skill is not simply knowing the current technology, but understanding where AI is heading and how to stay relevant as the field changes.

3. What New Zealand executives are asking about AI

Since joining Theta, Jake has been speaking with executives and business leaders across New Zealand. Two themes consistently came up as the areas of highest interest:

Agentic AI

The AI-enabled workforce

Executives are interested in the potential for AI to help work get done faster, but they are also cautious. Their concerns include:

• What happens to people and roles as AI becomes more widespread
• How to maintain quality
• How much human oversight is needed
• How governance should work
• How to separate genuine value from hype

Jake describes the general mood as positive and open, but careful.

4. What does agentic AI mean?

Jake explains agentic AI as a system of autonomous, action-oriented AI agents.

Traditional AI tools, especially in the first few years after ChatGPT, were mostly used like chatbots: a person asks a question, and the AI gives an answer. Agentic AI goes further. It can:

• Plan
• Act
• Review its own work
• Continue progressing towards a goal

Jake compares this shift to moving from a calculator-like tool to something closer to a junior colleague that can take an instruction and carry out work autonomously.

5. Where are organisations today?

Most organisations Jake has spoken with have already started with basic AI tools, particularly Microsoft 365 Copilot. Some are using Claude, and a smaller number have built their own agents using tools such as Microsoft Copilot Studio or AI Foundry.

Many of these are still relatively simple internal chatbots that search company documents. Jake identifies these as examples of RAG, or Retrieval Augmented Generation.

RAG is an AI architecture where the model searches an organisation’s own documents before answering, so its response is grounded in company-specific information rather than the wider internet.

However, many organisations are now hitting a wall. After deploying Copilot or basic document-search chatbots, they are asking:

• Where do we go from here?
• What are other organisations doing?
• How do we move beyond basic AI use?

Jake says the landscape can feel like being dropped into a new place without a map because there are so many tools, terms and competing narratives.

6. The ChatGPT shock and the shift to RAG

Jake reflects on the launch of ChatGPT in November 2022 as a major turning point. At the time, he was running an AI start-up in Korea that was building AI agents.

Before ChatGPT, one of the hardest problems was understanding user intent. His team had spent years building rule-based systems to classify what users wanted and decide which action to take. This rule book was effectively the company’s “crown jewel”.

When ChatGPT arrived, it became clear that much of that work had lost value almost overnight, because large language models could understand user intent far more effectively.

Although this was initially a shock, Jake says it also opened up new possibilities. His company moved quickly and became one of the first in Korea to build a RAG system. That experience taught him that today’s best practice can become outdated very quickly, so AI systems should be designed to be open and expandable.

7. The four-year evolution of AI engineering

Jake outlines how AI practice has changed year by year since ChatGPT launched.

2023 — Prompt engineering

The first year after ChatGPT was dominated by prompt engineering. Because models were less capable than they are now, the wording and structure of a prompt had a major impact on the quality of the answer.

The key skill was learning how to ask the model the right question in the right way.

2024 — Context engineering

The second year shifted towards context engineering. As people began putting more information into the request sent to the model, the key challenge became deciding what information should be included.

This could include:

• Conversation history
• Workflows
• Tool definitions
• Relevant business data

Because the model’s context window is limited, teams had to decide what mattered most and what should be left out.

2025 — Harness engineering

In the third year, the focus moved to harness engineering. As models became better at reasoning and planning, the challenge became less about giving more instructions and more about setting boundaries.

Harness engineering is about creating:

• Constraints
• Safety rails
• Operating rules

Limits on what the AI should and should not do

The goal is to stop AI systems from going off track, especially when they are given more autonomy.

2026 — Loop engineering

Jake says the current emerging topic is loop engineering. Rather than humans repeatedly checking an AI’s work and prompting it to continue, loop engineering automates that interaction.

A loop can keep an AI agent working through a task list until the work is complete. Jake mentions that this approach began with simple scripts and has now become a feature in major tools such as Claude Code and Codex.

The shift means people may no longer be writing prompts directly to the AI as often. Instead, they may be designing the loops that guide the AI’s work.

8. Main takeaway

The episode’s central message is that AI is changing in repeated waves. Each year since ChatGPT, a new practice has become important, only to be overtaken by the next shift about twelve months later.

Jake describes this as “four ChatGPT shocks in a row”. The lesson for organisations is not to bet everything on one tool, method or current best practice. Instead, they should build flexible, governed and expandable AI systems that can adapt as the technology changes.

‍

‍

Key themes

• Generative AI has become a board-level concern.
• Executives are interested in agentic AI but cautious about governance and workforce impact.
• Many organisations have started with Copilot or RAG-based chatbots but are unsure what comes next.
• AI best practice is changing quickly.
• Prompt engineering, context engineering, harness engineering and loop engineering show the evolution of AI capability.
• The next phase is likely to involve more autonomous agents, stronger governance, and systems designed to adapt over time.

‍

‍

Chapters:

00:00 Jake Kim’s AI journey

02:38 What leaders are asking about AI

03:18 Agentic AI explained

05:00 Where organisations are today

07:30 The ChatGPT turning point

09:56 Four waves of AI practice (2023 – 2026)

14:57 The takeaway – absorbing ChatGPT “shocks”