In this article, I'll show you how to use a custom GitHub Copilot agent to automatically upgrade a multi-module Spring Boot application from version 3.5.0 to 4.0.5, complete with dependency updates, code migrations, and test validations.

Prerequisites

Before we dive in, I'm assuming you:

• Have GitHub Copilot installed and activated in VS Code

• Are familiar with basic Copilot interactions (chat, inline suggestions)

• Understand what Spring Boot is and have worked with Maven projects

• Have a Spring Boot project that needs upgrading

If you're new to GitHub Copilot, check out the official documentation to get started.

What Are Custom Agents in GitHub Copilot?

GitHub Copilot Workspace introduces a powerful concept: custom agents. Think of them as specialized AI assistants with domain-specific expertise.

Understanding the Agent Hierarchy

Base Agent (GitHub Copilot)
Your primary AI coding assistant that handles general programming tasks, questions, and code generation.

Custom Agents
Specialized agents configured for specific tasks or domains. They extend Copilot's capabilities with:

• Domain-specific knowledge

• Predefined workflows

• Custom instructions and prompts

• Access to specialized tools

Sub-Agents
Purpose-built agents that work under a parent agent to handle specific subtasks. For complex operations, a custom agent might orchestrate multiple sub-agents, each handling a specific aspect of the task.

The Spring Boot Upgrader Agent Ecosystem

To address the complexity of Spring Boot upgrades, I built the Spring Boot Upgrader Agent—a purpose-built solution that transforms what used to be a multi-day manual task into an automated, reliable workflow. The agent is available as an open-source project at github.com/tanmoymandal/gh-copilot-agents.

The Spring Boot Upgrader is a parent agent that orchestrates several specialized sub-agents:

This orchestrated approach ensures each aspect of the upgrade is handled by a specialized component, resulting in a thorough and reliable upgrade process.

Getting the Spring Boot Upgrader Agent

The Spring Boot Upgrader agent is available as an open-source project that you can easily integrate into your workspace.

Step 1: Clone the Agent Repository

git clone https://github.com/tanmoymandal/gh-copilot-agents.git
cd gh-copilot-agents

Step 2: Understand the Agent Configuration

The agent is configured using a .agent.md file with YAML frontmatter. Here's a simplified view of the structure:

---
name: Spring Boot Upgrade to 4.0.x
description: >
  Use when upgrading a Spring Boot project to version 4.0.x. 
  Orchestrates full upgrade workflow: version detection, Java version selection, 
  dependency upgrade, test updates, vulnerability scanning, and upgrade report generation.
applyTo:
  - filePattern: '**/pom.xml'
  - filePattern: '**/build.gradle*'
tools:
  - runSubagent
  - read_file
  - replace_string_in_file
  - run_in_terminal
---

# Agent Instructions

[Detailed workflow instructions for the agent...]

Key Configuration Elements:

• name: The identifier you'll use to invoke this agent

• description: What the agent does and when to use it

• applyTo: File patterns that trigger the agent's availability (Maven/Gradle files)

• tools: Which Copilot tools the agent can use

• Instructions: Detailed workflow steps the agent follows

Step 3: Install the Agent in Your Project

There are two ways to make the agent available:

Option A: Workspace-Level (Recommended for Team Projects)

Copy the .agent.md file to a .github/copilot/ directory in your project:

mkdir -p <your-project>/.github/copilot/agents
cp spring-boot-upgrade/.agent.md <your-project>/.github/copilot/agents/spring-boot-upgrader.agent.md

Option B: User-Level (Recommended for Personal Use)

Place the agent in your global Copilot configuration:

mkdir -p ~/.copilot/agents
cp spring-boot-upgrade/.agent.md ~/.copilot/agents/spring-boot-upgrader.agent.md

After copying, reload VS Code or restart the Copilot extension to make the agent available.

Real-World Example: Upgrading my-awesome-app

Let's walk through upgrading a real multi-module Spring Boot application. The my-awesome-app is a sample project available at https://github.com/tanmoymandal/my-awesome-app.

Project Structure

my-awesome-app/
├── pom.xml                    # Parent POM (Spring Boot 3.5.0)
├── dataaccess/                # Shared JPA entities and repositories
│   ├── pom.xml
│   └── src/main/java/...
├── api/                       # REST API module
│   ├── pom.xml
│   └── src/main/java/...
└── batch/                     # Spring Batch jobs
    ├── pom.xml
    └── src/main/java/...

Initial State

• Spring Boot: 3.5.0

• Java: 17

• Spring Batch: 5.2.2

• Spring Data JPA: 3.5.0

• Known Vulnerabilities: 16 CVEs (5 High, 6 Medium, 5 Low)

Step 1: Clone the Sample App

git clone https://github.com/tanmoymandal/my-awesome-app.git
cd my-awesome-app

Step 2: Open in VS Code

code .

Step 3: Invoke the Spring Boot Upgrader Agent

Open the GitHub Copilot Chat (Cmd+Shift+I on Mac, Ctrl+Shift+I on Windows/Linux) and type:

@Spring Boot Upgrade to 4.0.x upgrade this project to Spring Boot 4.0.x

Or simply:

Upgrade this Spring Boot project to version 4.0.x

If the agent is configured correctly with the applyTo patterns, it will automatically detect that you're in a Maven project and activate.

Step 4: Watch the Magic Happen

The agent will:

1. Detect Current Versions 🔍

   Analyzing pom.xml files...
   Detected: Spring Boot 3.5.0, Java 17
   

2. Fetch Migration Documentation 📖

   Retrieving Spring Boot 4.0 migration guide...
   Analyzing breaking changes...
   

3. Upgrade Dependencies ⬆️

   Updating parent POM to Spring Boot 4.0.5...
   Updating Spring Batch 5.2.2 → 6.0.3...
   Resolving starter modularization changes...
   

4. Apply Code Migrations 🔧

   Updating Spring Batch imports (package restructure)...
   Replacing deprecated APIs...
   

5. Update Tests

   Fixing test dependencies...
   Running test suite... 23/23 tests passed
   

6. Vulnerability Scan

   Pre-upgrade: 16 CVEs (5 High)
   Post-upgrade: 0 CVEs
   

7. Generate Report

   Creating UPGRADE_REPORT.md...
   

What Actually Changed?

1. Parent POM Update

 <parent>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-parent</artifactId>
-    <version>3.5.0</version>
+    <version>4.0.5</version>
     <relativePath/>
 </parent>

 <properties>
-    <java.version>17</java.version>
+    <java.version>25</java.version>
 </properties>

2. Starter Modularization (api/pom.xml)

Spring Boot 4.0 decomposed monolithic starters:

 <dependency>
     <groupId>org.springframework.boot</groupId>
-    <artifactId>spring-boot-starter-web</artifactId>
+    <artifactId>spring-boot-starter-webmvc</artifactId>
 </dependency>

 <dependency>
     <groupId>org.springframework.boot</groupId>
-    <artifactId>spring-boot-starter-test</artifactId>
+    <artifactId>spring-boot-starter-webmvc-test</artifactId>
     <scope>test</scope>
 </dependency>

+<dependency>
+    <groupId>org.springframework.boot</groupId>
+    <artifactId>spring-boot-starter-data-jpa-test</artifactId>
+    <scope>test</scope>
+</dependency>

3. Batch Module Updates (batch/pom.xml)

 <dependency>
     <groupId>org.springframework.boot</groupId>
-    <artifactId>spring-boot-starter-batch</artifactId>
+    <artifactId>spring-boot-starter-batch-jdbc</artifactId>
 </dependency>

 <dependency>
-    <groupId>org.springframework.batch</groupId>
-    <artifactId>spring-batch-test</artifactId>
+    <groupId>org.springframework.boot</groupId>
+    <artifactId>spring-boot-starter-batch-test</artifactId>
     <scope>test</scope>
 </dependency>

4. Spring Batch 6.0 Package Restructure

Spring Batch 6.0 reorganized core types into sub-packages:

-import org.springframework.batch.core.Job;
-import org.springframework.batch.core.Step;
+import org.springframework.batch.core.job.Job;
+import org.springframework.batch.core.step.Step;

-import org.springframework.batch.item.ItemProcessor;
-import org.springframework.batch.item.ItemWriter;
+import org.springframework.batch.infrastructure.item.ItemProcessor;
+import org.springframework.batch.infrastructure.item.ItemWriter;

-import org.springframework.batch.item.data.RepositoryItemReader;
-import org.springframework.batch.item.data.builder.RepositoryItemReaderBuilder;
+import org.springframework.batch.infrastructure.item.data.RepositoryItemReader;
+import org.springframework.batch.infrastructure.item.data.builder.RepositoryItemReaderBuilder;

These changes were applied automatically across:

• ProductReportJobConfig.java

• UserSyncJobConfig.java

• Both batch job test files

The Final Result

After the agent completes its work, you get:

✅ Fully Upgraded Application

• Spring Boot 3.5.0 → 4.0.5

• Java 17 → 25

• All dependencies updated to compatible versions

✅ Zero Vulnerabilities

• Eliminated all 16 CVEs (5 High, 6 Medium, 5 Low)

✅ All Tests Passing

• 23/23 tests pass with zero failures

• Test infrastructure updated to Spring Boot 4.0 conventions

✅ Comprehensive Documentation

• UPGRADE_REPORT.md with full change log

• Before/after comparison tables

• CVE remediation details

When to Use This Agent

The Spring Boot Upgrader agent is ideal for:

✅ Major Version Upgrades - Spring Boot 3.x to 4.x migrations
✅ Multi-Module Projects - Handles complex Maven/Gradle structures
✅ Security Compliance - Eliminates known CVEs through upgrades
✅ CI/CD Modernization - Part of dependency update automation
✅ Java Version Migrations - Coordinated Java + framework upgrades

⚠️ Important Considerations

While the agent is powerful, keep in mind:

1. Always Review Changes - The agent makes extensive modifications. Review all changes before committing.

2. Test Thoroughly - While the agent runs tests, you should perform additional integration and manual testing.

3. Backup First - Commit your current state or work in a branch before running the upgrade.

4. Custom Code - The agent handles framework migrations but can't understand all custom business logic. Manual review is required.

5. Version Compatibility - Ensure your application is compatible with Java 25 and Spring Boot 4.0's requirements.

Inspecting the Upgrade Report

The generated UPGRADE_REPORT.md contains:

## Executive Summary
- Before/after version matrix
- Dependency upgrade count
- Vulnerability remediation summary
- Test pass rates

## Version Changes
- Core framework versions
- Spring ecosystem dependencies
- Third-party library updates

## Migration Changes
- API breaking changes applied
- Package reorganizations
- Code transformations performed

## Test Results
- Module-by-module test results
- Failure details (if any)
- Coverage statistics

## Vulnerability Assessment
- Before/after CVE comparison
- Severity breakdown
- Remediation details

## Recommendations
- Post-upgrade tasks
- Performance considerations
- Further improvements
- Detected areas for enhancement (based on project analysis)

The agent intelligently analyzes your codebase during the upgrade process and may suggest areas for improvement—such as deprecated patterns it detected, opportunities for modernization, or performance optimizations that align with the new Spring Boot version's capabilities.

Next Steps

After a successful upgrade:

1. Review the changes carefully using git diff

2. Run your full test suite (unit, integration, E2E)

3. Test in a staging environment before production

4. Update your CI/CD pipelines for Java 25

5. Review and commit the UPGRADE_REPORT.md

Conclusion

Custom GitHub Copilot agents represent a paradigm shift in how we approach complex, repetitive development tasks. The Spring Boot Upgrader agent demonstrates how domain-specific AI assistants can handle tasks that typically require hours of manual work—dependency analysis, migration guide research, code transformations, testing, and documentation—all in a single automated workflow.

By leveraging sub-agents for specialized tasks and orchestrating them intelligently, we can achieve results that are both faster and more reliable than manual upgrades.

Try It Yourself

1. Clone the agent repository: gh-copilot-agents

2. Try the sample app: my-awesome-app

3. Follow instructions in this article to see how you can test out this agent against the sample app

Resources

• GitHub Copilot Documentation

• Spring Boot 4.0 Release Notes

• Spring Batch 6.0 Migration Guide

• Custom Agent Repository

• Sample Application

Have you used custom GitHub Copilot agents in your workflow? What tasks would you like to automate? Share your thoughts.

Why This Matters

The Model Context Protocol (MCP) landed in late 2024 and spread fast. Within months, every major AI client — Claude Desktop, Cursor, Windsurf, and dozens of others — adopted it as the standard way for LLMs to talk to external tools.

The problem? Nearly all the tutorials are Python.

The Java ecosystem is massively underserved here. If you're running Spring Boot microservices — and a huge portion of the enterprise world is — you deserve a first-class, idiomatic MCP story. That's what this article delivers.

By the end, you'll have:

• A fully working MCP server exposing 9 Todo management tools

• A Spring AI 1.1.2 + Spring Boot 4.0.x setup using the official starters

• Clean service/tools separation so your business logic stays testable

• Unit tests with JUnit 5 + Mockito

• A clear mental model of how MCP fits into your architecture

What Is MCP? (The 90-Second Version)

MCP is a client–server protocol that lets AI models call external tools in a standardised way. Think of it as USB-C for AI integrations — one protocol, any tool, any client.

MCP Client (Claude Desktop / Cursor / your app)
        │
        │  SSE or STDIO transport
        │
MCP Server (your Spring Boot app)
        │
        ├── Tool: createTodo
        ├── Tool: getAllTodos
        ├── Tool: completeTodo
        └── Tool: getStats

When a user asks Claude "What are my critical tasks today?", the LLM:

1. Recognizes it needs external data

2. Calls your getTodosByPriority tool with priority="CRITICAL"

3. Reads the JSON response

4. Formulates a natural language answer

Your Java code runs. The AI gets the data. The user gets a useful answer. No hallucination, no guessing.

Project Overview

Stack:

• Spring Boot 4.0.3

• Spring AI 1.1.2 (latest stable as of March 2026)

• Spring Data JPA + H2 (swap to PostgreSQL for production)

• Java 25

MCP Tools exposed:

Project Structure

todo-mcp-server/
├── pom.xml
└── src/main/java/com/tanmoymandal/mcp/todo/
    ├── TodoMcpServerApplication.java       ← main class
    ├── model/
    │   └── Todo.java                       ← JPA entity + enums
    ├── repository/
    │   └── TodoRepository.java             ← Spring Data JPA
    ├── service/
    │   └── TodoService.java                ← business logic (no MCP dependency)
    ├── tools/
    │   └── TodoMcpTools.java               ← @Tool annotations live here
    └── config/
        ├── McpServerConfig.java            ← registers tools with MCP
        └── DataInitializer.java            ← seeds demo data on startup

Key design decision: TodoService has zero dependency on Spring AI. The TodoMcpTools class is the adapter between the AI world and your domain. This separation keeps your business logic independently testable and reusable outside of MCP context.

Step 1 — The pom.xml

The single most important thing here is the Spring AI BOM — it ensures all Spring AI artifacts are version-compatible with no manual version juggling.

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>4.0.3</version>
</parent>

<properties>
    <java.version>21</java.version>
    <spring-ai.version>1.1.3</spring-ai.version>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>${spring-ai.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <!-- MCP Server over HTTP/SSE using Spring MVC -->
    <dependency>
        <groupId>org.springframework.ai</groupId>
        <artifactId>spring-ai-starter-mcp-server-webmvc</artifactId>
    </dependency>

    <!-- Web layer (required by webmvc transport) -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- JPA + H2 -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>com.h2database</groupId>
        <artifactId>h2</artifactId>
        <scope>runtime</scope>
    </dependency>
</dependencies>

Why spring-ai-starter-mcp-server-webmvc? This starter provides the HTTP/SSE transport layer — the way most MCP clients connect to remote servers. It auto-configures the /sse and /mcp/messages endpoints for you. The alternative spring-ai-starter-mcp-server is STDIO only (suitable for local subprocess invocation). Use webmvc for anything network-accessible.

Step 2 — application.yml

server:
  port: 8080

spring:
  datasource:
    url: jdbc:h2:mem:tododb;DB_CLOSE_DELAY=-1
    driver-class-name: org.h2.Driver
    username: sa
    password:
  jpa:
    hibernate:
      ddl-auto: create-drop
  h2:
    console:
      enabled: true

  ai:
    mcp:
      server:
        name: todo-mcp-server
        version: 1.0.0
        type: SYNC
        instructions: |
          This server manages Todo/Task items.
          Tools available: createTodo, getTodoById, getAllTodos,
          updateTodo, deleteTodo, completeTodo, searchTodos,
          getTodosByPriority, getStats.
        capabilities:
          tool: true

The instructions field is important — it's sent to the AI client during the handshake and helps the LLM understand what your server does before it even reads the individual tool descriptions.

Step 3 — The Entity

@Entity
@Table(name = "todos")
public class Todo {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @NotBlank
    private String title;

    @Column(length = 2000)
    private String description;

    @Enumerated(EnumType.STRING)
    private Priority priority = Priority.MEDIUM;

    @Enumerated(EnumType.STRING)
    private Status status = Status.PENDING;

    private LocalDateTime createdAt;
    private LocalDateTime updatedAt;
    private LocalDateTime completedAt;

    public enum Priority { LOW, MEDIUM, HIGH, CRITICAL }
    public enum Status   { PENDING, IN_PROGRESS, COMPLETED, CANCELLED }

    @PrePersist
    protected void onCreate() {
        createdAt = updatedAt = LocalDateTime.now();
    }

    @PreUpdate
    protected void onUpdate() {
        updatedAt = LocalDateTime.now();
    }

    // ... getters/setters
}

Step 4 — The Tools Class (The Heart of It All)

This is where Spring AI's @Tool annotation does the heavy lifting. Every annotated method becomes a discoverable MCP tool with an auto-generated JSON Schema for its parameters.

@Service
public class TodoMcpTools {

    private final TodoService service;
    private final ObjectMapper objectMapper;

    public TodoMcpTools(TodoService service, ObjectMapper objectMapper) {
        this.service = service;
        this.objectMapper = objectMapper;
    }

    @Tool(name = "createTodo",
          description = """
              Creates a new Todo item. Returns the created Todo as JSON
              including the auto-generated id.
              Priority: LOW, MEDIUM, HIGH, CRITICAL (defaults to MEDIUM).
              """)
    public String createTodo(
            @ToolParam(description = "Short, descriptive title. Required.")
            String title,

            @ToolParam(description = "Optional detailed description.", required = false)
            String description,

            @ToolParam(description = "Priority: LOW, MEDIUM, HIGH, CRITICAL.", required = false)
            String priority) {

        try {
            Todo todo = service.create(title, description, priority);
            return toJson(todoToMap(todo));
        } catch (Exception e) {
            return errorJson("createTodo failed: " + e.getMessage());
        }
    }

    @Tool(name = "completeTodo",
          description = """
              Marks a Todo as COMPLETED and records the completion timestamp.
              Returns the updated Todo as JSON.
              """)
    public String completeTodo(
            @ToolParam(description = "The numeric id of the todo to complete.")
            Long id) {

        return service.complete(id)
                .map(t -> toJson(todoToMap(t)))
                .orElse(errorJson("Todo with id=%d not found".formatted(id)));
    }

    @Tool(name = "getStats",
          description = """
              Returns aggregate statistics: counts by status and priority.
              Great for summaries and dashboards.
              """)
    public String getStats() {
        return toJson(service.getStats());
    }

    // ... remaining tools follow the same pattern
}

Write tool descriptions for the LLM, not for humans. Be explicit about valid enum values, what null means, and exactly what the return value contains. The LLM reads these descriptions to decide which tool to call and how to call it correctly.

Step 5 — Register the Tools

@Configuration
public class McpServerConfig {

    @Bean
    public ToolCallbackProvider todoToolCallbacks(TodoMcpTools todoMcpTools) {
        return MethodToolCallbackProvider.builder()
                .toolObjects(todoMcpTools)
                .build();
    }
}

That's it. MethodToolCallbackProvider reflects over your @Tool methods, generates the MCP tool descriptors with full JSON Schema from the method signatures, and Spring AI's auto-configuration registers them on the MCP endpoint automatically.

Step 6 — Run It

./mvnw spring-boot:run

You'll see in the logs:

Registered MCP tools: [createTodo, getTodoById, getAllTodos,
  getTodosByStatus, getTodosByPriority, searchTodos,
  updateTodo, completeTodo, deleteTodo, getStats]
Started TodoMcpServerApplication on port 8080

The server is live at:

• SSE endpoint: http://localhost:8080/sse

• H2 Console: http://localhost:8080/h2-console

• Health check: http://localhost:8080/actuator/health

Connecting Claude Desktop

Add this to your Claude Desktop claude_desktop_config.json:

{
  "mcpServers": {
    "todo-manager": {
      "type": "sse",
      "url": "http://localhost:8080/sse"
    }
  }
}

Restart Claude Desktop. You'll see a hammer icon in the chat UI — that means your tools are live. Now ask:

"What are my critical priority tasks?""Mark task 5 as complete.""Give me a summary of all my todos."

Claude will call your Spring Boot server, get real data back, and answer with zero hallucination.

Unit Testing the Service Layer

Because TodoService has no Spring AI dependency, it tests exactly like any other Spring service:

@ExtendWith(MockitoExtension.class)
class TodoServiceTest {

    @Mock  TodoRepository repository;
    @InjectMocks TodoService service;

    @Test
    void create_shouldPersistAndReturnTodo() {
        Todo expected = new Todo("Fix bug", "NPE on line 42", Todo.Priority.HIGH);
        expected.setId(1L);
        when(repository.save(any())).thenReturn(expected);

        Todo result = service.create("Fix bug", "NPE on line 42", "HIGH");

        assertThat(result.getTitle()).isEqualTo("Fix bug");
        assertThat(result.getPriority()).isEqualTo(Todo.Priority.HIGH);
        verify(repository).save(any(Todo.class));
    }

    @Test
    void complete_shouldSetStatusAndTimestamp() {
        Todo todo = new Todo("Task", null, Todo.Priority.MEDIUM);
        todo.setId(1L);
        when(repository.findById(1L)).thenReturn(Optional.of(todo));
        when(repository.save(any())).thenAnswer(i -> i.getArguments()[0]);

        Optional<Todo> result = service.complete(1L);

        assertThat(result).isPresent();
        assertThat(result.get().getStatus()).isEqualTo(Todo.Status.COMPLETED);
        assertThat(result.get().getCompletedAt()).isNotNull();
    }
}

Production Considerations

When you're ready to move beyond the demo, here's what to address:

Database: Swap H2 for PostgreSQL by replacing the datasource config and adding the PostgreSQL driver. Zero code changes needed — that's the beauty of Spring Data JPA.

Security: The MCP spec requires OAuth2 for HTTP-exposed servers. Spring AI 1.1.x has a companion mcp-server-security module. Add it and a SecurityFilterChain bean — the Spring team published a detailed walkthrough on the Spring blog.

Observability: Add spring-boot-starter-actuator with Micrometer. Your MCP tool call counts, latencies, and error rates surface as Prometheus metrics automatically.

Packaging: Build a Docker image with ./mvnw spring-boot:build-image — Spring Boot's Cloud Native Buildpacks produce a production-grade container with no Dockerfile required.

What We Built

In one Spring Boot application we've produced an MCP server that:

• Exposes 9 fully-described tools discoverable by any MCP client

• Uses standard Spring idioms — JPA, @Service, @Bean, @Transactional

• Keeps business logic completely decoupled from the AI/MCP layer

• Is unit-testable without any AI dependencies

• Seeds itself with demo data for instant exploration

• Connects to Claude Desktop, Cursor, or any MCP-compatible client in 30 seconds

The Java ecosystem is ready for MCP. Spring AI 1.1.x gives you a first-class, annotation-driven path that feels exactly like the Spring you already know — no Python required.

Full Source Code

The complete project is available on GitHub: https://github.com/tanmoymandal/todo-mcp-server

Tags: #Java #SpringBoot #SpringAI #MCP #ModelContextProtocol #AI #LLM #Claude

From Transformers to RAG, Agents to Embeddings — decoded.

Whether you're diving into your first machine learning project or architecting enterprise AI systems, the landscape of AI terminology can feel overwhelming. This glossary cuts through the noise with clear, developer-friendly definitions — organized by category so you can navigate what you need.

Bookmark this. You'll be back.

🧠 Foundation: Core AI Concepts

Artificial Intelligence (AI)

The broad field of building systems that can perform tasks typically requiring human intelligence — reasoning, understanding language, recognizing patterns, and making decisions. AI is the umbrella; everything else in this glossary lives under it.

Machine Learning (ML)

A subset of AI where systems learn from data rather than following explicitly programmed rules. Instead of writing if (temperature > 100) return "hot", you feed examples and let the algorithm figure out the pattern.

Deep Learning (DL)

A subset of machine learning that uses neural networks with many layers (hence "deep"). Deep learning powers most modern breakthroughs — image recognition, speech synthesis, large language models.

Neural Network

A computational model loosely inspired by the human brain. It consists of interconnected nodes (neurons) organized in layers that transform input data into output predictions. Each connection has a weight that gets tuned during training.

Algorithm

A set of rules or instructions that a model follows to make decisions or learn from data. In ML, the algorithm defines how the model learns — gradient descent, backpropagation, etc.

Model

The trained artifact that results from running an ML algorithm on data. When people say "deploy the model," they mean the weights and architecture that now encode learned knowledge.

📊 Data & Training

Training Data

The dataset used to teach a model. Quality and quantity both matter enormously. Biased training data → biased model. Insufficient training data → an underfit model.

Test Data / Validation Data

Held-out datasets used to evaluate model performance after training. Validation data guides hyperparameter tuning during training; test data gives the final performance estimate.

Overfitting

When a model learns the training data too well — including its noise and quirks — and fails to generalize to new data. Classic symptom: 99% training accuracy, 60% test accuracy.

Underfitting

The opposite problem: the model is too simple to capture the underlying patterns. Both low training accuracy and low test accuracy.

Supervised Learning

Training with labeled examples — input/output pairs. The model learns to map inputs to correct outputs. Most classification and regression tasks are supervised.

Unsupervised Learning

Training on unlabeled data to discover hidden structure. Clustering (grouping similar items) and dimensionality reduction are common unsupervised tasks.

Reinforcement Learning (RL)

A paradigm where an agent learns by taking actions in an environment and receiving rewards or penalties. Used in game-playing AIs (AlphaGo) and increasingly in fine-tuning LLMs.

Fine-Tuning

Taking a pre-trained model and continuing to train it on a smaller, task-specific dataset. Much cheaper than training from scratch and usually yields excellent results for specialized domains.

RLHF (Reinforcement Learning from Human Feedback)

A fine-tuning technique where human raters score model outputs, and those scores train a reward model that guides further RL training. Core to how models like Claude and ChatGPT are aligned.

Batch Size

The number of training examples processed together before updating model weights. Larger batches = more stable gradients but more memory. Smaller batches = noisier gradients but potentially better generalization.

Epoch

One complete pass through the entire training dataset. Training typically runs for multiple epochs.

Learning Rate

A hyperparameter that controls how much to adjust model weights per update. Too high = unstable training. Too low = painfully slow convergence.

🤖 Large Language Models (LLMs)

Large Language Model (LLM)

A neural network trained on massive text corpora to understand and generate human language. "Large" refers to billions of parameters. GPT-4, Claude, Gemini, and Llama are LLMs.

Transformer

The neural network architecture that powers virtually all modern LLMs. Introduced in the 2017 paper "Attention Is All You Need", it replaced RNNs with a mechanism called self-attention that processes all tokens simultaneously.

Attention Mechanism

The core innovation of Transformers. Allows the model to weigh the importance of different parts of the input when generating each output token. "Attending" to the right context is what makes LLMs coherent.

Token

The basic unit of text that LLMs process. Not quite words — tokens are chunks of characters (e.g., "transformer" might be one token; "unbelievable" might be two). Most LLMs use ~4 characters per token on average.

Context Window

The maximum number of tokens an LLM can process in a single interaction — both input and output combined. GPT-4 Turbo has 128K tokens; Claude has up to 200K. Larger context = better for long documents.

Prompt

The input text you send to an LLM to get a response. Prompt design significantly affects output quality — hence the discipline of prompt engineering.

Prompt Engineering

The art and science of crafting prompts to elicit better responses from LLMs. Techniques include chain-of-thought prompting, few-shot examples, role assignment, and structured output requests.

Few-Shot Prompting

Including a few examples of the task in the prompt (e.g., 2–5 input/output pairs) to help the model understand what you want without any fine-tuning.

Zero-Shot Prompting

Asking the model to perform a task with no examples — just a description. Works surprisingly well with modern LLMs due to their broad pre-training.

Chain-of-Thought (CoT)

A prompting technique where you ask the model to reason step-by-step before giving its final answer. Dramatically improves performance on multi-step reasoning and math problems.

Temperature

A parameter (0.0 to 2.0) that controls output randomness. Temperature 0 = deterministic, always picks the most likely token. Temperature 1+ = more creative and varied. For code generation, use low temperatures; for brainstorming, use higher.

Top-P (Nucleus Sampling)

An alternative to temperature for controlling randomness. Instead of adjusting probabilities, it restricts sampling to the smallest set of tokens whose cumulative probability exceeds P. top_p=0.9 is a common default.

Hallucination

When an LLM confidently generates factually incorrect information. A fundamental challenge in LLMs — they optimize for plausible text, not necessarily true text.

Grounding

Connecting model outputs to verifiable, external sources of truth to reduce hallucinations. Retrieval-Augmented Generation (RAG) is the primary grounding technique.

🔍 RAG & Retrieval

RAG (Retrieval-Augmented Generation)

An architecture that combines an LLM with a retrieval system. Instead of relying solely on training knowledge, the model retrieves relevant documents at inference time and uses them as context. Dramatically reduces hallucinations for knowledge-intensive tasks.

Vector Database

A database optimized for storing and querying embeddings (high-dimensional vectors). Used heavily in RAG systems to find semantically similar documents. Examples: Pinecone, Weaviate, Chroma, pgvector.

Embedding

A numerical vector representation of text (or images, audio, etc.) that captures semantic meaning. Similar concepts cluster together in embedding space. "cat" and "kitten" will have similar embeddings; "cat" and "blockchain" will not.

Semantic Search

Search based on meaning rather than keyword matching. Uses embeddings to find documents that are conceptually relevant, even if they don't share the exact same words.

Chunking

The process of splitting large documents into smaller pieces before embedding and storing them in a vector database. Chunk size is a critical tuning parameter in RAG — too large loses precision, too small loses context.

Reranking

A second-pass step in RAG that takes the top-k retrieved chunks and re-scores them using a more powerful (but slower) cross-encoder model, before passing the best results to the LLM.

🛠️ Agents & Tools

AI Agent

An LLM-powered system that can reason, plan, and take actions autonomously — not just generate text. Agents decide what tools to call, observe results, and iterate until a goal is achieved.

Tool Use / Function Calling

The ability for an LLM to call external functions or APIs as part of generating a response. The model outputs a structured "call this function with these arguments" rather than raw text.

Agentic Loop

The iterative cycle an AI agent follows: observe → think → act → observe → repeat, until the task is complete or a stopping condition is met.

Multi-Agent System

An architecture where multiple specialized AI agents collaborate — one might browse the web, another writes code, another reviews it. Frameworks like LangGraph and AutoGen implement this.

ReAct (Reason + Act)

A prompting framework for agents that interleaves reasoning ("Thought: ...") with actions ("Action: search[...]") and observations. Makes agent behavior more transparent and debuggable.

MCP (Model Context Protocol)

An open protocol developed by Anthropic that standardizes how AI models connect to external tools, data sources, and services. Think of it as USB-C for AI integrations — a universal interface for connecting models to the world.

Orchestration

The layer that manages the flow of an AI system — routing between agents, managing state, handling retries, and coordinating tool calls. LangChain, LlamaIndex, and LangGraph are popular orchestration frameworks.

⚙️ Model Architecture & Inference

Parameters

The learned numerical weights inside a model. "A 70B model" has 70 billion parameters. More parameters generally means more capability, but also more compute and memory.

Inference

Running a trained model to generate predictions or responses. Distinct from training. When you call an LLM API, you're doing inference.

Quantization

Reducing the numerical precision of model weights (e.g., from 32-bit floats to 4-bit integers) to decrease memory usage and speed up inference. Essential for running large models on consumer hardware.

Latency vs. Throughput

Two key inference metrics. Latency is how long a single request takes (user-facing). Throughput is how many requests per second the system handles. There's often a tradeoff.

TTFT (Time to First Token)

The latency between sending a request and receiving the first token of the response. Critical for user experience in streaming applications.

Structured Output

Constraining an LLM to generate responses in a specific format (JSON, XML, etc.) rather than free text. Used when downstream code needs to parse the response programmatically.

System Prompt

Instructions sent to an LLM that set the context, persona, or rules for the conversation — separate from the user's message. Most API-based LLMs support a dedicated system prompt field.

🎨 Generative AI (Images, Audio, Video)

Generative AI

AI systems that can create new content — text, images, audio, video, code — rather than just classifying or analyzing existing content.

Diffusion Model

The architecture behind most modern image generation models (Stable Diffusion, DALL-E, Midjourney). Works by learning to reverse a noise-addition process — starting from random noise and gradually denoising into a coherent image.

Text-to-Image

Generating images from natural language descriptions. The prompt "a photorealistic astronaut riding a horse on Mars, golden hour lighting" produces an image.

Multimodal Model

A model that can process and generate multiple types of data — text, images, audio, video. GPT-4o and Claude 3.5 are multimodal — they can see images and respond in text.

Latent Space

The compressed, abstract representation of data learned by a model. Diffusion models generate images by navigating latent space. Embeddings are points in latent space.

🔐 Safety, Alignment & Ethics

Alignment

The challenge of ensuring AI systems behave in accordance with human values and intentions. Misaligned AI does what it was literally trained to do, not necessarily what we actually want.

Constitutional AI

An Anthropic technique where a model critiques and revises its own outputs based on a set of principles (a "constitution"), reducing reliance on human feedback for every edge case.

Guardrails

Constraints applied to model inputs or outputs to prevent unsafe, harmful, or off-topic responses. Can be implemented at the prompt level, via fine-tuning, or with a separate classifier.

Jailbreak

An attempt to bypass an LLM's safety guardrails through clever prompting — often by roleplay scenarios, hypothetical framings, or encoded instructions.

Bias

Systematic errors in model outputs reflecting unfair prejudices from training data. Algorithmic bias can perpetuate or amplify societal inequalities if left unchecked.

📐 Evaluation & Metrics

Benchmark

A standardized dataset and evaluation protocol used to compare model capabilities. Common LLM benchmarks include MMLU, HumanEval (coding), and MATH.

Perplexity

A measure of how well a language model predicts a sample of text. Lower perplexity = better. Mostly used internally during training; less useful for task-specific evaluation.

BLEU / ROUGE

Automated metrics for evaluating text generation quality by comparing to reference outputs. BLEU is common for translation; ROUGE for summarization. Both have limitations — high scores don't always mean high quality.

Evals (Evaluations)

The practice of systematically testing AI model outputs against desired behavior. Moving from vibes-based to eval-driven development is the mark of a mature AI engineering team.

🚀 Deployment & Infrastructure

API (Application Programming Interface)

The interface through which you call an LLM programmatically. Send a prompt → receive a response. OpenAI, Anthropic, Google, and others expose their models via REST APIs.

Self-Hosted / On-Premises

Running an LLM on your own infrastructure rather than via a cloud API. Required for air-gapped environments, data privacy requirements, or cost optimization at scale.

GPU (Graphics Processing Unit)

The hardware backbone of AI. GPUs excel at the massively parallel matrix multiplications that neural networks require. NVIDIA's H100 and A100 are the current gold standard for AI training and inference.

Model Serving

The infrastructure that takes a trained model and makes it available as a service — handling request routing, batching, scaling, and versioning. Tools include NVIDIA Triton, vLLM, and Ray Serve.

Streaming

Returning LLM output token by token as it's generated rather than waiting for the full response. Makes UX feel much more responsive.

🧩 Quick Reference Cheat Sheet

Wrapping Up

AI terminology evolves fast — new terms emerge with every major paper and product launch. The best way to stay current is to read primary sources (ArXiv, research blogs from Anthropic, Google DeepMind, Meta AI), build things, and stay curious.

Got a term that should be in here? Drop a comment below.