Test Frameworks for VSCode Extensions

You are a Java developer and want to not only write VSCode extension tests but also debug them comfortably? In this chapter, you will learn how to start tests directly from VSCode with F5, set breakpoints, and step through your code - exactly as you know it from Eclipse.

After this chapter you will be able to: - Start tests with F5 from VSCode (not just from the console!) - Set breakpoints in tests and inspect variables - Create the necessary configuration files (launch.json, tasks.json) - Understand the difference between GUI-based testing (development) and CLI testing (CI/CD)

43.2 Quick Start: The First Test with F5 Support

Before we dive into details, let’s set up a working test that you can immediately debug with F5.

43.2.1 Step 1: Create launch.json (REQUIRED for F5 support!)

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Run Extension Tests",
      "type": "extensionHost",
      "request": "launch",
      "runtimeExecutable": "${execPath}",
      "args": [
        "--extensionDevelopmentPath=${workspaceFolder}",
        "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
      ],
      "outFiles": [
        "${workspaceFolder}/out/**/*.js"
      ],
      "preLaunchTask": "npm: compile-tests"
    }
  ]
}

43.2.2 Step 2: Create tasks.json (REQUIRED for automatic compilation!)

Without this file, you need to compile manually! Create .vscode/tasks.json:

{
  "version": "2.0.0",
  "tasks": [
    {
      "type": "npm",
      "script": "compile-tests",
      "label": "npm: compile-tests",
      "group": "build",
      "problemMatcher": "$tsc"
    }
  ]
}

43.2.3 Step 3: Minimal Setup in package.json

{
  "scripts": {
    "compile-tests": "tsc -p ./",
    "test": "node ./out/test/runTest.js"
  },
  "devDependencies": {
    "@types/mocha": "^10.0.0",
    "@types/node": "16.x",
    "@types/vscode": "^1.73.0",
    "@vscode/test-electron": "^2.3.0",
    "mocha": "^10.0.0",
    "typescript": "^4.9.0"
  }
}

43.2.4 Step 4: Test Infrastructure

import * as path from 'path';
import { runTests } from '@vscode/test-electron';

async function main() {
    try {
        // Path to the extension under development
        const extensionDevelopmentPath = path.resolve(__dirname, '../../');
        // Path to the test suite entry point
        const extensionTestsPath = path.resolve(__dirname, './suite/index');
        
        await runTests({
            extensionDevelopmentPath,
            extensionTestsPath
        });
    } catch (err) {
        console.error('Failed to run tests');
        process.exit(1);
    }
}

main();

import * as path from 'path';
import * as Mocha from 'mocha';
import * as glob from 'glob';

export function run(): Promise<void> {
    // Configure Mocha test runner with appropriate settings
    const mocha = new Mocha({
        ui: 'tdd',    // Test-driven development interface
        color: true   // Enable colored output for better readability
    });

    const testsRoot = path.resolve(__dirname, '..');

    return new Promise((c, e) => {
        // Discover all test files matching the pattern
        glob('**/**.test.js', { cwd: testsRoot }, (err, files) => {
            if (err) {
                return e(err);
            }

            // Add each discovered test file to the Mocha suite
            files.forEach(f => mocha.addFile(path.resolve(testsRoot, f)));

            try {
                // Execute the complete test suite
                mocha.run(failures => {
                    if (failures > 0) {
                        e(new Error(`${failures} tests failed.`));
                    } else {
                        c();
                    }
                });
            } catch (err) {
                e(err);
            }
        });
    });
}

43.2.5 Step 5: The First Debuggable Test

import * as assert from 'assert';

suite('My First Test Suite', () => {
    test('Addition Test with Debugging', () => {
        // SET BREAKPOINT HERE! (Click left of line 6)
        const a = 5;
        const b = 3;
        const result = a + b;
        
        // In the debugger, you can now inspect a, b, and result
        assert.strictEqual(result, 8);
    });
});

43.2.6 NOW TEST IT!

43.3 GUI vs. Console: The Two Testing Worlds

43.3.1 GUI-based Testing in VSCode (Development)

Advantages: - ✅ Breakpoints work - ✅ Variable inspection possible - ✅ Step-by-step debugging - ✅ Integrated error analysis

How it works: 1. launch.json and tasks.json must be present 2. Set breakpoint (click left of line number) 3. Press F5 4. Use debug tools: - Variables Panel: Shows all local variables - Watch Panel: Monitor custom expressions - Call Stack: Shows call hierarchy - Debug Console: Interactive commands during pause

Debug Shortcuts: - F5: Start/Continue - F9: Set/Remove breakpoint - F10: Step Over (next line) - F11: Step Into (jump into function) - Shift+F11: Step Out (exit function)

43.3.2 Console-based Testing (CI/CD)

Advantages: - ✅ Scriptable for CI/CD - ✅ No GUI needed - ✅ Batch processing possible

# Run all tests
npm test

# With grep for specific tests only
npm test -- --grep "Addition"

Important: Console tests don’t show breakpoints! Use console.log() for debug output.

43.4 The Three Test Levels in Detail

43.4.1 1. Unit Tests (Fast & Isolated)

// src/utils/calculator.ts
export class Calculator {
    add(a: number, b: number): number {
        return a + b;
    }
    
    divide(a: number, b: number): number {
        if (b === 0) {
            throw new Error('Division by zero');
        }
        return a / b;
    }
}

// src/test/suite/calculator.test.ts
import * as assert from 'assert';
import { Calculator } from '../../utils/calculator';

suite('Calculator Unit Tests', () => {
    let calculator: Calculator;
    
    setup(() => {
        // Before each test - initialize fresh calculator instance
        calculator = new Calculator();
    });
    
    test('Addition with Debugging', () => {
        // Set a breakpoint here!
        const result = calculator.add(10, 5);
        
        // In the debugger you will see:
        // - calculator instance
        // - result = 15
        assert.strictEqual(result, 15);
    });
    
    test('Division by Zero', () => {
        // Breakpoint here shows exception handling in action
        assert.throws(
            () => calculator.divide(10, 0),
            /Division by zero/
        );
    });
});

Debug Tip: Set breakpoints in both the implementation AND the test to understand the flow!

43.4.2 2. Integration Tests (With VSCode API)

// src/test/suite/extension.test.ts
import * as assert from 'assert';
import * as vscode from 'vscode';

suite('Extension Integration Tests', () => {
    
    test('Command Registration', async () => {
        // Breakpoint here shows all registered commands
        const commands = await vscode.commands.getCommands(true);
        
        // In the Debug Console you can type:
        // commands.filter(c => c.startsWith('myExtension'))
        assert.ok(commands.includes('myExtension.helloWorld'));
    });
    
    test('Document Manipulation', async () => {
        // Create test document
        const doc = await vscode.workspace.openTextDocument({
            content: 'Hello Test',
            language: 'plaintext'
        });

        const editor = await vscode.window.showTextDocument(doc);
        
        // Breakpoint here - inspect editor and doc!
        await editor.edit(editBuilder => {
            editBuilder.insert(new vscode.Position(0, 0), 'Modified: ');
        });
        
        assert.ok(doc.getText().startsWith('Modified:'));
    });
});

43.4.3 3. Mock Tests (Simulated Dependencies)

// src/test/suite/mocked.test.ts
import * as assert from 'assert';
import * as sinon from 'sinon';
import * as vscode from 'vscode';

suite('Mock Tests', () => {
    let sandbox: sinon.SinonSandbox;
    
    setup(() => {
        // Create isolated sandbox for each test
        sandbox = sinon.createSandbox();
    });
    
    teardown(() => {
        // Clean up all mocks after each test
        sandbox.restore();
    });
    
    test('Mock VSCode Messages', async () => {
        // Create mock for VSCode API call
        const showInfoStub = sandbox.stub(vscode.window, 'showInformationMessage');
        showInfoStub.resolves('OK');
        
        // Call your function
        await vscode.window.showInformationMessage('Test');
        
        // Breakpoint here shows mock details!
        assert.ok(showInfoStub.calledOnce);
        assert.ok(showInfoStub.calledWith('Test'));
    });
});

43.5 Advanced Debug Configurations

43.5.1 Multiple Test Configurations in launch.json

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "All Tests",
      "type": "extensionHost",
      "request": "launch",
      "runtimeExecutable": "${execPath}",
      "args": [
        "--extensionDevelopmentPath=${workspaceFolder}",
        "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
      ],
      "outFiles": ["${workspaceFolder}/out/**/*.js"],
      "preLaunchTask": "npm: compile-tests"
    },
    {
      "name": "Unit Tests Only",
      "type": "extensionHost",
      "request": "launch",
      "runtimeExecutable": "${execPath}",
      "args": [
        "--extensionDevelopmentPath=${workspaceFolder}",
        "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
      ],
      "outFiles": ["${workspaceFolder}/out/**/*.js"],
      "preLaunchTask": "npm: compile-tests",
      "env": {
        "TEST_TYPE": "unit"
      }
    },
    {
      "name": "Debug Specific Test",
      "type": "extensionHost",
      "request": "launch",
      "runtimeExecutable": "${execPath}",
      "args": [
        "--extensionDevelopmentPath=${workspaceFolder}",
        "--extensionTestsPath=${workspaceFolder}/out/test/suite/index"
      ],
      "outFiles": ["${workspaceFolder}/out/**/*.js"],
      "preLaunchTask": "npm: compile-tests",
      "env": {
        "MOCHA_GREP": "${input:testName}"
      }
    }
  ],
  "inputs": [
    {
      "id": "testName",
      "type": "promptString",
      "description": "Test name to run",
      "default": ""
    }
  ]
}

43.5.2 Extended tasks.json for Different Scenarios

{
  "version": "2.0.0",
  "tasks": [
    {
      "type": "npm",
      "script": "compile-tests",
      "label": "npm: compile-tests",
      "group": "build",
      "problemMatcher": "$tsc",
      "presentation": {
        "reveal": "never"
      }
    },
    {
      "type": "npm",
      "script": "watch-tests",
      "label": "npm: watch-tests",
      "group": "build",
      "problemMatcher": "$tsc-watch",
      "isBackground": true,
      "presentation": {
        "reveal": "never"
      }
    },
    {
      "label": "Clean and Compile Tests",
      "dependsOn": [
        "npm: clean",
        "npm: compile-tests"
      ],
      "group": "build"
    }
  ]
}

43.6 Professional Debug Techniques

43.6.1 1. Conditional Breakpoints

test('Loop Debugging', () => {
    for (let i = 0; i < 100; i++) {
        // Right-click on breakpoint → Edit Breakpoint
        // Condition: i === 50
        processItem(i);
    }
});

43.6.2 2. Logpoints (Logging without Code Changes)

test('Async Debugging', async () => {
    // Right-click → Add Logpoint
    // Message: "Start: {new Date().toISOString()}"
    const result = await complexOperation();
    // Logpoint: "Result: {JSON.stringify(result)}"
});

43.6.3 3. Debug Console Commands

// Inspect variables
myVariable

// Call functions
calculator.add(1, 2)

// Test VSCode API
vscode.window.activeTextEditor?.document.fileName

// Explore objects
Object.keys(vscode.commands)

43.7 Troubleshooting

43.7.1 Problem: F5 doesn’t work

Checklist: 1. ✓ Does .vscode/launch.json exist? 2. ✓ Does .vscode/tasks.json exist? 3. ✓ Is "preLaunchTask": "npm: compile-tests" set? 4. ✓ Does the label match in tasks.json?

43.7.2 Problem: Breakpoints are ignored

// tsconfig.json
{
  "compilerOptions": {
    "sourceMap": true,  // MUST be true!
    "outDir": "./out"
  }
}

43.7.3 Problem: “Cannot find module ‘vscode’”

import * as vscode from 'vscode';  // ✓ Correct
// NOT: import vscode from 'vscode';  // ✗ Wrong

The key difference between this approach and traditional Java testing lies in the debugging capabilities. While JUnit tests in Eclipse can be debugged directly within the IDE, VSCode extension tests require this specific configuration setup to enable the same level of debugging comfort. The extension host architecture means tests run in a separate VSCode instance, but with proper configuration, you get the same debugging experience you’re accustomed to from Eclipse - breakpoints, variable inspection, and step-by-step execution all work seamlessly.

43 Test Frameworks for VSCode Extensions

43.1 Objective of This Chapter