Jenkins

Caching

Offical documentation

JobCacher Jenkins community plugin

arbitraryFileCache(
    path: 'my-cache',
    cacheValidityDecidingFile: 'package-lock.json',
    includes: 'node_modules',
    excludes: '**/*.generated'
)

Debugging Pipelines

Debugging Tools for Jenkins

Jenkins provides comprehensive debugging capabilities through console logs, pipeline replay, and workspace inspection.

1. Console Output

When to use: First stop for any Jenkins build failure. Provides complete build log with timestamps and color-coded output.

How to access:

Navigate to your build (job → build number)
Click “Console Output” in left sidebar
Or append /console to build URL

What you get:

Complete stdout/stderr from entire build
Timestamps for each line (if configured)
ANSI color codes for readability
Full stack traces for errors
Environment variable output

Enable timestamps:

// Jenkinsfile
pipeline {
  options {
    timestamps()
  }
}

Debugging tips:

// Add debug output in pipeline
stage('Debug') {
  steps {
    script {
      echo "=== Environment Variables ==="
      sh 'env | sort'

      echo "=== Tool Versions ==="
      sh '''
        node --version
        npm --version
        docker --version
      '''

      echo "=== Disk Space ==="
      sh 'df -h'

      echo "=== Current Directory ==="
      sh 'pwd && ls -la'
    }
  }
}

Search console output:

Use browser’s Find (Ctrl/Cmd+F)
Download console log and search locally
Use Jenkins “Console Output” search feature if available

2. Pipeline Replay

When to use: Need to test pipeline changes without committing to repository, quickly iterate on fixes, or modify pipeline script to add debug output.

How to use:

Go to a completed pipeline build
Click “Replay” in left sidebar
Modify the pipeline script directly in browser
Click “Run” to execute modified version

What you get:

Ability to modify Jenkinsfile without committing
Test fixes quickly before pushing to git
Add debug statements temporarily
Try different configurations

Common debugging modifications:

// Original failing step
stage('Test') {
  steps {
    sh 'npm test'
  }
}

// Modified for debugging via Replay
stage('Test') {
  steps {
    // Add environment inspection
    sh '''
      echo "NODE_VERSION: $(node --version)"
      echo "NPM_VERSION: $(npm --version)"
      echo "PATH: $PATH"
    '''

    // Run with more verbose output
    sh 'npm test -- --verbose'

    // Or run subset of tests
    sh 'npm test -- --testPathPattern=failing-test.js'
  }
}

Important notes:

Replay uses the same commit, workspace, and parameters
Changes are NOT saved - commit to Jenkinsfile when working
Can replay multiple times with different modifications

3. Workspace Inspection

When to use: Need to examine build artifacts, inspect generated files, check directory structure, or understand what files were created during build.

How to access:

Navigate to build (job → build number)
Click “Workspace” in left sidebar
Browse directory structure

Or use “Wipe Out Workspace” plugin to manually inspect:

// Add to Jenkinsfile to preserve workspace
options {
  skipDefaultCheckout(false)
  preserveStashes(buildCount: 5)
}

What you get:

Browse all files in workspace
Download individual files
View file contents directly in browser
Inspect build artifacts and logs

Access workspace via SSH/command line:

// Print workspace location for SSH access
stage('Debug') {
  steps {
    echo "Workspace: ${env.WORKSPACE}"
    sh 'echo "Access workspace at: $(hostname):$PWD"'
  }
}

Archive artifacts for later inspection:

post {
  always {
    archiveArtifacts artifacts: '''
      **/target/*.jar,
      **/build/libs/*.jar,
      **/*.log,
      test-results/**/*.xml
    ''', allowEmptyArchive: true
  }
}

Tips for workspace debugging:

Check directory structure: sh 'find . -type f | head -20'
Verify files were created: sh 'ls -la dist/ || echo "dist/ not found"'
Check file permissions: sh 'ls -la'
Search for files: sh 'find . -name "*.log"'

Local Validation

Local Validation with Docker

Jenkins doesn’t have an official local runner, but you can replicate your CI environment using Docker Compose to validate pipeline logic before pushing.

When to Use This Approach

Use Docker-based local validation when:

Testing Jenkinsfile script logic
Validating build commands and dependencies
Reproducing CI environment locally
Developing new pipeline stages

Note: This won’t execute Jenkins-specific features (pipeline syntax, plugins), but validates the actual work your pipeline does.

Working Example

Create docker-compose.ci.yml in your repository:

version: '3.8'

services:
  ci-validator:
    # Match your Jenkins agent image
    image: node:18-bullseye

    working_dir: /app

    # Mount your code
    volumes:
      - .:/app
      - /app/node_modules  # Use container's node_modules

    # Replicate your pipeline stages
    command: |
      bash -c "
        echo 'Stage: Install Dependencies' &&
        npm ci &&

        echo 'Stage: Lint' &&
        npm run lint &&

        echo 'Stage: Test' &&
        npm test &&

        echo 'Stage: Build' &&
        npm run build &&

        echo '✅ All stages passed'
      "

    # Set environment variables
    environment:
      NODE_ENV: test
      CI: true

Basic Usage

# Run full validation
docker-compose -f docker-compose.ci.yml run --rm ci-validator

# Run specific stage
docker-compose -f docker-compose.ci.yml run --rm ci-validator bash -c "npm test"

# Interactive shell for debugging
docker-compose -f docker-compose.ci.yml run --rm ci-validator bash

Customization for Your Stack

Match your Jenkins agent image:

# For Java projects
image: maven:3.9-eclipse-temurin-17

# For Python projects
image: python:3.11-slim

# For multi-language projects
image: ubuntu:22.04

Add service dependencies:

If your Jenkins pipeline uses databases or caches:

services:
  ci-validator:
    image: node:18
    depends_on:
      - postgres
      - redis
    environment:
      DATABASE_URL: postgres://postgres:password@postgres:5432/test
      REDIS_URL: redis://redis:6379
    command: |
      bash -c "
        npm ci &&
        npm test  # Tests now have access to postgres and redis
      "

  postgres:
    image: postgres:15
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_DB: test

  redis:
    image: redis:7-alpine

Mount additional volumes:

volumes:
  - .:/app
  - ~/.m2:/root/.m2  # Maven cache
  - ~/.gradle:/root/.gradle  # Gradle cache
  - ./build:/app/build  # Preserve build artifacts

Replicating Jenkins Stages

Map your Jenkinsfile stages to Docker Compose commands:

Your Jenkinsfile:

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
        stage('Test') {
            steps {
                sh 'mvn test'
            }
        }
        stage('Deploy') {
            steps {
                sh './deploy.sh'
            }
        }
    }
}

Equivalent Docker Compose:

services:
  ci-validator:
    image: maven:3.9-eclipse-temurin-17
    working_dir: /app
    volumes:
      - .:/app
    command: |
      bash -c "
        echo 'Stage: Build' &&
        mvn clean package &&

        echo 'Stage: Test' &&
        mvn test &&

        echo 'Stage: Deploy' &&
        ./deploy.sh
      "

Limitations

This approach has trade-offs:

What it validates:

✅ Build commands work
✅ Tests pass in similar environment
✅ Dependencies install correctly
✅ Scripts execute without errors

What it doesn’t validate:

❌ Jenkins pipeline syntax (use Blue Ocean Pipeline Editor instead)
❌ Jenkins plugin behavior
❌ Jenkins-specific features (credentials, shared libraries)
❌ Distributed builds across multiple agents

Maintenance required:

Keep Docker Compose config in sync with Jenkinsfile changes
Update base images to match Jenkins agent updates
Manually replicate environment variables

Tips

Start simple: Begin with a single stage, add complexity as needed
Match the environment: Use the same base image as your Jenkins agents
Test incrementally: Run individual commands interactively to debug
Version the config: Commit docker-compose.ci.yml alongside your Jenkinsfile
Document differences: Note any environment differences between Docker and Jenkins in README

Alternative: Jenkins CLI

For Jenkinsfile syntax validation only:

# Install Jenkins CLI
wget http://your-jenkins-server/jnlpJars/jenkins-cli.jar

# Validate pipeline syntax
java -jar jenkins-cli.jar -s http://your-jenkins-server \
  -auth user:token \
  declarative-linter < Jenkinsfile

This validates syntax but doesn’t run the pipeline. Combine with Docker Compose for complete validation.

Parallelisation

Offical documentation

Concurrent Step Jenkins community plugin

def barrier = createBarrier count: 3;
boolean out = false;
parallel(
        await1: {
            awaitBarrier barrier
            echo "out=${out}"
        },
        await2: {
            awaitBarrier (barrier){
                sleep 2 //simulate a long time execution.
            }
            echo "out=${out}"
        },
        await3: {
            awaitBarrier (barrier){
                sleep 3 //simulate a long time execution.
                out = true
            }
            echo "out=${out}"
        }
)

Continuous Integration

Continuous Delivery

Providers

Languages

Jenkins

Caching

Debugging Pipelines

Debugging Tools for Jenkins

1. Console Output

2. Pipeline Replay

3. Workspace Inspection

Local Validation

Local Validation with Docker

When to Use This Approach

Working Example

Basic Usage

Customization for Your Stack

Replicating Jenkins Stages

Limitations

Tips

Alternative: Jenkins CLI

Parallelisation