Background Jobs Guide

Overview

The Neuron CMS utilizes the Neuron Jobs component for asynchronous task processing and scheduled job execution. The system provides two complementary mechanisms: a cron-based scheduler for recurring tasks and a queue system for background job processing.

Key capabilities:

Job Scheduler: Execute tasks at scheduled times using cron expressions
Queue System: Process jobs asynchronously in the background
Multiple Queue Drivers: Database (recommended), File, Sync (testing)
Automatic Retry: Exponential backoff with configurable max attempts
Failed Job Management: Track, inspect, retry, and delete failed jobs
Delayed Jobs: Schedule jobs to run at specific future times
Horizontal Scaling: Run multiple workers concurrently

Architecture

Core Components

Scheduler: Cron-like task scheduling without system cron dependency
QueueManager (Neuron\Jobs\Queue\QueueManager): Orchestrates job dispatch and processing
Queue Drivers: Database, File, or Sync storage backends
Worker Processes: Long-running processes that execute queued jobs
Failed Job Tracker: Records and manages failed job attempts

System Flow

Scheduled Jobs: Defined in YAML configuration, executed by scheduler based on cron expressions
Queue Dispatch: Jobs can be queued directly or dispatched from scheduled jobs
Worker Processing: Background workers poll queues and execute jobs
Retry Logic: Failed jobs automatically retry with exponential backoff
Failed Job Handling: Jobs exceeding max attempts moved to failed jobs table

Installation

Install Component

The Jobs component is included via Composer:

composer require neuron-php/jobs

Database Setup

For the database queue driver (recommended for production), create the required tables.

Migration Method

Copy the migration stub:

cp vendor/neuron-php/jobs/resources/migrations/create_queue_tables.php.stub db/migrate/YYYYMMDDHHMMSS_create_queue_tables.php

Update the class name to match the filename, then run the migration:

./vendor/bin/neuron db:migrate

Manual SQL Creation

SQLite

-- jobs table
CREATE TABLE jobs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    queue VARCHAR(255) NOT NULL DEFAULT 'default',
    payload TEXT NOT NULL,
    attempts INTEGER NOT NULL DEFAULT 0,
    reserved_at INTEGER NULL,
    available_at INTEGER NOT NULL,
    created_at INTEGER NOT NULL
);

CREATE INDEX idx_queue_reserved ON jobs(queue, reserved_at);

-- failed_jobs table
CREATE TABLE failed_jobs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    queue VARCHAR(255) NOT NULL,
    payload TEXT NOT NULL,
    exception TEXT NOT NULL,
    failed_at INTEGER NOT NULL
);

MySQL

CREATE TABLE jobs (
    id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    queue VARCHAR(255) NOT NULL DEFAULT 'default',
    payload LONGTEXT NOT NULL,
    attempts TINYINT UNSIGNED NOT NULL DEFAULT 0,
    reserved_at INT UNSIGNED NULL,
    available_at INT UNSIGNED NOT NULL,
    created_at INT UNSIGNED NOT NULL,
    INDEX idx_queue_reserved (queue, reserved_at)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

CREATE TABLE failed_jobs (
    id BIGINT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    queue VARCHAR(255) NOT NULL,
    payload LONGTEXT NOT NULL,
    exception LONGTEXT NOT NULL,
    failed_at INT UNSIGNED NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

PostgreSQL

CREATE TABLE jobs (
    id BIGSERIAL PRIMARY KEY,
    queue VARCHAR(255) NOT NULL DEFAULT 'default',
    payload TEXT NOT NULL,
    attempts SMALLINT NOT NULL DEFAULT 0,
    reserved_at INTEGER NULL,
    available_at INTEGER NOT NULL,
    created_at INTEGER NOT NULL
);

CREATE INDEX idx_queue_reserved ON jobs(queue, reserved_at);

CREATE TABLE failed_jobs (
    id BIGSERIAL PRIMARY KEY,
    queue VARCHAR(255) NOT NULL,
    payload TEXT NOT NULL,
    exception TEXT NOT NULL,
    failed_at INTEGER NOT NULL
);

Configuration

Queue Configuration

Add queue settings to config/neuron.yaml:

queue:
  driver: database          # database, file, or sync
  default: default          # Default queue name
  retry_after: 90          # Seconds before retrying reserved jobs
  max_attempts: 3          # Maximum retry attempts
  backoff: 10              # Backoff multiplier for exponential backoff
  file_path: storage/queue # Path for file queue (if using file driver)

database:
  adapter: sqlite
  name: storage/database.sqlite3

Configuration Options:

Option	Type	Default	Description
`queue.driver`	string	`database`	Queue driver: database, file, or sync
`queue.default`	string	`default`	Default queue name
`queue.retry_after`	int	`90`	Seconds before job is retried
`queue.max_attempts`	int	`3`	Maximum retry attempts
`queue.backoff`	int	`10`	Backoff multiplier for exponential backoff
`queue.file_path`	string	`storage/queue`	Path for file queue storage

Queue Drivers

Database Driver (Recommended)

Best for production. Provides reliability, ACID guarantees, and supports concurrent workers.

queue:
  driver: database

Pros:

Persistent storage
Concurrent worker support
Reliable with transaction support
Queryable job history

Cons:

Requires database setup
Slight performance overhead

File Driver

Simple file-based queue. No database required, but not suitable for high-throughput applications.

queue:
  driver: file
  file_path: storage/queue

Pros:

No database required
Simple setup
Good for development

Cons:

Not suitable for high concurrency
File system locking issues possible
Harder to monitor

Sync Driver

Executes jobs immediately (synchronously). Useful for testing and local development.

queue:
  driver: sync

Pros:

Immediate execution
Perfect for testing
No setup required

Cons:

Not asynchronous
Blocks request processing
Not for production

Job Scheduling

Schedule Configuration

Create config/schedule.yaml:

schedule:
  # Quick job - runs directly in scheduler process
  log_cleanup:
    class: App\Jobs\CleanupLogs
    cron: "0 * * * *"              # Every hour
    args:
      max_age_days: 30

  # Long-running job - dispatch to background queue
  daily_backup:
    class: App\Jobs\BackupDatabase
    cron: "0 2 * * *"              # Daily at 2 AM
    queue: backups                 # Dispatches to 'backups' queue

  # Email job dispatched to queue
  send_reminders:
    class: App\Jobs\SendReminders
    cron: "0 9 * * *"              # Daily at 9 AM
    queue: emails

  # Quick sync - runs directly
  api_sync:
    class: App\Jobs\QuickSync
    cron: "*/15 * * * *"           # Every 15 minutes
    args:
      endpoint: https://api.example.com/sync

Key Points:

No queue field → Job runs directly in scheduler process (use for quick jobs < 1 second)
With queue field → Job dispatched to queue for background processing (use for long-running jobs)

Cron Expression Format

Cron expressions define when jobs should run:

* * * * *
| | | | |
| | | | +-- Day of week (0-6, Sunday = 0)
| | | +---- Month (1-12)
| | +------ Day of month (1-31)
| +-------- Hour (0-23)
+---------- Minute (0-59)

Common Patterns:

Pattern	Description
`* * * * *`	Every minute
`0 * * * *`	Every hour
`0 0 * * *`	Daily at midnight
`0 2 * * *`	Daily at 2 AM
`0 /6 * *`	Every 6 hours
`30 9 * * 1`	Every Monday at 9:30 AM
`0 0 1 * *`	First day of every month
`0 9 * * 1-5`	Weekdays at 9 AM
`/15 * * *`	Every 15 minutes

Use crontab.guru to build and test cron expressions.

Creating Jobs

All jobs must implement the Neuron\Jobs\IJob interface.

Basic Job Example

Create src/Jobs/SendEmailJob.php:

<?php

namespace App\Jobs;

use Neuron\Jobs\IJob;
use Neuron\Log\Log;

class SendEmailJob implements IJob
{
    public function getName(): string
    {
        return 'send_email';
    }

    public function run( array $args = [] ): mixed
    {
        $to = $args['to'] ?? '';
        $subject = $args['subject'] ?? '';
        $body = $args['body'] ?? '';

        if( !$to )
{
            throw new \InvalidArgumentException( 'Email recipient is required' );
        }

        Log::info( "Sending email to: {$to}" );

        // Send email logic here
        mail( $to, $subject, $body );

        Log::info( "Email sent successfully" );

        return true;
    }
}

Job with Error Handling

<?php

namespace App\Jobs;

use Neuron\Jobs\IJob;
use Neuron\Log\Log;

class ProcessImageJob implements IJob
{
    public function getName(): string
    {
        return 'process_image';
    }

    public function run( array $args = [] ): mixed
    {
        $path = $args['path'] ?? '';
        $operations = $args['operations'] ?? [];

        if (!file_exists( $path )) {
            Log::error( "Image not found: {$path}" );
            throw new \RuntimeException( "Image file not found" );
        }

        Log::info( "Processing image: {$path}" );

        try {
            foreach( $operations as $operation => $params )
{
                $this->performOperation( $path, $operation, $params );
            }

            Log::info( "Image processed successfully" );
            return ['success' => true, 'path' => $path];
        } catch( \Exception $e )
{
            Log::error( "Image processing failed: {$e->getMessage()}");
            throw $e; // Re-throw to trigger retry
        }
    }

    private function performOperation( string $path, string $operation, array $params ): void
    {
        match ($operation) {
            'resize' => $this->resize( $path, $params ),
            'crop' => $this->crop( $path, $params ),
            'watermark' => $this->watermark( $path, $params ),
            default => throw new \InvalidArgumentException( "Unknown operation: {$operation}" )
        };
    }

    private function resize( string $path, array $params ): void
    {
        // Resize implementation
    }

    private function crop( string $path, array $params ): void
    {
        // Crop implementation
    }

    private function watermark( string $path, array $params ): void
    {
        // Watermark implementation
    }
}

Job with External API Integration

<?php

namespace App\Jobs;

use Neuron\Jobs\IJob;
use Neuron\Log\Log;

class SyncDataJob implements IJob
{
    public function getName(): string
    {
        return 'sync_data';
    }

    public function run( array $args = [] ): mixed
    {
        $endpoint = $args['endpoint'] ?? '';
        $apiKey = $args['api_key'] ?? '';

        Log::info( "Starting data sync from: {$endpoint}" );

        $ch = curl_init( $endpoint );
        curl_setopt_array( $ch, [
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_HTTPHEADER => [
                "Authorization: Bearer {$apiKey}",
                "Content-Type: application/json"
            ],
            CURLOPT_TIMEOUT => 30
        ] );

        $response = curl_exec( $ch );
        $httpCode = curl_getinfo( $ch, CURLINFO_HTTP_CODE );
        curl_close( $ch );

        if( $httpCode !== 200 )
{
            Log::error( "Sync failed with HTTP code: {$httpCode}" );
            throw new \RuntimeException( "API request failed" );
        }

        $data = json_decode( $response, true );
        $this->processData( $data );

        Log::info( "Data sync completed successfully" );
        return ['synced' => count( $data ), 'timestamp' => time()];
    }

    private function processData( array $data ): void
    {
        // Data processing implementation
    }
}

Dispatching Jobs

Using Helper Functions

The Jobs component provides convenient global helper functions.

Basic Dispatch

use App\Jobs\SendEmailJob;

// Dispatch to default queue
dispatch( new SendEmailJob(), [
    'to' => '[email protected]',
    'subject' => 'Welcome!',
    'body' => 'Welcome to our application'
]);

Specific Queue

// Dispatch to 'emails' queue
dispatch( new SendEmailJob(),
    ['to' => '[email protected]', 'subject' => 'Welcome'],
    'emails'
);

Delayed Job

// Delay execution by 1 hour (3600 seconds)
dispatch( new SendReminderJob(),
    ['order_id' => 123],
    'default',
    3600
);

Immediate Execution

// Execute immediately, bypassing the queue
$result = dispatchNow( new ProcessDataJob(),
    ['data' => $data]
);

Using QueueManager

use Neuron\Jobs\Queue\QueueManager;

$queueManager = new QueueManager();

// Dispatch job
$jobId = $queueManager->dispatch( new SendEmailJob(),
    ['to' => '[email protected]'],
    'emails',
    0
);

// Execute immediately
$result = $queueManager->dispatchNow( new ProcessDataJob(),
    ['data' => $data]
);

Running the Scheduler

The scheduler executes jobs defined in config/schedule.yaml based on cron expressions.

Daemon Mode (Continuous Polling)

Runs continuously, checking for scheduled jobs at regular intervals:

# Start scheduler (polls every 60 seconds by default)
./vendor/bin/neuron jobs:schedule

# Custom polling interval (every 30 seconds)
./vendor/bin/neuron jobs:schedule --interval=30

# Poll every 5 minutes
./vendor/bin/neuron jobs:schedule --interval=300

Recommended for development and simple deployments.

Single Poll Mode

Executes a single check and then exits:

./vendor/bin/neuron jobs:schedule --poll

Use with system cron (recommended for production):

Add to crontab to run every minute:

* * * * * cd /path/to/project && ./vendor/bin/neuron jobs:schedule --poll >> /dev/null 2>&1

Queue Workers

Queue workers process jobs from the queue in the background.

Starting a Worker

# Process jobs from default queue
./vendor/bin/neuron jobs:work

# Process jobs from specific queue
./vendor/bin/neuron jobs:work --queue=emails

# Process multiple queues with priority (high first, then default, then low)
./vendor/bin/neuron jobs:work --queue=high,default,low

Worker Options

# Process one job then exit
./vendor/bin/neuron jobs:work --once

# Stop when queue is empty
./vendor/bin/neuron jobs:work --stop-when-empty

# Custom sleep time when queue is empty (default: 3 seconds)
./vendor/bin/neuron jobs:work --sleep=5

# Max jobs to process before stopping (default: 0 = unlimited)
./vendor/bin/neuron jobs:work --max-jobs=100

# Job timeout in seconds (default: 60)
./vendor/bin/neuron jobs:work --timeout=120

# Memory limit in MB
./vendor/bin/neuron jobs:work --memory=128

Running Multiple Workers

For high-throughput applications, run multiple workers concurrently:

# Terminal 1: High priority queue
./vendor/bin/neuron jobs:work --queue=high

# Terminal 2: Default queue
./vendor/bin/neuron jobs:work --queue=default

# Terminal 3: Low priority queue
./vendor/bin/neuron jobs:work --queue=low

Worker Management

Systemd Service (Recommended for Production)

Create /etc/systemd/system/[email protected]:

[Unit]
Description=Neuron Queue Worker %i
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/app
ExecStart=/usr/bin/php /var/www/app/vendor/bin/neuron jobs:work %i --sleep=3 --timeout=60
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Run multiple workers:

# Start 3 workers for default queue
sudo systemctl enable neuron-worker@default
sudo systemctl start neuron-worker@default

# Start workers for specific queues
sudo systemctl enable neuron-worker@emails
sudo systemctl start neuron-worker@emails

sudo systemctl enable neuron-worker@images
sudo systemctl start neuron-worker@images

Supervisor Configuration

Create /etc/supervisor/conf.d/neuron-workers.conf:

[program:neuron-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /var/www/app/vendor/bin/neuron jobs:work --queue=default --sleep=3
directory=/var/www/app
user=www-data
autostart=true
autorestart=true
numprocs=3
redirect_stderr=true
stdout_logfile=/var/www/app/storage/logs/worker.log

Reload supervisor:

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start neuron-worker:*

Scheduler Deployment

Systemd Service for Scheduler

Create /etc/systemd/system/neuron-scheduler.service:

[Unit]
Description=Neuron Job Scheduler
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/app
ExecStart=/usr/bin/php /var/www/app/vendor/bin/neuron jobs:schedule
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

Enable and start:

sudo systemctl enable neuron-scheduler
sudo systemctl start neuron-scheduler
sudo systemctl status neuron-scheduler

System Cron (Alternative)

Add to crontab:

* * * * * cd /var/www/app && /usr/bin/php vendor/bin/neuron jobs:schedule --poll >> /dev/null 2>&1

Failed Job Handling

Viewing Failed Jobs

./vendor/bin/neuron jobs:failed

Output example:

Failed Jobs (2):
────────────────────────────────────────────────────────────────────────────────

ID: job_67890.12345
Queue: emails
Job: App\Jobs\SendWelcomeEmailJob
Failed: 2025-01-08 10:30:00
Exception:
RuntimeException: SMTP connection failed
#0 /path/to/EmailService.php(123): send()
#1 /path/to/SendWelcomeEmailJob.php(45): sendEmail()
────────────────────────────────────────────────────────────────────────────────

Retrying Failed Jobs

# Retry specific job
./vendor/bin/neuron jobs:retry job_67890.12345

# Retry all failed jobs
./vendor/bin/neuron jobs:retry --all

Deleting Failed Jobs

# Delete specific failed job
./vendor/bin/neuron jobs:forget job_67890.12345

# Clear all failed jobs
./vendor/bin/neuron jobs:flush --failed

Queue Management

Queue Statistics

./vendor/bin/neuron jobs:stats

# Multiple queues
./vendor/bin/neuron jobs:stats --queue=default,emails,processing

Output example:

Queue Statistics
════════════════════════════════════════════════════════════

Queue: default
  Pending jobs: 42

Queue: emails
  Pending jobs: 15

Failed Jobs: 3

════════════════════════════════════════════════════════════
Total pending jobs: 57
Total failed jobs: 3

Clearing Queues

# Clear default queue
./vendor/bin/neuron jobs:flush

# Clear specific queue
./vendor/bin/neuron jobs:flush --queue=emails

# Clear all failed jobs
./vendor/bin/neuron jobs:flush --failed

Common Patterns

Email Notifications

// In controller
public function register(): void
{
    $user = $this->userRepository->create( $userData );

    // Queue welcome email
    dispatch( new SendWelcomeEmailJob(), [
        'email' => $user->getEmail(),
        'name' => $user->getUsername()
    ], 'emails');

    // Immediate response to user
    redirect( '/welcome' );
}

Event Listeners

use Neuron\Events\IListener;
use Neuron\Events\IEvent;

class SendOrderConfirmationListener implements IListener
{
    public function handle( IEvent $event ): void
    {
        $order = $event->getOrder();

        // Queue order confirmation email
        dispatch( new SendOrderConfirmationJob(), [
            'order_id' => $order->getId(),
            'email' => $order->getCustomerEmail()
        ], 'emails');
    }
}

Scheduled Jobs Dispatching Queue Jobs

use Neuron\Jobs\IJob;

class DailyReportJob implements IJob
{
    public function getName(): string
    {
        return 'daily_report';
    }

    public function run( array $args = [] ): mixed
    {
        $users = $this->getUsersForReport();

        // Queue individual report emails
        foreach( $users as $user )
{
            dispatch( new SendReportEmailJob(), [
                'user_id' => $user->getId()
            ], 'emails');
        }

        return count( $users );
    }
}

Image Processing

// Controller
public function uploadImage(): void
{
    $path = $this->saveUploadedFile( $_FILES['image'] );

    // Queue image processing
    dispatch( new ProcessImageJob(), [
        'path' => $path,
        'sizes' => ['thumb', 'medium', 'large']
    ], 'processing');

    respond()->json( ['status' => 'processing'] );
}

// Job
class ProcessImageJob implements IJob
{
    public function run( array $args = [] ): mixed
    {
        $path = $args['path'];
        $sizes = $args['sizes'];

        foreach( $sizes as $size )
{
            $this->resizeImage( $path, $size );
        }

        return true;
    }
}

Rate Limiting

class SendApiRequestJob implements IJob
{
    public function run( array $args = [] ): mixed
    {
        // Rate limit: 1 request per second
        sleep( 1 );

        $response = $this->apiClient->request( $args['endpoint'],
            $args['data'] );

        return $response;
    }
}

// Dispatch multiple requests
foreach( $endpoints as $endpoint )
{
    dispatch( new SendApiRequestJob(), [
        'endpoint' => $endpoint,
        'data' => $data
    ], 'api-requests');
}

Batch Processing

class ImportDataJob implements IJob
{
    public function run( array $args = [] ): mixed
    {
        $file = $args['file'];
        $batchSize = 1000;

        $rows = $this->readCSV( $file );
        $batches = array_chunk( $rows, $batchSize );

        foreach( $batches as $index => $batch )
{
            dispatch( new ProcessBatchJob(), [
                'batch' => $batch,
                'batch_number' => $index + 1
            ], 'processing');
        }

        return count( $batches );
    }
}

Error Handling and Retries

Automatic Retry

Failed jobs are automatically retried based on configuration:

queue:
  max_attempts: 3          # Maximum retry attempts
  backoff: 10              # Backoff multiplier for exponential backoff
  retry_after: 90          # Seconds before making reserved jobs available

Retry delays with backoff: 10:

Attempt 1: Immediate
Attempt 2: 10 seconds
Attempt 3: 20 seconds
Attempt 4: 40 seconds

After exceeding max_attempts, jobs are moved to the failed jobs table.

Job Exception Handling

public function run( array $args = [] ): mixed
{
    try {
        // Job logic here
        $this->performTask( $args );

        return ['success' => true];
    } catch( \Exception $e )
{
        Log::error( "Job failed: {$e->getMessage()}");

        // Re-throw to trigger retry mechanism
        throw $e;

        // Or return error without retry (don't re-throw)
        // return ['success' => false, 'error' => $e->getMessage()];
    }
}

Best Practices

1. Keep Jobs Small and Focused

// Good: Single responsibility
class SendEmailJob implements IJob { /* ... */ }
class ProcessImageJob implements IJob { /* ... */ }

// Avoid: Multiple responsibilities
class SendEmailAndProcessImageJob implements IJob { /* ... */ }

2. Handle Failures Gracefully

public function run( array $args = [] ): mixed
{
    try {
        $this->doWork( $args );
        return true;
    } catch( \Exception $e )
{
        Log::error( "Job failed: " . $e->getMessage());
        throw $e; // Re-throw for retry logic
    }
}

3. Use Specific Queues for Priority

// High priority
dispatch( new ProcessPaymentJob(), $data, 'high');

// Normal priority
dispatch( new SendEmailJob(), $data, 'default');

// Low priority
dispatch( new GenerateReportJob(), $data, 'low');

4. Monitor Failed Jobs

Regularly check for failed jobs and investigate failures:

./vendor/bin/neuron jobs:failed

Set up alerts when failed job count exceeds threshold.

5. Set Appropriate Timeouts

# Short jobs
./vendor/bin/neuron jobs:work --timeout=30

# Long-running jobs
./vendor/bin/neuron jobs:work --timeout=300

6. Use Delayed Jobs for Scheduled Actions

// Send reminder 24 hours later
dispatch( new SendReminderJob(),
    ['order_id' => $order->getId()],
    'default',
    86400  // 24 hours in seconds
);

7. Test with Sync Driver

# config/neuron.test.yaml
queue:
  driver: sync

This executes jobs immediately during tests, simplifying test assertions.

8. Implement Proper Logging

public function run( array $args = [] ): mixed
{
    Log::info( "Starting job: {$this->getName()}", $args);

    try {
        $result = $this->performWork( $args );
        Log::info( "Job completed successfully", ['result' => $result] );
        return $result;
    } catch( \Exception $e )
{
        Log::error( "Job failed: {$e->getMessage()}", [
            'exception' => $e,
            'args' => $args
        ]);
        throw $e;
    }
}

Monitoring

Queue Health Checks

Create a health check script:

<?php

require 'vendor/autoload.php';

// Check queue size
$queueSize = queueSize();
if( $queueSize > 1000 )
{
    echo "WARNING: Queue is backed up ($queueSize jobs)\n";
    exit( 1 );
}

// Check failed jobs
$queueManager = getQueueManager();
$failedCount = count( $queueManager->getFailedJobs());
if( $failedCount > 50 )
{
    echo "WARNING: Too many failed jobs ($failedCount)\n";
    exit( 1 );
}

echo "OK\n";
exit( 0 );

Monitoring Logs

Configure comprehensive logging:

logging:
  destination: \Neuron\Log\Destination\File
  format: \Neuron\Log\Format\Json
  file: storage/logs/jobs.log
  level: info

Monitor logs in real-time:

tail -f storage/logs/jobs.log

Metrics to Track

Queue depth: Number of pending jobs per queue
Processing rate: Jobs processed per minute
Failed job count: Number of failed jobs
Worker health: Number of active workers
Average job duration: Time to process jobs
Retry rate: Percentage of jobs requiring retry

Troubleshooting

Jobs Not Processing

Symptoms: Jobs queued but not executing

Solutions:

Check worker is running:
```
ps aux | grep "jobs:work"
```
Check queue size:
```
./vendor/bin/neuron jobs:stats
```
Check logs:
```
tail -f storage/logs/app.log
```
Verify database connection (for database driver)
Check file permissions (for file driver)

High Memory Usage

Symptoms: Workers consuming excessive memory

Solutions:

Set memory limit:

./vendor/bin/neuron jobs:work --memory=128

Restart workers periodically via systemd/supervisor
Check for memory leaks in job code

Use --once flag and run workers via cron:

* * * * * cd /path/to/app && php vendor/bin/neuron jobs:work --once

Jobs Timing Out

Symptoms: Jobs fail with timeout errors

Solutions:

Increase timeout value:

./vendor/bin/neuron jobs:work --timeout=300

Break large jobs into smaller chunks
Use batch processing pattern
Optimize job code

Failed Jobs Accumulating

Symptoms: Many jobs in failed jobs table

Solutions:

Review exception messages:
```
./vendor/bin/neuron jobs:failed
```
Fix underlying issues in job code
Retry failed jobs:
```
./vendor/bin/neuron jobs:retry --all
```

Clear old failed jobs:

./vendor/bin/neuron jobs:flush --failed

Scheduler Not Running Jobs

Symptoms: Scheduled jobs not executing at expected times

Solutions:

Verify cron expression syntax at crontab.guru
Check system timezone matches configuration:
```
system:
  timezone: America/New_York
```
Ensure scheduler is running:
```
ps aux | grep "jobs:schedule"
```
Check logs for errors:
```
tail -f storage/logs/app.log
```
Verify schedule configuration exists at config/schedule.yaml

Queue Performance Issues

Symptoms: Slow queue processing

Solutions:

Increase number of workers:

# Run 5 workers
for i in {1..5}; do
    ./vendor/bin/neuron jobs:work &
done

Use multiple queues with priority
Optimize job code for performance
Consider database indexing (for database driver)
Use dedicated queue servers for high throughput

CLI Commands Reference

Scheduler Commands

# Start scheduler in daemon mode
./vendor/bin/neuron jobs:schedule

# Single poll
./vendor/bin/neuron jobs:schedule --poll

# Set polling interval
./vendor/bin/neuron jobs:schedule --interval=30

Worker Commands

# Start worker
./vendor/bin/neuron jobs:work

# Work specific queue
./vendor/bin/neuron jobs:work --queue=emails

# Process one job and exit
./vendor/bin/neuron jobs:work --once

# Stop when queue empty
./vendor/bin/neuron jobs:work --stop-when-empty

# Set options
./vendor/bin/neuron jobs:work --timeout=120 --sleep=5 --memory=128

Queue Management Commands

# View queue statistics
./vendor/bin/neuron jobs:stats

# View failed jobs
./vendor/bin/neuron jobs:failed

# Retry a specific failed job
./vendor/bin/neuron jobs:retry <job-id>

# Retry all failed jobs
./vendor/bin/neuron jobs:retry --all

# Forget (delete) a failed job
./vendor/bin/neuron jobs:forget <job-id>

# Clear all jobs from queue
./vendor/bin/neuron jobs:flush

# Clear specific queue
./vendor/bin/neuron jobs:flush --queue=emails

# Clear only failed jobs
./vendor/bin/neuron jobs:flush --failed

Helper Functions API

dispatch()

Dispatch a job to the queue for background processing.

dispatch( IJob $job, array $args = [], ?string $queue = null, int $delay = 0 ): string

Parameters:

$job - Job instance to queue
$args - Arguments to pass to the job
$queue - Queue name (null = default queue)
$delay - Delay in seconds before job becomes available

Returns: Job ID

dispatchNow()

Execute a job immediately, bypassing the queue.

dispatchNow( IJob $job, array $args = [] ): mixed

Parameters:

$job - Job instance to execute
$args - Arguments to pass to the job

Returns: Job result

queueSize()

Get the number of jobs in a queue.

queueSize( ?string $queue = null ): int

clearQueue()

Clear all jobs from a queue.

clearQueue( ?string $queue = null ): int

Returns: Number of jobs cleared

getQueueManager()

Get the queue manager instance.

getQueueManager(): QueueManager