HeliumTS is under active development and may have bugs or breaking changes. Report issues on GitHub.
Background Workers
Overview
HeliumTS provides defineWorker for creating long-running background processes that start when the server starts and continue running until the server shuts down. This is ideal for:
- Queue consumers (Redis, RabbitMQ, SQS, etc.)
- Background task processors
- Scheduled jobs and cron-like tasks
- Real-time data synchronization
- Cache warmers and data pre-loaders
- Periodic service synchronization
Workers eliminate the need for separate microservices or monorepo setups like Turborepo - everything runs in the same process, sharing the same code, services, types, and models.
Basic Usage
Create a worker file in your src/server directory:
Server (src/server/workers/queueConsumer.ts):
1import { defineWorker } from "heliumts/server";23export const queueConsumer = defineWorker(4 async (_ctx, { signal }) => {5 console.log("Queue consumer started");67 while (!signal.aborted) {8 // Poll for jobs9 const job = await queue.pop();10 if (signal.aborted) break;1112 if (job) {13 await processJob(job);14 }1516 // Wait before polling again17 await new Promise((resolve) => setTimeout(resolve, 1000));18 }19 },20 { name: "queueConsumer" }21);
When the server starts, you'll see:
1Starting worker 'queueConsumer'
Worker Options
1interface WorkerOptions {2 /**3 * The name of the worker, used for logging and identification.4 * If not provided, the handler function name will be used.5 */6 name?: string;78 /**9 * Whether the worker should automatically restart if it crashes.10 * Default: true11 */12 autoRestart?: boolean;1314 /**15 * Delay in milliseconds before restarting the worker after a crash.16 * Default: 5000 (5 seconds)17 */18 restartDelayMs?: number;1920 /**21 * Maximum number of retries after the initial run before giving up.22 * Set to 0 for unlimited restarts.23 * Default: 0 (unlimited)24 */25 maxRestarts?: number;2627 /**28 * Whether to start the worker automatically on server startup.29 * Default: true30 */31 autoStart?: boolean;32}3334interface WorkerLifecycle {35 signal: AbortSignal;36 onCleanup(cleanup: () => Promise<void> | void): void;37}
The worker handler receives ctx as the first argument and a lifecycle object as the second:
1type WorkerHandler = (2 ctx: HeliumContext,3 lifecycle: WorkerLifecycle4) => Promise<(() => Promise<void> | void) | void> | (() => Promise<void> | void) | void;
signalis aborted when Helium stops the worker during shutdown or hot reload.onCleanup(...)registers teardown logic for change streams, intervals, event listeners, queue consumers, and similar resources.- Returning a cleanup function is shorthand for workers that start background resources and should stay alive until Helium aborts them.
maxRestarts: 1 permits the initial run and one retry. Stopping cancels pending restart delays. Cleanup registered or returned after asynchronous setup finishes still runs, even when the worker was stopped while setup was pending.
1export const heartbeat = defineWorker(() => {2 const timer = setInterval(() => reportHealth(), 30_000);3 return () => clearInterval(timer);4}, { name: "heartbeat" });
Example with Options
1import { defineWorker } from "heliumts/server";23export const dataSync = defineWorker(4 async (ctx, { signal }) => {5 while (true) {6 if (signal.aborted) {7 break;8 }910 await syncDataFromExternalAPI();11 await new Promise((resolve) => setTimeout(resolve, 30000)); // Every 30 seconds12 }13 },14 {15 name: "dataSync",16 autoRestart: true,17 restartDelayMs: 10000, // Wait 10 seconds before restarting18 maxRestarts: 5, // Give up after 5 restart attempts19 }20);
Context Access
Workers receive a HeliumContext object. HTTP and RPC middleware does not run for workers; import application services directly instead of assuming middleware populated the context:
1import { defineWorker } from "heliumts/server";23import { db } from "../database"; // Your application service45export const contextExample = defineWorker(6 async (ctx, { signal }) => {7 // Access context properties8 console.log("Worker context:", ctx);910 while (!signal.aborted) {11 await performTask(db);12 await new Promise((resolve) => setTimeout(resolve, 5000));13 }14 },15 { name: "contextExample" }16);
Error Handling
Workers automatically handle errors with configurable restart behavior:
1import { defineWorker } from "heliumts/server";23export const resilientWorker = defineWorker(4 async (ctx) => {5 while (true) {6 try {7 await riskyOperation();8 } catch (error) {9 console.error("Operation failed:", error);10 // The worker continues running11 }1213 await new Promise((resolve) => setTimeout(resolve, 1000));14 }15 },16 { name: "resilientWorker" }17);1819// If the entire worker crashes, it will restart automatically20export const crashingWorker = defineWorker(21 async (ctx) => {22 // This will crash and restart up to 3 times23 throw new Error("Something went wrong!");24 },25 {26 name: "crashingWorker",27 autoRestart: true,28 maxRestarts: 3,29 restartDelayMs: 5000,30 }31);
Graceful Shutdown
Workers are automatically stopped when the server shuts down (SIGINT/SIGTERM):
1^C2Shutting down...3Stopped 3 worker(s)4Server closed
Multiple Workers
You can define multiple workers in the same file or across different files:
Server (src/server/workers/index.ts):
1import { defineWorker } from "heliumts/server";23export const worker1 = defineWorker(4 async (ctx) => {5 // Worker 1 logic6 },7 { name: "worker1" }8);910export const worker2 = defineWorker(11 async (ctx) => {12 // Worker 2 logic13 },14 { name: "worker2" }15);1617export const worker3 = defineWorker(18 async (ctx) => {19 // Worker 3 logic20 },21 { name: "worker3" }22);
Startup output:
1Starting worker 'worker1'2Starting worker 'worker2'3Starting worker 'worker3'
Best Practices
- Use descriptive names: Give workers meaningful names for easy identification in logs
- Handle errors gracefully: Catch errors within your worker loop to prevent unnecessary restarts
- Use appropriate restart settings: Set
maxRestartsto prevent infinite restart loops - Clean up resources: If your worker allocates resources (connections, file handles), clean them up on errors
- Log important events: Add logging for visibility into worker behavior
- Use long polling: For queue consumers, use long polling to reduce CPU usage
- Monitor worker health: Use
getWorkerStatus()to monitor worker states
TypeScript Support
Workers are fully typed:
1import { defineWorker, WorkerOptions, HeliumWorkerDef } from "heliumts/server";23const options: WorkerOptions = {4 name: "typedWorker",5 autoRestart: true,6 maxRestarts: 5,7};89export const typedWorker: HeliumWorkerDef = defineWorker(async (ctx) => {10 // Fully typed worker11}, options);
Why Workers Instead of Monorepos?
Traditional approaches require:
- Separate repositories or monorepo tools (Turborepo, Nx)
- Separate deployment pipelines
- Code duplication or complex package sharing
- Multiple running processes
With Helium workers:
- Single codebase: Everything in one place
- Shared code: Workers use the same services, types, and models as your RPC methods
- Single deployment: Deploy once, run everything
- Simplified architecture: No inter-service communication needed
- Type safety: Full TypeScript support across the entire application