AI Infrastructure

From Nine Minutes to Ten Seconds: A Warm Server for LTX-2.5 Video Generation

Open-weights video models pay a brutal cold-start tax — ~67 GB of weights loaded on every CLI run. We built a warm HTTP server around LTX-2.5 that keeps everything resident, and generation dropped from minutes to seconds.

August 20, 2026 · 6 min read · Cartolinks Engineering

Photo by Syed Qaarif Andrabi on Pexels

Soloa, our AI media product, needs video generation: short cinematic clips, image-to-video, lipsync. The model we settled on is LTX-2.5 from Lightricks — an open-weights, DiT-based audio-video foundation model that generates video with synchronized audio in a single pass. Under the hood it's a 22B diffusion transformer, a Gemma 4 12B text encoder, separate video and audio VAEs, and a latent upscaler.

The cold-start tax

Those components add up to roughly 67 GB of weights, and the stock CLI loads all of it on every single run — about three minutes of disk-to-GPU shuffling before a single denoising step happens. Generating a 5-second clip cold on a Mac took us around nine minutes, and even on a datacenter GPU the load time dominates for short clips. That's fine for research. It is unusable behind a product where someone clicks "generate" and waits.

Keep the weights hot

So we forked the repo and built a warm HTTP generation server on top of the stock pipelines. It loads everything once at startup, runs a small warmup generation that triggers torch.compile (~40 seconds on our CUDA pod, ~100 on a Mac), and from then on every request goes straight to denoising.

• A job queue plus sync endpoints. POST /jobs returns immediately with a job id you poll; synchronous /t2v and /i2v endpoints cover quick drafts.
• Every mode the model supports: text-to-video, image-to-video, first/last-frame interpolation, pinning arbitrary keyframes anywhere on the timeline, audio-driven lipsync, and Dub-It re-voicing of existing footage.
• Replicate-style prediction objects. Responses use the same status / input / output / metrics shape Replicate returns, so any tooling that already speaks that dialect drops in unchanged.
• Production plumbing: API-key auth, webhooks with retries, and automatic uploads to object storage.

curl -s -X POST -H "X-API-Key: $KEY" $LTX_HOST/jobs \
  -d '{"prompt": "a girl running on a beach at sunset",
       "duration": 20, "fps": 24}'

curl -s -H "X-API-Key: $KEY" $LTX_HOST/jobs/take-0001

What "warm" buys you

The pod is an RTX PRO 6000 Blackwell with 96 GB of VRAM. A 5-second 720p clip with synchronized audio in about ten seconds is a different product category from nine minutes — it turns video generation from a batch job into an interactive loop.

The details that make it production-ready

• Two-mode VRAM management. Jobs above ~620 million output pixels automatically switch to a slower stage-by-stage memory mode instead of crashing with an OOM; ~1.02 billion pixels is the hard ceiling. The mode switch is noted in the job's logs, not surfaced as an error.
• Signed webhooks. Pass a webhook URL on any request and the server POSTs the finished prediction there, retrying three times with backoff. When auth is on, the body carries an HMAC-SHA256 signature keyed with the API key so receivers can verify deliveries.
• Storage with graceful degradation. Finished videos upload to Cloudflare R2 and output becomes a public URL. If the upload fails, the response falls back to a server-relative path and the failure lands in the logs — generation never fails because of storage.
• One render at a time. The GPU is the scarce resource, so extra requests queue in order and each job carries an ETA hint the UI can display.

One codebase, two chips

One change we're particularly happy with: marker-guarded dependencies in pyproject.toml, so the exact same lockfile installs on Apple Silicon (MPS) and on CUDA Linux. We develop and test the server on an M4 Max, then deploy the identical code to the CUDA pod. The fork also ships a serverless worker wrapping the same pipeline for burst capacity, and a small Next.js studio UI that proxies to the server.

Takeaways

• Open-weights video generation is production-usable in 2026. The missing piece isn't model quality — it's serving infrastructure.
• Cold starts are the real enemy. Keeping ~67 GB of weights resident is the single change that moved us from batch-job latency to interactive latency.
• Don't invent API shapes. Copying Replicate's prediction object meant our existing tools, webhooks, and UI code worked on day one.
• You don't have to build this yourself. Everything in this article — model hosting, warm serving, storage, the API layer — is infrastructure we set up for client teams.