Hardware-isolate,

Scale to Zero Compute

Serverless GPU Inference Cloud

Deploy production-grade AI inference endpoints that automatically scale down to zero. Pay only for the exact milliseconds your container executes.

Developer Integration

One API endpoint. Zero infrastructure code.

Drop-in compatible with standard OpenAI SDKs. Point your existing LangChain, LlamaIndex, or agent loops to Istidlal's high-throughput serverless gateway with zero code refactoring.

100% OpenAI Drop-in SDKMulti-Language Linting & Type-SafetySub-3s Cold Start · Scale to Zero
serverless-inference.ts
1
import OpenAI from "openai";
2
3
// Initialize OpenAI client pointed to Istidlal Serverless Gateway
4
const client = new OpenAI({
5
baseURL: "https://api.istidlal.com/v1",
6
apiKey: process.env.ISTIDLAL_API_KEY,
7
});
8
9
// Endpoint automatically cold-starts GPU & scales down to 0ms when idle
10
const completion = await client.chat.completions.create({
11
model: "llama-3.3-70b-instruct",
12
messages: [
13
{ role: "user", content: "Explain quantum GPU interconnects." },
14
],
15
temperature: 0.7,
16
});
17
18
console.log(completion.choices[0].message.content);
Zero code refactoring · Standard OpenAI SDK compatible
Scale to Zero Active
Why Serverless GPU?

Eliminate idle GPU overhead completely

Engineered from silicon up for modern AI workloads. Get dedicated hardware isolation, sub-3s cold starts, and zero idle charges.

// 01/< 2.4s Cold Start

Sub-3s Cold Starts from True Zero

Legacy GPU clouds spend 3 to 5 minutes provisioning nodes, pulling container images, and loading weights into VRAM. Istidlal streams cached model layers over NVMe-oF fabrics directly into GPU HBM in under 2.4 seconds.

Traditional Cloud (AWS / GCP)180s – 300s
VM Boot (45s) → Docker (60s) → Weights (90s)High Latency
Istidlal Serverless GPU< 2.4s (Warm: 18ms)
Pre-warmed Container + NVMe-oF Streaming75× Faster
// 02/0ms Idle Cost

Zero Waste Scale-to-Zero

Stop paying for inactive GPUs running 24/7 during off-peak hours. When incoming request volume drops to zero, billing drops immediately to exactly zero with zero lingering container reservations.

Simulate Traffic:
Active GPU Replicas:0 Replicas (Suspended)
Current Cost / Sec:$0.0000 / sec
Dedicated H100: $2,480/moSave up to 88%
// 03/MIG Protected

Hardware Slice Isolation

Powered by NVIDIA Multi-Instance GPU (MIG). Each container runs in physically isolated GPU memory controllers, SM clusters, and cache slices with 100% deterministic QoS and zero noisy neighbors.

Physical H100 Slicing100% Hardware Fenced
28 SMs · 840 GB/s
Mistral, Llama 3 8B, Qwen 2.5 14B
// 04/Micro-Metered

Per-Second Micro Billing

Billed strictly for the exact milliseconds your inference kernel executes. No rounding up to the nearest hour, no minimum commitments, and no cluster lock-in contracts.

Inference ExecutionActive Run
Execution Time:3.42 seconds
Total Cost:$0.00075
Sub-millisecond precision metering with automated budget caps
// 05/vLLM & SGLang

Pre-Built AI Runtimes

Container images pre-optimized with FlashInfer, PagedAttention, and continuous batching. Spin up leading model engines instantly with maximum tokens-per-second throughput.

Supported Serving Stacks
NVMe-oF Pre-Cached
Opt:Chunked PrefillContinuous BatchingFP8 & AWQ Kernels
$istidlal run --engine vllm --model llama-3.3-70b --slice 40gb
// 06/100% Drop-in SDK

Drop-in SDK Compatibility

Zero code refactoring required. Simply point your OpenAI client base URL to Istidlal's high-speed endpoint and keep using your existing LangChain, LlamaIndex, or agent loops.

OpenAI SDK
1from openai import OpenAI
2 
3client = OpenAI(
4 base_url="https://api.istidlal.ai/v1",
5 api_key=os.environ["ISTIDLAL_API_KEY"]
6)
7response = client.chat.completions.create(
8 model="istidlal/deepseek-r1",
9 messages=[{"role": "user", "content": "Hello!"}]
10)
Instant Runtimes

Serverless Model Performance

Model ArchitectureHardware SliceCold Start TimePer-Second RateStatus
Llama 3.3 70B Instruct40GB Slice2.8s$0.00044/secInstant Warm
DeepSeek R1 Sliced 32B20GB Slice1.9s$0.00022/secInstant Warm
Mistral NeMo 12B10GB Slice1.2s$0.00011/secInstant Warm
Qwen 2.5 32B Coder20GB Slice2.1s$0.00022/secInstant Warm