Streaming

Learn how to stream AI responses in real-time using Server-Sent Events for a better user experience.

What is Streaming?

Streaming allows you to receive AI responses token-by-token as they're generated, rather than waiting for the complete response. This creates a better user experience, especially for long responses.

Instead of showing a loading spinner for 10 seconds, users see text appearing in real-time - just like ChatGPT.

Why Use Streaming?

• Better UX: Users see responses immediately instead of waiting
• Perceived Performance: Feels faster even if total time is the same
• Early Cancellation: Users can stop generation if they got what they need
• Real-time Feedback: Users know the AI is working, not stuck

How Streaming Works

Cencori uses Server-Sent Events (SSE) for streaming:

1. Your app calls cencori.ai.chatStream()
2. Cencori forwards the request to the AI provider
3. As the provider generates tokens, Cencori streams them back
4. Your app displays each chunk as it arrives
5. Stream ends when generation completes

Client → Cencori → AI Provider
  ↓         ↓         ↓
Chunk 1 ← Chunk 1 ← Token 1, 2, 3
Chunk 2 ← Chunk 2 ← Token 4, 5, 6
  ...       ...       ...
Done    ← Done    ← Complete

Basic Usage

React/Next.js Example

Node.js Server Example

Stream Chunk Format

Each chunk in the stream contains:

Error Handling

Handle errors gracefully during streaming:

Best Practices

• Show Loading State: Display a typing indicator before first chunk
• Handle Connection Drops: Implement reconnection logic
• Allow Cancellation: Let users stop generation early
• Buffer Chunks: Avoid too many DOM updates by batching small chunks
• Error Messages: Show user-friendly errors if stream fails
• Timeout Protection: Set maximum generation time

Provider Support

All Cencori providers support streaming:

Troubleshooting