Text generation
Call leading large models behind one interface.
Frontier models sit behind a single interface, with routing, caching, rate limits, failover and cost control handled by the runtime. Switching models does not change your code.
The engineering around the model, handled by the runtime
The runtime handles rate limits under load, timeouts on long context, opaque billing, and rewriting a prompt pipeline just to change models. Semantic caching serves repeated requests, very long context is sharded and inferred in parallel, and a shaky model fails over to a hot standby in the same capability tier. Callers change nothing.
Built for production
Long context, consistent turns
Instructions hold across long conversations and large engineering context. Tool-call chains nest several levels deep and roll back on failure.
Low-latency streaming
Nearby entry points, warm connections and speculative decoding, delivered token by token.
Cost control built in
Semantic caching, context compression and batch merging stack together, with billing reconciled line by line.
Automatic failover
Models in the same capability tier back each other up, and degradation policy is configured per business line.
API capabilities
Streaming or complete response
Push tokens over SSE, or take the whole result in one call from the same endpoint.
Tool calls and structured output
Native function calling with JSON Schema constraints, ready to deserialise on arrival.
Very long context
Long documents are sharded and processed in parallel, with no splitting logic to write yourself.
Multimodal input
Images, PDFs, audio and text share one messages array.
Semantic cache
The similarity threshold is adjustable; a hit returns immediately and is not billed.
OpenAI SDK compatible
Existing code connects without changes and keeps the advanced features of the SDK.
Device voice assistants
Turn natural language into device commands, remembering which room and which device the last turn referred to.
In-app support and self-diagnosis
With the product manual and device logs connected, answer pairing failures, offline states and alerts, and say what to do next.
Scene and automation authoring
A user describes what they want in one sentence, and the conditions, actions and active hours of an automation come back for review.
Device operation reports
Turn energy use, runtime and alert history into a readable weekly report, grouped by room or by store.
starsg = Starsg(api_key="sk-...") stream = starsg.chat( model="claude-sonnet-4-6", messages=[{"role": "user", "content": "分析这份财报的风险点"}], attachments=["file_9f2a....pdf"], fallback="gpt-4o", # 上游抖动即热备 stream=True, ) for chunk in stream: print(chunk.delta, end="")
One call declares the behaviour
Streaming, attachments, tool calls and fallback policy are all declared in the same call — no extra orchestration layer.
Start building with Text generation
Sign up to create an API key. It shares the same auth, billing and observability as the other six capabilities.