Create an inference session
From Inference sessions, use the Create session form.
Leave Workload set to Inference.
Choose a Model.
Choose a Distribution plan for that model. Check both the number of workers and the memory required per contributor. Two workers at 4 GB, for example, means two contributors, each supplying 4 GB.
If Network path is shown, choose an available connection option below.
Select Create session.
Only enabled models and plans appear. If the model or plan you need is missing, contact the service operator.
Choose a network path¶
| Option | What it means |
|---|---|
| Default (relay) | Traffic passes through the coordinator. Use this when you do not need direct links. |
| Fast (direct) | Contributors try direct browser connections, with relay fallback. Neighbouring peers can see each other’s public IP addresses. Setup can take up to 30 seconds after loading. |
| Private TURN server | Browser traffic passes through a TURN server so contributors do not see each other’s IP addresses. Available only when the service enables it. |
The network path is fixed when you create the session. Create another session if you need a different path. The name “Fast” is an option label, not a guarantee that your connection will be faster. API requests still go to the coordinator.
Bring contributors online¶
On the session page, find Contributor access:
Select Copy link beside Invitation link and send it to your group. To join yourself, select I’ll contribute myself.
Each contributor opens the link, reviews the model, memory budget and connection details, and selects Accept and contribute when available. Direct connections require acknowledging the IP-address notice.
Their browser checks WebGPU support and loads its assigned model pieces. Download progress appears on the contributor page; cached pieces can be reused.
Keep the contributor tabs open and their computers awake while using the session.
Sharing the link does not make a worker ready immediately. Contributors need a browser and GPU that pass the session’s checks, enough memory and time to load the model.
Wait for readiness¶
Watch Worker assignments and the status bar on the session page:
Waiting: the stage does not yet have a ready contributor.
Preparing: the contributor is joining, checking its GPU or loading the model.
Ready: that contributor is ready to execute its assigned stage.
Wait for Open model chat to appear. Worker markers describe readiness, not whether a message has finished. Connection details shows how the contributors are connected.
Select the model name under Sessions whenever you need to return to this page. When you are finished, use Close session. It ends access to inference for that session; create a new session to start again.
Next: Use model chat or Use the API.