LLM hosting service

The current Spike LLM hosting service exposes SCC-managed models through one OpenAI-compatible gateway. Researchers call the models from code or an approved client; SCC AI Engineers operate the underlying Run:ai and vLLM deployments. You must not create, resize, restart, reconfigure, or delete those workloads.

The gateway is available on the TU/e network or VPN. The current gateway base URL, access procedures, and live service details are maintained in the Spike hosting project in TU/e GitLab.

Current service information

To use the service, connect to TU/e VPN and request a personal API key from your SCC AI Engineer or via the Spike hosting project in TU/e GitLab. The key gives access to the shared gateway; it does not grant permission to process data that has not been approved for the project.

Use the Spike hosting project in TU/e GitLab for the live model catalogue, endpoint base URL, availability, examples, and changelog. TU/e GitLab may ask you to sign in. Model revisions, request limits, and availability can change, so do not copy a model identifier from another project or an old example.

Use this page for safe client guidance and the GitLab project for live service information. If they appear to disagree, stop and ask your SCC AI Engineer before sending data or building a dependency on an endpoint.

Before sending a request

Confirm:

The model is approved for the intended research purpose.
The input and output data are allowed to be processed by this service.
The endpoint, authentication method, and request limits came from the SCC onboarding or support channel.
The client will not log tokens, sensitive prompts, or model responses.
A small harmless request succeeds before starting a batch or benchmark.

Discover available model identifiers

For an OpenAI-compatible endpoint, keep the issued values in local environment variables:

# Obtain the current gateway base URL from the Spike hosting project in TU/e GitLab
export HOSTED_MODEL_BASE_URL='https://<hosted-model-gateway-endpoint>/v1'
export HOSTED_MODEL_TOKEN='your-personal-api-key'

List the models currently exposed by that endpoint:

curl --fail --silent \
  -H "Authorization: Bearer ${HOSTED_MODEL_TOKEN}" \
  "${HOSTED_MODEL_BASE_URL}/models"

Use only an identifier returned by discovery and approved for the project. For a general request, best-live asks the gateway to select a model that is already running; specify an exact model only when the research requires it.

Send a small request

curl --fail --silent \
  -H "Authorization: Bearer ${HOSTED_MODEL_TOKEN}" \
  -H 'Content-Type: application/json' \
  "${HOSTED_MODEL_BASE_URL}/chat/completions" \
  -d '{
    "model": "best-live",
    "messages": [
      {"role": "user", "content": "Return the word ready."}
    ]
  }'

Set an application timeout and handle rate limits and service errors. Do not retry indefinitely. Large batches and load tests require prior coordination with the SCC AI Engineer managing the endpoint.

The backend uses vLLM’s OpenAI-compatible server, but the SCC-provided endpoint and policy remain authoritative. Compatible clients can be configured with the gateway URL and personal API key; keep the key in the client’s secret store rather than its exported configuration.

Shared-service behaviour

Inactive models can be moved out of GPU memory. A first request may therefore need warm-up time, and a busy or unavailable model can return HTTP 502 or 504. Retry only a small number of times with a delay, or use best-live; do not create an unbounded retry loop. Coordinate large batches, benchmarks, and load tests with the SCC AI Engineer managing the service.

The gateway returns token counts in the response usage field. You can review your request and token totals in the personal usage dashboard (linked in the Spike hosting project in TU/e GitLab) using your API key. The current service records usage metadata rather than prompt and completion content, but this does not relax the project’s data classification or approval requirements.

What to report when requesting help

Share the time of the request, the endpoint hostname, model identifier, HTTP status, and a redacted error message. Do not include the bearer token or sensitive prompt content. SCC AI Engineers can inspect and change the deployment; researchers should not attempt to repair it through Run:ai.