Paid customer API key & production usage guide

This guide explains how Research and Enterprise customers use their Terradev API key, activate hot-swap serving, and call the OpenAI-compatible endpoints for an adapter.

1. Quickstart: prompt → hot adapter in one script

The full automation pipeline is: generate an adapter from a prompt, wait for generation to complete, serve it on a GPU instance, wait for it to go hot, then call it like any OpenAI endpoint.

1.1 One-shot bash script

#!/usr/bin/env bash
set -euo pipefail

API_KEY="<your-api-key>"
BASE="https://terradev.cloud/api"

# Step 1 — Generate
echo "Generating adapter..."
ADAPTER_ID=$(curl -sf -X POST "$BASE/adapters/generate" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"base_model":"meta-llama/Llama-3.1-8B","text":"A helpful coding assistant","name":"coding-assistant"}' \
  | jq -r '.adapter_id')
echo "Adapter ID: $ADAPTER_ID"

# Step 2 — Poll until generation is ready (typically 10–60 s)
echo "Waiting for generation..."
while true; do
  STATUS=$(curl -sf "$BASE/adapters/$ADAPTER_ID" \
    -H "Authorization: Bearer $API_KEY" | jq -r '.status')
  echo "  status: $STATUS"
  [ "$STATUS" = "ready" ] && break
  [ "$STATUS" = "failed" ] && { echo "Generation failed."; exit 1; }
  sleep 5
done

# Step 3 — Activate hot-swap serving (paid tiers)
echo "Starting hot-swap serving..."
curl -sf -X POST "$BASE/adapters/$ADAPTER_ID/serve" \
  -H "Authorization: Bearer $API_KEY" > /dev/null

# Step 4 — Poll until the adapter is hot (cold-start: 2–10 min)
echo "Waiting for GPU instance to go hot..."
while true; do
  RESP=$(curl -sf "$BASE/adapters/$ADAPTER_ID" \
    -H "Authorization: Bearer $API_KEY")
  STATUS=$(echo "$RESP" | jq -r '.status')
  GATEWAY=$(echo "$RESP" | jq -r '.lorax_instance // empty')
  echo "  status: $STATUS"
  [ "$STATUS" = "hot" ] && [ -n "$GATEWAY" ] && break
  sleep 15
done

echo "Gateway live at: $GATEWAY"

# Step 5 — Call the OpenAI-compatible endpoint
echo "Calling adapter..."
curl -s -X POST "$GATEWAY/v1/chat/completions" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$ADAPTER_ID\",\"messages\":[{\"role\":\"user\",\"content\":\"What is a LoRA adapter?\"}]}" \
  | jq -r '.choices[0].message.content'

1.2 Full Python script

Requires: pip install requests openai

import time
import requests
from openai import OpenAI

API_KEY  = "<your-api-key>"
BASE_URL = "https://terradev.cloud/api"
HEADERS  = {"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"}


def generate_adapter(prompt: str, base_model: str = "meta-llama/Llama-3.1-8B", name: str | None = None) -> str:
    """Submit a generation request and return the adapter_id."""
    resp = requests.post(
        f"{BASE_URL}/adapters/generate",
        headers=HEADERS,
        json={"base_model": base_model, "text": prompt, "name": name or prompt[:40]},
    )
    resp.raise_for_status()
    return resp.json()["adapter_id"]


def wait_for_ready(adapter_id: str, poll_interval: int = 5, timeout: int = 300) -> None:
    """Block until adapter status is 'ready'. Raises on failure or timeout."""
    deadline = time.time() + timeout
    while time.time() < deadline:
        resp = requests.get(f"{BASE_URL}/adapters/{adapter_id}", headers=HEADERS)
        resp.raise_for_status()
        status = resp.json()["status"]
        print(f"  generation status: {status}")
        if status == "ready":
            return
        if status == "failed":
            raise RuntimeError(f"Adapter {adapter_id} generation failed.")
        time.sleep(poll_interval)
    raise TimeoutError(f"Adapter {adapter_id} not ready after {timeout}s.")


def serve_adapter(adapter_id: str) -> None:
    """Activate hot-swap GPU serving for the adapter."""
    resp = requests.post(f"{BASE_URL}/adapters/{adapter_id}/serve", headers=HEADERS)
    resp.raise_for_status()


def wait_for_hot(adapter_id: str, poll_interval: int = 15, timeout: int = 900) -> str:
    """Block until adapter is hot and return the gateway URL."""
    deadline = time.time() + timeout
    while time.time() < deadline:
        resp = requests.get(f"{BASE_URL}/adapters/{adapter_id}", headers=HEADERS)
        resp.raise_for_status()
        data   = resp.json()
        status  = data["status"]
        gateway = data.get("lorax_instance")
        print(f"  serving status: {status}")
        if status == "hot" and gateway:
            return gateway
        time.sleep(poll_interval)
    raise TimeoutError(f"Adapter {adapter_id} did not go hot after {timeout}s.")


def stop_adapter(adapter_id: str) -> None:
    """Stop the GPU serving instance."""
    requests.post(f"{BASE_URL}/adapters/{adapter_id}/stop", headers=HEADERS).raise_for_status()


# --- Full pipeline ---
if __name__ == "__main__":
    prompt = "A helpful coding assistant that explains Python errors clearly"

    print("Step 1: Generating adapter...")
    adapter_id = generate_adapter(prompt, name="coding-assistant")
    print(f"  adapter_id: {adapter_id}")

    print("Step 2: Waiting for generation to complete...")
    wait_for_ready(adapter_id)
    print("  ready.")

    print("Step 3: Starting hot-swap serving...")
    serve_adapter(adapter_id)

    print("Step 4: Waiting for GPU instance to go hot...")
    gateway_url = wait_for_hot(adapter_id)
    print(f"  gateway: {gateway_url}")

    print("Step 5: Calling adapter via OpenAI client...")
    client = OpenAI(base_url=f"{gateway_url}/v1", api_key=API_KEY)
    response = client.chat.completions.create(
        model=adapter_id,
        messages=[{"role": "user", "content": "What is a LoRA adapter?"}],
    )
    print(response.choices[0].message.content)

    # Optional: stop serving when done to avoid GPU costs
    # stop_adapter(adapter_id)

1.3 Download the LoRA adapter bundle

The download endpoint returns a .zip containing two files at the root:

• adapter_model.safetensors — the LoRA weights
• adapter_config.json — the PEFT config (base model, rank, target modules)

curl -s "https://terradev.cloud/api/adapters/${ADAPTER_ID}/download" \
  -H "Authorization: Bearer <your-api-key>" \
  -o adapter.zip
unzip adapter.zip -d ./my-adapter

The unzipped directory is a standard HuggingFace PEFT adapter. You can:

• Load it locally: PeftModel.from_pretrained(base_model, "./my-adapter")
• Push it to HF Hub and then import into Fireworks AI or Baseten (see 8. Push to Hugging Face Hub)
• Upload the .zip to S3 with a presigned URL and import into Together AI directly

2. Get an API key

1. Sign up or log in at https://terradev.cloud.
2. Subscribe to a paid tier (Research or Enterprise) if you have not already.
3. Go to /dashboard.
4. Click Generate API key in the Production API key card.
5. Copy the key immediately — it is the only time it is shown in full.

You can also regenerate the key from the same card or by calling:

curl -X POST https://terradev.cloud/api/auth/api-key \
  -H "Cookie: session=<your-session-cookie>"

3. Authenticate requests

Every paid-tier request must include your API key in one of these headers:

X-API-Key: <your-api-key>
# or
Authorization: Bearer <your-api-key>

This key is used both for the Terradev backend and for the dedicated vLLM/SGLang gateway that is spun up when an adapter is active.

4. List your adapters

curl https://terradev.cloud/api/adapters \
  -H "Authorization: Bearer <your-api-key>"

Each adapter has:

• id — the adapter identifier used as the OpenAI model name.
• status — generating, ready, hot, failed.
• lorax_instance — the active OpenAI-compatible gateway URL (set only when status is hot).
• gateway_url — same value, kept for backward compatibility.
• rank — LoRA rank when known.
• base_model — base model the adapter was trained on.

5. Activate hot-swap serving

Only paid adapters can be served. Click Activate or toggle Hot-swap in the dashboard, or call:

curl -X POST "https://terradev.cloud/api/adapters/<adapter-id>/serve" \
  -H "Authorization: Bearer <your-api-key>"

The backend will:

1. Pick a GPU instance that fits the base model (T4 → A10G → A100 → H100).
2. Start a vLLM or SGLang pod with the base model.
3. Load your LoRA adapter dynamically.
4. Return the gateway URL when healthy.

Note: Adapter generation runs on CPU-only hypernetworks. GPUs are used only for hot-swap serving/inference (/serve), not for generation. GPU instance selection is the fallback order the backend uses when it provisions an EC2 GPU instance. This requires GPU_POOL_ENABLED=true and valid AWS credentials. If the GPU pool is disabled, the serving container runs on the same host as the API instead of provisioning a separate GPU.

Cold-start time depends on the GPU type and base model size, typically 2–10 minutes. Poll GET /adapters/<adapter-id> until status is hot and lorax_instance is set.

6. Call the OpenAI-compatible gateway

When an adapter is hot, the dashboard shows a dropdown with these endpoints:

The full URL is shown in the dashboard and returned as lorax_instance.

cURL example

curl "<lorax_instance>/v1/chat/completions" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<adapter-id>",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello!"}
    ]
  }'

Python example

from openai import OpenAI

client = OpenAI(
    base_url="<lorax_instance>/v1",
    api_key="<your-api-key>",
)

response = client.chat.completions.create(
    model="<adapter-id>",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
)
print(response.choices[0].message.content)

Replace <lorax_instance> and <adapter-id> with the values from the dashboard.

7. Stop serving

To stop the GPU instance and stop billing:

curl -X POST "https://terradev.cloud/api/adapters/<adapter-id>/stop" \
  -H "Authorization: Bearer <your-api-key>"

Or toggle Hot-swap off in the dashboard.

8. Push to Hugging Face Hub

Research and Enterprise customers can push a ready adapter directly to a Hugging Face Hub repository. Both files land in a single atomic commit — either the full adapter is pushed or nothing changes.

curl -X POST "https://terradev.cloud/api/adapters/<adapter-id>/push-to-hub" \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "repo_id": "your-username/my-lora-adapter",
    "hf_token": "hf_...",
    "private": true
  }'

Returns:

{ "url": "https://huggingface.co/your-username/my-lora-adapter" }

• repo_id must be username/repo-name. The repo is created if it does not exist.
• hf_token must have write scope. Generate one at huggingface.co/settings/tokens.
• private defaults to true.
• The HF token is never stored server-side.

Once pushed, the HF URL can be used directly:

You can also push from the dashboard — click the upload icon (↑) on any ready adapter (paid tiers only).

9. Import a PEFT adapter from Hugging Face

Research and Enterprise customers can import a LoRA directly from Hugging Face:

curl -X POST https://terradev.cloud/api/adapters/import \
  -H "Authorization: Bearer <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "hf_repo_id": "organization/adapter-name",
    "hf_token": "hf_...",
    "subfolder": "",
    "name": "My imported adapter",
    "base_model": "meta-llama/Llama-3.1-8B-Instruct"
  }'

• hf_token is optional. If omitted, the server uses its own HF_TOKEN.
• base_model is optional if adapter_config.json contains base_model_name_or_path.
• The imported adapter counts against your tier adapter cap (Research 10, Enterprise 50).

10. Production operator checklist

If you are running the backend yourself:

1. Set a strong random SESSION_SECRET and API_KEY env vars if applicable.
2. Export HF_TOKEN so gated base models and PEFT imports work.
3. Choose a serving engine:

   SERVING_ENGINE=vllm
   SERVING_IMAGE=vllm/vllm-openai:v0.11.2
   

4. Configure GPU pool fallback order:

   GPU_POOL_ENABLED=true
   GPU_INSTANCE_SEQUENCE=g4dn.xlarge,g5.xlarge,p4d.24xlarge,p5.4xlarge
   

5. Ensure the API host has Docker, the NVIDIA container runtime, and AWS credentials for EC2 provisioning.
6. Put the API behind HTTPS (e.g. Caddy or a reverse proxy). The dashboard only calls /api/* on the same origin.
7. For pricing, serving costs, and fallback behavior, see aws/GPU_POOL.md and aws/SERVING.md.

11. Limits by tier

Need help?

For dedicated Enterprise support, use the contact card on the dashboard or email theo@southfacing.app.