Hexy Templates

vLLM

vLLM is a high-throughput and memory-efficient inference and serving engine for LLMs with an OpenAI-compatible API.

Not yet tested ai Based on TrueNAS app v1.0.18 (vLLM v0.30.0)
Project website TrueNAS app source Script on GitHub Report a problem
  1. 1
    Copy the script below
  2. 2
    In HexOS, open Apps, find vLLM and choose Custom Install
  3. 3
    Paste, then press Install — full guide with screenshots
vllm.json
Download 
1{
12  "version": 5,
23  "script": {
4    "version": "0.1.0",
5    "changeLog": "Auto-generated draft from TrueNAS vllm 1.0.18"
6  },
37  "installation_questions": [
8    {
9      "question": "Model name",
10      "type": "text",
11      "key": "model_name",
12      "required": true
13    }
14  ],
15  "requirements": {
416    "locations": ["ApplicationsPerformance"],
517    "specifications": ["2CORE"],
618    "permissions": ["READ_WRITE_LOCATIONS"],
719    "ports": [30422]
20  },
821  "ensure_directories_exists": [
22    {"path": "$LOCATION(ApplicationsPerformance)"},
23    {"path": "$LOCATION(ApplicationsPerformance)/vllm"},
24    {
25      "path": "$LOCATION(ApplicationsPerformance)/vllm/data",
26      "owner": {"user": "apps"},
27      "snapshot": {"id": "data"}
28    }
29  ],
30  "app_values": {
931    "vllm": {"model_name": "$QUESTION(model_name)"},
32    "storage": {
1033      "data": "$HOST_PATH($LOCATION(ApplicationsPerformance)/vllm/data)"
34    },
35    "network": {
1136      "web_port": {"bind_mode": "published", "port_number": 30422}
37    },
38    "resources": {
1239      "limits": {"cpus": 2, "memory": "$MEMORY(10%, 4096)"},
1340      "gpus": "$GPU_CONFIG()"
41    }
42  }
43}

What this script does

Hover or tap a box to see the lines it explains.

  1. Script format

    Tells HexOS this is a version 5 install script, so it knows how to read the rest.

    Line 2

  2. Script revision

    This copy of the script is revision 0.1.0. Latest change: “Auto-generated draft from TrueNAS vllm 1.0.18”.

    Lines 3–6

  3. Questions you'll be asked

    Before installing, HexOS will ask you for: Model name. Your answers are slotted into the settings further down wherever you see $QUESTION(...).

    Lines 7–14

  4. Storage locations it needs

    The app uses these HexOS storage locations: App storage (fast). If one isn't set up yet, HexOS will ask you to pick a place for it.

    Line 16

  5. Minimum hardware

    Recommended minimum: 2 CPU cores.

    Line 17

  6. Permission to use your folders

    Allows the app to read and write files inside the storage locations listed above — and nowhere else.

    Line 18

  7. Network ports

    The app needs port 30422 to be free on your server. A port is like an apartment number for your server's address.

    Line 19

  8. Folders created on install

    Creates the folders the app needs before it starts: App storage (fast) › vllm and App storage (fast) › vllm › data. Some folders are handed to a specific system user so the app is allowed to write to them. Folders marked “snapshot” are included in HexOS snapshots, so you can roll them back.

    Lines 21–29

  9. Your answer: model name

    Filled in with what you typed for “model name” when you installed.

    Line 31

  10. Where the app's main data is saved

    Where the app's main data is saved on your server: App storage (fast) › vllm › data.

    Line 33

  11. Web port: 30422

    The app's web page is published on port 30422. You don't need to remember it — clicking the app's card in HexOS takes you there.

    Line 36

  12. Resource limits

    Caps how much of your server the app can use: 2 CPU cores and 10% of your server's memory, but never more than 4 GB. This keeps one busy app from slowing everything else down.

    Line 39

  13. Graphics card access

    If your server has a supported graphics card, HexOS shares it with the app (useful for video conversion or AI features).

    Line 40