Run your own LLM on a cloud server with Ollama (CPU)

Steps

1. 1Install Ollama

   The script installs Ollama as a systemd service; by default it listens only on 127.0.0.1:11434.

   Copy

   curl -fsSL https://ollama.com/install.sh | sh
   systemctl status ollama --no-pager

2. 2Pull a model and try it

   Pick a model that fits your RAM. Roughly (4-bit): 3B ≈ 3 GB, 8B ≈ 6 GB, 14B ≈ 10 GB, 32B ≈ 20 GB of memory.

   Copy

   ollama pull llama3.1:8b
   ollama run llama3.1:8b "Bulut sunucu nedir? Kısaca anlat."
   # Daha küçük / smaller: ollama pull llama3.2:3b

3. 3Use the API

   Your apps on the same server can call the model over the HTTP API.

   Copy

   curl http://127.0.0.1:11434/api/generate -d '{
     "model": "llama3.1:8b",
     "prompt": "Merhaba!",
     "stream": false
   }'

4. 4Secure remote access

   Don't expose the API to the internet. Connect from your computer through an SSH tunnel; for a web UI, run Open WebUI bound to a local address.

   Copy

   # Kendi bilgisayarından / from your computer:
   ssh -L 11434:127.0.0.1:11434 deploy@SUNUCU_IP
   
   # İsteğe bağlı web arayüzü (sunucuda, yalnız yerel adres) / optional web UI bound to localhost:
   docker run -d -p 127.0.0.1:3000:8080 --add-host=host.docker.internal:host-gateway \
     -v open-webui:/app/backend/data --name open-webui --restart always \
     ghcr.io/open-webui/open-webui:main

Updated: 2026-10-23