Discussion
Loading...

Post

  • About
  • Code of conduct
  • Privacy
  • About Bonfire
Rost Glukhov
@ros@techhub.social  ·  activity timestamp 19 hours ago

Install llama.cpp, run GGUF models with llama-cli, and serve OpenAI-compatible APIs using llama-server. Key flags, examples, and tuning tips with a short commands cheatsheet

#Cheatsheet #GGUF #AI #LLM #DevOps #OpenAI #API #SelfHosting #CUDA #Prometheus #llama.cpp

https://www.glukhov.org/llm-hosting/llama-cpp/

Rost Glukhov | Personal site and technical blog

llama.cpp Quickstart with CLI and Server

Install llama.cpp, run GGUF models with llama-cli, and serve OpenAI-compatible APIs using llama-server. Key flags, examples, and tuning tips with a short commands cheatsheet
  • Copy link
  • Flag this post
  • Block
Log in

Encryptr.net Social

This is a forward thinking server running the Bonfire social media platform.

LGBTQA+ and BPOC friendly.

Encryptr.net Social: About · Code of conduct · Privacy ·
Encryptr.net social · 1.0.0-rc.3.6 no JS en
Automatic federation enabled
  • Explore
  • About
  • Code of Conduct
Home
Login