Skip to content

Instantly share code, notes, and snippets.

@cgmb

cgmb/build-llama-cpp.sh

Last active April 17, 2024 23:27
Show Gist options
  • Save cgmb/be113c04cd740425f637aa33c3e4ea33 to your computer and use it in GitHub Desktop.
Save cgmb/be113c04cd740425f637aa33c3e4ea33 to your computer and use it in GitHub Desktop.
Download ZIP
How to build llama.cpp on Debian
Raw
build-llama-cpp.sh
#!/bin/sh
# Build llama.cpp on Debian 13 or Ubuntu 23.10 and later
# Tested with `docker run -it --device=/dev/dri --device=/dev/kfd --security-opt seccomp=unconfined --volume $HOME:/mnt/home debian:sid`
apt -y update
apt -y upgrade
apt -y install git hipcc libhipblas-dev librocblas-dev cmake build-essential
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp/
git checkout b2110
CC=clang-15 CXX=clang++-15 cmake -H. -Bbuild -DLLAMA_HIPBLAS=ON -DAMDGPU_TARGETS="gfx803;gfx900;gfx906;gfx908;gfx90a;gfx1010;gfx1030" -DCMAKE_BUILD_TYPE=Release
make -j16 -C build
build/bin/main -ngl 32 --color -c 2048 --temp 0.7 --repeat_penalty 1.1 -n -1 -m ~/Downloads/dolphin-2.2.1-mistral-7b.Q5_K_M.gguf --prompt "Once upon a time"
@cgmb
Copy link
Author

cgmb commented Mar 4, 2024 •
edited

@userbox020, you need to apt-get install -y hipcc libhipblas-dev librocblas-dev and use a newer OS and compiler. I suggest FROM ubuntu:mantic and CLANG_VERSION=15. The necessary packages are not available for Debian Bullseye or clang-11. Those are too old.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment