Running LLaMA, a ChapGPT-like large language model released by Meta on Android phone locally.
I use antimatter15/alpaca.cpp, which is forked from ggerganov/llama.cpp.
Alpaca is the fine-tuned version of LLaMA which was released by Stanford University. The alpaca.cpp could run on CPU only mode.
Please install Termux first. Then choose one methods from below to compile Alpaca.cpp on your device.
Alpaca requires at leasts 4GB of RAM to run. If your device has RAM >= 8GB, you could run Alpaca directly in Termux or proot-distro (proot is slower). Devices with RAM < 8GB are not enough to run Alpaca 7B because there are always processes running in the background on Android OS. Termux may crash immediately on these devices.
Or you could root your phone and setup a chroot environment. Then mount a swapfile to get more RAM on your device.
pkg install clang wget git cmake
wget https://github.com/lzhiyong/termux-ndk/releases/download/ndk-r23/android-ndk-r23c-aarch64.zip
unzip android-ndk-r23c-aarch64.zip
export NDK=~/android-ndk-r23c-aarch64
https://github.com/lzhiyong/termux-ndk/releases
git clone https://github.com/rupeshs/alpaca.cpp.git
cd alpaca.cpp
mkdir build-android
cd build-android
cmake -DCMAKE_TOOLCHAIN_FILE=$NDK/build/cmake/android.toolchain.cmake -DANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-23 -DCMAKE_C_FLAGS=-march=armv8.4a+dotprod ..
make -j8
wget https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin
./chat
Install Chroot Ubuntu and log in to Ubuntu.
If your devices has RAM lower than 8GB, it is recommened to mount a SWAP file.
dd if=/dev/zero of=/swapfile bs=1M count=8192 status=progress
chmod 0600 /swapfile
mkswap /swapfile
swapon /swapfile
apt install build-essential wget
git clone https://github.com/antimatter15/alpaca.cpp
cd alpaca.cpp
make chat
wget https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin
./chat
proot-distro login debian --shared-tmp
apt install build-essential wget
git clone https://github.com/antimatter15/alpaca.cpp
cd alpaca.cpp
make chat
wget https://huggingface.co/Sosaka/Alpaca-native-4bit-ggml/resolve/main/ggml-alpaca-7b-q4.bin
./chat
Now you can start chattig with Alpaca.
It will takes Alpaca 30 seconds to start answering your questions.
What can you do with Termux? Well, this looks like the answers from Termux official website :)
](https://links.melissandre.dev//en-us/posts/minecraft-java-edition-termux-proot/)
The rise of Chinese AI startup DeepSeek has been nothing short of remarkable. After surpassing ChatGPT on the App Store, DeepSeek sent shockwaves to the tech world, triggering a frenzy in the market. But the attention hasn’t all been positive. DeepSeek’s website faced an attack that forced the company to suspend registrations, and some skeptics