FireflyChat

FireflyChat is a graphical application platform for LLM developed by
the Firefly team. Users can quickly experience the actual use of
various large models on the visual WebUI provided by FireflyChat.


Installation

You need to obtain the required installation package from the

Download Center

 to install FireflyChat. There are two installation packages, which
are "fireflychat*.deb" and "libfirefly-ai*.deb".

After obtaining the installation package, transfer it to AIBOX-1684X
and run the following command to install it:

   # Install venv
   sudo apt update
   sudo apt install python3-venv

   # Install FireflyChat
   sudo dpkg -i libfirefly-ai*.deb
   sudo dpkg -i fireflychat*.deb

After installing FireflyChat, you need to import the model file to
FireflyChat to use it properly.

Model import

LLM

Users can obtain the model files (.bmodel) provided by the Firefly
team from the LLM model resource in the

Download Center

.

After obtaining the model file, transfer it to AIBOX-1684X and run the
following command to import it:

   # Move the model file location, where <model_path> replace with the actual model file path
   sudo mv <model_path> /models/LLM/

Embedding model

Users can obtain the model files (.bmodel) provided by the Firefly
team from the embedding model resource in the

Download Center

.

After obtaining the model file, transfer it to AIBOX-1684X and run the
following command to import it:

   # Move the model file location, where <model_path> replace with the actual model file path
   sudo mv <model_path> /models/embedding/

Restart FireflyChat

After adding or modifying the required model files, you need to
restart FireflyChat for the new model to take effect:

   sudo systemctl restart FireflyChat

Software usage

WebUI usage (Main usage methods)

FireflyChat will run automatically after it is installed successfully.
You can directly access FireflyChat by entering "<ip_address>:7860" in
the browser.

Note: <ip_address> must be replaced with the actual IP address, for
example: "172.16.11.18:7860".

Software background management

FireflyChat is an application registered as the systemd service. You
can control the running status of FireflyChat by running the following
commands:

   # Stop FireflyChat
   sudo systemctl stop FireflyChat

   # Restart FireflyChat
   sudo systemctl restart FireflyChat

   # Disable the automatic startup of FireflyChat
   sudo systemctl disable FireflyChat

   # Viewing run logs
   sudo journalctl -u FireflyChat