Kastalia Knowledge Management System · Glasperlenspiel template · knot 1513
mid-sized LLMs
🌐 public
· created AE540923 (23.09.2024)
· by DDH
· open in the standard editor view
· 📽 open as presentation
We use mid-sized (< 8 billion parameters) large language models derived from Llama 3.1 8B and Mistral 7b base models.
On top of these models, we subsequently train specific adapters by means of Low Rank Adaptation (LoRA) methodology.
Additionally, Retrieval Augmented Generation (RAG) is also deployed in order to increase response accuracy.
Ancestors (1 superordinated path)