initial commit

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,33 @@
 ---
 license: apache-2.0
 ---
+This is a GGUF-formatted checkpoint of
+[rnj-1-instruct](https://huggingface.co/EssentialAI/rnj-1-instruct) suitable
+for use in llama.cpp.  This has been quantized with the Q4\_K\_M scheme, which
+results in model weights of size 4.8GB.
+
+Support for this model has been PR'd upstream, but in the meantime you can
+build from source with these instructions for MacOS.  For Linux, install cmake
+using your package manager.  For Windows, consult the llama.cpp [build
+guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md).
+
+```bash
+brew install cmake
+git clone https://github.com/Essential-AI/llama.cpp
+cd llama.cpp
+git checkout rnj-1
+cmake -B build
+cmake --build build --config Release
+```
+
+To run this model in a web UI, use this command and open your browser to http://localhost:8080:
+
+```
+build/bin/llama-server -hf EssentialAI/rnj-1-instruct-GGUF -c 0
+```
+
+To run it in the CLI, use this command:
+
+```
+build/bin/llama-cli -hf EssentialAI/rnj-1-instruct-GGUF
+```

rnj-1-8B-instruct-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89b8e7e212062029fb9fe3cec95a9eeaeeddde63d3b4c728107afa1eccf28e4f
+size 5113914272