Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.
Example:
fltr --file emails.txt --prompt "Is the following email spam? Email:" --batch-size 32
It will output all lines in the file where the answer is yes.
Text file input tokens per second:
- Nvidia RTX 3070 with 8GB memory: Mistral 7B: ~52 tok/s, Mixtral 8x7B: ~28 tok/s
- Intel I5-6500 with 8GB memory: Mistral 7B: ~5 tok/s, Mixtral 8x7B: ~2 tok/s
Supports: Linux (x86_64) & macOS (x86_64 & arm64)
