tomasmcm/loyal-piano-m7

Source: chargoddard/loyal-piano-m7 ✦ Quant: TheBloke/loyal-piano-m7-AWQ ✦ Intended to be a roleplay-focused model with some smarts and good long-context recall

  • Public
  • 36 runs

Run time and cost

This model runs on Nvidia A40 GPU hardware. Predictions typically complete within 4 seconds. The predict time for this model varies significantly based on the inputs.

Readme

Built with Axolotl

Experimenting with dataset ratios. Intended to be a roleplay-focused model with some smarts and good long-context recall.

Not sure if I’ve succeeded on the roleplay front, but something sure went right! Currently the #4 7B model on the leaderboard as of 11/30/2023. Going to riff on this and see where it goes.

model Average ARC HellaSwag MMLU TruthfulQA Winogrande GSM8K DROP
fblgit/juanako-7b-UNA 59.91 68.17 85.34 62.47 65.13 78.85 20.7 38.74
Intel/neural-chat-7b-v3-1 59.06 66.21 83.64 62.37 59.65 78.14 19.56 43.84
Weyaxi/OpenHermes-2.5-neural-chat-7b-v3-1-7B 58.6 66.55 84.47 63.34 61.22 78.37 23.58 32.66
chargoddard/loyal-piano-m7 58.42 66.72 85.03 64.43 60.03 79.08 25.7 27.92
Gryphe/MythoMist7b 58.26 65.87 83.55 62.32 59.98 78.06 20.24 37.82

Dataset composition: | dataset | rows used | percent of total | | — | — | — | | PIPPA | 14.6k | 43% | | summarize_from_feedback | 9k | 26% | | orca_mini_v1_dataset | 5.6k | 17% | | rpguild | 2.86k | 8% | | LimaRP | 2k | 6% |