multitrickfox / janeway-6.7b-16bit

just a testbed for new stuff

  • Public
  • 0 runs
  • GitHub
  • Paper
  • License
Playground Examples README Versions

No versions have been pushed to this model yet.

Readme

Model Description

Fairseq-dense 6.7B-Janeway is a finetune created using Fairseq’s MoE dense model.

Training data

The training data contains around 2210 ebooks, mostly in the sci-fi and fantasy genres. The dataset is identical as dataset used by GPT-Neo-2.7B-Janeway. Some parts of the dataset have been prepended using the following text: [Genre: <genre1>,<genre2>]

Limitations and Biases

Based on known problems with NLP technology, potential relevant factors include bias (gender, profession, race and religion).