Hybrid models incoming!
Added 2022-05-13 07:34:01 +0000 UTCI couldn't be more proud of this one: I have a new fairseq-dense model in the pipeline called "Nerys". It's a combination of the "Pike" dataset (20% bigger than Janeway) with some Asian novels, the CYS dataset AND some good stories from Shinen. All in all, it's quite a nice dataset. The Pike dataset is good for around 80% of the full dataset, the rest is LN, CYS and adult novels. I also thought about adding some IRC chatlogs, but I found out that Janeway already supports that format...
I am currently in the process of building the 2.7B Fairseq-dense model, which should technically be done by the end of the day, and after that, I might do the 13B Fairseq-dense model if I can get the time to work a bit on it.
Note that these will be released first "for patreons only", so if you like early access to test them out and to report bugs, let me know and I will add you to the Huggingface team.
Just to be fair, if the model starts spitting out SCP-like stories, I will start pruning into the dataset. I know that it's an artefact from Picard, but if it's getting out of hand, let me know and I will remove it from the Pike dataset. I love good feedback, and I love reading your stories.
