I sent @theshawwn my huge video game walkthrough dump that I've been retraining "medium" GPT-2 on, and he used Special Computing Resources at his disposal to train the "full" 1.5 billion parameter model (too big to train on commodity hardware) on it
Shawn Presser
 @theshawwn
16 Jan 2020
Trained a GPT-2 1.5B on 50MB of videogame walkthroughs. The results are *very* interesting. Someone could parse this and generate game levels, or mechanics. Thanks to @me_irl for the dataset, and for giving permission to release everything publicly. I'll post instructions soon.

Jan 16, 2020 · 2:37 PM UTC

3
11
1
30
Interestingly the 1.5B model appears to remember a lot of video game related information from the initial GPT-2 training set. @theshawwn repeated my experiment of prompting with an incomplete list of Overwatch heroes and it ends up completing the list somewhat accurately
1
3
whereas in my experiments, it would seem to sort of vaguely remember Junkrat's correct bio and then veer off into making up new ones
1
1
2
Here is just the first sample I got just now with the same prompt, for comparison. The red line is where I stop writing and the model picks up
2
1
8
loller
1
1
3
No!
1
1
5
These samples are actually still from my 355M version, if that wasn't clear
1
1
One thing that is fun to do is write the intro to an imaginary FAQ and have the Ultimate Gaming Mind write the rest. Some examples of that (again from my 355M version) here. It gets really into writing the story synopsis for the game in some of these. pastebin.com/JqL4zGVH

[in these first several examples, I wrote the text up to "millennia" and prompte - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

pastebin.com
1
2
A town built specifically for the purpose of collecting Blue Present.
1
5
11
Oh yeah, here are some of my previous experiments about a week ago in a different thread
the government man @me_irl
9 Jan 2020
i have been playing around with text generation models again
1
1
Top 10 anime betrayals
1
3
Getting some good item lists out of 355M/69MB dataset Ultimate Gaming Mind right now
3
5
Still need to get around to downloading the 1558M one to play with but I'm still too busy getting good-ass item concepts out of the one I cooked earlier
2
5
makes sense
2
1
Some monsters. Im the Bombokiller
3
2
3
1
4
5
Comparing and contrasting
1
3
I have been able to elicit false Kongs
2
7
1
9
Please watch out for GURCHIN MAN
1
2
5