OpenAI Scholars Project

My blog post is split up into three parts:

Literature Review (incomplete, this is a very large field of study): Lenses of Analysis on Opinion Spread. This post is an older version that contains more detailed discussion and why I didn't include some things (like advertising), for those that are interested.

Experiments and Analysis Section: Training models on their own outputs.

Speculative Section: What we'll (probably) see before we build artificial cultures. This is just some research ideas and thoughts I had after doing lots of reading, take them with a grain of salt.

Comments

Post a Comment

Popular posts from this blog

My prediction of AGI in 2021-2023

In 2020 I made a prediction on Facebook that AGI was only 1-3 years away. Probably time to follow up on that. At the time, Gwern’s “You can only prove the presence of an ability, not the absence of it” seemed solid. Someone would say “GPT-3 can’t do X”, and then later someone would find a prompt where it could reliably do X! There was a lot of talk about becoming a good “prompt engineer”: being good at wording things in a way that cause language models to reliably do what you want. Relatedly, these days my favorite way to think about language models is as a Simulator (Janus popularized this framing here  https://generative.ink/posts/simulators/ ). Inside them is all these different persons/personalities, each with different abilities (try character.ai to experience this). Prompt engineering pulls out a specific simulated person and inhibits many others. If you get the wrong person, it’ll fail at things other simulated people are very capable of! But honestly, I think ChatGPT proved Gwe
Read more

Inductive Linguistic Biases

Image
(unfinished blog post, posting for reference) Introduction Humans have "inductive biases". When humans see data that they could draw multiple conclusions from, they often pick one of those conclusions. There are certain kinds of conclusions that humans are more likely to come up with, and those conclusions are our inductive biases. For example, humans tend to assume the sequence (1,2,3) will be followed by (4,5,6,7,...) even though there are many other conclusions you could come to. These inductive biases are very important for understanding how humans can learn from so little data: many mechanisms in our society involve learning to coordinate , instead of learning the "correct" behaviour. Often these norms determined based on the previous generation's "best guesses", and since you share the same inductive biases, you are likely to make those same guesses, and so can learn to coordinate easier. This means that if you can create AI models that have ve
Read more