Lenses on Analysis of Opinion Spread
(This is gonna be a post I keep updating as I learn new lenses, so this is a working document that'll never be complete. The goal of this research direction (for me) is ultimately to figure out what we need to get AIs to have a society of comparable complexity to human society)
There are a few different ways of "looking at" opinion spread. Some of these are subsets of the others, and others can be usefully combined, but each perspective still provides a set of unique insights that aren't provided by the others. I'll mostly constrain myself to talking about social media, but when the insights carry over into other places I'll try to mention that as well.
Note that when I talk about "opinions", precisely what I mean by "opinion" is fairly loose. Much of this post is describing the attempts I'm familiar with so far of trying to formalize that notion.
Distributional Lens
The simplest way to do this is to train a language model on the set of utterances produced by people on a social media platform over a period of time. This language model provides a "snapshot" of the distribution of utterances at that time.
Insights:
- Can help you see biases that exist in the data (what gender is associated with certain professions, word correlations, etc.)
- Different social media seem to promote different distributions. Sometimes this might just be due to the user base, but there also seems to be this intuition that "Twitter is good for news" whereas "Stack exchange is good for objective answers"
- Looking at scaling behavior at different scales might give you insight into the spectra (and thus, some symmetry/geometric properties) of the data at different scales
- Model interpretability methods may help us understand one way of decomposing the distribution into an algorithmic form. That may not be the form that humans are using, but it's at least one representation, which is better than no representation.
Downsides:
- Doesn't capture the dynamic processes of opinion change over time
- It doesn't give many insights as for why certain social media websites are better at different kinds of things, it just says "they are different"
This is just language modeling and large pre-trained models like GPT-3, there's a wealth of papers on this subject.
Modeling Lens
You can try and use more sophisticated models to reproduce the behavior of a network.
Insights:
- Much of the modeling ends up being centered around "spikes" where external information caused a large change in distribution. In other words, social media is not self-contained, patterns are significantly influenced by the outside world, which means that models that don't consider external information will be missing out
- Some of those spikes can be generated by insights formed "in network" - such as some new meme someone comes up with, a new idea someone has, something controversial someone says, etc.
- These "spikes due to external information" are very reminiscent of trying to predict the stock market. Because there are various competing interests that want to sway opinion discourse, it's also probably similarly anti-inductive. Anti-inductive means that it actively resists attempts at modeling because any use of models to influence or participate in the system change the nature of the system itself, making those models less useful in an ever increasing arms race. This is a really informative insight, as it suggests that opinion modeling can never be "complete" in the same way that we can never have a perfect model of the stock market. I think this is true even if the models aren't being used to influence opinions, because the world itself is getting more complicated over time, and opinions are being formed about the world.
Downsides:
- Complex models tend to give shallow insights because formal analysis without making any assumptions is very difficult
See Facebook's approach to simulate social media behavior and the recent DARPA SocialSim project for simulating GitHub
Diffusion Lens
Think of social media as a graph. Vertices are people, edges are opportunities to interact with others. One very simple model of opinion spread is a diffusion model, where an opinion has some probability p of spreading to a node's neighbors. More generally, you can use techniques that people use to model the spread of viruses. Going Critical has some great interactive visualizations for those that are interested.
Insights:
- Percolation theory has developed some pretty detailed analysis, since this is a fairly simple model
- There's a "critical threshold" T for spreading, where any p < T will fizzle out, whereas any p > T will persist and eventually infect everyone
- The more neighbors you have, the more opportunity it has to spread, and the lower the critical threshold is. This means that things that wouldn't "go viral" in loose networks (Discord, rural areas) will go viral in dense networks (Facebook, cities), which also could imply a faster mutation rate depending on the mutation dynamics.
- When recommendation algorithms dynamically adjust who is exposed to content, they are influencing the critical threshold
- Depending on the structure of the network, providing information at different places can result in different people learning the information. Those "insertion points" can also affect how likely it is that the information spreads successfully: the more connected the initial network, the more chance you have to spread. These sort of insights are used in Fair Influence Maximization: A Welfare Optimization Approach to consider how to spread information about emergencies fairly
Downsides:
- Not much, mostly they just fail to capture some important phenomena, but diffusion models can be augmented to include those so this isn't a fundamental limitation. The main downside is that this model helps you understand what's happening, but it's still hard to say if "things go viral easier" is good or bad because you don't have any nuanced way of talking about value assignment on behavior.
Feedback Lens
This is essentially thinking about very expressive models seeing outputs of their previous data, as a way of thinking about what kind of behavior that might lead to, and what the limits of that process are. This was my scholars project, and writeup is in progress.
Insights:
- Each step of updating the model on its data represents a random walk
- Multiple models seeing each other is still a random walk, just look at the "center of mass" (it's a bit messy and not as clean, but the intuition is sensible enough)
- Without pulls towards certain distributions, most discrete models will slowly lose entropy as they do a random walk around the probability space. Continuous ones will just do a random walk and drift far away, but this also depends on the distribution of data points at each step. If that distribution of input points doesn't change, it can walk too far away and classify all the points as the same.
- Those results depend on if you keep your data around or not. If you keep the data around, it's a decaying step size (1/n, so it technically never converges, but practically it starts being very very slow). If you use momentum on past data/use momentum on your current model, it's basically the same as not keeping the model around, just with a smaller step size (as the influence of past models approaches zero exponentially quickly).
- Sampling with temperature != 1.0 will increase the step size, perpetuating biases and speeding up entropy loss
- More data between each step will decrease the step size. Just a very small amount of "true data" can effectively anchor a model and prevent it from ever drifting too far away, even 1/100 is often enough as long as your step size isn't too large, as this acts as a bias to pull the random walk in the right direction (this alongside the previous point implies that if you use temperature != 1.0, you'll need more of your data as the true distribution)
Downsides:
- None of these settings got open-ended behavior, so there's something serious missing here as a modeling paradigm. Yet it seems very general, so I'm still trying to figure out what that is
Auto-induced Distributional Shift Lens
Auto-induced Distributional Shift refers to settings where an algorithm is incentivized to change the distribution of inputs to make the task easier to solve.
Insights:
- Sometimes what changes on a platform is not the opinions of the users, but instead the users themselves. If you don't model this phenomenon, you'll be missing out of much of what determines the eventual population of users
- Some recommendation algorithms can unintentionally be incentivized to change user's opinions, so you need to consider that effect when designing them
Systems Lens
We should think more carefully about the interactions between opinions. Some opinions promote other opinions (promoting/accelerating spread is also known as catalyzing), while some opinions inhibit other opinions. For different sets of inhibit/promote, we can see different kinds of behaviors that arise in real world systems.
Insights:
- Sometimes it's easier to model things in terms of their relationships, and it's harder to explain things when you try and decompose them and model them individually. Rationality/the scientific worldview has a natural tendency to want to break things down into individual independent components (a formal model of trust, a formal model of cooperation, a formal model of reputation, etc.), but often that can lead to worse explanations and more fragile models that need more careful variable tuning.
- RAF networks provide a way to model getting order from random noise, and a way of modeling "spikes" that are entirely within system, not being caused by external influences. The idea is that "breakthroughs" can be represented by some chemical A that has a very low chance of forming, but once it forms, it catalyzes the production of chemical B, which then catalyzes the production of chemical A, etc. These "self catalyzing cycles" (which can be larger than size 2) will then persist and spread once they initially form. This insight is also applicable to opinions. technologies, etc.
- TODO: Study how inhibiting and catalyzing setups affect the analysis of diffusion models
Downsides:
- It's very easy to say "I have a model that explains all these diverse phenomena" where the majority of actual state space is unrealistic or degenerate. These models can tend to be too expressive if you aren't careful, and then it's hard to say anything concrete about them.
- Many of these are good at describing the flow of existing opinions, but don't model the flow of new opinions
See A Systems Approach To Cultural Evolution for more detailed thoughts in this direction. I think there's still a big open question here about finding a decent space that is expressive enough, but not too expressive. Iterative prisoners dilemma algorithms (or some related thing) that grow in decision tree depth over time seem the most promising to me as a potential expressive opinion space that implicitly defines opinion relationships. (I wrote this paragraph a while ago in an earlier draft, and now I'm less convinced, tying structure to the problem the distributed thinking system is solving makes more sense to me then trying to somehow emerge it from some simple system. But maybe iterated prisoners dilemma algorithms are a one of the simplest example of open-ended complexity by progressively countering the solutions of past generations. I'd just like to understand if this can be done in a more cooperative setting, but maybe there's other simple games that would facilitate that).
Metrics Lens
(This section is still "todo", currently pasted from past post but I want to rework this into insights once I have more of a chance to go through exactly what those laws imply)
Attention Bottleneck/Distributed Hivemind Lens
(I'll probably split this into a separate blog post at some point and just put the summary here. I'm mixing up multiple perspectives here that I think each deserve their own section)
None of these models so far have really addressed the question of "what is the purpose of cultural evolution/opinion spread". One answer that appears in some of the literature is "thinking through other minds". For some good discussion and references, see Thinking through other minds: A variational approach to cognition and culture (the free energy stuff is less relevant than the discussion of prior work).
I'm still working through the more detailed thoughts other people have on this subject, so some of these insights are me guessing and this section is subject to change as I further develop my understanding.
Insights:
- Try thinking of things in terms of "what are the constraints". In opinions, one major constraint is that humans have finite time to spend thinking about things (attention), so they need to decide what to spend their attention on. Cultural technologies that effectively distribute the "thinkwork" would have persisted more than cultural technologies that don't distribute thinkwork, and there could also be a selective pressure at play (for example, social status may have emerged partially as a way to distribute thinkwork). Thus, the cognitive technologies we have inherited and makeup what we think of as "culture" are those that are are decent at distributing thinkwork. In other words, we are a giant distributed hivemind that tries to distribute "thinkwork" effectively among individuals that are each limited by the amount of thinking time they have.
- Some of the ways in which we try to solve problems end up creating new problems, which is part of the reason for ever growing complexity
- Much of our thinking is done on an intuitive level, yet to communicate we need to turn those intuitive fuzzy ideas into concrete statements, and this process is lossy. This is probably a source of some open-endedness, as we'd continually invent and communicate new concepts as we try and get a better and better approximation of our underlying nuanced, context sensitive, association based system 1 understanding.
- Plenty of the ways we developed/evolved for distributing thinkwork in small tribes may not scale effectively to very large groups of people or significantly different network structures, which may explain some of the difficulties we have with social media.
We can make an assumption that people try to spend attention to maximize some reward. Economics has learned that this "rational agent" model isn't great when studying humans - and I think this is especially true with something as fuzzy as opinions - but it's a decent start for thinking about analysis.
Taking this perspective, there seem to be a few important curves you could draw, where the y axis is reward:
- X axis=time spent thinking about it: How much reward does someone get from spending lots of time paying attention to a single thing? Something like a meme has very little benefit from spending more than 5 minutes, whereas a scientific topic could be benefited from someone spending multiple years of their lives studying it.
- X axis=how many people think about it: Some topics might be hurt from too many people thinking about them (being scooped sucks and ends up duplicating/wasting effort), some topics seem pretty neutral, and some topics gain a lot from more people thinking about them (in particular, entertainment media which is very social).
- X axis=something about diversity of attention spread?: This one is a little more vague. The point is that for those topics where spending lots of attention gains the individual lots of utility, they can spread those insights with others so many people share the benefits of the investment of attention. If they spend all their time around other people that also think about the same things, those insights may be less useful since others would have come to similar conclusions already, thus, diversity in a social network is helpful for effectively "distributing the work". This sharing sometimes requires some level of social trust ("I'm going to trust you worked out the details without checking them myself"), and platforms that help amplify this social trust like Stack Exchange help with this mechanic. There also seems to be a higher level thing of "we need to share enough of the general ideas in order for the nuanced bits we spent different time on to be useful", and in cases like that experts spending time together is still good (two mathematicians working on slightly different problems can use each other's expertise).
I think that there's a lot of analysis you could do here. For different curves, you can talk about what kind of distribution algorithm would best maximize value. If you make some assumptions about what distribution algorithm humans are using you could even suggest particular network topologies and recommendation algorithms that best facilitate that distribution algorithm.
Two relevant thoughts here would be the Good Regulator Theorem which could be argued to suggest that your social network is less helpful the less it "looks like" the underlying problems it's thinking about, and Conway's Law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure". These suggest that it is important to let individuals design things that work for them and help them in their specific context and problem, instead of some organization imposing some top down system. See On centering, solutionism, justice and (un)fairness and Algorithmic injustice: a relational ethics approach. (Still iffy about this paragraph tbh, it seems complicated and also computational universality results suggest to me maybe the difference is just a constant factor, but maybe that's not true for distributed algorithms and the particular heuristics humans use?)
I also think that "value" is a little too abstract here, and you could imagine something that provides perceived value to an individual while hurting someone else. For example, imagine someone has a simplifying model of the world that includes viewing minorities as a scapegoat for many problems (this is one form of racism). The individual gets utility through having an (incorrect) simplified model of the world and thus feeling like they are better able to understand the world around them, while the people the model is racist about get hurt by this opinion being widespread.
I'm also interested in looking at constraints around cognitive dissonance (which may encourage development of opinions that inhibit inconsistent opinions with one's currently held set of opinions, and protective opinions), lack of exposure to certain ideas, how much certain ideas feedback into reinforcing themselves (see Weaponized Sacredness), and difficulty in creating variations. Trapped Priors also seem like they'd create a bottleneck in some of these systems if not thought about carefully.
Still some open questions I have:
1. Why does solving problems for us not taper out at some height, and instead keep leading to new interesting things? Just saying "it's zero sum" doesn't seem strong enough, as the multi-agent team said they have a "love hate relationship" with zero sum games. They are great because they make a curriculum (two dumb agents can improve by one just being a little less dumb), however usually they seem to cap out after a point, and it's tricky to tune things properly. It's possible that that "cap out" point is just a problem with not having good enough learning methods, not having complex enough environments, or not putting enough compute into the system, but ideally they'd like to have good signs that we don't need to dump tons of compute into things.
2. When do anti-inductive systems (which are sorta zero sum) help for problem solving? They seem like a natural choice for a zero sum game that is open-ended, but it's unclear exactly how they relate to solving problems.
3. How does self organization fit into all of this?
4. Is the relationship between converting our nuanced system 1 intuition into a set of linguistic symbols an important piece of all this? (constantly trying to compress and distill intuition, but always being behind compared to the intuition (since any distillation will also be captured by the intuition) might be a source of constant open-ended understanding) Or is it just an artifact of the way we think and our evolutionary history?
5. What are the origins of the "attention distribution algorithm" we are using on a cultural level? How much of it is evolved vs learned? How can we break it apart into concrete research problems? I think that Kamal's work on Learning Social Learning is a great start in this direction. Update: See my post on my current answer to this question.
Comments
Post a Comment