Continuing building the research mountain

(not my biweekly post, just some thoughts I've had about research)

Previous posts: Beware the important problem: aka Scope Creep in Research, Letting the problem shape the direction you go, first part of Research Processes and Inductive Linguistic Biases

My outside view of research was that it was like "slowly chiseling away at problems, breaking off pieces of them until eventually we break down the whole thing".

But these days, I feel like a much better analogy is "building mountains". We have some really high up cloud we are trying to reach, in a pretend world where clouds are fixed and don't move. We can't jump up there immediately (there's no viable approach), so we need to start with some simple sub problem.

It makes sense to pick the smallest sub problem you can find, a minimum viable product, some tiny spec of cloud slightly off the ground. You build a small mound around that research direction, and now you can reach a little higher. You look for the next minimum piece of work you can do to get higher up, and you build some more mountain there.

Sometimes there's a nice connection, and you can use a huge chunk of someone else's mountain to bridge up much further really quickly. But ultimately you are still constrained by whichever points you can climb up. Sometimes it's hard to even know if you are getting closer, and that's okay. Measuring yourself by "am I closer" is much less helpful then measuring yourself by how many new mountains you have built, but of course this is a tricky balance. If those mountains end up not being as useful for your target problem, that's unfortunate, but just part of how research works.

As Hamming said, "Education is what, when, and why to do things, training is how to do it. Most courses are training. There are very technical people who applied their methods and technology to the wrong problem and it had to be undone. There are other people with all kinds of theories that couldn't do anything. Neither is useful. You need both theory to guide you, and skill and technique to do well."

Once you've made a small MVP trying to tackle some small piece of your problem, a few useful questions to ask are:

"Can this be stated in a way that's general and useful for other people, instead of being specific to this problem?"

"What's the smallest generalization that we can't disprove?" Keep working through them until you find something you can't disprove/prove. Make sure to state things crisply so you know precisely what you're talking about. Keep building the smallest MVPs that build off your current thing.

"What are all the things that it seems like we need? Do we actually need to solve those problems? Or can we sidestep them?"

"Does a single concept actually represent multiple distinct things that could both be active or inactive at different times?"

"What are other objects/concepts this principle might apply to?" Generating and working through patterns, and abstracting principles and patterns from those examples are one of the best ways to gain more knowledge about something.

"Does this theory match how I'm thinking about things? If not, what's missing?" (especially relevant if there seems to be a "theory gap" between theory and practice, as people build up informal intuitions that can often diverge from the formal theory)

"Hey, wouldn't it be fun if?" (research is ultimately a creative process, so it's important to be in Adventure Mode where you can generate good, new ideas. From that video, which I highly recommend: "Once you lay down the bare bones of your project, then you can tidy up and take time to polish. But until then, do experimentation without judgement. Novelty, playing around, chaos. Chaos and fun before everything. If you know there's something to an idea but it's not working, it's because you're trying to polish the table without putting the legs on.")