Peaks & dips in financial markets, timescales between perilous events, evolutions/extinctions in ecology, the coordination between neurons before they collectively fire, and even learning/training processes within AI model architectures, share a couple of underlying themes.

1) They are confusing. The lack mechanistic explainability and are challenging to model -- it isn’t obvious from their random fluctuations how to discern signal from noise. Broadly speaking, it’s safe to categorize these system as disordered, given the difficulty in mapping exact structural causes to their dynamics.

2) It’s presumably worth dealing with how confusing they are. Even though the individual fluctuations are random, there are macroscopic tendencies that can be defined probabilistically, which then leads to the question: what governs these tendencies in the first place? The benefit to figuring this out is that we might be able to predict, even control what is putatively random, or as the saying goes, “find order in disorder”. Note that we can only hope to better control the systems where the dynamics are driven (make learning algorithms converge faster). We cannot control whether natural events happen although we can prepare around them.

I primarily used the energy landscape framework, which is a way to think about complex system in terms of all their possible states and their relative stability (related to potential energy).

Mapping all the possible states of a complex system and their corresponding potential energies may result in a rugged energy landscape with peaks (for its most unstable positions) and valleys (for it’s most stable positions). This framework can be extended to pretty much any system whose configuration spans a number of possibilities. Seminal examples include constraint satisfaction problems such as the traveling salesman, or the Levinthal paradox for protein folding where there are a number of solutions but one optimal solution that is incredibly difficult to compute. That is of course, unless we have a bird’s eye view of the landscape (which we do not!).

Studying stochastic processes is like studying motion along the surface of this landscape. If we move about the rugged landscape randomly and forgetfully, we would only encounter a solution by “luck”. It might take us anywhere between seconds or years to land where we want. To make an explicit connection to modern machine learning, training a model can be viewed as searching for good solutions on a loss landscape.If we had global knowledge of the landscape, then we would methodically be able to travel to our best solution. Taking heed to that logic, it becomes clear that geometrically distinct landscapes indicate that some problems are harder to solve because of the terrain (convex problems), whereas others are easier (concave problems).

My projects fall under two approaches to tackle the problem of extracting structure from disorder.

The first category is to use statistical physics to understand whether seemingly random behavior conceals deeper signals about what governs the disorder. Using Monte Carlo simulations of Ising spin models, I studied how movement across rugged energy landscapes gives rise to rare events, collective behavior, and memory.

The second category approaches the problem from the opposite direction – if we have observational data in a disordered state, then what is the minimal amount of information we need to recover it’s true structure?