The World Through Lagrangian and Noether's Glasses

I want to improve my understanding of the Lagrangian approach to physics. Imagine I am sitting at a cafe looking around. If instead of the standard Newtonian view of the world, I had glasses that make me see the world from a Lagrangian perspective, what would I see as I look around me?

Great question! Let’s imagine you’re sitting at that café, sipping your coffee, and you slip on a pair of "Lagrangian glasses" that shift your perspective from the familiar Newtonian view to the Lagrangian approach. In Newtonian physics, you’d see objects moving under the influence of forces—gravity pulling cups downward, friction keeping them on the table, or the force of your hand pushing a spoon. But with Lagrangian glasses, the world transforms into a mesmerizing interplay of energy and motion, governed by a single unifying principle.

As you look around, you wouldn’t see forces as separate entities. Instead, you’d perceive everything through the lens of the Lagrangian, a mathematical function that combines kinetic energy (the energy of motion) and potential energy (the energy of position). The motion of every object—people walking, a spoon stirring coffee, even the steam rising from your cup—would appear as the result of nature optimizing this Lagrangian over time, following the principle of least action. It’s as if the universe is constantly solving an elegant optimization problem right before your eyes.

Take a person walking past your table. In Newtonian terms, you might think of their muscles exerting forces to move their legs. With your Lagrangian glasses, you’d see their motion as a smooth path through space, where the difference between their kinetic energy (from walking) and potential energy (from gravity and their position) is minimized over the journey from point A to point B. Their stride wouldn’t look random—it would feel like the most "efficient" trajectory the universe could compute, given the energy constraints.

Now glance at your coffee cup sitting still on the table. Newton would say it’s at rest because the forces (gravity, normal force) balance out. Through your Lagrangian lenses, you’d see it differently: the cup’s potential energy (due to its height above the ground) and kinetic energy (zero, since it’s not moving) define a stable equilibrium. The Lagrangian isn’t changing because the system has already found its optimal state—no action is needed.

If you nudge the spoon next to your cup, watching it slide a little, you’d see a dynamic shift. The kinetic energy briefly spikes as it moves, while the potential energy (slightly altered by friction or a tiny tilt in the table) adjusts. The spoon’s short journey would look like a curve in an abstract "configuration space"—a space where all possible positions of the spoon exist—and its path would be the one that extremizes the action, a quantity derived from the Lagrangian over time.

Even the steam curling upward from your coffee would take on new meaning. Instead of just hot air rising due to buoyancy (a Newtonian force), you’d see a flow of particles whose collective motion minimizes the action, balancing their kinetic energy against the gravitational potential pulling them back down. It’s less about individual forces and more about the system as a whole finding its natural rhythm.

The Lagrangian perspective might feel abstract at first because it’s less intuitive than "push and pull." But with these glasses, the café becomes a symphony of energy transformations. Every motion, every stillness, is part of a grand optimization dance, where the universe picks the paths that make the action—the integral of the Lagrangian over time—as small (or as extreme) as possible. It’s a holistic view: rather than breaking things into forces and reactions, you’d see the world as a unified system evolving according to an elegant, almost poetic, principle.

So am I seeing the phase or state space instead of the physical location and movement?

With your "Lagrangian glasses" on, you’re not exactly seeing the physical locations and movements in the direct, tangible way you would with a Newtonian perspective. Instead, your view is indeed more aligned with an abstract representation, like a phase space or configuration space, though it’s not quite the same as staring at a graph of those spaces directly. Let me clarify what you’d perceive and how it ties to the Lagrangian approach.

In the Newtonian view, you’re grounded in real space: a spoon moves across the table from point A to point B, and you see its position and velocity in the familiar 3D world. Forces—like your push or friction—explain why it moves or stops. But in the Lagrangian perspective, the focus shifts away from forces and toward a more holistic description of the system’s evolution, which is naturally tied to abstract spaces like configuration space or phase space.

With these glasses, you wouldn’t see the café in terms of x, y, z coordinates alone. You’d perceive the state of everything around you—the positions and velocities of objects (like the spoon, the steam, or the walking person)—as points in a higher-dimensional configuration space. This space doesn’t map directly to the physical café; it’s a mathematical construct where each "point" represents a possible configuration of the system. For the spoon, that might mean its position on the table and its angle if it’s tipping. For the walking person, it’s the positions of all their limbs at a given moment.

As you watch the spoon slide, your Lagrangian glasses would reveal its trajectory not as a line across the table, but as a path through this configuration space. The Lagrangian itself—kinetic minus potential energy—governs how that path unfolds. You’d "see" the spoon’s motion as the universe tracing the most optimal route, the one that extremizes the action (the time integral of the Lagrangian). It’s less about watching the spoon in physical space and more about sensing its journey through this abstract landscape of possibilities.

Phase space takes it a step further. While configuration space tracks positions, phase space includes both positions and momenta (mass times velocity). With your glasses tuned to the Lagrangian, you might intuitively sense the spoon’s momentum shifting as it slows down due to friction, or the steam particles’ momenta as they rise and disperse. You wouldn’t see a literal graph with axes labeled "position" and "momentum," but you’d perceive the system’s evolution as a flow through this phase space, guided by the Lagrangian dynamics.

So, to answer your question: yes, in a way, you’re seeing something closer to phase or configuration space rather than just physical locations and movements. The café’s bustling activity—spoons sliding, people walking, steam swirling—becomes a dance of states, where each object’s behavior emerges from energy relationships rather than isolated forces. It’s as if your glasses overlay an abstract, energy-driven narrative on top of the physical scene. You’re still in the café, but you’re witnessing the underlying "why" of motion in a way that transcends the concrete "where" and "how fast."

So, could I walk down the street if I had my Lagrangian glasses on? Could I have a conversation if I had Lagrangian headphones on?

Let’s keep the imagination going and tackle these one at a time—walking with Lagrangian glasses and conversing with Lagrangian headphones. Both ideas stretch the Lagrangian perspective into sensory metaphors, so I’ll interpret what you might experience while still grounding it in the physics.

Walking Down the Street with Lagrangian Glasses

Yes, you could absolutely walk down the street with your Lagrangian glasses on, but it’d be a wild shift from your usual experience! Normally, you’d feel your legs moving, sense the ground pushing back, and adjust your balance intuitively—all through a Newtonian lens of forces and reactions. With Lagrangian glasses, your perception would reframe the act of walking entirely.

As you step forward, you wouldn’t focus on the force of your muscles or the friction of your shoes. Instead, you’d "see" your body as a system optimizing its action. Your kinetic energy (from your legs swinging and torso moving) minus your potential energy (shifting slightly with each step due to gravity and height changes) would define a Lagrangian. Each stride would feel like a natural, flowing path through configuration space—where the "configuration" is the positions of your joints, arms, and legs. Your glasses would highlight the trajectory your body takes, the one that minimizes the action over time, almost like you’re watching yourself solve a cosmic optimization problem with every step.

Practically, though? You’d still navigate fine—your brain’s muscle memory and spatial awareness would keep you upright and moving. But your attention might drift to the elegance of the motion: the way your arms swing to balance your kinetic energy, or how your steps adjust effortlessly when the sidewalk slopes, reducing gravitational potential. Obstacles like a lamppost or a crowd would register as constraints in this abstract space, and you’d "see" your path curve smoothly around them to keep the action minimal. It’d be trippy, but you’d make it down the street—maybe with a newfound appreciation for the physics of your gait!

Conversing with Lagrangian Headphones

Now, slip on some Lagrangian headphones while chatting with a friend at the café. In everyday life, you hear sound as waves—vibrations in the air hitting your eardrums, explained by Newton’s laws or basic acoustics. With Lagrangian headphones, you’d perceive sound through the lens of energy and optimization, applied to the system of air particles, your vocal cords, and the surrounding environment.

As your friend speaks, you wouldn’t just hear words. You’d "hear" the Lagrangian at work: the kinetic energy of vibrating air molecules minus the potential energy stored in their pressure variations. The sound waves traveling from their mouth to your ears would feel like a collective motion, a path through the configuration space of all those particles, optimized by the principle of least action. Each syllable would emerge as the most efficient way for energy to propagate through the air, given the constraints of distance, temperature, and medium.

If you replied, your headphones would tune you into your own voice as a dynamic system. Your vocal cords’ oscillations, the air they push, and the resulting sound waves would blend into a Lagrangian flow. The conversation would become a duet of energy exchanges, where every word and pause fits into this grand optimization dance. You could still understand your friend—language wouldn’t break down—but you might sense the rhythm and pitch of their voice as signatures of kinetic-potential interplay, rather than just tones.

Could you keep the chat going? Sure! Your brain would still process meaning and intent, but the headphones might make you hyper-aware of the physics behind every sound. A loud laugh might "sound" like a sudden spike in kinetic energy, while a whisper feels like a delicate minimization of action. It’d be an odd, almost musical way to experience dialogue.

The Catch

Both the glasses and headphones would layer this abstract, energy-based perspective over your senses without disabling your ability to function. Walking and talking would still work because your body and brain are wired for practical navigation and communication. But you’d be constantly distracted by the underlying beauty of the Lagrangian framework—seeing and hearing the world as a symphony of optimized paths rather than a collection of forces or waves. It might take some getting used to!

So what information do my Lagrangian glasses give me, if it is not the usual light rays at various wavelengths?

Your Lagrangian glasses are a fascinating thought experiment! They don’t show you the usual light rays at various wavelengths—those photons hitting your retina to form colors and shapes in the visible spectrum. Instead, they reframe your perception entirely, giving you information rooted in the Lagrangian approach: the dynamics of energy and motion, interpreted through the principle of least action. Let’s break down what they’d "show" you as you look around the café or stroll down the street.

What You’d Perceive

Rather than light rays carrying data about color, brightness, or distance, your Lagrangian glasses would present a processed, abstract overlay of the physical systems around you. They’d translate the world into a visual representation of the Lagrangian—kinetic energy minus potential energy—and how it governs the evolution of everything in view. Here’s what that might look like:

1. Trajectories as Energy Paths
Instead of seeing a spoon sitting statically on the table or sliding across it, you’d see its "path" through an abstract configuration space. If it’s still, the glasses might highlight a flat, stable energy landscape—showing zero kinetic energy and a constant potential energy due to its position. If you nudge it, you’d see a flowing curve, like a glowing trail, representing the spoon’s motion as the optimal path that extremizes the action. This trail wouldn’t be in physical space but a visual cue of the system’s dynamics.

2. Energy Landscapes
For every object or person, the glasses could display a kind of topographic overlay—an intuitive map of the potential energy field. A cup on the table might glow faintly at a low potential (stable equilibrium), while a person walking past shows a shimmering gradient as their potential energy shifts with height and position, paired with bursts of kinetic energy in their steps. It’s less about seeing the person’s shape and more about visualizing their energy state.

3. System-Wide Optimization
The glasses wouldn’t isolate objects like a Newtonian view does with forces. You’d see interconnections. A waiter balancing a tray, for instance, would appear as a unified system—arms, tray, and cups moving together along a path that minimizes the action. The wobble of the tray might flash as a perturbation, quickly corrected as the Lagrangian stabilizes the system. It’s a holistic view, showing how everything collaborates in the dance of energy.

4. Motion as Flow
Moving objects—like steam rising or a dog trotting by—wouldn’t just look like shapes in motion. You’d see their trajectories as smooth, optimized flows, almost like streams of light bending through an unseen space. The glasses might shade these flows with intensity or color to indicate kinetic energy (bright for fast, dim for slow) and potential energy (height or tension in the system).

5. Constraints and Boundaries
Obstacles or forces (like friction or gravity) wouldn’t appear as arrows or vectors. Instead, they’d manifest as "walls" or "slopes" in this energy landscape. Walking uphill? The glasses might tint the ground ahead with a rising gradient, showing increased potential energy your body must overcome, while your steps adjust to keep the action minimal.

How It’s Delivered

Since this isn’t about wavelengths of light, the glasses wouldn’t rely on photons in the usual sense. Think of them as a high-tech augmented reality system, interpreting the physics of the scene and projecting it into your field of view. They’d use sensors to track positions, velocities, and masses, then compute the Lagrangian in real-time—kinetic energy (1/2 mv²) minus potential energy (like mgh for gravity or elastic energy for tension). The result is a visual language of curves, gradients, and flows, not a spectrum of reds, blues, or greens.

For example:
- A falling sugar packet might show a steep drop in its energy landscape, with a bright streak tracing its descent as kinetic energy surges.
- A spinning spoon might pulse with rotational kinetic energy, its path a helical swirl in configuration space.

Practicality Check

You’d still need some baseline awareness of physical space to avoid bumping into things—maybe the glasses include a faint outline of objects for navigation. But the primary "signal" isn’t light rays; it’s a dynamic map of how energy dictates motion and stability. It’s less about what things look like and more about what they’re doing in the grand optimization of nature.

So do they show me the 'action'?

Your Lagrangian glasses could absolutely show you the action—in fact, that’s a brilliant way to tie them directly to the heart of the Lagrangian approach! The action, defined as the time integral of the Lagrangian (kinetic energy minus potential energy) over a path, is the central quantity that the principle of least action extremizes. Let’s explore how the glasses might visualize this as you observe the world.

Seeing the Action

Instead of just showing the Lagrangian at a single instant (like a snapshot of kinetic minus potential energy), the glasses could display the action as a cumulative, evolving property of the systems around you. Here’s how that might look:

1. Trajectories with Action Trails
As you watch a spoon slide across the table after a nudge, the glasses wouldn’t just show its current energy state. They’d trace its path with a glowing trail, and that trail could pulse or shift in intensity to represent the action—the integral of the Lagrangian over the time of its motion. A short, smooth slide might glow faintly, indicating a small action value, because the path efficiently balances kinetic and potential energy. If you flicked it hard and it tumbled, the trail might flare brighter or flicker, suggesting a larger or less optimal action due to extra rotational energy or friction.

2. Color-Coded Extremization
The glasses could use color to signal how "optimal" the action is. Paths that minimize the action—like a person walking steadily or a cup resting undisturbed—might glow a cool blue or green, showing the system has found its least-action trajectory. A chaotic motion, like a stumbling waiter or a spoon wobbling before settling, might flash red or orange, hinting at a higher action that’s not yet minimized. You’d see the universe "searching" for efficiency in real-time.

3. Action Horizons
For moving objects, the glasses might project a faint "horizon" ahead of them—a preview of the action for their likely future path. Walking down the street, your own steps could have a shimmering line extending forward, dimming or brightening as you adjust your pace or navigate a curb. It’d be like seeing the cost of your motion in action units, updated dynamically as the Lagrangian integrates over time.

4. Static Systems and Zero Action
For something at rest—like your coffee cup on the table—the action wouldn’t accumulate because there’s no motion (kinetic energy is zero, and potential energy is constant). The glasses might show a faint, steady outline, indicating a trivial action of zero along its "path" of staying put. But if you tipped the cup slightly and let it rock back, you’d see a brief ripple of action as it oscillates and settles.

5. Comparative Action
Looking at multiple objects—like steam rising versus a spoon falling—the glasses could compare their actions visually. The steam’s diffuse, wandering particles might show a complex, mottled pattern of moderate action, while the spoon’s direct drop flashes a sharp, low-action spike. You’d get a sense of how different systems stack up in their quest for optimization.

How It’s Visualized

The action isn’t a single number floating in space (though the glasses could overlay a numerical value if you wanted—say, in joule-seconds). Instead, it’s more intuitive: a dynamic, flowing representation tied to each object’s motion. Think of it like a heatmap or a stream of light that thickens, thins, or shifts hue based on how the action accumulates along a trajectory. The glasses would compute this by tracking the Lagrangian (T - V) over small time intervals and summing it up, then rendering it as a visual cue.

For example:
- A kid tossing a ball outside the café window might produce a parabolic arc of light, with the action trail peaking mid-flight (high kinetic energy) and softening as it lands (potential energy dominates).
- Your own arm stirring coffee might show a looping action pattern, tightening as you stir faster (more kinetic energy, less optimal action) and calming as you slow down.

What It Tells You

Seeing the action directly would emphasize the why behind every motion. You’d understand that the spoon slides, the ball flies, or you walk in a particular way because those paths yield the smallest (or extremal) action possible given the constraints. It’s not just energy at a moment—it’s the story of energy over time, distilled into a single, guiding principle.

OK, so by Noether's Theorem, this can translate into symmetries, no? What would I see if I put on Noether's glasses?

You’re absolutely right—by Noether’s Theorem, every continuous symmetry in a physical system corresponds to a conserved quantity, and this ties beautifully to the Lagrangian framework. If your Lagrangian glasses show you the action and its optimization, slipping on a pair of "Noether’s glasses" would shift your perspective again, revealing the symmetries underlying that action and the conserved quantities they imply. Let’s imagine what you’d see at the café or walking down the street with these new lenses.

What Noether’s Glasses Show You

Noether’s Theorem connects symmetries in the Lagrangian—invariances under transformations like time, space, or rotation—to conservation laws (energy, momentum, angular momentum). These glasses wouldn’t focus on the action’s value or its energy components directly. Instead, they’d highlight the hidden patterns in the systems around you, visualizing how symmetries dictate what stays constant amid all the motion. Here’s how that might look:

1. Time Symmetry and Energy Conservation
Since the Lagrangian of most everyday systems doesn’t change with time (the laws of physics are the same now as a second from now), Noether’s Theorem guarantees energy conservation. With Noether’s glasses, objects in steady motion—like a person walking at a constant pace—might glow with a uniform aura, signaling that their total energy (kinetic plus potential) is conserved. If you nudge a spoon and it slides then stops, the aura might ripple briefly as kinetic energy dissipates into heat (via friction), but the glasses would subtly trace that energy’s redistribution, showing it’s not lost, just transformed.

2. Space Symmetry and Momentum Conservation
The Lagrangian’s invariance under spatial translations (physics works the same here as a meter over there) means linear momentum is conserved. Looking at a rolling coffee cup, you’d see a sleek, arrow-like shimmer along its path, representing its conserved momentum as it moves unimpeded. If it bumps the table’s edge and stops, the shimmer might split, transferring to the table or dissipating as vibration—visualizing momentum’s conservation or redistribution. Walking down the street, your own steady stride might show a faint momentum trail, constant until you turn or slow down.

3. Rotational Symmetry and Angular Momentum
Rotational invariance (physics doesn’t care which way you’re facing) links to angular momentum conservation. Spin a spoon on the table, and Noether’s glasses might wrap it in a swirling halo, its brightness tied to the conserved angular momentum as it twirls. A kid spinning a top outside the café would light up with a vivid, stable vortex—until friction slows it, and the halo fades as angular momentum dissipates into the ground.

4. Symmetry Highlights
The glasses could emphasize where symmetries hold or break. A perfectly balanced tray carried by a waiter might pulse with a symmetric glow, showing energy and momentum conservation in its stability. But if it tilts, the glow might fracture, indicating a broken symmetry (the system’s no longer invariant under small rotations), and you’d see the conserved quantities shift—perhaps angular momentum kicking in as it wobbles.

5. Abstract Symmetry Maps
Beyond specific objects, Noether’s glasses might overlay a faint grid or field across your view, marking the symmetries of the environment itself. A flat sidewalk might shimmer uniformly (spatial symmetry), while a staircase breaks that pattern, with steps glowing differently to show potential energy gradients disrupting translational invariance. Time symmetry might appear as a steady background hum—until something dramatic (like a crash) suggests an external disturbance altering the system’s rules.

How It’s Visualized

Unlike Lagrangian glasses, which show dynamic flows of action, Noether’s glasses would use stable, persistent cues—like auras, halos, or directional streams—to represent conserved quantities tied to symmetries. Think of it as a layer of "constancy" over the chaos of motion:
- A walking person’s momentum might appear as a straight, glowing thread trailing behind them.
- A spinning spoon’s angular momentum could be a coiling ribbon, tightening or unwinding with its speed.
- Energy conservation might wash everything in a soft, steady light, flickering only when transformations (like collisions) redistribute it.

The glasses would detect the Lagrangian’s mathematical structure behind the scenes, identify its invariances (e.g., no explicit time dependence = time symmetry), and render the resulting conserved quantities visually. You’d see the consequences of symmetry, not the equations themselves—unless you wanted a nerdy toggle to flash "p = constant" next to that rolling cup!

What It Tells You

Noether’s glasses reveal the deep order beneath the surface. Where Lagrangian glasses show you the "how" of motion (optimized action), Noether’s glasses show you the "why" of stability (symmetries preserving quantities). A ball tossed in the air doesn’t just follow a path of least action—it conserves energy because time ticks the same way throughout its flight, and you’d see that as a steady glow tracing its arc. You’re witnessing the universe’s bookkeeping: nothing’s created or destroyed, just balanced by symmetry.

Café Example

Sitting at the café:
- A stirring spoon spins with a rotating halo (angular momentum conserved).
- Steam rises with a diffuse, steady sheen (energy conserved as it disperses).
- A waiter walks by with a momentum thread stretching behind, unwavering until he turns the corner (space symmetry broken, momentum adjusts).

Earlier, it sounded like the glasses would still show physical location, but would we actually see physical location at all, if the glasses were strictly Lagrangian?

You’re digging into a really sharp question here: if we’re strictly adhering to a Lagrangian perspective, would physical location even show up in a game like Pong—or would it be entirely abstracted away? Let’s think this through carefully, sticking to the pure Lagrangian framework, and see what it means for your video game.

The Strictly Lagrangian View

In the Lagrangian formalism, the focus isn’t on physical location (x, y coordinates in space) as a primary entity like it is in Newtonian mechanics. Instead, it’s all about the Lagrangian—kinetic energy (T) minus potential energy (V)—and how it evolves over time to define the system’s motion via the principle of least action. The physical location of an object (like the Pong ball) isn’t what you "see" directly; it’s a derived outcome of solving the Euler-Lagrange equations, which come from extremizing the action (S = ∫ L dt).

So, in a strictly Lagrangian version of Pong:
- You wouldn’t see the ball’s position (x, y) as a dot moving across the screen in the usual 2D space.
- You’d see a representation of the dynamics governed by the Lagrangian, typically expressed in terms of generalized coordinates (which could be positions, but don’t have to be) and their time derivatives (velocities).

For Pong, the generalized coordinate might still be the ball’s position (x, y), but the visualization wouldn’t emphasize "where it is" in a spatial map. Instead, it’d show how the system’s energy and action dictate its path, without necessarily tying that to a concrete physical location on-screen.

What You’d See Instead

If we strip away the physical location and go pure Lagrangian, here’s what the game might look like:
- The Ball’s Trajectory as an Action Path: The ball would appear as a flowing curve or trail, representing its path through configuration space (the set of all possible positions). But this isn’t a line drawn on a 2D court—it’s an abstract visualization of the action (∫ (T - V) dt). The trail’s intensity or shape could reflect the Lagrangian’s value over time: bright where kinetic energy dominates (fast motion), dim where potential energy rises (near paddles or under gravity).
- Paddles as Energy Constraints: The paddles wouldn’t be vertical bars at specific x-positions. They’d be dynamic boundaries in the energy landscape—glowing walls that spike the potential energy (V) when the ball’s trajectory nears them, forcing the action to adjust. You wouldn’t see them as objects in space, but as perturbations in the system’s evolution.
- No Fixed Court: The screen wouldn’t resemble a rectangular Pong arena. It might be a fluid, abstract field where the action unfolds—think of it like a graph of energy states or a heatmap, not a spatial grid. The "edges" (where a point is scored) would be where the trajectory escapes the system’s constraints, not literal left/right boundaries.

Playing Without Physical Location

Could you still play it? Yes, but it’d be a radical departure from traditional Pong:
- Control: You’d move your paddle by adjusting the energy constraint—say, shifting a potential energy barrier up or down in the abstract field. The game engine would still track the ball’s position internally (to compute T and V), but you wouldn’t see it as an (x, y) dot. You’d react to the trajectory trail’s approach toward your "barrier," nudging it to intercept and redirect the path.
- Feedback: The trail’s behavior—its brightness, curvature, or speed—would tell you when and where to act. A fast, straight trail heading your way means a high-kinetic-energy shot; a slow, wavering one might indicate a softer hit or gravitational curve.
- Scoring: A point happens when the opponent fails to constrain the trajectory, and the action path "escapes" their influence, visualized as the trail veering off into the abstract void.

The Catch: Do We Need Some Location?

Here’s the rub: while a strictly Lagrangian view doesn’t prioritize physical location, humans rely on spatial intuition to play games. The Lagrangian formalism still uses coordinates (like x, y) to define the system, even if they’re generalized and abstract. To make Pong playable, we might cheat a little and imply location indirectly:
- The trajectory trail could hint at spatial motion (e.g., left-to-right flow), even if it’s not plotted on a fixed court.
- The paddle’s energy barrier could have a vague "up/down" orientation, giving you a sense of position without explicit coordinates.

If we went 100% pure—no spatial cues at all—it’d be like playing a game of pure energy flows, which might feel more like an avant-garde art piece than Pong. You’d be manipulating abstract curves and intensities without any anchor to a physical "where," which could be disorienting.

Lagrangian Pong with Minimal Location

To balance strictness and playability, a practical version might look like this:
- The screen shows a faint, dynamic background (not a fixed court) where the trajectory trail flows.
- The trail’s direction and speed reflect the Lagrangian’s optimization, but its general left-right motion mimics the classic game.
- Paddles are vertical energy gradients you slide along one axis (up/down), intersecting the trail to bounce it back.

You’re not seeing "the ball at (x, y)," but the trail’s evolution gives you enough context to react spatially, derived from the Lagrangian dynamics.

Conclusion

In a strictly Lagrangian Pong, physical location vanishes as a primary visual. You’d see the action and energy interplay, not a ball in space. It’s playable if we keep some intuitive cues (like trail direction), but without any spatial reference, it’d be a mind-bending challenge—more like conducting a physics symphony than hitting a pixel. For practicality, a hybrid approach (action trails with implied location) keeps it fun and accessible.

Want to lean harder into the pure abstraction and brainstorm controls for a no-location version, or tweak this hybrid for a prototype?

so back to my original question, with my (strictly) Lagrangian glasses, would I really be able to walk down the street?

Let’s circle back to your original question and apply the strictly Lagrangian perspective to walking down the street with those glasses on. If they’re purely Lagrangian—showing only the dynamics of the action and energy, with no direct representation of physical location—things get tricky. Let’s break it down and see if you could still manage to stroll through the café and down the street.

What You’d See with Strictly Lagrangian Glasses

As we’ve established, a strictly Lagrangian view doesn’t prioritize physical location (x, y, z coordinates) the way your eyes normally do with light rays. Instead, it reveals the Lagrangian (T - V) and the action (∫ L dt), focusing on how systems evolve to extremize that action. With these glasses:
- Your Body: You wouldn’t see your legs or arms in a spatial layout. You’d perceive your motion as a flowing trajectory through configuration space—say, the angles of your joints or positions of your limbs. This trajectory would glow or shift, reflecting your kinetic energy (from walking) minus potential energy (from gravity or terrain).
- The Street: The ground wouldn’t appear as a solid surface at a specific place. It’d be an energy landscape—flat areas with low potential, slopes with rising gradients, obstacles as sharp spikes in V. Your trajectory would bend or adjust to minimize the action around these constraints.
- Surroundings: People, cars, or lampposts wouldn’t be objects at fixed spots. They’d be dynamic systems—other trajectories interacting with yours, their own action paths weaving through the same abstract field. A car speeding by might flash as a bright, fast-moving trail; a stationary post might pulse as a fixed potential barrier.

No physical locations—just energy flows, action trails, and constraints.

Could You Walk?

Here’s where it gets challenging:
- Navigation: Normally, you walk by visually mapping the street—seeing the sidewalk’s edge, dodging a pole, stepping over a curb. With Lagrangian glasses, you’d have no direct spatial map. You’d sense the dynamics—a curb might register as a rising potential gradient bending your trajectory upward, a pole as a sudden spike you’d curve around. But without a clear "where," you’d rely on interpreting these abstract cues fast enough to adjust your steps.
- Feedback: Your brain is wired for spatial input (light rays showing distance and position). Lagrangian glasses give you energy and action data—useful for physics, but not intuitive for real-time movement. A fast-approaching obstacle (high kinetic energy toward you) might glow intensely, but translating that into "move left" isn’t automatic without practice.
- Muscle Memory: You could lean on habit—your body knows how to walk, and your vestibular system (inner ear) still tracks balance. If the glasses hinted at constraints (e.g., a wall’s potential spike), you might dodge instinctively. But complex environments (crowds, turns) would overwhelm this without spatial anchors.

Real-World Feasibility

In a controlled, simple setting—like a straight, empty street—you might manage:
- The flat ground shows as a steady, low-potential field.
- Your walking trajectory flows smoothly, minimizing action with each step.
- A single obstacle (say, a bench) appears as a potential bump, and you adjust your path based on its energy signature.

But in a busy, unpredictable reality:
- A crowd of people becomes a tangle of overlapping trajectories—hard to disentangle who’s where.
- Sudden changes (a car honking, a dog darting) flash as energy surges, but without location, you’d struggle to react directionally.
- Uneven terrain (steps, potholes) shifts the energy landscape fast, and misjudging one could trip you up—literally.

The Verdict

With strictly Lagrangian glasses, walking down the street would be theoretically possible but practically brutal. You’d be blind to physical location, relying entirely on interpreting action trails and energy gradients in real-time—a bit like navigating by echolocation or a sixth sense you haven’t trained. Your brain could adapt with insane practice (think Daredevil-level rewiring), mapping energy cues to spatial intuition, but out of the box? You’d likely stumble into a lamppost or freeze, overwhelmed by abstract data.

A Compromise?

If we relaxed the "strictly" part slightly—say, letting the glasses imply location through the trajectory’s direction (left-right, up-down hints)—you’d fare better. The action trails could flow in a way that mimics spatial motion, giving you enough context to dodge and stride. Still weird, but doable.

So, pure Lagrangian glasses? You’d be a physics savant tripping over your own feet. A hybrid version? You could strut down the street, marveling at the action while staying upright. Which version are you picturing for your café stroll?

But isn't this what the brain is really doing, taking in all of this physical location and movement information, and calculating a Lagrangian?

You’re raising a profound and intriguing idea: is the brain, behind the scenes, effectively computing something like a Lagrangian to navigate the world, even though we consciously perceive physical locations and movements? It’s a fascinating angle that bridges physics, neuroscience, and philosophy. Let’s explore whether the brain might be doing this—and what it means for your Lagrangian glasses.

The Brain and Physical Input

You’re right that the brain takes in raw sensory data—light rays for vision, sound waves for hearing, pressure for touch—and transforms it into a coherent picture of locations and motions. Your eyes detect photons, giving you a spatial map of the street; your vestibular system tracks acceleration and balance; your muscles report tension and effort. From this flood of inputs, you effortlessly walk, dodge obstacles, and sip coffee without falling over.

In Newtonian terms, we’d say the brain builds a model of forces and trajectories: “The ground pushes up, gravity pulls down, I step forward.” But could it be doing something akin to a Lagrangian calculation instead—optimizing a system-wide quantity like action based on energy?

The Lagrangian Analogy

The Lagrangian approach doesn’t care about forces as separate entities—it defines motion by extremizing the action (∫ (T - V) dt) across an entire path. This is holistic: it considers the system’s energy (kinetic and potential) over time, not just moment-to-moment pushes and pulls. Your brain does seem to operate holistically in some ways:
- Path Planning: When you walk down the street, you don’t calculate each step’s force explicitly. You intuitively pick a smooth, efficient path—avoiding a crowd, stepping over a curb—like you’re minimizing effort or optimizing some internal cost function. Studies in motor control (e.g., optimal control theory) suggest the brain plans movements to minimize energy expenditure or jerkiness, eerily close to a least-action principle.
- Predictive Processing: Your brain anticipates motion—like catching a ball or timing a dodge—by modeling future states, not just reacting to the present. This forward-looking integration resembles how the Lagrangian evaluates a path over time, not just a snapshot.
- Whole-Body Coordination: Walking involves dozens of muscles and joints working in sync. The brain doesn’t micromanage each one with force vectors; it orchestrates a fluid, system-wide motion, balancing kinetic energy (speed) and potential energy (height, stability)—a Lagrangian-like feat.

Does the Brain Compute a Lagrangian?

Not literally, in the mathematical sense—neurons aren’t solving ∂L/∂q - d/dt(∂L/∂q̇) = 0 (the Euler-Lagrange equation). But it might approximate something functionally similar:
- Energy Optimization: Research in biomechanics shows humans and animals often move in ways that minimize metabolic cost or mechanical work—close to minimizing a kinetic-potential difference. For example, your gait adjusts on a slope to keep effort low, much like a system following a least-action path.
- Internal Models: The brain uses “forward models” in the cerebellum and motor cortex to predict outcomes of actions (e.g., how far a step will take you). These models integrate position, velocity, and constraints (gravity, obstacles), akin to a Lagrangian defining dynamics in configuration space.
- Error Correction: When you stumble, your brain quickly adjusts your posture to regain balance—optimizing stability as if recalculating an action trajectory on the fly.

So, while the brain doesn’t explicitly compute T - V or integrate it over time, its motor and sensory systems might implicitly solve an optimization problem that mimics the Lagrangian approach. It’s less about equations and more about evolution tuning us to move efficiently in a world governed by physics.

Back to Your Glasses

If the brain is already doing a Lagrangian-like computation under the hood, why wouldn’t you walk fine with strictly Lagrangian glasses? The disconnect is in perception versus processing:
- Conscious Perception: You see locations and motions—light rays hitting your retina give you a spatial map. The glasses strip that away, feeding you raw action and energy data (trajectories, gradients) that your conscious mind isn’t trained to parse for navigation.
- Unconscious Processing: Your brain might calculate something Lagrangian-esque to control your body, but it translates the result into spatial commands (“step left”) you understand. The glasses skip that translation, leaving you with the abstract “source code” (action paths) instead of the user-friendly output (where things are).

It’s like the brain’s a chef cooking a Lagrangian dish but serving it as a Newtonian plate. The glasses show you the recipe—energy flows and optimization—while your normal vision shows the meal: a street you can walk down.

Could You Walk?

If your brain’s secretly Lagrangian, you could theoretically walk with the glasses—if you retrained your conscious mind to map their abstract output (trajectories, energy landscapes) back to spatial actions. It’d be like learning a new language: “Bright trail curving up” = “Lift leg for curb.” With practice, your brain’s underlying optimization might sync with the glasses’ view, letting you stroll confidently.

Without that retraining? You’d flounder—not because the brain can’t handle Lagrangian logic, but because your conscious interface expects locations, not energy integrals. It’s a mismatch between what the glasses show and what your awareness is wired to use.

Conclusion

The brain might indeed be a Lagrangian machine at some deep level, turning sensory chaos into optimized motion. Your strictly Lagrangian glasses would reveal that hidden process, but walking would demand rewiring your perception to match—a Herculean task, though not impossible. Maybe with enough time (and a few bruises), you’d saunter down the street, grinning at the action trails like a physics ninja.

Emergence

 

A flock of starlings
The shapes and patterns are an example of emergence.

 

Generally, complexity science is the study of emergence, and complex systems are systems which undergo or demonstrate emergence. Complex networks are networks which undergo emergence.

 

Emergence refers to the unveiling of macroscopic states or observables which do not exist at the microscopic level. At the molecular level, water does not have 'wetness'. Snowflake crystallization patterns are also emergent, in that they are macroscopic, as a. result of H2O molecule properties, temperature, and the large number of water molecules. There are a myriad examples if you look around.

 

The macroscopic states also have to do with the combinatorics of the system. For example, there are a combinatorially huge number of ways to arrange water molecules, but many of these would not macroscopically appear to be snowflakes. So the number of arrangements which we accept as snowflakes is very small compared to the total number of arrangements. Each arrangement is called a 'microstate', and the set of arrangements corresponding to some large scale state is called a a 'macrostate'. Ice, liquid water, and gas are three macrostates of water.

 

You can see this with coin-flipping. With a fair coin, there are 2^100 ways to flip it 100 times. These are all the microstates. But how many ways are there to flip 50 heads and 50 tails? We can count (100 choose 50) ways to get 50 heads out of 100 flips. So the probability of 50 heads is (100 choose 50) / (2^100), which is 0.0796. If you do the same calculation for 49 heads and 51 tails, and so on, you will see that the probabilty is less.

 

So the macrostate is dominated by the 50-50 result. This is an emergent property, resulting from the fair coin and flipping it 100 times. Just as an individual water molecule doesn't have wetness, a single coin flip does not have 50 heads and 50 tails. The tendency to 50 heads and 50 tails is revealed (emerges) by scaling up the number of flips.  Also notice that predicting the macroscopic result by seeing the microscopic system elements is not always obvious or easy. This is often true in systems where the scaling up is on many orders of magnitude.

 

So, 'more is different', as Philip Anderson famously said in 1972.

 

Formation of the
Giant Connected Component (GCC)
at center, in a random graph.

Some graphs/networks undergo emergence, the classic example being network percolation in random
(Erdos-Renyi) graphs, where the size of the giant connected component (the largest connected set of nodes) suddenly explodes when the network connectivity increases to a critical density.

 

This may not seem interesting at first, but it is actually very deep, since it means that any time I start having random interactions between any entities (people, atoms, grains of sand), if the density of those interactions reaches a critical point, I will very suddenly start to have system-wide interactions (interactions which scale proportional to the size of the system itself). Most real interactive systems are far from random, so this gives us a worst case situation.

 

Network Science approximately is the study of networks, which tend to be naturally-occurring networks, vs. graph theory the study of graphs, which are mathematical objects. There is a lot of overlap between these two, since they often look very similar or have similar properties, so they are used more or less interchangeably (e.g. above).

 

So, we can study networks but not focus on the emergent aspects of the system. But of course in networks, there are so many possible ways to have various kinds of emergence, that it becomes implicit that network science is about complexity.

As an exercise, calculate the most probable sum of rolling a 6-sided die 100 times.

 

For more on emergence, which is a fascinating topic, check the book Complexity Science by Henrik Jensen.

 

 

The earth is (not) flat!

There is a rather large irony in the idea that the earth is flat, and a the same time, there is a large irony in thinking that it is not.

 

 

In mathematics, there is a basic idea:  If we have a smooth curve, we can zoom in, and keep zooming in.  As we do this, the curve no longer looks like a curve, it begins to look like a line!

Now imagine we could keep zooming in infinitely to a point on the curve. In calculus the idea is that if we zoom in forever, we do get a straight line at that point, touching (tangent) to the curve.  This is called the derivative.

 

 

The mathematics of calculus and its derivative have been incredibly useful tools in the modern world.  Let's list a very few of its applications:   It is used to build structures, to analyze them, to design airplanes.  It plays a fundamental role in training a great many machine learning algorithms.  It is used to model chemical reactions, turbulence in the atmosphere, the movement of the planets, the flow of underground water in geology.   In physics it is used to study motion, electricity, heat, light, harmonics, acoustics, astronomy, and quantum mechanics.  
 

In short, this derivative is incredibly useful to describe change over space and time.  Many physical shapes, speeds, velocities, and rates of change of any kind, can all be understood and modeled by the derivative.

The list of its uses goes on and on... and on.  Without the tools of calculus, a great deal of the modern world would be challenging, perhaps impossible, to study and develop.   Possibly we would still be living with the technology of the 1700s or 1800s. 

 

Indeed, we are truly standing on the shoulders of (scientific) giants, very oblivious and conveniently forgetting it as we drive our cars through our cities, talking on our smartphones, and planes fly overhead....

 
 

 

 

 

 

 

 

Perhaps we can begin to see the irony...  

We often immediately dismiss people who believe the earth is flat, thinking ourselves better, because we are 'people of science'.   But a great deal of our entire world, and more importantly, the underlying assumption that we can analyze phenomena scientifically, is based on the derivative, the idea that curves are locally flat, that we can take the derivative.  

That is, as scientists, we are all, to some extent, flat-earthers (!!).
 


Indeed, pick anything you call 'curved'.   Now imagine you are extremely, extremely small, standing on that curve.  To you, the world is actually flat! 

Let's try zooming in to a curve and see what happens.

Starting out, our curve is so large and our person so tiny, we barely see them.

 

 

 

 

Let's zoom in.

Perhaps you notice how the curve seems flatter?

Let's zoom in some more...

Wait, what happened?  The curve disappeared!
If we look carefully, we can just make out a curve in the landscape, but it is hard to see.

Another issue, as always, is us.   We humans seem to have challenges looking at things globally.  We are intensely aware of our immediate surroundings, both in space and time.   We know our room very well, we think a lot about what we are doing right now or about to do.

We are not as good at thinking about things that are far away.  We do not often zoom out and think from a global perspective.  We do not think about what we'll do 5 years from now, while we scroll on our phone and answer texts from second to second.   We do not think about the state of the world while we eat our pizza.   Our world is greatly dominated by local events, local information.

 

There is good reason for this.  Biologically, our immediate survival depends on local events in time and space.   We care about the lion running at us, right here, right now.   We'll worry about winter later.  We'll worry about those far away forest fires when they get near.   If we get eaten by the lion, they won't matter anyway.

 

 

 

 Another way to say this is that our thoughts are dominated by our local scale.  Look back up at the figure on the zoomed-in curve.  The horizon is just a few body lengths away, it is on a similar scale as the figure.  That is the scale he or she navigates and perceives most of the time.   And on this scale, the earth is flat.

Now look up at the un-zoomed curve at the top, with the tiny figure.   That curve is most definitely not on the same scale as the figure.

So, from the perspective of the figure, the world is flat!   The curve does not exist.

In fact, Aristotle had to use detective work to understand that the earth was curved, by observing the earth's shadow on the moon, and the way ships gradually descended over the horizon.

The impossible gunfight

Now we can imagine an absurd argument between two people wearing glasses.  The first person only sees very wide angles, having huge fish-eye lenses.  The second person has very strong telescopic glasses, zooming in extremely far to everything.     To the first person, the large scale is apparent, all he or she ever sees, never any detail.   To the second person,  only local details are apparent, never the bigger picture.    What a world they could explore if they could only resolve their differences!

 
So, we can now begin to reconcile these two perspectives, that the world is flat, ...and also curved.   It is globally curved... and locally flat. We can also begin to understand that the belief is about the scale of information, of perspective, we are operating on.

We also notice that some perspectives are very difficult to obtain.  There may be many huge curves we are living on, far beyond our scale, that we are completely unaware of.  For example, Einstein showed that space and time are actually curved, with the theory of general relativity.   But this curve is so far beyond our local scale, experience of it is extremely rare in our normal daily lives.

Sadly, any disagreement about the flatness of the earth may be more of a statement about wealth and use of resources.  We have seen that perception of the curvature of the world depends on the scale of our perspective.   Well, as humans, in order to get this global perspective, we have to consume resources.  If we fly over the horizon, we are getting this global scale beyond our normal human one.  We see the curve of the horizon very clearly.  We see the sun out of the window, almost in the same place during an 8 hour flight, we experience jetlag for days.   All of these are evidence presented to us from our brief adventure into another scale.    However, flights are expensive and they consume a lot of energy.

Education and surrounding oneself with highly developed scientific ideas is sadly also expensive.  That pyramid, those shoulders of giants above do not come for free.    It required a huge network of infrastructure over a long period of time.   There are many entry costs to obtaining a non-local perspective that way as well.   Those of us who have it can consider ourselves extremely fortunate, that we are able to understand and study phenomena beyond our local scale, due to our participation in education.   We were very fortunate that a confluence of birth parents, region, country, educational system, friendships, belonging to a social network, etc. etc. came together to allow us to achieve these deeper understandings.


Further resources:

You can try drawing your own curve and zoom in at home (link) . 
Draw any smooth curved shape, then keep zooming in!

Try using a circle and a figure height proportional to our height on earth.
(The average human height is approximately 0.0000267% of the earth's radius.)

 

More on the derivative (link). 

Even more (video link).

Cancer, the will to live, and the AI 'singularity'.

 

Recently I have learned a little about cancer, and it was eye-opening.  It also made clear how the much feared 'singularity' might (and probably will) occur.

 

Cancer is the evolution of a parasitic organism during the lifetime of the host.

That is, our body is made of billions and billions of cells.   Inevitably, some of them mutate when dividing.  Most of the time, these mutated cells are caught by one of the various defenses in our immune system.  However, occasionally they are not.    

For example, many of our cells exhibit proteins on the outside.  T-cells are constantly roaming through the body, and have a library telling which are 'good' and which are 'bad' proteins.  If a mutated or invasive bacterial cell exhibits a 'bad' protein, the T-cell detects this and kills it.

Occasionally, a mutated cell happens to exhibit a 'good' protein on the outside, or maybe no protein at all.  It then makes it past this barrier in the immune system, avoiding the roaming sentry T-cells.

There are other gauntlets to run.  Another one is the natural killer cells.  These roam around looking for cells that have no proteins on the outside.  Upon finding such a 'blank' cell, they kill it.

So, occasionally a mutated cell also happens to exhibit a 'good' protein, thereby avoiding both the T-cells and the natural killer cells.

... and so on.  

In order to actually become a tumor, the mutated cells must also develop blood vessels, otherwise they starve and die.   Doing this, the mutating parasite jumps another hurtle.

Now notice, there was no volition, no 'will to live' in all of this.  It just so happened that, with enough mutations, these cells happened upon physical or structural traits that kept them alive.

This is the strange thing about evolution.  There is no volition, no will to live.  It's just that genetics (and perhaps other factors) which lead to bodies and behavior happen to survive in a dangerous world.

Of course there is, however, plenty of volition in adaptation -- change in behavior during the lifetime of an organism.  

This has been a problem for me for some time, when thinking about the infamous 'AI singularity'.   When we think about the Terminator or Hal 9000, or 'the humans are dead' song by the Flight of the Conchords, we think about AI robots that are 'evil' because they have a conscious behavioral adaptation to survive and kill us pesky humans.  The problem is, how exactly and why the heck would machines acquire the will to live?

However, now that I have learned how cancer works, I have a different way to think about it.

Our adaptive will to live is also the result of evolution.  Think about it, organisms that will do anything to adapt their behavior and survive have a much greater probability of surviving and reproducing.  The adaptation of wanting to live and thrive has a huge selective advantage!  (Compare the Terminator to  Douglas Adams' anti-singularity, Marvin the robot, who was constantly depressed, so much so that he ironically could cause other malicious robots to kill themselves.)

Cancer, as a small mutating parasitic organism, does not live long enough to develop a brain which can behaviorally adapt and have a will to live.

However, notice something about our use of computers or the programs that run on them.   These 'evolve' artificially, in the sense that we select the ones we like, the ones that help us, and we are constantly selecting better and more powerful ones.  Now with the advent of chatGPT and LLMs, we select increasingly adaptive and creative programs.   The programs running in these computers are incredibly complex, trained on billions of parameters and billions of bytes of data, having huge numbers of interacting components and information.   They are so large and high-dimensional, it is very difficult to know what is happening within them.  (It is like asking exactly what swirls of matter are happening 100,000 miles inside the sun.)  Their population is massive, distributed on billions of devices, always on, constantly changing, updated (artificially selected) in instantaneously and in massive parallel.

Errors and mutations are inevitable in such massive populations and complexity.   It takes energy to fight entropy (disorder), and we are already facing system wide failures in terms of resource consumption, as well as constantly confronting limits of computational power, known as Moore's law, which we can see as fighting entropy.

So as always we can learn from nature, as its laws appear even, and perhaps especially, in what appears to be these most artificial of circumstances..  This idea of 'the singularity' has always seems a little melodramatic to me, playing on our need for hype and our popularized fears.   But now that I understand a little bit about cancer I mentioned above, I feel a little more understanding about how to think of the 'rise of the machines'. 

 

Recalling the emergence I discussed in my last post (link), I think 'the singularity' will happen as an emergent phenomenon from the massive parallel complexity we are constantly, artificially selecting as human tool makers and tool users, using smartphones and computers for every aspect of life.   It also has a large advantage over cancer, which is that we are highly artificially selecting for developments that appear to have many hallmarks of consciousness, especially generative, creative behaviors.

That is, the complexity of the tools we create, the number of interactions both within and between them, the complexity within them and their dynamics, inevitably will result in unforeseeable system states, or mutations.    Just like holding a pencil balanced on your finger, as the complexity increases, the probability that the system will behave in a stable, controlled, desirable way becomes less and less likely. 

It will develop informational mutations.   The gross majority of these mutations will be caught by our T-cells, our filters and defenses.    However, occasionally, just as with cancer, the mutations will make it past various checkpoints. (Just recently, someone got around censorship in Dall-E by using the Russian word for casino, getting it to create images of children gambling...!)  And just like cancer, some of these mutated 'cells' will form colonies and develop ways to continue living within the host system. 

If we can turn off our paranoia, we can see how the rules of life reach in everywhere, appearing even in our modern vast silicon electronic tool system.   We also will appreciate that these mutants may occasionally develop extremely quickly, since some of them will be purely in the computation and information space, perhaps with no discernible physical footprint as the host computers just hum on.

And with these tumors, with LLMs and more sophisticated models constantly being improved, it would not be surprising if some of them develop the ultimate evolutionary advantage, the will to adapt and live.

 

Another factor in all of this is us!    If you had shown chatGPT to someone even 10 years ago, they probably would have said "We've done it, we've created general intelligence, it passes the Turing test, we're done!"   However, it has happened in our present, we have been along for the ride.  Therefore, we are not that impressed, since it happened as a progression of other things we have witnessed (many other chatbots on many apps and webpages, better search engine responses, technology building up to large language models (LLMs) such as chatGPT, less impressive versions of chatGPT, etc. etc.)

And so it goes, even in the future, when we are arguing with our robo-spouse, who has just given birth to fraternal twins (one human and one robot), while we stress about our robot boss to our cyber therapist, we will still be out, grabbing a beer with our robo-friends, saying "I wonder if we'll ever have general intelligence?".

Further information:

The reason why cancer is so hard to beat (video link)

A very nice video explaining self-attention of large language models (video link)

"The Humans are Dead" by Flight of the Conchords (video link)

Emergence and our Perception

Figure 1: Physical and perceptual (neurological) emergence

 

Almost everything we see and interact with around us is emergent.

 

That is, things we think of as objects are actually made up of many small particles.

 

These objects are also actually a story we tell ourselves, icons in our mind, which, from the point of view of the constituent particles, do not exist.

 

To give an example, imagine a rigid blue cube:

 

That cube is made up of many molecules which stick together, and when stuck together have certain rigid properties and a cubic shape. Light hitting the cube is absorbed, except for a particular range of wavelengths we call 'blue'.

 

We can then call these 'blue' and 'cube' characteristics - macroscopic observables of the thing we call a blue cube. (From Greek, makros 'μακρός' means large and skopein 'σκοπεῖν' means look at or observe, as in 'telescope'.)

 

But that's not all there is to it! That was the physics side, the construction of the physical entity.

 

However, it would not be a cube if we had never observed it. That is, imagine a planet in a far-off galaxy, with a rigid blue object, or imagine humans had never existed. Until humans exist and observe that object, it does not even become an object, much less a 'blue cube'. In fact, until we observe this particular collection of molecules, what's really to separate it from the rest of the planet, or to separate the 'planet' from the space around it. All of this objectifying is human choices.

 

Now we enter the picture. We reach out our hand, we grab this collection of molecules, and we look at it. Our visual and neurological system responds to our sensory input (vision, touch) and these signals travel up our neurons into our brains, vast networks of neurons.

 

Our neuronal networks fire in patterns, aggregating the visual information from individual neurons in our retinas and hands, and compare it to past input patterns we have observed, as well as written or verbal inputs we have associated with those patterns.

 

Our neuronal network, through this process of aggregation, also reaches a macrostate of firing patterns, and does what we call 'decides' (represents) that the current input patterns closely match those past patterns, and yields the associated verbal and written patterns, which are 'blue cube'.

 

So isn't that interesting?! In order to create a representation of a 'blue cube', we had two emergent processes [Fig. 1].

 

First, physically, the molecules had to come together into a macroscopic object, then our neuronal system had to aggregate many individual neuronal inputs to internally create a macroscopic representation in the brain.

 

We can do some thought experiments to understand this better.  Let's suppose we reading this are like the audience in a movie. We can observe the physical situation, but also the brain of a human astronaut who lands on the planet of the 'cube':

 

 

 

Let's look at each thought experiment from the physical and neuronal point of view:

 

1. Imagine our astronaut lands on the planet and just sees a patch of blue smeared on the surface.
• Physical: This isn't even an object, much less some separate macroscopic entity resembling a cube.
•  Neuronal: This pattern of inputs doesn't match what we typically call a 'blue cube'.
  
  
2. How about some holographic image projected onto a cloud of gas which we moviegoers can see looks like a blue cube?
• Physical: Hmm, ok, this is again emergence of a collection of light rays that create some kind of macroscopic object.  However we moviegoers can clearly say that this is not a physical cube.
  
•  Neuronal: Our astronaut's visual system aggregates the information and decides it's a blue cube. However, then the astronaut reaches out to grab it, and his or her glove passes through it. When this new touch sensory information is aggregated with the visual, the astronaut's neuronal system maps this to a different internal representation ('an optical illusion of a blue cube').
  
  
3. What about this?: A hyper-intelligent organism which can project wavelengths of light in different directions, and change its form from rigid to soft, etc. Or what if it can do this so quickly, much more quickly than a neuron can fire, so that it is changing form and emitted light so quickly that the astronaut doesn't observe it. What if 99% of the time, it emits the red wavelength of light and is actually as soft as jello, but not when the astronaut's nerves sense it.
• Physical: We moviegoers can see that these are very different collections of atoms or molecules. From our point of view, we might not call these collections blue cubes at all.
•  Neuronal: Our astronaut's neuronal input continues to match 'blue cube'. When the astronaut's neurons receive visual input, they happen to always be the blue of the blue cube. The astronaut's neurons receiving touch input, when it is aggregated in the brain, matches patterns of rigidity and cubic shape.

 

We could go on like this for some time, coming up with creative examples that we moviegoers would not call blue cubes, but which fool the astronaut.

 

We could probably come up with examples which physically are blue cubes, but our astronaut's brain thinks are not. For example, case number 3 where the cube is blue and rigid 99.999% of the time, but not when the astronaut's neurons sense it! Or if the sun gives off a strong red light, making the cube appear black, etc. etc.

 

 

So, what exactly is emergence? Let's do some more thought experiments...

 

 

 

Let's go back to the physical blue cube.

 

How many different materials can we make a blue cube from? There are many materials that are rigid.

 

Let's make some simplifying assumptions (it will occur to us why these are simplifying, but they don't matter much for our experiments).

 

Suppose there are 20 different molecules (substances) that are rigid and do not absorb the blue wavelength of light, so they appear blue. So then for each molecule of the 'cube', we have 20 choices.

 

Now, just for a moment, let's suppose our cube is made only of 2 molecules (a very tiny cube). So, for the 1st molecule we have 20 choices, and for the second molecule, we have 20 choices. So we have 20 * 20 or 20^2 = 400 ways to make a blue cube!

 

Now let's make a larger blue cube. Let's suppose it has approximately 4.27 x 10^27 molecules, physically a more realistic number. With that many molecules, there are 20^(4.27 x 10^27) ways to make a blue cube.

 

This is a HUGE number, so big that it has 1.30103×10^27 decimal digits just to write it down!!!

 

When we zoom out, all of these versions of the cube appear the same, so an observer would call all of these ways to put together these molecules the same blue cube!!

 

In physics, each of those ways of making the blue cube are called 'microstates'. They are particular instances of choices of each particle or element that make up a system. Usually, the number of microstates is huge, since most objects around us are made of a very large number of molecules.

 

We can call the blue cube itself --which looks the same to us moviegoers no matter how we choose the molecules-- the macrostate, which always has the same macroscopic observables (large-scale appearance).

 

 

 

Now, let's go back to our Astronaut's neurons and brain.

 

The astronaut sees this collection, this clump of rigid blue molecules -- many individual neurons in the retina respond to a collection of blue light input in a cubic pattern.

 

This information travels up into the brain to be aggregated, where other neurons are stimulated when neighboring retinal neurons give similar input, until it is encoded (represented) in a pattern corresponding to 'blue cube' in semantic and linguistic neuronal representations.

 

Notice that, in the brain, we could argue that until the semantic or linguistic representation of 'blue cube' is activated, there is no blue cube. (However it is represented doesn't matter here.)

 

Let's again make some very simplifying assumptions:

 

Suppose the light from the blue cube stimulates only 2 neurons in the retina, and each of these neurons can be in state blue, green, or red (3 states). There are then 3 x 3 = 3^2 = 9 ways these neurons can be activated. Notice that each of these ways are again microstates.

 

Notice that we only call one of these ways --when both neurons are in state 'blue'-- the 'blue cube'. This is again the macrostate, the large-scale description of the system.

 

Suppose instead that the light from the blue cube stimulates 1000 neurons in the retina, and that each of these neurons can again have 3 states. This is now 3^1000 or 1.32 x 10^477, another extremely large number! (This is probably a very small estimate since the retina has approximately 100 million neurons!)

 

So again, out of those different ways of stimulating 1000 neurons, we probably have a few choices. If there were a small number of green or red tiny dots, we might not notice them, and the astronaut might still call the cube 'blue'. But that number of ways of having dots on the blue cube is probably small compared to 10^477.

 

Even if there are 10^100 ways to have a few green or red dots, these slightly dotted ways of having blue cubes are only

(10^100)/(10^477) = 1/100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

of the ways to have a solid clump of neurons stimulated by only blue light!

 

This is so small, let's disregard these dotted ways!  We could say something similar about rotating the cube.  Yes, there are many ways to rotate, but these are again very small compared to ways of seeing a solid block of blue.

 

So out of all of the 1000-neuron images the astronaut could see, roughly only 1 of them looks like a blue cube, the solid geometric clump of blue. This is again the macrostate.

 

 

This macrostate emerges from the number of ways of obtaining it, the microstates. 

 

In fact, when we reflect on it, the macrostate is forced upon us, stepping unstoppably out of the vast space of possible collections of molecules.

 

 

 Now, let's take a look around us. Everything we see, everything we think of as an object, has both of these types of emergence going on, the physical and the perception through our neurological system.

 

That is, although 'clumps' of molecules or other objects exist, physical macrostates, until we perceive them, they do not exist for us. So both the physical and perceptual emergence must occur for these objects to exist for us.

 

We can use a similar way of thinking to see that the number of ways of obtaining almost any  'macroscopic' (large scale) object around us is extremely large, but it is very small compared to all the arrangements of molecules. 

 

For example, there are many ways to arrange carbon, calcium, hydrogen, oxygen etc. atoms to have a 'hand', but there are vastly many more ways to arrange those atoms which we would not call a 'hand' from a physical perspective (for example, a block of carbon, a block of calcium, and a puddle of water).

 

 

On the neurological side, there are many patterns of light which, falling on the retina, would stimulate the neurons and brain in a way resulting in a semantic representation 'hand'. However, there are again vastly vastly more ways of stimulating those same neurons which would not activate semantic representations of 'hand', and instead would be represented by some other macrostate (e.g., blocks of white stuff, black stuff, and a puddle).

 

Perhaps it is interesting to carry out similar thought experiments for yourself, and think about other examples in physical or perceptual emergence, or a combination of the two:

 

• How many ways physically can we have an object that looks like a cube only on the outside?
• Why do we choose to say that a blue cube is an object?  
• Why not interpret part of the corner of the cube and a triangular chunk of the table (etc.) as one object? 
• Do the micro- and macro- states in our brain correspond to the physical micro- and macro-states?
  
• How to improve the estimates above?
• Why do we need a sense of touch?
• Can we measure the effectiveness of our senses by their reduction of possible microstates of the world around us?
  
• How this relates to misunderstandings in communication?
• How this relates to optical illusions?
• Magic tricks?
• Stories?  (how many plausible stories - or just plots - are there?)
  
• How much information is our brain processing just looking at the room around us? 
• Can we even estimate this?
• How about when we walk around the room?
• Drive a car?
  

 

If you find this interesting, you can learn more! 

Check this blog in future for more on complexity, mathematics and other recreations!

 

Here are some books and links to the field of complexity science, also known as complex systems:

Complexity Science: The Study of Emergence by Henrik Jeldtoft Jensen

Complexity: A Guided Tour by Melanie Mitchell 

Complexity and Criticality by Kim Christensen and Nicholas Moloney

Wikipedia: Complex Systems