- Oct 10, 2025
The Progress Engine: Turning Problems into Compounding Progress
- Kostakis Bouzoukas
- 0 comments
A Contaminated Petri Dish That Changed Everything
In 1928 the Scottish bacteriologist Alexander Fleming returned from holiday to find that one of his Petri dishes had been contaminated. Instead of discarding the plate, he noticed that the Staphylococcus colonies near the invading mold had dissolved, leaving a clear halo[1]. The mold produced an antibacterial substance that killed common pathogens[1]. Fleming’s decision to investigate an unexpected failure rather than throw it away led to the discovery of penicillin—a drug that has saved millions of lives. What made this happen? He treated the problem as data and followed a sequence of conjecture, test, correction and iteration. This essay argues that progress is not the result of positive thinking or blind optimism; it is the product of deliberately transforming problems into better explanations through repeated loops.
Why Avoidance Slows Innovation
Philosopher Karl Popper wrote that all life is problem solving[2]. In his framework of critical rationalism, knowledge grows through cycles of conjecture and refutation: we propose bold ideas and actively look for errors[3]. Because any explanation can be improved, progress is open‑ended. Yet history shows that institutions often resist this engine.
Science and Paradigms. In The Structure of Scientific Revolutions, Thomas Kuhn observed that scientific communities operate within paradigms—shared assumptions and methods. During periods of normal science anomalies are treated as nuisances; researchers refine existing puzzles rather than question foundational ideas[4]. When enough anomalies accumulate, they trigger a crisis and eventually a paradigm shift[4]. If anomalies are ignored, disciplines can fall into “eternal normalcy”: incremental work continues but transformational progress stalls.
Risk Aversion and Bureaucracy. Contemporary research culture suffers from a similar malaise. A 2024 City Journal article notes that despite more journals, papers and scientists than ever before, major breakthroughs are rarer because researchers are bound by risk‑averse funding, bureaucratic procedures and hyper‑specialisation[5]. Scientists spend more time writing grant proposals and chasing citations than conducting risky experiments[5]. Metrics such as citation counts reward safe, incremental research and discourage bold conjectures[6]. When quantity is prized over insight, the engine slows.
Unsovable Problems and Hubris. Some challenges are not just hard but fundamentally unsolvable. A 2025 Quanta Magazine article recounts that mathematics contains undecidable statements—propositions that cannot be proven or disproven[7]. Kurt Gödel’s incompleteness theorems and Alan Turing’s work on computability show that there is no algorithm that can solve every Diophantine equation[8]. Recognising that certain problems lie beyond our current knowledge helps us prioritise and avoid futile searches.
Moral Limits. Progress is not purely technical. Philosopher Nick Bostrom warns that some technological solutions can create existential risks, from misaligned artificial intelligence to engineered pandemics[9]. Treating every challenge as solvable without considering downstream consequences can be dangerous. The progress engine must therefore include ethical reflection alongside error correction.
Together these arguments show that ignoring problems—whether by suppressing anomalies, rewarding busywork or assuming everything is solvable—slows innovation. A culture that treats problems as fuel rather than roadblocks is required for open‑ended progress.
The Progress Engine: A Five‑Stage Cycle
Foundations
The progress engine rests on three epistemic pillars. Popper argues that knowledge grows through trial and error[3]. Physicist David Deutsch extends this by emphasising better explanations: progress happens when we generate ideas that not only solve existing problems but survive critical testing[10]. He contends that, because explanations can always improve, progress is potentially unbounded[11]. Physicist Richard Feynman reminds us that theories must be tested against observation; if a prediction disagrees with experiment, the theory is wrong[12]. These principles converge in a simple mechanism: propose conjectures, criticise them through tests and learn from the resulting errors.
The Five Stages
The progress engine turns problems into progress through five distinct, iterative stages. Each stage links directly to our anchor story—Fleming’s discovery—and applies equally to business, organisational learning and personal growth.
Problem Recognition. Progress begins when we notice a mismatch between expectations and reality. Fleming noticed the clear halo around the mold instead of discarding the contaminated dish[1]. In business, executives might observe that growth has plateaued. At a personal level, a writer might feel stuck halfway through a draft. The key is to record the anomaly rather than rationalising it away.
Conjecture. We then generate hypotheses about what might explain or fix the problem. Fleming hypothesised that the mold secreted an antibacterial substance. Conjectures should be bold enough to expose themselves to criticism and varied enough to avoid tunnel vision.
Test. We design experiments, prototypes or feedback mechanisms that could falsify our conjectures. Fleming tested his hypothesis by applying the mold extract to various bacteria[1]. In product development this might be an A/B test; in personal life, a deliberate habit change.
Error Correction. We compare the results of our tests to our predictions. Any discrepancy is a gift: it tells us where our explanation is wrong. Toyota’s Andon cord system encourages workers to pull a cord when they spot any abnormality; this stops the assembly line and triggers a “five why” root cause investigation[13]. Google’s blameless postmortems document incidents, identify multiple causes and generate preventative actions without scapegoating[14]. Error correction is most effective when it focuses on systemic factors rather than individual blame.
Next Problem. Every correction surfaces new problems. Fleming’s discovery raised questions about mass production, distribution and antibiotic resistance. Each answer leads to better questions, and the cycle continues. There is no final solution; progress is an infinite game.
Measuring Learning: Error‑Correction Velocity
To ensure the engine does not become a platitude, we need a way to measure its pace. Error‑Correction Velocity (ECV) quantifies how quickly and deeply we move through the cycle. It combines three elements:
Number of cycles completed in a given period (e.g., week or quarter). More loops mean more learning opportunities.
Correction depth, rated on a scale (e.g., 1 for superficial fixes; 5 for systemic changes). This captures the quality of corrections.
Lead time, the average duration from recognising a problem to implementing a correction. Shorter lead times indicate faster learning.
The formula is:
ECV = (Number of cycles × Average correction depth) / Average lead time
Like Mean Time to Recovery (MTTR) in reliability engineering, ECV provides a speedometer for progress. For example, a team completing 10 cycles per month with an average correction depth of 3 and lead time of 7 days (0.23 months) would have an ECV of about 130. Improving their ritual—documenting problems, conducting blameless reviews and quickly testing conjectures—might allow them to complete 15 cycles with a depth of 4 and lead time of 4 days (~0.13 months), raising ECV to about 462. The absolute numbers matter less than the trend: rising ECV indicates compounding learning.
From Lab Bench to Boardroom: Cases of the Progress Engine
Science: Penicillin’s Birth
Our anchor case illustrates the engine in action. Problem recognition occurred when Fleming noticed the bacteria‑free halo[1]. He formed a conjecture that the mold produced an antibacterial agent. He tested it by applying the mold extract to various microbes, demonstrating that it killed them[1]. Error correction involved refining the extraction process and working with Howard Florey and Ernst Chain to purify and stabilise the compound for mass production. Each loop raised new problems, including scaling up production and addressing bacterial resistance. These loops paved the way for the antibiotic era.
Business: From DVDs to Streaming
In the mid‑2000s, Netflix’s DVD‑by‑mail service began to plateau. Executives observed broadband usage surging and recognised that physical discs would soon become obsolete—a clear problem[15]. Their conjecture was that customers would watch movies online if streaming were reliable and affordable. They tested this by launching a streaming service in 2007 despite technical and licensing hurdles. Initial buffering issues and a limited catalogue highlighted deficiencies. Through error correction, Netflix invested in content delivery networks, negotiated broader licensing deals and developed better compression algorithms[15]. Each improvement uncovered new problems, from rising content costs to the need for original programming. By cycling through these stages, Netflix reinvented itself as a streaming leader.
Organisations: Blameless Learning
Large‑scale systems fail. Google’s Site Reliability Engineering (SRE) teams accept this and treat incidents as opportunities for learning. After an outage, they conduct a blameless postmortem: engineers document what happened, its impact and the contributing causes[14]. The process emphasises systemic factors rather than individual error[16]. Corrective actions address root causes and often uncover deeper issues such as inadequate monitoring or unclear responsibilities. Each postmortem feeds new problems into the next iteration. Over time, MTTR decreases, reliability increases and engineers are more willing to surface anomalies.
Personal: Writing as Iteration
Creativity thrives on messy iteration. Author Anne Lamott notes that all good writers produce “shitty first drafts”[17]. The first draft is intentionally rough; its purpose is to get ideas onto the page. Subsequent drafts refine structure and clarity, and later drafts polish details[18]. This mirrors the progress engine: recognise your discomfort with a scene, hypothesise a better version, test it by rewriting, learn from feedback and tackle the next problem. By embracing flaws rather than fearing them, writers transform problems into prose.
When Engines Stall: Failure Modes and Limits
Even a well‑designed engine can stall. Here are the common pitfalls:
Ignoring anomalies. During normal science, anomalies are suppressed to protect the paradigm[4]. Businesses may ignore early signs of market saturation. Individuals may dismiss discomfort as minor instead of investigating it. The engine fails when problems are not harvested.
Timid conjectures. Risk‑averse cultures prioritise incremental work. Scientists may avoid bold hypotheses because citations and funding reward safe projects[6]. Without daring conjectures there is little to test.
Lack of critical testing. Conjectures validated by confirmatory experiments waste time. Feynman warns that theories must be tested rigorously and abandoned when they disagree with observations[12].
Blame and shame. Cultures that punish errors discourage problem recognition. Google’s blameless postmortems exist precisely because blaming individuals prevents learning[16].
Static solutions. Treating a fix as final stops progress. After each correction, new questions should be generated. The engine stalls when teams move on after a single fix.
Beyond these practical failure modes lie intrinsic limits. Some problems are unsolvable with current knowledge; undecidable statements in mathematics show that there are questions for which no algorithm can provide an answer[7][8]. Ethical constraints also limit what should be attempted. Bostrom’s work on existential risks reminds us that certain solutions can create new dangers[9]. Recognising these limits helps direct attention toward problems that are tractable and beneficial.
Tools for Putting the Engine to Work
The Progress Engine Canvas
The canvas is a structured worksheet that guides you through each stage of the engine:
Define the problem. Describe the mismatch, failure or constraint in clear terms. Avoid vague diagnoses; “customers abandon the checkout at step 3” is more actionable than “checkout is bad.”
Generate conjectures. List multiple possible explanations or solutions. Encourage diverse perspectives and avoid premature consensus. Draw on existing knowledge but allow creative leaps.
Design tests. For each conjecture, design an experiment or feedback loop that could refute it. Establish success and failure criteria before starting.
Execute tests and collect errors. Run your tests quickly. Gather quantitative data (metrics) and qualitative observations (surprises, emotions). Document everything; as Google’s SRE practice emphasises, incidents recur unless formally captured and analysed[14].
Correct and learn. Analyse discrepancies between predictions and results. Ask “Why?” multiple times to reach root causes. Develop corrective actions that address systemic factors rather than symptoms. Record lessons learned.
Capture new problems. Each correction reveals additional mismatches. Explicitly list these and feed them into the next cycle. Treat improvements as starting points, not endpoints.
Building an ECV Dashboard
To monitor your progress, create a dashboard or spreadsheet with the following columns:
Cycle count: number of engine loops completed in the period.
Average correction depth: your rating of each correction’s depth.
Lead time: average days between problem recognition and implemented correction.
New problems identified: count of fresh problems unearthed per cycle. A low count may indicate you are closing loops without expanding the frontier.
Qualitative notes: reflections on surprises, emotions and ethical concerns.
Plot these metrics over time. A rising ECV indicates faster learning and deeper corrections. A plateau suggests suppressed problems, superficial fixes or increasing lead times. The dashboard is not for judgement but for reflection and course correction.
The 15‑Minute Progress Review
Busy schedules often crowd out reflection. A short weekly ritual keeps the engine humming:
Set the scene (2 minutes). In a quiet space, recall the past week and choose one significant problem or frustration. Write it down.
Brainstorm conjectures (3 minutes). List at least three possible explanations or solutions. Include a wild idea to encourage creativity.
Design a mini‑test (3 minutes). Pick one conjecture and design a small experiment to run in the coming week. Define how you will know it failed.
Reflect on errors (5 minutes). Review last week’s test. What surprised you? Identify root causes and consider how to correct them. Note any new problems discovered.
Commit and schedule (2 minutes). Decide on one action for the coming week. Schedule it. A commitment increases follow‑through.
Repeating this ritual builds a habit of problem harvesting. Over time your ECV will rise as you complete more cycles, correct more deeply and shorten lead times. More importantly, the ritual nurtures a mindset that treats problems as invitations to learn rather than nuisances to avoid.
Conclusion: Compounding Hope
Progress is not a linear march or a feel‑good mantra. It is a mechanical process that turns friction into insight. Fleming’s contaminated Petri dish became a miracle because he noticed the anomaly, formed a bold conjecture, tested it, learned from errors and then sought the next problem[1]. Netflix reinvented itself by recognising a plateau, conjecturing a streaming future, testing it and iterating through many failures[15]. Toyota’s Andon cord and Google’s blameless postmortems institutionalise this cycle[13][14]. Writers like Anne Lamott embrace messy drafts as the raw material for polished prose[18].
The Progress Engine unites these stories. It channels Popper’s trial‑and‑error, Deutsch’s better explanations and Feynman’s demand that theories face experiments[3][10][12]. It warns against stagnation when anomalies are ignored[4], risk aversion is rewarded[5] or unsolvable problems are pursued without regard to limits[7]. It insists on ethical vigilance to avoid progress that harms more than it helps[9].
In practical terms, the engine invites you to pull your Andon cord. Notice the next small frustration in your work or life. Describe it. Invent a daring explanation. Design a test to refute it. When it fails, smile—the error is your teacher. Correct it and list the next problem. Track your cycles, depth and lead time. Over months, your Error‑Correction Velocity will climb, and with it your capacity to innovate. Problems are not obstacles; they are the raw material of hope. By embracing them, we join an unbroken lineage of progress that began with a contaminated Petri dish and continues with every question we dare to ask.
[1] Penicillin | Discovery, History, Uses, Types, Side Effects, & Facts | Britannica
https://www.britannica.com/science/penicillin
[2] Review of Popper’s Essay, “All Life is Problem Solving” in a Collection of Essays Under the Same Title » Imphal Review of Arts and Politics
[3] Critical rationalism - the epistemology of trial and error | DIIS
https://www.diis.dk/en/research/critical-rationalism-the-epistemology-of-trial-and-error
[4] Thomas Kuhn (Stanford Encyclopedia of Philosophy)
https://plato.stanford.edu/entries/thomas-kuhn/
[5] [6] How to Accelerate Science
https://www.city-journal.org/article/how-to-accelerate-science
[7] [8] New Proofs Probe the Limits of Mathematical Truth | Quanta Magazine
https://www.quantamagazine.org/new-proofs-probe-the-limits-of-mathematical-truth-20250203/
[9] Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards
https://nickbostrom.com/existential/risks.pdf
[10] [11] The Beginning of Infinity | Summary, Quotes, FAQ, Audio
https://sobrief.com/books/the-beginning-of-infinity
[12] Richard Feynman Quotes about Wrong - Lib Quotes
https://libquotes.com/richard-feynman/quotes/wrong
[13] The Systems Thinker – Pulling the Andon Cord: Toyota Responds to Challenge and Change - The Systems Thinker
https://thesystemsthinker.com/pulling-the-andon-cord-toyota-responds-to-challenge-and-change/
[14] [16] Google SRE - Blameless Postmortem for System Resilience
https://sre.google/sre-book/postmortem-culture/
[15] Case Study: Netflix’s Transition from DVD Rental to Streaming - Oxford Executive Institute
https://oxfordexecutive.co.uk/case-study-netflixs-transition-from-dvd-rental-to-streaming/
[17] [18] Anne Lamott — Bird by Bird: Some Instructions on Writing and Life – David Labaree on Schooling, History, and Writing
https://davidlabaree.com/2021/11/11/anne-lamott-bird-by-bird-some-instructions-on-writing-and-life/