The Art of Doing Science: How to Think Better

You probably think great ideas are a stroke of luck, but Richard Hamming spent a lifetime proving they are actually a result of a specific, trainable style of thinking. Today, we’re breaking down his masterwork, The Art of Doing Science and Engineering, to show you how to move from just admiring "mighty deeds" to performing them yourself. We’ll explore why Hamming’s 1986 "You and Your Research" sermon is the ultimate blueprint for avoiding the common mistake of doing mediocre work. By the end of this, you’ll have a framework for systematic learning that draws on Shannon’s information theory and Grace Hopper’s programming breakthroughs. Ready to see why your spectacular failures might actually be your best teaching tools? Let’s dive in.

To truly master the art of doing science and engineering, you have to start by reconsidering your relationship with failure and the very structure of your problem-solving process. We often view an engineering failure as a catastrophe—a bridge collapsing or a software system crashing—but if you shift your perspective, these moments become the most information-dense data points in your career. There is a fundamental principle in engineering that suggests we learn far more from things that go wrong than from things that go right. When a design works, it only tells you that the specific parameters you chose were sufficient; it doesn't tell you how close you were to the edge of disaster or where the true limits of the system lie. However, when something breaks, the universe is giving you a very specific, non-negotiable piece of feedback about the laws of physics or the logic of your system. You are forced to look at the "unwritten laws" that govern how professionals behave and how systems interact. This involves a level of intellectual honesty that is rare. You cannot simply patch a hole and move on; you have to perform a forensic analysis of the underlying assumptions. This mirrors the way we should approach our own learning. If you solve a math problem correctly on the first try, you’ve confirmed what you already know. If you struggle, fail, and eventually find the error in your reasoning, you have expanded the boundaries of your competence. This is why systematic learning requires a "design thinking" approach where you treat your own brain as a prototype. You are constantly iterating, testing hypotheses about how you learn best, and being willing to scrap a mental model that no longer serves the complexity of the problems you are facing. You have to be willing to be a "streetfighter" with your logic—using approximations, back-of-the-envelope calculations, and gut-check heuristics to see if a result even makes sense before you dive into the granular details. This foundation is built on the realization that engineering and science are not just about following a recipe; they are about managing uncertainty through rigorous, creative discipline. You are building a mental scaffolding that allows you to handle increasingly abstract concepts without losing your footing.

One of the greatest barriers to high-level engineering and scientific work is a "math phobia" that stems from how we were taught in school—focusing on rote memorization rather than the underlying "mindware" that makes math a powerful tool for decision-making. To think like a master, you need to treat mathematics not as a series of hurdles to jump over, but as a language for describing reality and identifying errors in your own intuition. Think about the concept of regression to the mean or the way we often misunderstand basic probability in our daily lives. We see a "hot streak" and assume it will continue, or we see a failure and assume a permanent decline, forgetting that extreme performances are naturally followed by more average ones. By integrating these statistical mental models into your daily routine, you stop being a victim of "wrongness" and start seeing the hidden patterns in data. This is where "street-fighting mathematics" comes into play. It’s the ability to look at a complex differential equation and, instead of getting bogged down in the calculus, use dimensional analysis or extreme cases to guess the form of the answer. Does the result go to zero when the input goes to infinity? If not, your equation is probably wrong. This kind of "sanity checking" is what separates the experts from the amateurs. It’s about having a toolkit of "super-thinking" models—like understanding the difference between a linear and an exponential process—so you don't get blindsided by how quickly a small problem can scale into a systemic collapse. You also have to embrace the "math for dummies" philosophy in the sense of breaking down every complex operation into its most basic, foundational parts. If you don't understand the arithmetic of a problem, you have no business trying to solve its calculus. This rigorous attention to the basics ensures that your more advanced conclusions aren't built on a foundation of sand. It’s about developing a "mathematical degree" of certainty in your logic, where you can trace every step of your reasoning back to first principles. This doesn't mean you never make mistakes; it means you’ve built a system that makes those mistakes visible to you before they reach the final design.

There is a persistent myth that creativity is a "eureka" moment that strikes like lightning, but the reality of top-tier science and engineering is that creativity is a systematic process of iteration and "creative confidence." It’s about believing in your ability to create change and having a reliable method to get there. This involves moving through distinct phases: observing the world with a "curious people" mindset, framing the right questions, and then rapidly prototyping solutions. Most people fail because they try to get the "right" answer on the first attempt, which leads to a fear of starting. Instead, you should adopt the design method of "failing fast" and "failing forward." Every failed prototype is a piece of information that narrows the search space for the eventual solution. You have to be comfortable with the "messy middle" of the creative process where things don't make sense yet. This is where you use "mental models" to bridge the gap between a vague idea and a technical reality. For instance, you might use the "five whys" to get to the root of a design flaw, or apply "lateral thinking" to see if a solution from an entirely different field—like biology—can be applied to a mechanical engineering problem. The goal is to build a "design thinking workbook" for your own life, where you are constantly documenting your observations, your failed attempts, and your "aha" moments. This documentation is crucial because the "eureka factor" often comes from the subconscious mind finally connecting two pieces of information you’ve been dwelling on for weeks. By systematically feeding your brain high-quality "source material" and then giving it the space to iterate, you turn creativity from a mystical event into a predictable output of your working style. You learn to trust that the process will yield a result if you stay disciplined in your observations and honest in your evaluations. This is how you build the confidence to tackle "wicked problems"—those that are ill-defined, shifting, and deeply complex—because you know you have a system that can digest that complexity and turn it into actionable insights.

In the world of high-level performance, we often attribute success to innate talent—the "Mozart effect" or the "born engineer." But a deep dive into the science of success reveals that "bounce" or resilience and deliberate practice are the true drivers of mastery. It’s not about how smart you are; it’s about how you study and how you respond to the plateau. When you hit a wall in your learning, do you assume you've reached your "natural limit," or do you analyze your practice methods to see where the "friction" is? Mastery in science and engineering requires a "growth mindset" where every challenge is seen as a way to upgrade your internal "software." This means engaging in practice that is purposefully difficult—pushing yourself just beyond your current capabilities so that your brain is forced to adapt. This is the difference between "studying" for a math degree and "doing" mathematics. One is passive consumption; the other is active reconstruction of ideas. You have to be willing to be "wrong" frequently and publicly, because that is the only way to get the feedback necessary for growth. Think about the most successful athletes or scientists; they aren't the ones who never failed, but the ones who used their failures as "fuel" for their next iteration. They understood that "to engineer is human" means acknowledging our fallibility and building systems—both technical and psychological—to account for it. This involves a level of self-awareness where you recognize your own cognitive biases, like the "confirmation bias" that makes you overlook evidence that your favorite theory is wrong. By training yourself to seek out disconfirming evidence, you align your thinking more closely with reality. This psychological rigor is what allows you to maintain focus over the decades it takes to make a truly significant contribution to your field. You aren't just learning facts; you are learning how to manage your own mind, your own ego, and your own energy so that you can stay in the "game" long enough for your "compounding interest" of knowledge to pay off.

Beyond the technical skills, there is a set of "unwritten laws" that govern how successful engineers and scientists navigate the human element of their work. You can be the most brilliant mathematician in the world, but if you cannot communicate your ideas or work within a team, your impact will be severely limited. Professionalism in these fields is about "engineering the self"—treating your career and your interactions as a system to be optimized. This means understanding the "social science" of the workplace: how to manage your manager, how to lead without authority, and how to build a reputation for reliability. It’s about recognizing that "how we think" is often influenced by the environment we are in. If you are in a culture that punishes failure, you will stop innovating. Therefore, part of your job as a "knowledge creator" is to foster an environment where "intellectual rigor" is valued over "being right." You must learn the "etiquette" of engineering, which isn't about being polite for the sake of it, but about ensuring that communication is clear, concise, and accurate so that errors aren't introduced through misunderstanding. This involves a "design thinking" approach to your own professional development. Are you "designing" your daily schedule to prioritize "deep work"? Are you "prototyping" different ways of presenting your data to see which one leads to the best decision-making by your stakeholders? By applying the same "engineering mindset" to your career that you apply to your technical problems, you create a feedback loop that constantly improves your "output." You start to see your professional life not as a series of random events, but as a "system" that can be analyzed, debugged, and upgraded. This includes being "street-smart" about the organizational politics and the "hidden curriculum" of your industry, ensuring that your technical brilliance isn't sidelined by a lack of "mindware" regarding human behavior.

When you are faced with a massive, multi-faceted problem—the kind that defines a career—you cannot rely on intuition alone; you need a "toolkit" of design methods that allow you to deconstruct complexity. This starts with "sensing" the environment—gathering data not just from books, but from direct observation and "empathy" for the end-user of your work. Whether you are designing a bridge or a piece of software, you have to understand the human context in which it will exist. This is where "101 design methods" come in, providing structured ways to "frame" the problem before you ever start looking for a solution. If you frame the problem incorrectly, no amount of technical brilliance will save you. You have to "map" the system, identifying the "leverage points" where a small change can have a large impact. This is "super-thinking" in action—using models like "first-principles thinking" to strip a problem down to its core physical constraints and then building back up from there. You also need to employ "divergent thinking" to generate a wide range of possibilities, followed by "convergent thinking" to narrow them down based on feasibility and impact. This process is inherently "mathematical" in its logic but "creative" in its execution. You are essentially running a "simulation" in your mind—and through your prototypes—to see how different variables interact. By using "design thinking workbooks," you can track these variables and ensure that you aren't missing a critical connection. This systematic approach prevents the "tunnel vision" that often leads to engineering disasters. You learn to see the "big picture" and the "minute details" simultaneously, moving between the two with the ease of a master. It’s about being "not wrong" by being "comprehensive" in your search for potential points of failure. This level of rigor is what allows for the "eureka factor" to occur, as your brain finally finds the "elegant" solution that satisfies all the complex constraints of the problem.

We’ve all experienced that sudden flash of clarity where a difficult problem suddenly becomes simple. While these "eureka" moments feel accidental, research into "the eureka factor" shows they are the result of specific neural processes that can be encouraged. Insight often happens when the brain is in a state of "relaxed alertness"—when you’ve done the hard, "street-fighting" work of wrestling with the data, but then step back to let your "subconscious mind" take over. This is why many of the greatest scientific breakthroughs happened during walks, in the shower, or just as the scientist was falling asleep. To facilitate this, you have to feed your mind a diverse diet of "source materials." The more "mental models" you have from different fields—psychology, business, social science, and literature—the more "raw material" your brain has to make unexpected connections. This is the "art" of science and engineering. It’s not just about crunching numbers; it’s about "pattern recognition" across disciplines. You are looking for the "mindware" that allows you to see a mechanical problem through the lens of a biological system, or a social problem through the lens of thermodynamics. This "cross-pollination" is what leads to truly innovative thinking. However, you cannot have the "insight" without the "effort." The "eureka factor" is the "reward" for the thousands of hours of "deliberate practice" and "systematic learning" you’ve put in. It’s the moment when your "internal map" of the problem finally aligns with the "territory" of reality. By understanding the "psychology" of these moments, you can structure your work to maximize their frequency—alternating between periods of intense "deep work" and periods of "incubation" where you allow your mind to wander. This is how you "engineer" your own genius, turning the "luck" of insight into a "methodology" for discovery.

Now, how do you take all of these "big ideas" and turn them into a daily practice? The "practical playbook" for the engineering mindset starts with a commitment to "intellectual honesty." Every day, ask yourself: "What do I believe to be true, and what evidence would it take to prove me wrong?" This is the core of "how not to be wrong." Second, adopt a "prototype-first" mentality. Don't wait until you have the "perfect" plan to start. Build a "minimum viable model" of your idea—whether it’s a sketch, a simple script, or a mathematical approximation—and "stress-test" it immediately. Use "street-fighting mathematics" to do quick "sanity checks" on your assumptions. Third, keep a "learning journal" where you document your "failures" with as much detail as your "successes." Analyze the "root cause" of your mistakes—was it a flaw in your "mental model," a lack of "source material," or a "cognitive bias"? Fourth, diversify your "mental toolkit." Read outside your field. If you’re a coder, read about "structural engineering" to understand how physical systems handle stress. If you’re a physicist, read about "design thinking" to understand how to frame human-centric problems. Fifth, practice "deliberate reflection." At the end of every project, perform a "post-mortem" to see what the "unwritten laws" of that specific situation were. What did you learn about the "system" that you didn't know at the start? Sixth, prioritize "foundational mastery." Don't move on to "complex calculus" until your "basic arithmetic" is flawless. Seventh, build "creative confidence" by setting small, achievable goals that push you slightly out of your comfort zone. Eighth, use "mental models" like "first principles" and "second-order thinking" to evaluate every major decision. Don't just think about the immediate effect; think about the "effect of the effect." Finally, remember that "to engineer is human"—embrace your fallibility, but build a "rigorous system" to catch your errors. This is the "art" of doing science and engineering: it’s the constant, disciplined effort to refine your "thinking machine" so that you can produce work that is not only "accurate" but "meaningful" and "impactful."

As we wrap up this exploration of the "art of doing science and engineering," it’s worth taking a moment to reflect on why this level of "intellectual rigor" matters. We live in a world of increasing complexity, where the "systems" we rely on—from our infrastructure to our information networks—are more interconnected than ever. In this environment, "mediocre thinking" is not just a personal failing; it’s a systemic risk. By adopting the "engineering mindset," you are choosing to be a "builder" rather than just a "consumer" of ideas. You are taking responsibility for the "quality" of your own mind and the "impact" of your work. This isn't a destination you reach; it’s a "way of being" in the world. It’s about having the "courage" to be wrong, the "discipline" to keep learning, and the "creativity" to see possibilities where others see "roadblocks." Think about the "source materials" of your own life—the books you read, the people you talk to, the "failures" you’ve endured. How are you "synthesizing" those into a unique "mental model" for solving the problems only you can see? The "unwritten laws" of success are not secret; they are hidden in plain sight, waiting for someone with the "intellectual honesty" and "persistence" to apply them. Whether you are at the start of your career or a seasoned professional, there is always a "new level" of mastery to reach, a "new tool" to add to your "mathematical toolkit," and a "new way" to frame a "wicked problem." I encourage you to take one idea from today—perhaps it’s the "five whys," or the "sanity check" of "street-fighting math," or the "growth mindset" of "deliberate practice"—and apply it to a challenge you are facing right now. See what happens when you treat your "thinking" as an "engineering project" that can be "optimized" and "upgraded." Thank you for spending this time diving deep into the "mechanics" of "great thinking" with me. Your commitment to "learning how to learn" is the most important "investment" you will ever make, and the "compounding interest" of that effort will shape not just your career, but the very way you experience the world. Reflect on how your "perspective" has shifted today, and how you can use that "new lens" to build something truly "extraordinary."