I've read parts of several algorithms books covering familiar topics such as sorting, searching, graph traversal, trees, and hash tables. They explain how standard algorithms work, but I still struggle with the creative process behind inventing or decomposing a solution.
For example, I recently needed to create an attractive and intuitive layout for a control-flow graph. This turned out to be a difficult problem, and I eventually learned that Sugiyama had developed a solution to it decades ago. What I don't understand is how to break down a problem like that independently and gradually arrive at a workable algorithm.
How can I practice and improve the skill of inventing algorithms without immediately getting stuck or looking up the established solution?
3 Answers
There isn't a reliable step-by-step recipe for algorithm invention, but the skill can be developed through repeated practice. Try solving problems yourself before reading the standard solution, and don't confuse understanding an existing answer with being able to generate one.
When you're stuck, simplify the problem. Solve a smaller input, remove some constraints, or focus on a restricted version. The easier version often reveals the idea needed for the original problem. Leave difficult problems alone for a while and return to them later instead of immediately searching for the answer.
Programming challenges can provide useful practice, but algorithmic thinking also appears in everyday work such as API design, service development, and managing complex systems. You don't need to independently rediscover famous algorithms to become good at problem solving; practice and reflection matter more than producing a groundbreaking result.
Start with a concrete problem rather than memorizing algorithms. For instance, take a list of numbers and design a sorting method yourself. Then ask how your approach changes if you need fewer operations, less memory, or support for parallel hardware.
Games and programming challenges that impose resource constraints can also be useful because they turn algorithm design into an iterative optimization problem. After attempting a solution, study the established algorithm and compare its choices with yours. That helps you understand not just what the algorithm does, but why it was designed that way.
Algorithm design is essentially writing a precise sequence of steps that solves a problem under particular constraints. Begin with manageable problems, experiment with different approaches, and gradually add requirements such as speed, memory limits, or scalability. The ability to decompose larger problems tends to emerge from building up that experience over time.
A good summary is the old advice about performing at Carnegie Hall: practice, practice, practice.