[Ed. note: As we take a break to recharge and gear up for the upcoming year, we’re revisiting our top ten posts of the year. We hope you enjoy this piece, one of our favorites from 2023, and we look forward to connecting with you again in 2024.]
Artificial intelligence (AI) is everywhere in the headlines these days, touting incredible advancements and sparking debates about the future of work. Among software developers, a common concern is whether AI could soon make our jobs obsolete, capable of directly translating business needs into fully functional applications. The vision is that executives and product managers will bypass developers entirely, instructing AI to build precisely what they envision. However, for those of us with years of experience transforming vague specifications into tangible software, this worry feels somewhat premature.
While coding itself can present intricate puzzles, the technical aspects are rarely the primary bottleneck. In my 15 years in the field, I can’t recall a coding problem that remained unsolved for more than a couple of weeks. Once you master the syntax, logical structures, and common programming patterns, the act of writing code becomes relatively systematic—most of the time. The real hurdles, the ones that truly test a software developer, invariably revolve around the software’s intended purpose. The most challenging part of software creation isn’t the coding; it’s deciphering and defining the requirements – and these requirements are still, fundamentally, a human endeavor.
This article will explore the critical relationship between software requirements and the actual code, and why even the most sophisticated AI needs clear human direction to produce meaningful results, especially in complex domains like self-driving cars.
“It’s Not a Bug, It’s a Feature”… Or Is It? The Requirement Conundrum
Early in my career, I joined a project already in progress, tasked with boosting the team’s development speed. The software was designed to configure customized products on e-commerce platforms. My specific task was to generate dynamic terms and conditions, varying based on the product type and the customer’s US state, due to differing legal stipulations.
During development, I identified what seemed like a significant flaw. The system would correctly generate terms based on the initial product selection. However, later in the user workflow, it allowed users to change the product type without updating the terms and conditions. This directly contradicted a key, signed-off business requirement.
Naively, I approached the client, a senior executive, and asked, “Should we prevent users from overriding the correct terms and conditions?” His response, delivered with absolute certainty, is etched in my memory:
“That will never happen.”
This was a seasoned executive, deeply familiar with the company’s operations and handpicked to oversee the software. The ability to override terms, ironically, had been explicitly requested by him. Who was I, a junior developer, to question such authority? I dismissed my concern and moved on.
Months later, mere weeks before the software launch, a client-side tester reported a bug, assigned directly to me. Reading the bug report, I couldn’t help but laugh. The “never happen” scenario – users overriding terms and conditions – was precisely what was happening. And guess who was tasked with fixing it? And who was subtly blamed for not catching it earlier?
The fix itself was straightforward, and the bug’s impact was minimal. Yet, this experience became a recurring theme throughout my software development journey. Talking to colleagues, I realized I wasn’t alone. The problems grew larger, more complex, and costlier, but the root cause often remained the same: unclear, inconsistent, or simply incorrect requirements.
Software development lifecycle showing a bug being introduced at the requirements stage and propagating through design, coding, testing, and maintenance.
AI’s Current Landscape: Chess Mastery vs. Self-Driving Car Challenges
Artificial intelligence, while a long-standing concept, has recently surged into public consciousness with highly publicized advancements, triggering both excitement and apprehension. AI has already achieved remarkable success in specific domains, with chess being a prime example.
AI’s application to chess dates back to the 1980s. Today, it’s widely accepted that AI surpasses human chess-playing ability. This isn’t surprising given chess’s inherent nature: FINITE parameters (though not yet “solved”). Chess begins with 32 pieces on a 64-square board, operates under clearly defined, universally accepted rules, and has a singular, unambiguous objective: checkmate. Each turn offers a finite set of possible moves. Chess, at its core, is a rules-based system. AI excels by calculating the consequences of each move, selecting the option most likely to capture pieces, gain positional advantage, and ultimately win.
Another area of intense AI development is self-driving cars. Manufacturers have been promising autonomous vehicles for years. While some cars now offer self-driving capabilities, they often come with significant limitations. Many systems require constant driver supervision, demanding hands on the wheel; true autonomy remains elusive.
Similar to chess AI, self-driving cars rely heavily on rules-based engines to make decisions. However, unlike chess, the rules for navigating every conceivable driving situation are far from clearly defined. Human drivers constantly make thousands of subtle judgments – avoiding pedestrians, maneuvering around obstacles, navigating complex intersections. The accuracy of these judgments is the difference between a safe arrival and a potential accident.
In technology, “availability” is a critical metric, often measured in “nines” – five nines (99.999%) or even six nines (99.9999%) of uptime. Achieving the first 99% availability isn’t overly challenging. It allows for over three days (87.6 hours) of downtime annually. But each additional “nine” exponentially increases the complexity and cost. Reaching 99.9999% availability, for instance, permits only 31.5 seconds of downtime per year, demanding vastly more sophisticated planning and resources. While achieving 99% might be relatively straightforward, that final fraction is orders of magnitude more difficult and expensive.
365 days/year 24 hours/day 60 minutes/hour = 525,600 minutes per year
- 99% availability: 5256 minutes downtime (87.6 hours)
- 99.9% availability: 526 minutes downtime (8.76 hours)
- 99.99% availability: 52 minutes downtime (less than 1 hour)
- 99.999% availability: 5.2 minutes downtime
- 99.9999% availability: 0.52 minutes downtime (roughly 31.5 seconds)
No matter how advanced AI becomes in driving, the risk of accidents and fatalities persists. Humans, of course, cause accidents daily. While the acceptable accident and fatality rate for self-driving cars is yet to be determined by regulators, it’s reasonable to assume it must be at least as good as, if not better than, human driving.
The difficulty in achieving this level of safety lies in the vastly greater, and importantly, non-finite, variables in driving compared to chess. The first 95% or 99% of driving scenarios might be predictable and programmable. However, the remaining few percent are filled with countless edge cases, each subtly unique: other drivers’ unpredictable behavior, road closures, construction zones, accidents, weather events, even faded road markings. Training AI to recognize and react appropriately to these anomalies and edge cases is immensely challenging because while they may share some characteristics, they are rarely identical, making it difficult for AI to generalize and apply learned responses effectively.
AI Can Generate Code, But Can It Create Software for Self-Driving Cars?
Creating and maintaining software shares more similarities with driving than with chess. It involves far more variables, and decisions are often based on nuanced judgment calls. While software development has a desired outcome, it’s rarely as singular as winning a chess game. Software is rarely “finished”; it evolves with new features and bug fixes – a continuous process. Chess games, in contrast, end definitively with a win or loss.
In software development, we strive to emulate the controlled rules of chess through technical specifications. Ideally, specs detail expected user interactions and program workflows – “to buy an e-sandwich, click this button, create this data structure, run this service.” However, reality rarely matches this ideal. More often, developers receive feature wishlists, napkin sketches of wireframes, and ambiguous requirement documents, leaving them to fill in the gaps with their best judgment.
Worse, requirements often shift or are disregarded entirely. Recently, I was asked to assist a team developing a system to provide COVID-19 health information via SMS in areas with unreliable Wi-Fi. Initially, I was enthusiastic about the project’s potential impact.
However, as the team described their vision, red flags emerged. Asking a retail customer to rate their shopping experience on a 1-10 scale is vastly different from conducting multi-step surveys with multiple-choice questions about COVID-19 symptoms via text message. While I didn’t refuse outright, I raised numerous potential points of failure and urged the team to clearly define how the system would handle incoming responses for each question. Should answers be comma-separated numbers corresponding to options? What happens if a response doesn’t match any valid options?
After these discussions, the team concluded that proceeding was too risky. Incredibly, this was deemed a success. Moving forward without addressing potential data errors would have been far more wasteful than halting the project.
So, is the vision of AI-driven software development to simply let stakeholders directly instruct a computer to create an SMS-based survey? Will AI proactively ask probing questions about handling data validation and error scenarios in SMS surveys? Will it anticipate and address the myriad potential human errors and edge cases inherent in such a system?
To create functional software, especially for complex systems like self-driving cars, AI needs precise, unambiguous instructions. Even seasoned developers sometimes uncover unforeseen challenges only after starting to code.
Over the past decade, the software industry has largely shifted from the Waterfall methodology to Agile. Waterfall emphasizes complete upfront specification of requirements before any coding begins. Agile, in contrast, embraces flexibility and iterative adjustments throughout development.
Many Waterfall projects failed because stakeholders, despite believing they understood and accurately documented their needs, were ultimately disappointed with the final product. Agile was intended as a remedy.
AI might excel at rewriting existing software for new hardware or modern programming languages. Many organizations still rely on COBOL, but COBOL programmers are becoming scarce. If the goal is clearly defined and unchanging, AI could potentially produce software faster and cheaper than human teams. AI could likely recreate existing software more efficiently than humans precisely because the hard work of defining what the software should do has already been done.
AI might actually be well-suited for the Waterfall process, sometimes jokingly called the “death march.” But humans struggle with Waterfall, not with the coding phase after requirements are finalized. The problem lies in everything before coding – in the inherently human process of understanding needs, anticipating complexities, and defining clear, complete, and consistent requirements, especially for something as nuanced and unpredictable as self-driving car behavior in the real world. Artificial intelligence is capable of extraordinary feats, but it cannot read minds or tell us what we truly need, particularly when it comes to navigating the infinite variables of the open road.
