But when I think about what killed most of the startups in the e-commerce business back in the 90s, it was bad programmers. A lot of those companies were started by business guys who thought the way startups worked was that you had some clever idea and then hired programmers to implement it. That's actually much harder than it sounds—almost impossibly hard in fact—because business guys can't tell which are the good programmers. They don't even get a shot at the best ones, because no one really good wants a job implementing the vision of a business guy.
-- excerpt from The 18 Mistakes That Kill Startups by Paul Graham
This is a spot on concern as it relates to start-ups, and is a pretty good summation that the world at large has with separating real technical talent from mediocre technical resources. To help with this, I've written down what I consider to be the key points that separate the great engineers from the herd.
First and foremost it has to be the Love of the Game. Some people call this Passionate or Enthusiastic. Regardless of how you refer to it, you know it when you see it. When an engineer truly likes being an engineer, they exercise that passion whether they are getting paid for it or not. These are the people who light up when they have a chance to explain some nuance of a solution or problem they are working on. Even when polite company would have recognized that a topic was becoming to specific or technical and gracefully glossed over the details, an engineer of passion may just not care about the social graces and bore full steam ahead unconcerned.
The inverse of this is the engineer who has a job. The get enough tech talk at work and go to "training" when they are told they need to learn something new. Engineering pays their bills, if they could make as much money doing something else, they would. When you see someone who has had the same job for years and years, has worked with the same technology, on the same application or platform and doesn't see the need to change, recognize that they may be competent but they lack the potential for greatness.
An enthusiastic engineer naturally leads to a Self-Reliant engineer. When you love something you want to know all about it. You genuinely like that there is always something new to learn. With engineering, there is the added benefit that technology is always evolving. When their heart is in it, nobody has to tell an engineer how to keep up. They do it naturally, on their own. They know that the best way to learn is just to leap in and embrace it. They don't wait for someone to send them to training, they experiment and create of their own accord. When they come upon a new technology they don't wait for the explanation, they don't need to be taught, they just jump right in and learn.
The inverse of this is the engineer who constantly needs help. They need the complete manuals, they need training materials, and samples. If they are always talking about how difficult something is to understand because of the lack materials at their disposal, if they want to get trained before they embark with a new technology, they may be competent, but they lack the potential for greatness.
Engineers who are or can be great, seek to be Wise. Anyone who has learned to be self-reliant understands (even if only subconsciously) the difference between Wisdom and Knowledge. Knowledge is the set of facts, data, or concepts that can be learned, taught, understood. Wisdom is the application of knowledge. For most people, with experience comes wisdom. The more they attempt to apply their knowledge to the world, the more they the learn about which knowledge is meaningful and practical. Knowing what you need to know and how to identify what you don't know in a particular circumstance is the most useful thing to understand when faced with the new and interesting. It is this constant self-refinement and search for the application of knowledge which can easily lead engineers to be considered socially inept. Socializing is often about compensating for differences, and celebrating commonalities and trivialities. This is completely juxtaposed with the engineers pursuit of practicality and usefulness. Engineers who are great will be able to switch their communication style between the practical and the social when appropriate.
The inverse of this is the engineer who knows only for the sake of knowing. When the emphasis is on the tests they've passed or certifications they hold, you might take a closer look. If they can't focus quickly on the criteria for success (or risks of failure), or if they aren't objective about the usefulness of their new widget then they may be competent, but they lack the potential for greatness.
To really leverage these talents an engineer should be Diverse, they should have a wide Breadth. Even early in their career an engineer with great potential will find opportunities to jump from problem space to problem space. They will find that they need more than one toolset on one platform. They'll have experience in applying their knowledge to more than one type of opportunity or industry. Any engineer who is truly experienced, will resemble an onion with layer after layer of different experiences, most of which won't be covered by a resume. They will invariably be able to find parallels with their past experiences, and will constantly be remembering skills and aspects of their work that are applicable now but which weren't significant enough to write into their cover sheet. To a great engineer, the fact that they had to write a multithreaded performance test harness, or an attribute injection lex, or an exception word map isn't really important. It was the sum of the parts they delivered which was interesting. To them it is just an assumption that to get the end result they'll have to do all the other things in between. Many of these in-between problems are significant and meaningful in other contexts all by themselves.
The inverse of this the engineer who spends all their time in the parts or focused on one technology, platform, or industry. They know all about how to make NUnit do interesting things, but have never shipped anything. They know how to build data entry applications but have never concerned themselves with how the reporting system does what it does. That database guy who doesn't really know how the front-end works should give you pause. The UI guy who draws brilliant graphic but stays away from "the backend" may be competent, but they lack the potential for greatness.
A caveat with this last one is that we all have specialties. From time to time we all need a real pro at just one thing, and it can take time to truly master some aspects of engineering. The distinction isn't that they don't have a specialty, it is that they have branched out into all the supporting and complementary areas. If you want to truly master something technical you need to understand how it overlaps with, supports, or is supported by other technologies. An amazing looking UI that takes to long to boot up, or doesn't integrate seamlessly with its data providers will ultimately be useless. Likewise the most efficient library design is worthless if consuming engineers find it hard to use.
Being having these other attributes becomes less useful if the engineer isn't Social. To be valuable long-term they need be someone that people can work with and form relationships with. In any environment, Trust is essential to mitigating risks and allowing velocity to increase. Further, independent contributors without a sense of community or social responsibility will leave their biggest value on the table. It is one thing to be able to do a thing well, it is much more valuable to be able to teach others to do it equally well. Sometimes this is referred to as leadership or mentoring but ultimately it is about being able to inspire trust and earn rapport. Often this is a latent talent that can be spotted very early by contributions that person makes to the community at large. A willingness to be trusted, a desire to connect with others about their contributions are often signs of a valuable resource.
The inverse of this is the engineer who only works independently, has trouble establishing rapport or eliciting trust, and isn't interested in the contributors around them. If they don't see the value in community boards, hoard their specialized knowledge and aren't interested in teaching or mentoring, then you have some warning signs. The loner who solves hard problems but doesn't care to explain how and is less effective as part of a team may be competent, but they lack the potential for greatness.
An engineer with a majority of the previous traits will likely be Opinionated. Simply put, the pursuit of greatness requires an ability to recognize when things are not great. An engineer who doesn't have strong preferences and habits is neither going to be decisive nor efficient. Efficiencies are only capable when you can rely on habits, known patterns, and economies of scale. These things require some level of standardization and best practices in place. A key trait of someone with the potential for greatness is that while they may often be wrong, they are rarely in doubt. It is this confidence in their ability, proven by their experiences, backed with their reason and intellect that allows them to make progress where others have stalled, to act quickly while others are paralyzed. It isn't that they are rigid in their preferences and opinions, quite the opposite, they may change their course much more often than seems normal. Being able to have an opinion is important, so is being able to change or let that opinion go when appropriate. The best engineers can assimilate the opinions and preferences of others which allows them be more accurate in their reasoning, even on an unknown landscape.
The inverse of this is the engineer whose opinions are too rigid, who holds to standards and habits long after the need or advantage for change is apparent. If the mind is too closed, reason will inevitably fail. Consider the age of toolset the engineer chooses and the terminology they use. If you arbitrarily alter the assumptions in a problem space and witness a lot of discomfort or inability to change tactics and approach quickly then you should be cautious. An engineer who presents the same tools and approach to every problem may be very competent in that space, but they lack the potential for greatness.
This list is more a guide to potential. The wrong situation, personality fit, or environment can make a great engineer useless. A healthy environment can take someone with mediocre skills but strong potential and give them an avenue to be great. Of course, that's just been my experience, YMMV.
Hopefully this list can help you spot competent engineers if that is what you are looking for; they certainly have their uses. After all, everyone can't meet the standard of great. You don't have to staff your team or project or company with only great people. If you have competent people, and a few people with the capability to become great, then the right environment will allow their potential to be realized. If you can't afford the risk, get at least one great engineer, and let them find the other competent engineers to round out their team.