Two Unicorns of Tech: Full-Stack Engineers and General Data Scientists

On hiring those with deep knowledge and specialized talents

By construction and definition, a full technology stack must exist for any company that relies upon hardware and software to interface with their customers.  That is, every modern company has a full stack of some sort.  When we construct buildings–complex and interwoven hardware and software edifices in their own right–we might have a general contractor but it would be silly to have one person responsible for personally building all the sub-systems. There are probably no master plumbers who are also great electricians who also build retaining walls to code (well, maybe Weird Al is an exception).

With this in mind, and recognizing the growing complexity of full-stack technology, it should come as no surprise that successful full-stack engineers would have to be pretty special. And in the “The Rise And Fall Of The Full Stack Developer”, Peter Yared makes the case that the modern notion of a full-stack engineer does not exist at all in practice. By full-stack engineers we mean those not only capable of building responsive user interfaces connecting to back-end infrastructure in cloud-based platforms, but those who can push envelopes and develop novel approaches to help their company stay ahead in industries with rapidly evolving technology. It would be like asking your plumber/electrician/mason to also invent a new desalination technique for captured rainwater. Yared argues that there are: “zero individuals with advanced-level knowledge in each of these areas that would be capable of single-handedly delivering this next generation kind of application. Just keeping up with the advancements and new programming interfaces in each category is almost a full-time job.” I agree: what a major onus we put on such folks and ourselves for trying to hire them.

Although a unicorn may exist, there are at least vanishingly few of them in nature.

What struck me is that Yared’s argument is also relevant to data science. I’ve argued before that I believe that there is no data scientist (at least in the Conway conception of the field), only data science: it’s a collaborative effort almost by definition. The idea is preposterous that one person would have the deep knowledge and skillset to perform all needed tasks to innovate machine learning algorithms, building statistically sound models for a real-world business problem, code, and be able to operationalize and curate an entire workflow. Data scientists are unicorns, a romantic embodiment of an amalgam that only exists in fairy tales and job boards. Steve Geringer, a machine learning consultant, also has also argued for the mythical beast nature of a modern data scientist.

Just like there was a time when a scientist could be an expert in all aspects of known physics, there may have indeed been a time when a full-stack engineer and a data scientist could have been the world’s expert in all of the different facets encompassing their respective disciplines. But that time of industrious polymaths has passed.

Still there is more need than ever in getting innovative data science into production and doing so with a differentiated full stack. At Wise.io, I see in my role the need to be the general contractor – building, organizing, and motivating a team of highly skilled people with world-class specialties. When hiring for the full stack and for data science, it’s natural to search for generalists, but there is a certain richness and beauty in finding and managing those who complement what you already have in these areas. This means hiring not just those with deep knowledge and specialized talents but those that know how to work with others with orthogonal skillsets.

Originally posted at wise.io/blog … see the archive.org link.