ToT : Techies Overvalue Technology

Techies overvalue technology, and sometimes it hurts them badly !

.

Story begins

.

John and team received a freshly minted contract to build an application. The required app was medium complexity. And, the contract was the first major one the organisation had received.

.

As it were, not only the contract, but also the team was fresh. Such a team wanted to impress the client. In their fervor to make a great first impression, the team chose a new programming language – Scala. By and large the choice was against the tried-and-tested languages like Java, .Net, Python, C, C++ etc.

.

For non-tech readers: Scala is a JVM ( Java Virtual Machine ) based programming language. It is known to be blazing fast and requires fewer lines of code. Additionally, Scala’s style is a mix of object-oriented and functional programming.

.

Tech break …

.

Functional programming languages are cool to “look at”. They use 2 lines of code where a regular one takes 20.

.

Yet most people don’t realize that – while the complexity of writing 10 lines gets hidden. The ease of debugging the same lines also gets hidden. Essentially the trade is:

.

Meaning – While it might be easy to code in a functional programming language, it is damn hard to debug in it.

.

Complexity like energy is never destroyed. While it can change its shape, form or location, it will never be destroyed.

.

… Continued

.

The team went to work, with their chosen tech stack. Of course development went smoothly in the beginning. Until the day bugs started pouring in.

.

Let’s recollect – debugging functional code is difficult. Suddenly, developer’s lives became hard. Endlessly, they worked day and night, weekends and holidays. After a while, as most do in such a scenario, developers started quitting en-mass. Compounding the attrition, the company could not hire new developers.

.

.

Any new programming has a smaller ecosystem around it. That is, there are few developers, still maturing tools, smaller support communities. Using the supply-demand formula we know that, when something is scarce it costs more. Meaning, when a programming language has few developers, they command high salaries.

.

Back to our story – the marketplace did not have many Scala developers in the first place. Unfortunately, the few who were found and did qualify, could not be hired. Their asking salaries were way about the company’s budget. So much so that the work came to a halt.

.

Ending

.

Increasing complexity and hiring challenges forced the company to abandon Scala.

As a result the whole app was re-done in a more “regular” programming language – Python. That is, the team re-started from scratch.

.

Of course the rewrite meant –

Essentially – Techies were eager to ride the “new technology bandwagon”. The same wagon became the biggest bottleneck in the software development machine.

.

Moral of the story : Techies overvalue technology, and sometimes it hurts them badly !

.

Why

.

This is a common affair in the IT industry; as much as I’d like it to not be. But why ?

 

Undoubtedly the decision makers at John’s organisation were experienced professionals. For the most part, they knew what they were doing ( or thought so ). That being the case, what explains such blunders seemingly qualified professionals make ?

.

I have a few ideas …

.

POBO ( Plain Old Bias Object )

.

A new technology is like a shiny new toy that every techie wants to get. Until 10 years ago, it was Android, Mongo, Cassandra, Go, Scala. Now we have Artificial Intelligence, Blockchain, Virtual Reality, Artificial Intelligence. In the next 10 years, newer techs will come up.

.

Learning and adopting new technologies is great. But, the problem occurs when techies only see the green grass on the other side. That is to say that, techies overestimate new technologies and underestimate existing ones.

.

Since we become biased to anything we stay in proximity to, such bias shouldn’t be a surprise. (it also explains the poor decisions). Instead the surprise is – that organisations do not take proactive steps to avoid the biases.

.

PS: Pobo sounds like a Java phrase – Pojo ( Plain Old Java Object), hence funny to techies.

I know this is a PJ ! I too am speechless with my sense of humour sometimes. 😶😶😶😶

.

Pluralistic ignorance

.

During the same time, a competitor was pushing his developers towards Scala too. The idea was – if John’s and co. are using it, then it must be good. In this case let’s not be ‘left behind’ in technology fashion.

.

Keeping up with the Joneses, organisation style !

.

Monkey see monkey do

.

The competitor wanted to chase Scala because John’s team was using it, not because they needed it. Interestingly John’s team might’ve been using Scala copying another team. And, the other team might’ve been doing it because of yet another team. Together, all teams might’ve been making a poor choice. But done so with the “social proof” of their collective choices.

.

When my decision to choose technology is based on the other guy’s choice, then I am being ignorant. Such ignorance spread across the industry is called ‘Pluralistic ignorance’.

.

.

Pluralistic ignorance is pretty common. For instance let’s take a look at most resumes floating around. We find the “Objective” section, explaining something like:

On one hand this section adds no value to a resume, on the other is available in most of them.

Why ? This is because everyone thinks the other guy knows and copies him/her.

This is pluralistic ignorance in action.

.

Tech journals, conferences and fashion impact many tech decisions. While the journals and conferences are great, they never wrote an actual program. Meaning, they have not had skin in the game.

.

Techies foolishly think – “Google uses Scala, so let’s use it too”. Never mind the fact that-

.

Pluralistic ignorance is another reason for poor tech decisions. That is – It ignores real proof to social proof, only to pay later.

.

Blindspots

.

Every new technology has blindspots, and takes time to mature through them. Part of our job, as techies, is to recognize these blindspots.

.

For example, MQ based systems existed earlier too. But they seem to be promoted excessively nowadays. One such famous example is – Kafka. As the ‘in fashion’ technology, everyone wants to do everything on Kafka:

Unsurprisingly Kafka as a new technology does ‘sound’ wonderful. But will it ?

As an MQ system, Kafka is designed to be a ‘transient’. Meaning …

This is to say that while kafka might be good, it has its blindspots.

.

Another example is JavaScript based mobile apps. While great, they will never provide the performance of native technologies (Android/ iOS ). On this occasion performance is the blindspot.

.

While each mentioned blindspot can be mitigated by custom solutions. The need for custom solutions isn’t obvious during the trial run of the tech stack. In other words, new technologies have blindspots. And the blindspots don’t show until technology has charmed us into marriage.

.

Due to our technology bias, we select the best horse in a world of cars; the best, but not relevant.

.

This is another reason for poor technology decisions.

.

Step children

.

Having understood the ‘overrated’ aspects of technology, let’s talk about some ‘underrated’ ones. In the world of new, all love goes to the newborn techs. During such times what are treated like stepchildren ?

.

.

Oldies

.

DotNet has flying reviews. Here is one from StackOverflow. As the highest ranked framework, it is loved by a third of developers worldwide. (review link)

.The liking rose more, when the survey considered professional developers. (link)

.

In existence for a long time, .Net has covered most of its blindspots. For example there aren’t many threading surprises, memory leaks, garbage collection issues. That is, once something is written it works. No ifs, no butts.

.

For these reasons, DotNet apps work like a charm. This makes leaders happy and keeps developers’ life peaceful. Also explaining the love for such old technology.

.

While so, techies are weaning away from .Net. Infact, I know organisations that chose newer technologies despite being .Net strong.

.

DotNet is treated like a stepchild. Why ? Because it is old.

.

Similarly, relational databases are starting to get treated as stepchildren. In spite of the fact that they play a central role in many great apps. And are a more polished product than most realize. Infact, if the DB structure is mostly fixed, a relational DB is a better choice by a big margin.

.

A successful telecom support organisation told me that they use MySql. Although treated as a stepchild by others, it works great for the org with millions of records each day. Unlike many other organisations, they did not want to touch no-sql DBs.

.

Though relational DBs are step-children at many , they are gold at this organisation.

.

The most unloved stepchild – Engineers

.

We tend to forget that the human component is always the trump card.

.

Let’s say you implement a scheduler using Spring Boot. Of course the scheduler will run on its schedule. But, the following considerations will not be obvious …

Does it pick one from the application’s pool ? or does it fetch a new one from the JVM ? Too many threads might cause the app to slow down.

Spring does not provide any such provision. So one will have to be designed and custom built.

Solving such problems is an engineer’s job. This goes to prove that, despite the best libraries, we depend on humans for the best solutions.

.

Here are some other examples …

Data might be repeated, but who cares !

A single thread might do the job.

If so, an event-based high tech system isn’t needed.

.

Each of the above points has a common decision maker – us Engineers. Translates to, while each technology has one strength area, but one has many – Engineers.

.

Finally, let me ask – are Engineers valued as the best solution providers ?

.

Nopes. They are treated as a stepchild.

.

Exactly my point !

.

Wrap up

.

We started by discussing how John’s organisation hurt itself by its tech bias. Then went on to understand the causes of such poor decisions. Finally we looked at the other side of the bias i.e. the undervalued resources.

.

Wrapping up, I mention the point again … we techies overvalue technology. And, the bias comes back to bite us in the ass. Number #1 thing we can do is, acknowledge our biases. Once we do, steps towards mitigating them become easier.

.

So, along with asking which technology to use, let us also ask what might be some of the drawbacks of using it ? Only when we accept that we are biased towards the pretty tech ( or chick ), can we ask such questions.

.

Although I am all for the new technologies. Let’s use them with the understanding of our biases. Acknowledging our biases and using checks to counter them will keep us from …

.

.

Or more !

 

.