First Love Vs Acquired Love - which is better?

This is a continuation of the last blog, where I said I would explain why I am attracted to Data, unwillingly, because it has qualities I want in my life. What I really want to say is that Software Engineering is my first love, and Data Engineering is who I married. But it is with very great difficulty that I can say the marriage part, because it seems like a commitment for life, when who knows, I might fall in love with AI or quantum engineering next. So - read the blog knowing that I am allowed to change my mind. Never Say Never. With that full disclosure, let’s get to the things I want more in life, which exist in the field of Data Engineering.

I have a theory about why opposites attract. I think its because we have a deep desire to get on each other’s nerves.

If I were to describe myself, I would say that when I am focused, I am like an arrow - once shot, it makes its way to the target. Software Engineering is the same - you know exactly what you want to build - an app, a webpage, and you build it (Of course, the person who shot the arrow keeps changing their mind, but that is not the arrow’s fault.) Now, Data Engineering, on the other hand, has no idea what the end game is - it is a field of exploration. Yes, you need the data because you want to track your weight and height, but you may realise something about your fat content along the way. So, it is more like a meandering river.

The other thing is that Software Engineering has defects, like the button on the screen is the wrong colour. You fix it, and it is done. And it never comes back. You know you have fixed it - once and done. But, with data, you never know - what was working for months will suddenly stop working when it’s a bank holiday or if a federal holiday falls on a leap day, which happens to be a Sunday. The only constant being uncertainty is not my cup of tea. I hate things that I cannot control - haha!

Software Engineering is code - like lines of code which you neatly bundle, put in a folder (deploy into production) and you are done. Data is about volume - so you built a pipeline which moves data from one place to another, but it’s the volume through the pipe that gets you. Sometimes, it’s so much data that it clogs the pipe, or the data contains chemicals that corrode it, or there is so little data that it never reaches the other end. Sheer volume gets you here - if you are in the business of acing knowing everything, then you are in the wrong line of engineering.

In the physical and material world, opposites attract. In the spiritual and emotional world, likes attract.

In some ways, Software Engineering is the opposite of Data Engineering. And, if you do not take the time to appreciate the differences, you won’t appreciate the intricacies of Data. When I first started in Data, my team would say things like the batch did not run today, and in my mind, I was thinking, all you are doing is moving data from one place to another, how difficult can it be? Despite all that I knew, there was something there - so I dug my heels in and before I knew, I was in love with Data. It was an acquired love. If you asked me if I love data, I probably would say I have a love-hate relationship with it, unlike Software engineering, which is my first love. I grew up as a software engineer. Some things about data annoy me, but that’s why I love it: it helps me be a better version of myself every day.

Even as I write this, a part of me is thinking, what if people think I am weird for using words like ‘love’ and ‘marriage’ for Data or Software Engineering? All I have to say is this is my truth now, and I own it. It is my creative expression.

What is your creative expression?