In this post, I’m going to focus for a moment on my full-time job. I know that this blog is mostly filled with my hobbies and personal projects so it might seem like those are the only things I do. However, a good chunk of my life actually revolves around my career in tech.
I started my internship as a data scientist at the beginning of the month. It’s my 2nd full-time job as a computer scientist, and in some ways, I cannot help but compare it to my 1st full-time job. I worked as a front-end software engineer for over two years in a smaller company. Both companies are great, filled with talented people I get along with. More importantly, at both companies I am doing work that I am passionate about even though they are different.
And that’s what I want to focus on in this post: the difference between my experience as a front-end software engineer from a small start-up(-ish) company, and my impression so far as a data science in a much, much larger company.
I knew that the work would be different. And yet, I think I naively assumed that the job would be similar enough that I could measure my productivity in the same way. In my old job, I knew I was being productive when I managed to finish my assigned tickets. Depending on the tasks, I could finish about five moderate bugs in a day; for new features, I could at least get some new code out to be reviewed within a day or two at most.
In my new job, the process is entirely different. We’re working in Kanban style, rather than sprints. My tasks are a little more vague. Instead of having a specific goal I could measure, like changing the header background from white to grey, or adding a pop-out to a link, I’m assigned tasks like visualizing the clusters of similar items. As you can see, this task is less measurable. For one thing, the end goal isn’t to just have a nice visualization, right? Underneath that statement, I know that my goal is also to analyze the visualization, to obtain insights from the clustering. And this means that I have to find out a clustering algorithm that can actually give me a good visualization; it means that I have to find the data that can work with such an algorithm; it means that I have to find a visualization that can actually give me insights. And in the end, how do I know if the insights are meaningful or not?
For the past four weeks, I have struggled a little with this vagueness. Yes, I know I could ask, but I get the impression that it’s also part of the job of a data scientist to figure out these things. Whenever I’m assigned a task, it’s no longer up to a product manager to break down that task for me. It’s up to me to figure out what’s involved in that process.
I think this is the biggest difference between my previous job and the current one. As a junior software developer, my job was to implement whatever the product managers told me to. This is in contrast with a research position, where my job is to discover what must be implemented.
I worry that I’m not being as productive as I can be, and my worries are compounded with the fact that I don’t actually know how to measure my productivity as a data scientist. Which leads me to this passage from The Lean Startup by Eric Ries.
When I worked as a programmer, that meant eight straight hours of programming without interruption. That was a good day. In contrast, if I was interrupted with questions, process, or — heaven forbid — meetings, I felt bad. What did I really accomplish that day? Code and product features were tangible to me; I could see them, understand them, and show them off. Learning, by contrast, is frustratingly intangible.
Wow. This book is required reading for my Technical Entrepreneurship course that runs alongside the internship. I don’t have much of an entrepreneurial spirit in me, but when I read this, I thought, “Aha! This is why this book is required reading!” I never realized what it was that bothered me as I started my career in data science, until I found this passage in the book. I could have never put it in a better way.
Learning, by contrast, is frustratingly intangible.
I realized much of what I do in my new job as a research intern is learning. When you’re researching, what you’re doing is learning. You’re learning what works and what doesn’t. I was so used to measuring my productivity in terms of how much code I write or how many tasks I finished. Now I have to figure out a way to measure my productivity in terms of learning and the return value from what I learn.