Today, one of the sexiest job titles is “Data Scientist.” Have you ever wondered why there is a sudden increase in the demand for this position over the past few years?

A very simple answer would be that over the last decade there’s been a gigantic explosion in the data generated (Big Data) and retained by companies, and by us. And data scientists are the people who make sense out of all this data and figure out just what can be done with it.

Setting Internship Goals

I am glad that I got an opportunity to intern as a Data Scientist at AT&T with a team working on projects related to big data. For any individual who wants to gain experience in a field, an internship opportunity is the best way to learn from pros and gain practical understanding. But your goal has to be the make the most of your internship opportunity.

My goal for this summer was to learn from experts about how to apply data science concepts to real-world problems. More specifically, I wanted to learn how to apply machine learning techniques, integrate data sources, and design visualizations to gain insights.

My initial assignment was to understand the client requirements. I had to create an information management tool to track data metrics of Advertisement ID and other influencing key data points. My goal was to reduce the amount of time and effort compiling reports and signing into multiple analytics services.

Working like a Data Scientist

Using dashboards, I was able to share the hourly data ingestion metrics to a wider audience within the organization.

The dashboard I created gathered data from multiple hourly log files from different nodes and was inserted into tables in PostgreSQL. I have been working on Hadoop and used PySpark to process the data and perform data cleaning and parsing of the data in the log files.

I then worked on Grafana to create a dashboard with data visualization to simplify complex log data and provide users with the current performance of the system and analyze the trends. The dashboard included world map, heat-map, graphs, and plots of hourly, daily, and monthly status.

I am grateful that I got a chance to design this whole process on my own from the start. I was able to design and create, from the back-end database to the front-end visualization.

Applying iSchool Coursework

This internship program helped me understand in-depth and real-life application of the concepts learned in IST -718 Big Data Analysis class. Courses like IST- 659 Database Management helped me design and create backend database design and IST-719 Data Visualization helped create Dashboards to enhance the user interface experience.

During my internship, I used Agile project management. Each sprint lasted about two weeks. Tasks are assigned and the progress is monitored using JIRA.

Courses like IST – 654 Information System Analysis and IST – 645 Project management gave me an edge. I was able to fit into the workflow process quickly. Having my business partner and mentor appreciate and use the dashboard to make key decisions was a proud moment for me.

Innovation Challenge

Apart from the project assignments at AT&T, the interns have an innovation challenge. I got to work on designing and creating AR (augmented reality) and VR (virtual reality) projects.

Some of the new skill set that I got to explore during my internship were:

  • Linux
  • Postgres
  • Grafana
  • Unity
  • Android Studio

And on a lighter note, I did get a chance to work on Mac! I’m a life-long Windows user, so it’s not something I would have normally volunteered to do.

A key lesson I have learned over the internship is taking initiative and tracking those goals. Opportunities will seldom come to anyone by chance. Seek out different opportunities apart from the day-to-day activities. Explore different technologies and roles. This will help in creating new opportunities for growth. Once you achieve your goals are you will feel a greater sense of accomplishment.