TADHACK GLOBAL 2017 Experience

TADHACK GLOBAL 2017 September 23-24 Auckland

Attending TAD Hack Global 2017 was one of most thrilling experience I enjoyed last weekend. Before that, I attended a lot of competitions with my own team or with proper preparation, this time It was totally different experience. When I went to Oracle New Zealand on Saturday morning, I hadn’t any idea what I was going to do or whom I was going to meet. I met first time with my five unknown team-mates, TAD Hack Coordinators. It was a global competition. Therefore, we can see each other activities on Live Stream telecast. TAD Hack is the widely global competition for all kind of IT professionals included students, business executive, or anyone.

When people meet with a common interest, they don’t need to know each other for starting a conversation. When we started sharing our ideas, we even didn’t know each other name. We didn’t know where we came from or what is our background. We were all focusing on our ideas and how to implement it.

After a few hours, we got to collaborate. We planned, designed and started implementing. We solved and troubleshoot problems, supported each other’s, wrote code, deployed the project, and celebrated each successful moment. We spent 2 days, almost 18 hours together. We never felt like uncomfortable, or we met the first time.  When we finished our competition, wrapped up our bag, and left Oracle New Zealand office, feelings were like we know each other.

Our story is: Six different unknown IT nerds, met somewhere, took a common challenge, and completed it.

I learned one thing: Passion and dedication can take you anywhere you wish to.

We won Local TADHack Auckland Prize and 1st Prize from Temasys Global Winners.

TADHACK Auckland MediConnect Team Member


PowerPoint Slide

 Thank you 🙂 


Week Twelve Activities: Conclusion Chapter

Well, it was a long journey to write a thesis report! I learned a lot of new things, faced problems and solved it. I enjoyed it except burned my food sometimes  :).

The Conclusion chapter, I planned to give a brief overview of my whole report to the reader. As it is a long report, the reader may lose their concentration at the end. Therefore, conclusion chapter is a good place, to summarize the report and giving the reader a fresh reminder of ending.

I hope my journey of “Graduate Diploma Project PRJ-702″” experience will help you to plan and write your final year report.

Poster Link:


Thank you 🙂


Week twelve Activities: Data transfer to HDFS Directory Storage System

In this Implementation part, I am going to share how to create a directory inside of HDFS default file system and how to transfer data from HDInsihgt Azure storage to directory location using SSH command line.

I will create two directories: one is uploaded data and another one for output of data

Data directory, I will move the raw datasets of 5 CSV files for hive job

Step 1: Connect to HDInsight head node using ssh connection via PuTTY 1.jpg

Step 2: Run directory create command line


Step 3: Run the following command to check directories are created or not!


Step 4: Run the following script to move data from HDInsight Storage to directory location of “Data folder”


Step 5: Run the following command to check files were moved or not to the data directory folder.


Step 6: You can check the files from the Azure Storage account dashboard from your directory location



Thank you 🙂

Week twelve Activities: Upload data into HDInsight Microsoft Storage Account from local machine using Powershell

In this implementation part, I am going to demonstrate how to upload files from local machine to HDInsihgt Microsoft Azure Storage account using the command prompt.

Step 1: Open your Windows Powershell. Run as Administrator



Step 2: Run the following script to upload data from local machine to HDInsihgt Hadoop HDFS default file system. I am going to upload 5 CSV files from January to May 2017 flight delay performance data. I downloaded that data from the websites :


Sytax: pscp file-location.format-of-file username@hostname:/location
pscp E:\newdata\01.csv sshuser@flightdelayh-ssh.azurehdinsight.net:

Step 3: CSV data uploaded to HDInsight storage through SCP protocol


Thank you 🙂


Week thirteen activities: Challenges and learning experience of the final year project

The first challenge was to select a completely new topic for final year project. There was the option for failure, but I was looking for something fascinating, exciting, challenging, and I can start from root. I studied more than 100 reports, journals, articles, web published, cloud vendors websites, and attended online seminar to educate me with big data analytical technologies. I had to concerned varies areas in this short period of time to complete this project because it was practical implementing plus research study. Solution analytical architecture, and compare each tool with others out there, and pick the best one and integrated it each other was thrilling excitement and big challenges. Learning each application, the failure results, testing repeatedly, searching for best option, solving systematic and analytical problems were the best experience and my learning part from this project.

Learning discovery area: Big data Ecosystem, HDInsight Microsoft Azure services, Hadoop framework, Hive, Power BI, Microsoft SQL Data Warehouse, System development and deployment.

Thank you 🙂

Week Twelve Activities: Data Visualization via Microsoft Power BI

This is the final implementation phase of my project. In this step, I am going to show how we can visualize analyzed data that we received from Big data ecosystem. I will show different visualize model in graphical that will help to simplify business decision.

The task lists:

  1. Visualize data in different graphical mode with some sample examples
  2. Power BI Share and publish facilities

Task: Visualize data in various graphical mode with some sample examples

Example 1: Total average flight delay performance from Origin and destination cities from January to May 2017

total number of delay performace of Flightdelay.jpg

Example 2: To view the top 5 origin cities Security delay with other flight delays performance from January to May 2017. The delay calculation is counted in minutes for  Carrier Delay, Weather Delay, National Air System (NAS) Delay, Security Delay,  and Late Aircraft Delay.


Example 3: Top 5 Security delay destination cities experience with other delays performance.


Example 4: To find out the flight delay performance percentage for origin cities and destination cities during January to May 2017


Task 2: Power BI Share and publish facilities 

The sharing and publishing facilities of the Power BI is to share dashboard, individual report across the organization with a bunch of co-workers in real time. This facility help organization to take a decision together without sitting the same room. The receiver can view the report from his/her Power BI services online account anywhere of the world, and able to represent it different graphical mode without changing data. That helps to business to simplify business decision and more data insight.

Example 5: Now I am going to share and publish the percentage of top 5 cities security delay from January to May 2017.


Step 1: First you need to click “Publish” icon and sign-in-your Power BI service on line account. It must be your organization account and have Power BI license.11.jpg

Now you will get the following publishing box


You will receive success notification!


Step 2: View the Same report from recipient end

If receipt opens his/her Power BI, the receiver will see the sharing report on the dashboard.


Now you can see the same report is displaying from my workplace account. I can share this report with other co-workers as well.


Step 3: Now I can save the report, print it, publish to the web, Export to PowerPoint, Download.16.jpg

Step 4: Now I am showing how to “Export to PowerPoint”. Click on the file -> Export to PowerPoint.


Now you can export the report to powerPoint.



Directly open with PowerPoint


Therefore, It is given you the facility to show the graph in your PowerPoint Presentation slide. Besides that, you can add the graph report directly to blog and able to publish any web.


Step 4: Publish to the web.







Web link:


Thank you 🙂 

Week Eleven Activities: Power BI integration with Microsoft Azure Database

In this Implementation phase, I will discuss how to integrate Microsoft Azure SQL database with Power BI desktop.

The tasks list for this section:

  1. Integration with Azure SQL database
  2. Data visualization in different graphical mode

Task 1: Integration with Azure SQL database

You need to install Power BI application system on your machine if you don’t have installed it. Download it from Microsoft Power BI website and install it on your computer. You need to open an account with your email address, that email ID must be similar to the Azure account.

Step 1:  After opening your Power BI desktop application system, You will get “Get Data” option. Select “Azure” and Azure SQL Database, Click connect1.jpg

Step 2: Now provide your Azure SQL Server connecting string, database name for establishing connection. Data connectivity mode “Import”, click “Ok”.2.jpg

Step 3:  Now you need to navigator with the tables you want to import. You can select more than one tables for importing. 3.jpg

Step 4: It will take some time to fetch the data from Azure SQL to Power BI Application system.5.jpg

Step 5: Now you can see your data is imported into Power BI and ready for further analyzing and data visualizations.



Thank you 🙂