Learn PySpark basics in 30 minutes!

Minute 1–5: Create an account on Databricks community edition.

Here’s the link: https://community.cloud.databricks.com/login.html

You have to start a cluster(red icon selected below)before running a notebook; see below the screenshot.

Minute 6–10: Create DataFrame

There are several ways to create a dataframe in PySpark, here we will look at only 2 ways: Create DataFrame with schema and Create DataFrame from CSV.

Find all code here: https://github.com/Srikanth421/PySpark


A short story.

They say that when old friends meet, you forget your sorrows and happiness becomes you. Jake gazed out of the window as the train approached the station. The conductor said, “Next stop Belleville! Arriving in 5 minutes.” Jake was eager to meet his two old pals and kept bending down to look out of the windows, it wasn’t difficult with his small stature, as he walked towards the exit door.

“There’s the bloody rascal!”, a voice yelled across the station platform. “Ryan!”, waved Jake. They both walked towards the car that was waiting with Lauren in the…


Learn Scala basics in 30 minutes!

Minute 1–5: Create an account on Databricks community edition.

Here’s the link: https://community.cloud.databricks.com/login.html

You have to start a cluster(red icon selected below)before running a notebook; see below the screenshot.


Let’s continue our journey from last time and in case you missed:

In the third part, we will focus on the following:

  1. SAS do-loop in Python
  2. SAS simple arrays in Python
  3. SAS multiple arrays in Python

SAS do-loop in Python

do-loops in SAS follow the syntax shown above with do statement and end statement. In the above example we are trying to square first 5 integers. In Python, we will have to create a function to do this:


A brief overview.

Before we talk about DSGE models, we need to first talk about macroeconomics. Macroeconomics is simply a way of understanding the behavior and performance of the economy as a whole.¹

A DSGE model is a method in macroeconomics that incorporates the evolution of economy, random shocks(like COVID-19) to economy, and general equilibrium theory. As per Wikipedia, “DSGE models share a structure built around three interrelated “blocks”: a demand block, a supply block, and a monetary policy equation. Formally, the equations that define these blocks are built on microfoundations and make explicit assumptions about the behavior of the…


A template to success.

If you are looking to break into data science, let me share my story as well as give you a template to success. Let me share my suggestions and share my story along the way. Before we begin — Why is it that employers nowadays ask a thousand programming languages for a job like Data Scientist? Here’s the secret: the modern meaning of the term “Data Science” was coined as recently as in 2008 by D.J. Patil, and Jeff Hammerbacher, who at that time were working at LinkedIn and Facebook.

Types of Data Scientists

Listen carefully! Each employer has their…


There are many recommendations on Netflix for you to watch. Netflix has an algorithm to recommend movies to you but you didn’t get recommended these 5 movies which I absolutely loved! (No spoilers given!)
1. Ala Vaikunthapurramuloo (A regional film from Southern India)
2. Spirited Away (A Japanese animation)
3. Zodiac (Thriller)
4. Passengers (Space drama)
5. Replicas (Sci-Fi Thriller)

First we have Ala Vaikunthapurramuloo:


The aim of this article is to introduce basic data exploration in python for SAS programmers. Every other job nowadays asks for python programming experience and before python craze there was a leading programming language for data analysis: SAS! (R programmers please calm down! SAS is older than R and R was typically restricted to academic world.)

First things first: I am using a Jupyter notebook to write python code. I prefer anaconda’s distribution of python and you can download it here: https://www.anaconda.com/products/individual

How to import data in python? There are many types of files to import and I will…


Writing functions in python is a skill acquired over time. In this article, I attempt to write about writing python functions for beginners as well as experts. (If you are new then download it from here: https://www.anaconda.com/products/individual)

Let’s start by a simple task of adding the first 100 positive integers. You might remember the story of how Gauss, the amazing German mathematician, was once asked to add the numbers at age 7! Gauss simply figured out that adding the first 50 numbers with the last 50 numbers(in reverse i.e., 1+100,2+99,3+98, etc) all result in 101; so we end up with…

Srikanth Potukuchi

I am working as a Data Scientist at a bank in Canada. Passionate about data science since 2014. I started with SQL , R and SAS; later picked up Python.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store