From task automation to data science, computer programming is now genuinely a “must-have” asset
Feature Image Credit: Marissa Anderson (Flickr)
What I am about to say has been said millions of times: the Internet has changed everything.
This is quite obvious to everyone. The world we live in now is vastly different than the one I grew up in in the late 80s and 90s. But while I open this post with something you’ve all heard before, I challenge you to think of how this has impacted practically everyone you know.
Do you remember the cost of making a long distance call 25 years ago? I don’t, but my parents sure do. I remember being told not to stay on the phone too long with my father or grandparents, who lives a 4-hour drive away in Quebec’s Eastern Townships because of the long distance fees. Today, I live on the opposite end of the country; three timezones away — and that same call is free.
How about sending mail to someone in a faraway land. I’m sure glad I didn’t know anyone in Korea when I was growing up, because it would have cost me a small fortune in postage stamps. Now, we have E-Mail, instant messaging applications and social media.
Checking in on your investments? My grandfather used to walk down to the bank twice a week to ensure his portfolio was allowing him to stay retired. Today, with a click of a button you can not only check on your investments, but deposit or withdraw funds and even change your investment portfolio altogether.
Finally, remember having to tune the television to Channel 3 to play video games? Not that it was much of a hassle back then, but now gaming exists virtually everywhere, on almost any device that is connected to the Internet.
Of course I can go on about how connectivity has changed the world for the better and, in some respects, for the worse, but I think you all get the point. But what about those people who are not satisfied with simply being consumers of what’s already online — what about those problems that are so unique that no one else has come up with an application or website to assist you with that challenge.
Truthfully, this is something that most people encounter everyday, and while most get by with doing things either the hard way or accepting the impossibility of the task without some sort of software, there is a solution for people willing to put in the time and energy into learning a skill that will only become more essential as time goes by.
The solution lies in the often mysterious world of computer programming.
I’ve taken an interest in computer programming over the last few years and while I am no expert, I find an element of peace when I’m trying to tackle a challenge by coding-it-out. It is an in-demand skill for job-seekers, with one employment website saying that in Canada, those with some computer programming knowledge make an average of $9,000 more per year than those in the same job with no programming ability.
I had always known that coding was something powerful. A successful computer program can thousands of bits of information in a matter of seconds. Just how powerful it was; however, was something I had to recently discover on my own while working on a project.
Started with an idea, and coded the rest
A couple of weeks ago, I had it in mind to put together a map of all Canadian acute care hospitals. I got the idea from the kind of work that I do. While this is a project that will only add functionality as I move forward, the initial program was to do one simple thing: place a marker on a map of Canada denoting the location of each Canadian acute care hospital. For those who know how to do it, this seems simple enough.
Not so fast though — while the idea for the project was well-established, I had one very large problem right from the start that threatened to derail the whole project. I did not have a data set. I didn’t have a list of hospitals in Canada, nor addresses to where they all were, nor all their names.
One would think you could just look online for a list, but that’s not always easy. As healthcare is administered by each Canadian province, and within the provinces oversight is delegated to regional health authorities, there is no central list once can just look up. Ultimately, I had to create one of my own. Thankfully, I was able to find something to work from at the CBC, from the documentary they did on rating Canadian hospitals.
My idea, though well-defined, seemed to become a project that would require multiple programs to complete. It ultimately required four different programs.
STEP ONE: Create a web scraping program.
Web scraping is an incredibly powerful tool for any computer programmer. It allows a developer to scan a website to and access specific information they would find useful when compiling their data set. In my case, I needed the name of the hospital and the address, but I also was able to pull information on the hospital type, the number of beds in each and whether the facility had an emergency department.
The web scrape requested the URLs of 635 different websites, scanned each website to isolate the information I needed, and wrote it into an Excel spreadsheet for me. The whole process took about 8 minutes to complete. Once it was done, I had an Excel file containing the names and addresses of 634 Canadian hospitals and a bit of information about each one.
STEP TWO: Formatting the address to create a geocoder-readable string
Once I had a list of hospitals and their and their addresses, I need to do some string concatenation. Concatenation is a fancy word for bringing together text elements as one variable.
In the screen grab of the spreadsheet above, you’ll notice that the province listed in the first column is separate from the address, shown partially in the last column visible. What I needed to do was essentially to take the information I already had in the spreadsheet and combine them for use in another program.
A geocoder can provide information, like coordinates, based on text input you feed it. However, for the most accurate reading, you want to ensure the geocoder has complete information. Simply feeding it “899 W. 12th Street” will not be enough to have the geocoder output information for use later in the program. In other words, I needed to create a new column which displayed:
ADDRESS + PROVINCE + COUNTRY
Since I had the address, I concatenated the information from the province column and added “Canada” to the end of the text. Since I was looking for Canadian hospitals, I was able to hard-code the country in the program. This provided an input, for Vancouver General Hospital, for example, which looked like:
899 W. 12th Street + Vancouver + BC + Canada
STEP THREE: Acquire latitude and longitude coordinates for each hospital
Now that we have these long strings containing addresses, cities, provinces and country, we need to acquire geographical information. Specifically, we need the code to return coordinates for each hospital in our data. To do this, we need to acquire longitude and latitude coordinates.
You might be thinking by now that this is science fiction, but this part was actually one of the easier steps of the program.
First, I needed to feed the spreadsheet into what’s called a pandas data frame. This is geek-speak for creating another Excel-type spreadsheet that can be read by computer code. The cell-format of the data frame looks identical to an Excel spreadsheet, but the data frame can be manipulated by the code itself. For example, you can code which columns appear, which don’t; which columns need to be added and which rows are acted upon in the program.
Once the data frame is created, we create two new columns: one for longitude and one for latitude. And then, we feed the geo-string for each hospital into the geocoder (the Google Maps API, in this instance) to get the coordinates. Once the coordinates are acquired, write them into the appropriate column in the data frame.
STEP FOUR: Generate the map and add the hospital markers
Finally, we get to the visually-appealing part. Creating the map and adding the markers.
In Python, there are a number of ways to accomplish this task, but what I did was use a package called Folium. This generates an HTML map image for the developer. This; however, is no ordinary image. The map it generates comes with the knowledge of coordinates on the map, and complete with zoom functions. In other words, if you tell the map you want a marker at a certain set of coordinates, the map will know where those coordinates are in the HTML image.
Once the map is generated, the developer simply needs to write the script that loops through all of the hospitals you have coordinates for, and place a marker on the map. The specifics of how to write it is beyond the scope of this article, but if you have a rudimentary understanding of control flow (if, elif, else) and for loops, you’re well on your way.
If there’s anything I would want someone to take away from this post, it’s that the only limits to what you can accomplish in computer code is your own imagination. I used to think about a problem, and how great it would be if someone came along with a solution for it. Then, when I started coding, I asked myself what the real-world applications of control flow and for loops were, outside of building basic decision-making programs.
In this project, we have an example of literally zero data to a functional, interactive map.
Having an understanding of computer science in this data-driven world is critical in a number of areas. It is vital in the workplace; it is essential if you are an entrepreneur; it is even useful for your own personal purposes.
What are some digital problems that you encounter that you’d like to see solved? If you can think of one, you have what it takes to get started in computer programming.