My first Kaggle submission - Titanic Survival Rate

If you don't not it yet, Kaggle is a platform that organizes data-oriented competition. There are small and large sets of data, and there are competitions for fun, for swag or for money. If you are interested in data-science, wanting to learn or to improve, or simply looking for programming challenges, this is a good place to start.

The competitions can originate from a specific requester, but can also be more general: for example there is currently a competition to improve lung cancer detection. There are also introductory competitions meant for beginners.

When I registered on Kaggle, five years ago, this was to access some large datasets to play with in R, but I have not tried any competition. 

I made my first submission in the Titanic: Machine Learning from Disaster competition. This is one of the competitions for learning (no ranking, no swag, no money). In it, you are presented with a training dataset that contains several features as well as if the person survived or not. The goal is to build a model that will predict in the testing set whether the person survived or died, based on the same feature. 

My notebook is publicly available here. Ultimately I opted for Python instead of R as I wanted to play with pandas, numpy and scikit-learn. Pandas is a library providing data frames to Python (among other), numpy an octave-like interface, structures and functions, and scikit-learn various algorithms revolving around machine learning, be them machine learning (kNN, SVM ...), preprocessing or encoders. 

I am quite pleased as my refined model has an accuracy of around 80%. Some of the missing data are imputed using a random process, so the repeated accuracy can change.

Feel free to comment on it!

What did I learn?

A lot! But more specifically ...

Exploration and visualization are important

This was actually my biggest mistake: I started the competition without really looking at the data, neither exploring it nor visualizing it. As a result, my first models' accuracy were below 75%. Looking at various examples and blogs, I saw people splitting the data and making it tell as much as possible, revealing in the process which features were important and which ones had less significance.

Get to know the context

For example, in 1912, during sea accidents, the principle of "women and children first" was still in place, which reflects in the data with a survival rate higher for women and children than for adult male. 

Another example is looking at the ship's plans: the second and third classes were far from the level with the lifeboat, which may also account for the lower survival rate of these populations. This could also have been provided by the cabin number, which started with the deck letter. Unfortunately that information was seldom provided in the dataset.

Understand your tools, and shun away from "easy" simplicity

Initially, I used a LabelEncoder from sklearn.preprocessing,  which translates categorical variable into an integer per category, for example "Black","White","Green","Black","Green" would become 0,1,2,0,2. This has the advantage of being quick, but unfortunately, this also makes possible things such as computing the average of "Black" and "Green" as "White", which makes no sense. I switched to a simple dummy encoding, i.e. converting all the factors to a 0/1 variable, which improved the accuracy.

SANS - Securing the Human

The SANS has an interesting section about securing the weakest point in Computer Security: the user.

For the last few years, the attackers have focused on trying to fool users into clicking on link or executing programs on their computers, either by sending e-mails, leaving USB thumbdrives in parkings or even mailing CDs. Combined with vulnerabilities in common desktop applications such as Adobe Acrobat, Adobe Flash or Oracle Java, this proved to be an optimal process: the attacker, instead of trying to pry the perimeter open, tried to have his payload be directly injected at the hearth of the network. To that, you have to add the "mellow cake" network: hard at the perimeter but gooey inside.

While there is a huge room for improvement on many network (segmentation/segregation of machines, network access control and so forth), securing the human is by far the most efficient way of raising the security level of a network.

Let's make a thought experiment: what if on your organization's networks, no one would be to click on links in e-mails, no one would ever connect a USB thumb drive or device to any computer and surfing would be limited to corporate/professional website? What would the result be? I claim it would lower the risk of compromise by multiple orders of magnitude.

Dedicated to security, the SANS has started a series of advices to "secure the human": there is a monthly video and various resources, such as guides and documents. For example, December's is about  the seven steps to secure a computer.

Aimed primarily at CSOs and technical security personel, I think everybody will gain by getting there and reading some of the docs.

The worse that could happen is that our security level will be raised.

How to Process, Analyze and Visualize Data

Another great course on MIT OpenCourseWare: How to process, analyze and visualize data. Through different examples, fundamental concepts of data analysis and visualisation are presented in a clear way. My favorite part deals with the analysis of text documents.

All the codes are written in python, and it assumes the reader has a working knowledge of the language plus the ability to install some development libraries. A basic understanding of statistics is also of great help.