Fork me on GitHub

kaggle-1-Titanic

Posted on | In , |
Words count in article: 2.5k | Reading time ≈ 13

kaggle上的Titanic数据处理、特征工程，建模等

  • 中位数填充缺失值
  • 特征工程处理
  • 建模过程

导入相关库

1
2
3
4
5
6
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import seaborn as sns
sns.set() # setting seaborn default for plots
1
2
train = pd.read_csv("/Users/peter/data-visualization/train.csv")
test = pd.read_csv("/Users/peter/data-visualization/test.csv")

查看数据信息及缺失值

1
train.head(3)

1
train.info()  # age 字段非常缺失(714)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.7+ KB
1
test.head()

1
2
print(train.shape)
print(test.shape)  # 少了预测的结果列
1
2
(891, 12)
(418, 11)
1
test.info()   # age 字段缺失
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 418 entries, 0 to 417
Data columns (total 11 columns):
PassengerId    418 non-null int64
Pclass         418 non-null int64
Name           418 non-null object
Sex            418 non-null object
Age            332 non-null float64
SibSp          418 non-null int64
Parch          418 non-null int64
Ticket         418 non-null object
Fare           417 non-null float64
Cabin          91 non-null object
Embarked       418 non-null object
dtypes: float64(2), int64(4), object(5)
memory usage: 36.0+ KB
1
train.isnull().sum()  # 查看缺失总数
1
2
3
4
5
6
7
8
9
10
11
12
13
PassengerId      0
Survived         0
Pclass           0
Name             0
Sex              0
Age            177
SibSp            0
Parch            0
Ticket           0
Fare             0
Cabin          687
Embarked         2
dtype: int64

Bar Chart for Categorical Features

  • Pclass
  • Sex
  • SibSp ( # of siblings and spouse)
  • Parch ( # of parents and children)
  • Embarked
  • Cabin
1
2
3
4
5
6
7
def  bar_chart(feature):
    # 定义两个列字段：survived, dead
    survived = train[train['Survived'] == 1][feature].value_counts()
    dead = train[train['Survived'] == 0][feature].value_counts()
    df = pd.DataFrame([survived, dead])
    df.index = ['Survived', 'Dead']
    df.plot(kind='bar', stacked=True, figsize=(10,5))
1
bar_chart("Sex")

1
bar_chart('Pclass')

1
bar_chart('SibSp')

1
bar_chart('Parch')

1
2
# 先找出存活的所有数据，再找出属性P（1，2，3）中存活的人，然后统计属性P的分类人数
train[train['Survived']==1]['Pclass'].value_counts()
1
2
3
4
1    136
3    119
2     87
Name: Pclass, dtype: int64
1
train[train['Survived']==1]['Pclass']
1
2
3
4
5
6
7
8
9
10
11
12
1      1
2      3
3      1
8      3
9      2
      ..
875    3
879    1
880    2
887    1
889    1
Name: Pclass, Length: 342, dtype: int64

特征工程

Feature engineering is the process of using domain knowledge of the data
to create features (feature vectors) that make machine learning algorithms work.

特征工程的处理：如何将原始数据中的字符串数据转换成数值类型

Name

1
2
3
train_test_data = [train, test]  # 将测试集和训练集合并
for dataset in train_test_data:
    dataset["Title"] = dataset["Name"].str.extract('([A-Za-z]+)\.', expand=False)  # str.extract  从正则表达式中返回第一个匹配中字符
1
train["Title"].value_counts()  # 统计个数 train["Title"].value_counts()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
Mr          517
Miss        182
Mrs         125
Master       40
Dr            7
Rev           6
Major         2
Col           2
Mlle          2
Lady          1
Capt          1
Sir           1
Countess      1
Mme           1
Don           1
Jonkheer      1
Ms            1
Name: Title, dtype: int64
1
test["Title"].value_counts()  # 统计个数
1
2
3
4
5
6
7
8
9
10
Mr        240
Miss       78
Mrs        72
Master     21
Rev         2
Col         2
Dr          1
Ms          1
Dona        1
Name: Title, dtype: int64

Title map

  • Mr : 0
  • Miss : 1
  • Mrs: 2
  • Others: 3
1
2
3
4
title_mapping = {"Mr": 0, "Miss": 1, "Mrs": 2, "Master": 3, "Dr": 3,
                 "Rev": 3, "Col": 3, "Major": 3, "Mlle": 3,"Countess": 3,
                 "Ms": 3, "Lady": 3, "Jonkheer": 3, "Don": 3, "Dona" : 3,
                 "Mme": 3,"Capt": 3,"Sir": 3 }
1
dataset['Title']  # 取出每个名字的title_map部分
1
2
3
4
5
6
7
8
9
10
11
12
0          Mr
1         Mrs
2          Mr
3          Mr
4         Mrs
        ...
413        Mr
414      Dona
415        Mr
416        Mr
417    Master
Name: Title, Length: 418, dtype: object
1
2
3
for dataset in train_test_data:
    # 将title_map和定义的title_mapping 进行对比
    dataset['Title'] = dataset['Title'].map(title_mapping)
1
dataset.head(3)  # 最后面新增了一列：title

1
bar_chart("Title")

1
2
3
# 删除某个非必须属性
train.drop('Name', axis=1, inplace=True)
test.drop('Name', axis=1, inplace=True)
1
train.head()

Sex

  • male:0
  • female:1
1
2
3
sex_mapping = {"male": 0, "female": 1}
for dataset in train_test_data:
    dataset['Sex'] = dataset['Sex'].map(sex_mapping)  # 从dataset中取出Sex属性的值再和 map函数中定义的字典进行对比，找出符合要求的，再赋值给Sex 属性
1
bar_chart('Sex')

Age

Age字段中有很多缺失值，用中位数进行填充

fillna函数后中位数进行填充
1
2
3
4
5
# 某个字段用中位数进行填充 fillna 函数
# transform之前要指定操作的列（Age），它只能对某个列进行操作，用于求最值、方差、中位数等
train['Age'].fillna(train.groupby('Title')['Age'].transform("median"), inplace=True)

test['Age'].fillna(train.groupby('Title')['Age'].transform("median"), inplace=True)
1
train.groupby("Title")["Age"].transform("median")
1
2
3
4
5
6
7
8
9
10
11
12
0      30.0
1      35.0
2      21.0
3      35.0
4      30.0
       ...
886     9.0
887    21.0
888    21.0
889    30.0
890    30.0
Name: Age, Length: 891, dtype: float64
1
2
3
4
5
6
facet = sns.FacetGrid(train, hue="Survived",aspect=4)
facet.map(sns.kdeplot,'Age',shade= True)
facet.set(xlim=(0, train['Age'].max()))
facet.add_legend()

plt.show()

1
2
3
4
5
facet = sns.FacetGrid(train, hue="Survived", aspect=4)
facet.map(sns.kdeplot, "Age", shade=True)
facet.set(xlim=(0, train['Age'].max()))
facet.add_legend()
plt.xlim(0, 20)   # 分段画出来 plt.xlim(20, 30)，(30, 40)，(40, 60)
(0, 20)


1
train.info()   # Age 字段已经填充
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Sex            891 non-null int64
Age            891 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
Title          891 non-null int64
dtypes: float64(2), int64(7), object(3)
memory usage: 83.7+ KB
如何将一个属性变成一个分类变量形式
1
2
3
4
5
6
7
8
# 将 Age 年龄变成 Categorical Variable 分类变量
# 根据不同的年龄段划分成不同的数值
for dataset in train_test_data:
    dataset.loc[dataset['Age'] <= 16, 'Age'] = 0,
    dataset.loc[(dataset['Age'] > 16) & (dataset['Age'] <= 26), 'Age'] = 1,
    dataset.loc[(dataset['Age'] > 26) & (dataset['Age'] <= 36), 'Age'] = 2,
    dataset.loc[(dataset['Age'] > 36) & (dataset['Age'] <= 62), 'Age'] = 3,
    dataset.loc[ dataset['Age'] > 62, 'Age'] = 4
1
train.head()

1
bar_chart("Age")

Embarked

根据属性的多种不同取值来绘制图形
1
train[train['Pclass']==1]['Embarked']  # 找出P属性中值为1的每个 Embarked 属性值
1
2
3
4
5
6
7
8
9
10
11
12
1      C
3      S
6      S
11     S
23     S
      ..
871    S
872    S
879    C
887    S
889    C
Name: Embarked, Length: 216, dtype: object
1
2
3
4
5
6
7
8
# 找出P属性中值为1的每个 Embarked 属性值,再进行分类统计
Pclass1 = train[train['Pclass']==1]['Embarked'].value_counts()
Pclass2 = train[train['Pclass']==2]['Embarked'].value_counts()
Pclass3 = train[train['Pclass']==3]['Embarked'].value_counts()

df = pd.DataFrame([Pclass1, Pclass2, Pclass3])  # 将新生成的3个数据组成DF数据框
df.index = ['1st class','2nd class', '3rd class']  # 添加上索引
df.plot(kind='bar',stacked=True, figsize=(10,5))   # 通过叠加的方式绘制柱状堆叠图

1
df

1
2
3
# fill out missing embark with S embark
for dataset in train_test_data:
    dataset['Embarked'] = dataset['Embarked'].fillna('S')  # 用S来填充缺失值
如何将属性中的字符串转成数值型？
1
2
3
embarked_mapping = {"S": 0, "C": 1, "Q": 2}
for dataset in train_test_data:
    dataset['Embarked'] = dataset['Embarked'].map(embarked_mapping)  # map函数进行匹配

Fare

缺失值填充中位数
1
2
3
4
# fill missing Fare with median fare for each Pclass

train["Fare"].fillna(train.groupby("Pclass")["Fare"].transform("median"), inplace=True)   # 对Fare属性进行缺失值填充：通过 Pclass 属性分组，指定Fare的中位数填充缺失值
test["Fare"].fillna(test.groupby("Pclass")["Fare"].transform("median"), inplace=True)
绘制图形
1
2
3
4
5
6
facet = sns.FacetGrid(train, hue="Survived",aspect=4)
facet.map(sns.kdeplot,'Fare',shade= True)
facet.set(xlim=(0, train['Fare'].max()))
facet.add_legend()

plt.show()

将Fare属性分段
1
2
3
4
5
for dataset in train_test_data:
    dataset.loc[ dataset['Fare'] <= 17, 'Fare'] = 0,
    dataset.loc[(dataset['Fare'] > 17) & (dataset['Fare'] <= 30), 'Fare'] = 1,
    dataset.loc[(dataset['Fare'] > 30) & (dataset['Fare'] <= 100), 'Fare'] = 2,
    dataset.loc[ dataset['Fare'] > 100, 'Fare'] = 3

Cabin

1
train.Cabin.value_counts()
1
2
3
4
5
6
7
8
9
10
11
12
B96 B98        4
C23 C25 C27    4
G6             4
E101           3
F33            3
              ..
C62 C64        1
D28            1
D46            1
B41            1
E17            1
Name: Cabin, Length: 147, dtype: int64
1
2
3
for dataset in train_test_data:
    # 取出Cabin中的第一个字母
    dataset['Cabin'] = dataset['Cabin'].str[:1]   # Cabin 已经变成了单个字母
1
train[train['Pclass']==1]['Cabin'].value_counts()
1
2
3
4
5
6
7
C    59
B    47
D    29
E    25
A    15
T     1
Name: Cabin, dtype: int64
1
2
3
4
5
6
7
8
9
# 统计每个Pclass中的每个字母各出现多少次
Pclass1 = train[train['Pclass']==1]['Cabin'].value_counts()
Pclass2 = train[train['Pclass']==2]['Cabin'].value_counts()
Pclass3 = train[train['Pclass']==3]['Cabin'].value_counts()

# 生成数据框和行索引，绘图
df = pd.DataFrame([Pclass1, Pclass2, Pclass3])
df.index = ['1st class','2nd class', '3rd class']
df.plot(kind='bar',stacked=True, figsize=(10,5))

1
2
3
4
5
6
cabin_mapping = {"A": 0, "B": 0.4,
                 "C": 0.8, "D": 1.2,
                 "E": 1.6, "F": 2,
                 "G": 2.4, "T": 2.8}  # 每个字母匹配不同的数字
for dataset in train_test_data:
    dataset['Cabin'] = dataset['Cabin'].map(cabin_mapping)  # 将Cabin中的字母变成数字
1
2
3
# fill missing Fare with median fare for each Pclass
train["Cabin"].fillna(train.groupby("Pclass")["Cabin"].transform("median"), inplace=True)
test["Cabin"].fillna(test.groupby("Pclass")["Cabin"].transform("median"), inplace=True)

FamilySize

添加属性familysize
1
2
train["FamilySize"] = train["SibSp"] + train["Parch"] + 1
test["FamilySize"] = test["SibSp"] + test["Parch"] + 1
1
2
3
4
5
facet = sns.FacetGrid(train, hue="Survived",aspect=4)
facet.map(sns.kdeplot,'FamilySize',shade= True)
facet.set(xlim=(0, train['FamilySize'].max()))
facet.add_legend()
plt.xlim(0)
(0, 11.0)


1
train.head()  # 最后添加了familysize属性

1
2
3
family_mapping = {1: 0, 2: 0.4, 3: 0.8, 4: 1.2, 5: 1.6, 6: 2, 7: 2.4, 8: 2.8, 9: 3.2, 10: 3.6, 11: 4}
for dataset in train_test_data:
    dataset['FamilySize'] = dataset['FamilySize'].map(family_mapping)
删除某些属性
1
2
3
4
5
# 使用drop删除不需要的属性
features_drop = ['Ticket', 'SibSp', 'Parch']
train = train.drop(features_drop, axis=1)
test = test.drop(features_drop, axis=1)
train = train.drop(['PassengerId'], axis=1)
1
2
3
4
train_data = train.drop('Survived', axis=1)  # train_data是删除3个属性后的数据
target = train['Survived']

train_data.shape, target.shape
1
((891, 8), (891,))
1
train_data.head(10)

建模

导入各种模型

1
2
3
4
5
6
# Importing Classifier Modules
from sklearn.neighbors import KNeighborsClassifier  # K近邻
from sklearn.tree import DecisionTreeClassifier   # 决策树
from sklearn.ensemble import RandomForestClassifier   # 随机森林
from sklearn.naive_bayes import GaussianNB   # 贝叶斯分类器
from sklearn.svm import SVC  # 支持向量机
1
train.info()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 9 columns):
Survived      891 non-null int64
Pclass        891 non-null int64
Sex           891 non-null int64
Age           891 non-null float64
Fare          891 non-null float64
Cabin         891 non-null float64
Embarked      891 non-null int64
Title         891 non-null int64
FamilySize    891 non-null float64
dtypes: float64(4), int64(5)
memory usage: 62.8 KB

交叉验证

1
2
3
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score
k_fold = KFold(n_splits=10, shuffle=True, random_state=0)
1
2
3
4
5
# KNN
clf = KNeighborsClassifier(n_neighbors = 13)
scoring = 'accuracy'
score = cross_val_score(clf, train_data, target, cv=k_fold, n_jobs=1, scoring=scoring)
print(score)
1
2
[0.82222222 0.76404494 0.80898876 0.83146067 0.87640449 0.82022472
 0.85393258 0.79775281 0.84269663 0.84269663]
1
2
3
round(np.mean(score)*100, 2)  # KNN score

82.6
1
2
3
4
5
6
7
## 决策树
clf = DecisionTreeClassifier()
scoring = 'accuracy'
score = cross_val_score(clf, train_data,
                        target, cv=k_fold,
                        n_jobs=1, scoring=scoring)
print(score)
[0.76666667 0.82022472 0.76404494 0.7752809  0.88764045 0.76404494
 0.83146067 0.82022472 0.74157303 0.79775281]

1
2
3
round(np.mean(score)*100, 2)

79.69
1
2
3
4
5
### 随机森林
clf = RandomForestClassifier(n_estimators=13)  # 13 个评估器
scoring = 'accuracy'
score = cross_val_score(clf, train_data, target, cv=k_fold, n_jobs=1, scoring=scoring)
print(score)
[0.8        0.82022472 0.79775281 0.76404494 0.86516854 0.82022472
 0.80898876 0.80898876 0.75280899 0.80898876]

1
2
3
round(np.mean(score)*100, 2)  # Random Forest Score

80.47
1
2
3
4
5
#### 贝叶斯
clf = GaussianNB()
scoring = 'accuracy'
score = cross_val_score(clf, train_data, target, cv=k_fold, n_jobs=1, scoring=scoring)
print(score)
[0.85555556 0.73033708 0.75280899 0.75280899 0.70786517 0.80898876
 0.76404494 0.80898876 0.86516854 0.83146067]

1
2
3
round(np.mean(score)*100, 2)

78.78
1
2
3
4
5
##### SVM
clf = SVC()
scoring = 'accuracy'
score = cross_val_score(clf, train_data, target, cv=k_fold, n_jobs=1, scoring=scoring)
print(score)
[0.83333333 0.80898876 0.83146067 0.82022472 0.84269663 0.82022472
 0.84269663 0.85393258 0.83146067 0.86516854]

1
2
3
round(np.mean(score)*100,2)

83.5

testing

从上面的结果中观察到使用支持向量机的效果是最好的。

1
2
3
4
5
clf = SVC()
clf.fit(train_data, target)

test_data = test.drop("PassengerId", axis=1).copy()
prediction = clf.predict(test_data)
1
2
3
4
5
6
submission = pd.DataFrame({
        "PassengerId": test["PassengerId"],
        "Survived": prediction
    })

submission.to_csv('submission.csv', index=False)  # 将最终的结果文件写入csv
1
2
submission = pd.read_csv('submission.csv')
submission.head()  # 读取文件的前5行数据

本文标题:kaggle-1-Titanic

发布时间:2019年12月19日 - 23:12

原始链接:http://www.renpeter.cn/2019/12/19/kaggle-1-Titanic.html

许可协议: 署名-非商业性使用-禁止演绎 4.0 国际 转载请保留原文链接及作者。

Coffee or Tea