# Some brief concepts on numerical python - NUMPY

<div class="alert alert-warning" style = "border-radius:10px">
<b>MAIN QUEST:</b> Exercise about training, test and validation. (1 week)
<p>
<b>SUB QUEST:</b> NONE
</div>

Python is not designed specifically for mathematical and scientific computing.
In particular, Python lists are very flexible containers, but they are poorly suited to efficiently represent common mathematical constructs like vectors and matrices. 

Fortunately, the **numpy** package (module) exists. It is a package that provide high-performance vector, matrix and higher-dimensional data structures for Python. It is implemented in C and Fortran so when calculations are vectorized (formulated with vectors and matrices), performance is very good. It is used in almost all numerical computation using Python.


Why not simply use Python lists for computations instead of creating a new array type?

There are several reasons:

* Python lists are very general. They can contain any kind of object. They are dynamically typed. They do not support mathematical functions such as matrix and dot multiplications, etc. Implementating such functions for Python lists would not be very efficient because of the dynamic typing.
* Numpy arrays are statically typed and homogeneous. The type of the elements is determined when array is created.
* Numpy arrays are memory efficient.
* Because of the static typing, fast implementation of mathematical functions such as multiplication and addition of numpy arrays can be implemented in a compiled language (C and Fortran is used).

## Basics of Numpy

To use **numpy** it is needed to import the module:


```python
import numpy as np
```

## Creating numpy arrays
There are a number of ways to initialize new numpy arrays, for example from

1. A Python list or tuples
2. Using array-generating functions, such as `arange`, `linspace`, etc.
3. Reading data from files


numpy.array is a function that returns a numpy.ndarray. There is no object type numpy.array.



### 1. From a list
For example, to create new vector and matrix arrays from Python lists we can use the `numpy.array` function.


```python
# a vector: the argument to the array function is a Python list
v = np.array([1,2,3,4])
v
```




    array([1, 2, 3, 4])




```python
type(v)
```




    numpy.ndarray




```python
v.dtype
```




    dtype('int32')




```python
# a matrix: the argument to the array function is a nested Python list
M = np.array([[1, 2], [3, 4]])
M
```




    array([[1, 2],
           [3, 4]])



If we want, we can explicitly define the type of the array data when we create it, using the `dtype` keyword argument: 


```python
M = np.array([[1, 2], [3, 4]], dtype=int)
M
```




    array([[1, 2],
           [3, 4]])



Common type that can be used with dtype are: int, float, complex, bool, object, etc.

We can also explicitly define the bit size of the data types, for example: int64, int16, float128, complex128.

### 2. Using array-generating functions
For larger arrays it is inpractical to initialize the data manually, using explicit python lists. Instead we can use one of the many functions in `numpy` that generates arrays of different forms. Some of the more common are:

**Zeros and Ones**


```python
np.zeros(5, dtype=float)
```




    array([0., 0., 0., 0., 0.])




```python
np.ones(5,dtype=float)
```




    array([1., 1., 1., 1., 1.])




```python
a = np.zeros((2,3),dtype=np.int64)
a.shape
```




    (2, 3)



**arange**


```python
x = np.arange(0, 20, 1) # arguments: start, stop, step
x
```




    array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
           17, 18, 19])



**linspace and logspace**


```python
print("A linear grid of 5 elements between 0 and 1:")
print(np.linspace(0, 1,5))
```

    A linear grid of 5 elements between 0 and 1:
    [0.   0.25 0.5  0.75 1.  ]


**Creating random arrays**


```python
# uniform random numbers in [0,1]
np.random.rand(5,5)
```




    array([[0.00594451, 0.07510901, 0.44177116, 0.9870086 , 0.48399101],
           [0.03449061, 0.56418171, 0.74848344, 0.82323553, 0.51349367],
           [0.18969478, 0.85156371, 0.5108862 , 0.6572645 , 0.56298342],
           [0.93090994, 0.04054921, 0.3798979 , 0.11006237, 0.86609357],
           [0.8468136 , 0.43201715, 0.08609731, 0.69660495, 0.989217  ]])




```python
# 5 samples from a normal distribution with a mean of 10 and a variance of 3:
np.random.normal(10, 3, 5)
```




    array([13.24030147,  6.97283327, 10.02331165, 11.99978282,  7.13639879])



Although out of the scope, we will need permutations at some point. Thus, here comes how to create a permutation of the elements of a given list.


```python
np.random.permutation([1,2,3,4,5,6,7])
```




    array([2, 6, 4, 1, 7, 3, 5])



Observe that each time you execute the former line, the result changes. If you want to get the same permutation everytime, we can set the seed of the Pseudo-Random Number Generator.


```python
np.random.seed(1)
np.random.permutation([1,2,3,4,5,6,7])
```




    array([7, 3, 2, 1, 5, 4, 6])



*Few remarks on NANs:*

By definition, NaN is a float point number which is not equal to any other number 


```python
np.nan != np.nan
```




    True



Thus, the equal operator can not be used for detecting NaN

Instead, isnan function is used:


```python
np.isnan(x)
```




    array([False, False, False, False, False, False, False, False, False,
           False, False, False, False, False, False, False, False, False,
           False, False])



## Manipulating arrays


```python
lst = [10, 20, 30, 40] #python list
arr = np.array([10, 20, 30, 40],dtype='int64') #numpy array
M = np.array([[10, 20, 30, 40],[50, 60, 70, 80]]) #numpy matrix
```

** Element indexing **



```python
#get the first element of list
lst[0]
```




    10




```python
#get the first element of array
arr[0]
```




    10




```python
# M is a matrix, or a 2 dimensional array, taking two indices 
print(M)
print(M[0,0]) # element from first row first column 
print(M[1,1]) # element from second row second column
print(M[1,2]) 
```

    [[10 20 30 40]
     [50 60 70 80]]
    10
    60
    70


If we omit an index of a multidimensional array it returns the whole row
(or, in general, a N-1 dimensional array)


```python
M[1] # second row
```




    array([50, 60, 70, 80])



The same thing can be achieved with using `:` instead of an index: 


```python
M[1,:] # second row, all columns 
```




    array([50, 60, 70, 80])




```python
M[:,3] # all rows, fourth column 
```




    array([40, 80])



We can assign new values to elements in an array using indexing:


```python
M[0,0] = 1
M
```




    array([[ 1, 20, 30, 40],
           [50, 60, 70, 80]])




```python
# also works for rows and columns
M[1,:] = 0
M
```




    array([[ 1, 20, 30, 40],
           [ 0,  0,  0,  0]])



Arrays are homogeneous; i.e. all elements of an array must be of the same type



```python
#Lists are heterogeneous
lst[1] = 'a string inside a list'
lst

```




    [10, 'a string inside a list', 30, 40]




```python
#Arrays are homogeneous
arr[1] = 'a string inside an array'
```

Once an array has been created, its dtype is fixed and it can only store elements of the same type. For this example where the dtype is integer, if we store a floating point number it will be automatically converted into an integer:


```python
arr[1] = 1.234
arr
```




    array([10,  1, 30, 40], dtype=int64)



** Index slicing **

Index slicing is the technical name for the syntax `M[lower:upper:step]` to extract part of an array:


```python
A = np.array([1,2,3,4,5])
#slice from second to fourth element, step is one
A[1:3:1]
```




    array([1, 3, 5])




```python
A[1:3:1] = [-2,-3]
A
```




    array([ 1, -2, -3,  4,  5])



We can omit any of the three parameters in `M[lower:upper:step]`, by default `lower` is the beginning , `upper` is the end of the array, and `step` is one


```python
A[::2] # step is 2, lower and upper defaults to the beginning and end of the array
```




    array([ 1, -3,  5])




```python
A[:3] # first three elements
```




    array([ 1, -2, -3])




```python
A[3:] # elements from index 3
```




    array([4, 5])



Negative indices counts from the end of the array:


```python
A[-1] # the last element in the array
```




    5




```python
A[-3:] # the last three elements
```




    array([-3,  4,  5])




```python
A[::-1] #Step backwards, it returns an array with elements in reverse order
```




    array([ 5,  4, -3, -2,  1])



Index slicing works exactly the same way for multidimensional arrays, but every dimension separated by comma:


```python
M
```




    array([[ 1, 20, 30, 40],
           [ 0,  0,  0,  0]])




```python
#a block from the original array
#all rows, two central columns
M[:, 1:3]

```




    array([[20, 30],
           [ 0,  0]])




```python
# all row, skiping even columns
M[:, ::2]
```




    array([[ 1, 30],
           [ 0,  0]])



** Indexing with other arrays  (*Fancy indexing*)** 

Arrays allow for a more sophisticated kind of indexing: you can index an array with another array, and in particular with an array of boolean values.  This is particluarly useful to **filter**
information from an array that matches a certain condition.


```python
import numpy as np
arr = np.array([10,1,30,40])
print(arr)
mask = arr < 9 # construct a boolean array 
               #where i-th eleement is True if the i-th element of arr is less than 9
mask
```

    [10  1 30 40]





    array([False,  True, False, False])




```python
print('Values below 9:', arr[mask])
```

    Values below 9: [1]


The index mask can be converted to position index using the `where` function


```python
indices = np.where(mask,99999.,0.)
indices
```




    array([    0., 99999.,     0.,     0.])




```python
print('Resetting all values below 9 to 10...')
arr[arr < 9] = 10
print(arr)
arr < 9
```

    Resetting all values below 9 to 10...
    [10 10 30 40]





    array([False, False, False, False])



It is also possible to select using integer arrays that represent indexes.


```python
print(arr)
row_indices = [1, 2 ,3]
arr[row_indices]
```

    [10 10 30 40]





    array([10, 30, 40])




```python
a = np.array([2, 4, 6, 8], float) 
b = np.array([0, 0, 1, 3, 2, 1], int)  # the 0th, 0th, 1st, 3rd, 2nd, and 1st elements of a
a[b] 
```




    array([2., 2., 4., 8., 6., 4.])



For multidimensional arrays, we have to send multiple one-dimensional integer arrays to the 
selection bracket, one for each axis.


```python
a = np.array([[1, 4], [9, 16]], float) 
print(a)
b = np.array([0, 0, 1, 1, 1], int) 
c = np.array([0, 1, 1, 1, 0], int) 
a[b,c] 
```

    [[ 1.  4.]
     [ 9. 16.]]





    array([ 1.,  4., 16., 16.,  9.])



`np.where` can also be used as a conditional statement.


```python
print (a)
print (np.where(a==1.0,1,0))

```

    [[ 1.  4.]
     [ 9. 16.]]
    [[1 0]
     [0 0]]


# The machine learning pipeline

Modeling churn means to understand what keeps the customer engaged to our product. Its analysis goal is to predict or describe the **churn rate** i.e. the rate at which customer leave or cease the subscription to a service. Its value lies in the fact that engaging new customers is often more costly than retaining existing ones. For that reason subscription business-based companies usually have proactive policies towards customer retention.

In this case study, we aim at building a machine learning based model for customer churn prediction on data from a Telecom company. Each row on the dataset represents a subscribing telephone customer. Each column contains customer attributes such as phone number, call minutes used during different times of day, charges incurred for services, lifetime account duration, and whether or not the customer is still a customer.

This case is partially inspired in Eric Chiang's analysis of churn rate. Data is available from the University of California Irvine machine learning repositories data set.

## Goal
 + Implement a full machine learning pipeline.
 + Understand the concepts of training, validation, and test.


```python
%load datalist.py
```


```python
%reset -f
from datalist import *

dl = DataList()
dl.read_csv('./files/churn_curated_numerical.csv',hasHeader = False)
```


```python
#IF YOU DON'T HAVE DATALIST READY YOU CAN USE:

import pandas as pd

dl=pd.read_csv('./files/churn_curated_numerical.csv',header=-1)
```


```python
data = dl.get_values()
```


```python
data.shape[0] * data.shape[1]
```




    63327




```python
X = data[:,:-1]
y = data[:,-1]
```


```python
np.unique(y)
```




    array([0., 1.])




```python
%matplotlib inline
import numpy as np
import matplotlib.pyplot as plt
plt.pie(np.c_[len(y)-np.sum(y),np.sum(y)][0],labels=['No Churn','Churn'],colors=['r','g'],shadow=True,autopct ='%.2f' )
fig = plt.gcf()
fig.set_size_inches(6,6)
np.array([len(y)-np.sum(y),np.sum(y)])
```

## Data

Observe data


```python
import numpy as np

np.mean(X,axis=0)
np.var(X,axis=0)
```




    array([1.58532433e+03, 1.79478761e+03, 8.75182028e-02, 2.00104799e-01,
           1.87315130e+02, 2.96580639e+03, 4.02647298e+02, 8.57114045e+01,
           2.57112237e+03, 3.96791913e+02, 1.85762804e+01, 2.55694661e+03,
           3.82815581e+02, 5.17804314e+00, 7.79202952e+00, 6.05575823e+00,
           5.68002683e-01, 1.72999748e+00])



A problem in Scikit-Learn is modeled as follows:

+ Input data is structured in Numpy arrays. The size of the array is expected to be [n_samples, n_features]:

    + *n_samples*: The number of samples ($N$): each sample is an item to process (e.g. classify). A sample can be a document, a picture, a sound, a video, an astronomical object, a row in database or CSV file, or whatever you can describe with a fixed set of quantitative traits.
  
    + *n_features*: The number of features ($d$) or distinct traits that can be used to describe each item in a quantitative manner. Features are generally real-valued, but may be boolean, discrete-valued or even cathegorical.

$${\rm feature~matrix:} {\bf X}~=~\left[
\begin{matrix}
x_{11} & x_{12} & \cdots & x_{1d}\\
x_{21} & x_{22} & \cdots & x_{2d}\\
x_{31} & x_{32} & \cdots & x_{3d}\\
\vdots & \vdots & \ddots & \vdots\\
\vdots & \vdots & \ddots & \vdots\\
x_{N1} & x_{N2} & \cdots & x_{Nd}\\
\end{matrix}
\right]$$

$${\rm label~vector:} {\bf y}~=~ [y_1, y_2, y_3, \cdots y_N]$$
    

The number of features must be fixed in advance. However it can be very high dimensional (e.g. millions of features) with most of them being zeros for a given sample. 

Create and fit a decision tree (you can find it in the module sklearn.tree and the name is DecisionTreeClassifier)


```python
from sklearn import tree

clf = tree.DecisionTreeClassifier()
clf.fit(X,y)
```




    DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,
                           max_features=None, max_leaf_nodes=None,
                           min_impurity_decrease=0.0, min_impurity_split=None,
                           min_samples_leaf=1, min_samples_split=2,
                           min_weight_fraction_leaf=0.0, presort=False,
                           random_state=None, splitter='best')



Predict the data you used for training/fiting the classifer


```python
yhat = clf.predict(X)
```

<div class="alert alert-success" style = "border-radius:10px"><b>EXERCISE:</b> We need a measure of how well the classifier is performing. `yhat` is a list and our target outcome `y`is also a list. Create a measure of accuracy. </div>


```python
#Your code here
```

    accuracy: 100.0%


One sensible way of measuring the goodness of a classifier is measuring the error rate, i.e. the number of times the classifier fails divided by the total number of elements.

$$err = \frac{1}{N}\sum \mathbb{1}_{\tilde{y}!=y}$$

where $\mathbb{1}_{\text{cond}}$ is the indicator function given a condition, $\text{cond}$, defined as 

$$\mathbb{1}_{\text{cond}}=\left \{\begin{align} 1 & \quad\text{if cond = True}\\ 0 & \quad\text{otherwise} \end{align}\right.$$
Alternatively, we can report the accuracy, defined as the rate of success

$$acc = \frac{1}{N}\sum \mathbb{1}_{\tilde{y}==y}.$$

Observe that $acc = 1-err$.

`sklearn` reports this result using the method from module `metrics`, `.accuracy_score`.


```python
from sklearn import metrics

metrics.accuracy_score(y,yhat)
```




    1.0



<div class = "alert alert-info" style="border-radius:10px"> <b>QUESTION:</b> Is this a good result?</div>

<div class = "alert alert-info" style="border-radius:10px"> <b>QUESTION:</b> Is this the value we expect to have when we apply this method in production?</div>

In real applications we will train a classifier on a given data set but then apply the classifier to unseen data. Let us simulate this process by spliting the data set in two sets. We will call data we use for fiting the classifier training and data used for assessing the performance, test data.

<div class="alert alert-success" style = "border-radius:10px"><b>EXERCISE:</b> Split the data set 70% for training purposes and the rest for test purposes. You should end up with four variables `X_train`, `y_train`, `X_test`, `y_test`. <p>
<b>OTHER REQUIREMENTS:</b> Reshuffle data using a permutation of the indexes (`np.random.permutation(...)`) and set the seed of the random number generator using `np.random.seed(42)`
</div>


```python
#Your code here
```

Split data in training and set, use the module cross_validation, train_test_split , use random_state=42 as an argument for reproductibility.


```python
from sklearn import model_selection

X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, train_size=0.7, random_state=42)
```

Let us try a new algorithm, nearest neighbor, with parameter n_neighbors = 1


```python
from sklearn import neighbors

clf = neighbors.KNeighborsClassifier(n_neighbors=1)
clf.fit(X_train,y_train)
yhat = clf.predict(X_train)
from sklearn import metrics

metrics.accuracy_score(y_train,yhat)

```




    1.0




```python
yhat = clf.predict(X_test)
from sklearn import metrics

metrics.accuracy_score(y_test,yhat)
```




    0.811



<div class = "alert alert-info" style="border-radius:10px"> <b>QUESTION:</b> Is this a good result?</div>


```python
#INSERT SNOOPING CODE

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()


X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, train_size=0.7, random_state=42)

X_train_scaled = scaler.fit_transform(X_train)


clf = neighbors.KNeighborsClassifier(n_neighbors=1)
clf.fit(X_train_scaled,y_train)
yhat = clf.predict(X_train_scaled)
from sklearn import metrics

metrics.accuracy_score(y_train,yhat)


X_test_scaled = scaler.transform(X_test)

yhat = clf.predict(X_test_scaled)
from sklearn import metrics

metrics.accuracy_score(y_test,yhat)
```




    0.866




```python
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)

clf = neighbors.KNeighborsClassifier(n_neighbors=1)
clf.fit(X_train_scaled,y_train)
yhat = clf.predict(X_train_scaled)
from sklearn import metrics

metrics.accuracy_score(y_train,yhat)

```




    1.0




```python
from sklearn.preprocessing import StandardScaler

X_test_scaled = scaler.transform(X_test)

yhat = clf.predict(X_test_scaled)
from sklearn import metrics

metrics.accuracy_score(y_test,yhat)
```




    0.866



This result is ok, but how consistent is it? Maybe we have been lucky with the train-test partion. We can repeat this process for different values of the random_state (or just use random permutations) and report the average result.

<div class="alert alert-success" style = "border-radius:10px"><b>EXERCISE:</b> We want a vector of accuracies, `acc`, of shape (10,1) with the values of testing a 1-NN classifier on a `train_size=0.7` for the different random_states stored in the array `r_state`. Use `sklearn.cross_validation.train_test_split` function.
</div>


```python
r_state = [0,1,2,3,4,5,42,43,44,45]
```


```python
acc = []
for r_s in r_state:
    X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7, random_state=r_s)
    scaler = StandardScaler()
    X_train_scaled = scaler.fit_transform(X_train)

    clf = neighbors.KNeighborsClassifier(n_neighbors=1)
    clf.fit(X_train_scaled,y_train)
    yhat = clf.predict(X_train_scaled)
    from sklearn import metrics

    metrics.accuracy_score(y_train,yhat)


    X_test_scaled = scaler.transform(X_test)

    yhat = clf.predict(X_test_scaled)
    from sklearn import metrics

    acc.append(metrics.accuracy_score(y_test,yhat))
acc = np.array([acc]).T
```

Check your code with the following visualization code:


```python
import matplotlib.pyplot as plt
fig = plt.figure()
plt.boxplot(acc)
plt.scatter(np.ones(acc.shape)+0.01*np.random.normal(size=(10,1)),acc,alpha = 0.5,color='r')
```

<div class = "alert alert-info" style = "border-radius:10px"> <b>QUIZ:</b> Report the average accuracy.</div>


```python
#Your code
value = (sum(acc)/len(acc))[0]
```


```python
assert np.abs(value-0.8691)<0.0001
```

# Model selection (I)

We have tried a 1-Nearest Neighbors classifiers but we could try also different values for the Nearest Neighbors. The selection of a model between different alternatives is called model selection. We can use the same strategy as before and report accuracies for the three models. Let us do it comparintg 1-NN, 3-NN and a DecisionTree.

<div class="alert alert-success" style = "border-radius:10px"><b>EXERCISE:</b> We want a matrix of accuracies, `acc`, of shape (10,3) with the values of testing a decision tree, a 1-NN, and a 3-NN classifier on a `train_size=0.7` for the different random_states stored in the array `r_state`. 
</div>


```python
#Your code

from sklearn import metrics
from sklearn import tree
from sklearn import neighbors
from sklearn.preprocessing import StandardScaler

r_state = [0,1,2,3,4,5,42,43,44,45]

acc = np.zeros((len(r_state),3))

for i in range(len(r_state)):
    X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, train_size=0.7, random_state=r_state[i])
    
    #Your code here
    scaler = StandardScaler()
    X_train_scaled = scaler.fit_transform(X_train)
    
    clf = tree.DecisionTreeClassifier()
    clf.fit(X_train_scaled,y_train)
    X_test_scaled = scaler.transform(X_test)
    yhat_tr = clf.predict(X_test_scaled)

    clf = neighbors.KNeighborsClassifier(n_neighbors=1)
    clf.fit(X_train_scaled,y_train)
    X_test_scaled = scaler.transform(X_test)
    yhat_nn1 = clf.predict(X_test_scaled)
    
    clf = neighbors.KNeighborsClassifier(n_neighbors=3)
    clf.fit(X_train_scaled,y_train)
    X_test_scaled = scaler.transform(X_test)
    yhat_nn3 = clf.predict(X_test_scaled)    
    
    acc[i,0] = metrics.accuracy_score(y_test,yhat_tr)
    acc[i,1] = metrics.accuracy_score(y_test,yhat_nn1)
    acc[i,2] = metrics.accuracy_score(y_test,yhat_nn3)
```


```python
import matplotlib.pyplot as plt
fig = plt.figure()
plt.boxplot(acc)
plt.scatter(np.tile(np.array([1,2,3]),(10,1))+0.01*np.random.normal(size=(10,3)),acc,alpha = 0.5,color='r')
```

<div class = "alert alert-info" style = "border-radius:10px" ><b>QUIZ:</b > What is the best of the three methods?</div>

<div class = "alert alert-info" style = "border-radius:10px"><b>QUIZ:</b> What is the expected accuracy of the selected method in exploitation over unseen data? </div>


```python
acc[:,2].sum()/len(acc[:,2])
```




    0.8952



<div class="alert alert-danger" style = "border-radius:10px"><b>MAIN QUEST EXERCISE:</b> The  `breast_cancer` dataset from `datasets` (check `load_breast_cancer`) reports a set of clinical trials with the outcome of breast cancer detection. We want to build a method to predict whether a patient has a potential cancer or not according to her clinical trials.

<p>

For that purpose we will use two different models, a support vector machine and a gradient boosting machine. We will train different settings of a support vector machine (`svm.SVC`) and a single gradient boosting machine (`ensemble.GradientBoostingMachine`) with the following parameters:

<ul>
<li>
`svm.SVC(C=10.0,gamma = 1e-5,random_state=42)`
</li>
<li>
`svm.SVC(C=100.0,gamma = 1e-5,random_state=42)`
</li>
<li>
`svm.SVC(C=1000.0,gamma = 1e-6,random_state=42)`
</li>
<li>
`ensemble.GradientBoostingClassifier(random_state=42)`
</li>
</ul>
<p>
For selection purposes and accuracy evaluation we will use `model_selection.train_test_split`.
The data set will be divided using parameters `test_size = 100` and `random_state=42`. As a result of this first division we will have a big training set and a 100 samples test set. Following that and using the same settings we will divide the remaining training set into the final training set and the validation set with 100 samples again.

<p>

Prepare the following answers:

<ol>
<li>
Check the sizes of the training, validation and test sets.
</li>
<li>
Report the training accuracy of all four methods.
</li>
<li>
Report the validation accuracy for all methods.
</li>
<li>
Report the performance of all methods using the test set.
</li>
</ol>

<p>
In the light of the answers obtained from the exercise:
<ul>
<li>
Question 1: What is the size of the training set? 
</li>
<li>
Question 2: What method do you select?
</li>
<li>
Question 3: What is the expected performance of the method selected?
</li>
</ul>

</div>


```python
from sklearn.datasets import load_breast_cancer

data = load_breast_cancer()
X = data.data
y = data.target

X.shape
```




    (569, 30)




```python
#Your code
from sklearn import svm
from sklearn import ensemble

X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size = 100, random_state=42)
X_train, X_val, y_train, y_val = model_selection.train_test_split(X_train, y_train, test_size = 100, random_state=42)
    
```


```python
print(X_train.shape[0])
print(X_val.shape[0])
print(X_test.shape[0])
```

    369
    100
    100



```python
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
clf = svm.SVC(C=10.0,gamma = 1e-5,random_state=42)
clf.fit(X_train_scaled,y_train)
X_val_scaled = scaler.transform(X_val)
yhat_val = clf.predict(X_val_scaled)
print(f"svm.SVC(C=10.0,gamma = 1e-5,random_state=42): acc={metrics.accuracy_score(y_val,yhat_val)}")

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
clf = svm.SVC(C=100.0,gamma = 1e-5,random_state=42)
clf.fit(X_train_scaled,y_train)
X_val_scaled = scaler.transform(X_val)
yhat_val = clf.predict(X_val_scaled)
print(f"svm.SVC(C=100.0,gamma = 1e-5,random_state=42): acc={metrics.accuracy_score(y_val,yhat_val)}")

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
clf = svm.SVC(C=1000.0,gamma = 1e-6,random_state=42)
clf.fit(X_train_scaled,y_train)
X_val_scaled = scaler.transform(X_val)
yhat_val = clf.predict(X_val_scaled)
print(f"svm.SVC(C=1000.0,gamma = 1e-6,random_state=42): acc={metrics.accuracy_score(y_val,yhat_val)}")

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
clf = ensemble.GradientBoostingClassifier(random_state=42)
clf.fit(X_train_scaled,y_train)
X_val_scaled = scaler.transform(X_val)
yhat_val = clf.predict(X_val_scaled)
print(f"ensemble.GradientBoostingClassifier(random_state=42): acc={metrics.accuracy_score(y_val,yhat_val)}")
```

    svm.SVC(C=10.0,gamma = 1e-5,random_state=42): acc=0.76
    svm.SVC(C=100.0,gamma = 1e-5,random_state=42): acc=0.97
    svm.SVC(C=1000.0,gamma = 1e-6,random_state=42): acc=0.97
    ensemble.GradientBoostingClassifier(random_state=42): acc=0.99



```python
#Choosen is GBC with acc 99%
#lets check performance
X_test_scaled = scaler.transform(X_test)
yhat_test = clf.predict(X_test_scaled)
print(f"Acc={metrics.accuracy_score(y_test,yhat_test)}")
```

    Acc=0.96


ada659ceeda88da99fbd877404289a9013fecaf2
ipynb
Jupyter Notebook
pill2_first_steps/pill2_first_steps_in_machine_learning_student.ipynb
SaxMan96/ML-UB
67d51bcfcd37664b3cd057b52c8a4d07ee2f8b08
MIT
pill2_first_steps/pill2_first_steps_in_machine_learning_student.ipynb
SaxMan96/ML-UB
67d51bcfcd37664b3cd057b52c8a4d07ee2f8b08
MIT
pill2_first_steps/pill2_first_steps_in_machine_learning_student.ipynb
SaxMan96/ML-UB
67d51bcfcd37664b3cd057b52c8a4d07ee2f8b08
MIT
Qwen/Qwen-72B
1. YES
2. YES