# Objective: Semi-supervised classification of MNIST Dataset

Less than 1.5 % labelling in each class


```python
# load the required  libs
from tqdm import tqdm
import numpy as np
import mnist
import torch
import faiss
from matplotlib import pyplot as plt
from torch_pdegraph.utilities import *
from torch_pdegraph.operators import MeanCurv, GradPlusInfNorm, GradMinusInfNorm
```


```python
# create a smaller dataset
# We will take all the 70000 images and label only 100 in each class later
images_tr = mnist.train_images()
labels_tr = mnist.train_labels()
images_te = mnist.test_images()
labels_te = mnist.test_labels()
images = np.concatenate((images_tr,images_te),axis=0)
labels = np.concatenate((labels_tr, labels_te),axis=0)
images_flatten = np.reshape(images, (images.shape[0], images.shape[1]*images.shape[2])).astype(np.float32) * 1000
```

# Graph construction:

- In order to simply show the effectiveness of PDEs on graph,  I am only creating a simple K-NN based graphs. This may or maynot be the best graph for a given problem at hand.

- One can create graph using whatsoever apt approach or one can even use third-party network datasets and run a PDE on that graph.  PDEs are extensible to any given graph/network at hand as long as that graph has edges and weights( edge_index and edge_attr).

Although torch_cluster comes with a knn-graph method. I found it to be limited and  slow when the node-features have high dimensions. We shall be using facebook's faiss library which is blazingly fast for a KNN-graph construction.


```python
# Create the intial front(level-sets) for each class with 100 labels(seeds) in each class
"""
Initial front creation process in literature it is 
also known as intial seed or level-set creation process.
"""
Front = genInitialSeeds(**dict(labels = labels, num_seeds = 100))

# Create the Knn graph of the flattened image features and 
# assign weights to the edges
res = faiss.StandardGpuResources()
index = faiss.IndexFlatL2(images_flatten.shape[1])
gpu_index_flat = faiss.index_cpu_to_gpu(res,0,index)
gpu_index_flat.add(images_flatten/1000)
k = 30
D, I = gpu_index_flat.search(images_flatten/1000,k+1)

#Graph 
edge_index = np.vstack((I[:,1:].flatten(), np.repeat(I[:,0].flatten(),k)))
edge_attr = np.exp(-(D[:,1:].flatten()/505000))

edge_index = torch.tensor(edge_index, dtype=torch.long).to('cuda:0')
edge_attr = torch.tensor(edge_attr, dtype=torch.float32).to('cuda:0')
edge_attr = edge_attr.view(-1,1)
graph = Graph(edge_index, edge_attr)
```

    Success!


# Run a manually defined PDE:



\begin{equation}
\mathbf{x}^{n+1}_{i} = \mathbf{x}^{n}_{i} + \Delta t \kappa_w(i,\mathbf{x}^{n}) \|\nabla^{+}_w\mathbf{x}^{n}_{i}\|_{\infty}, \quad if\quad \kappa_w(i,\mathbf{x}^{n}) > 0
\end{equation}



\begin{equation}
\mathbf{x}^{n+1}_{i} = \mathbf{x}^{n}_{i} + \Delta t \kappa_w(i,\mathbf{x}^{n}) \|\nabla^{-}_w\mathbf{x}^{n}_{i}\|_{\infty}, \quad if\quad \kappa_w(i,\mathbf{x}^{n}) < 0
\end{equation}

- $\mathbf{x}_{i}$ is the node feature/signal at the $i^{th}$ node
- $\nabla^{-}_{w}$, $\nabla^{+}_{w}$ are the negative and positive directional gradients on weighted graphs respectively.
- $\kappa(i,\mathbf{x})$ is the mean curvatrue operator.

**Example:**

```python
from torch_pdegraph.operators import GradMinusInfNorm, GradPlusInfNorm, MeanCurv 
# Instantiate the operators
ope_normM = GradMinusInfNorm.OPE(graph) 
ope_normP = GradPlusInfNorm.OPE(graph)
ope_curv = MeanCurv.OPE(graph)

# Run the above explicit PDE on intial front(level-set) on graph
Ip = (ope_curv(fr) > 0.0)
Im = (ope_curv(fr) < 0.0)
fr = fr + dt * ope_curv(fr) * (ope_normP(fr) * Ip.type(torch.int) + ope_normM(fr) * Im.type(torch.int)
```


To know more ref [DEL11](https://ieeexplore.ieee.org/document/6116433), PDE level-set method in section 6.3 of [M. Toutain's thesis](https://tel.archives-ouvertes.fr/tel-01258738)


```python
# Params
itr = 500
dt = 0.05

# Instantiate the operators
ope_curv = MeanCurv.OPE(graph)
ope_normP = GradPlusInfNorm.OPE(graph)
ope_normM = GradMinusInfNorm.OPE(graph)

#Evolved front
newFront = []

#Run the custome PDE on each initial front (initial level-set)
for fr in tqdm(Front):
    fr = torch.tensor(fr, dtype=torch.float32).to('cuda:0')
    for i in range(itr):
        Ip = (ope_curv(fr) > 0.0)
        Im = (ope_curv(fr) < 0.0)
        fr = fr + dt * ope_curv(fr) * (ope_normP(fr) * Ip.type(torch.int) + ope_normM(fr) * Im.type(torch.int)) 
    newfr = fr.to('cpu')
    newFront.append(newfr.numpy())
```

    100%|██████████| 10/10 [00:24<00:00,  2.42s/it]



```python
# Now see the results
from sklearn.preprocessing import MinMaxScaler
scalar = MinMaxScaler()
norm_fr = []

for fr  in newFront:
    scalar.fit(fr) 
    norm_fr.append(scalar.transform(fr))


m = np.argmax(norm_fr, axis=0)
m = m[:,0]

import collections
from sklearn.metrics import confusion_matrix
print(f"The original number of elements in each class is : \n {collections.Counter(labels)}")
print(f"The predicted number of elements are: \n {collections.Counter(m)}")
print(f"The matrix of confusion: \n {confusion_matrix(labels,m)}")

nmask = (m == labels)
nmask = nmask.astype("int")
print("The accuracy of the classification is: \t {acc}".format(acc=np.sum(nmask)*100/len(labels)))
```

    The original number of elements in each class is : 
     Counter({1: 7877, 7: 7293, 3: 7141, 2: 6990, 9: 6958, 0: 6903, 6: 6876, 8: 6825, 4: 6824, 5: 6313})
    The predicted number of elements are: 
     Counter({1: 8618, 7: 7521, 9: 7515, 0: 7207, 6: 7076, 3: 7016, 2: 6375, 4: 6353, 5: 6329, 8: 5990})
    The matrix of confusion: 
     [[6839    7    4    1    1    9   34    4    1    3]
     [   0 7825   15    1    3    2    6    9    1   15]
     [ 114  226 6290   21   25   12   34  223   30   15]
     [  23   60   28 6717    6  136    6   78   41   46]
     [   6   88    1    0 6135    0   34   10    1  549]
     [  45   44    0   75   16 5945  111    7    6   64]
     [  44   24    0    0    8   20 6779    0    1    0]
     [   4  141   11    2   25    2    0 6997    1  110]
     [  97  167   21  117   67  187   66   59 5898  146]
     [  35   36    5   82   67   16    6  134   10 6567]]
    The accuracy of the classification is: 	 94.27428571428571


### NOTE: Here I have used a simple knn-graph construction method but with a more sophisticated method of graph creation like [two-sided tangent distance](http://www.keysers.net/daniel/files/ICPR2000.pdf) one can achieve  more than 98% of classification accuracy with 1% labeling using this PDE level-set method, as it was shown in the section 6.4.5 in the thesis of [M. Toutain](https://tel.archives-ouvertes.fr/tel-01258738)

0132502c42e16257364cbf48a290a972fb76173d
ipynb
Jupyter Notebook
applications/4_classification.ipynb
aGIToz/Pytorch_pdegraph
fade6817e437b606c43221a5ca13bdaeec563fff
MIT
2020-08-24T09:04:48.000Z
2022-03-19T03:46:07.000Z
applications/4_classification.ipynb
aGIToz/Pytorch_pdegraph
fade6817e437b606c43221a5ca13bdaeec563fff
MIT
applications/4_classification.ipynb
aGIToz/Pytorch_pdegraph
fade6817e437b606c43221a5ca13bdaeec563fff
MIT
2020-08-27T15:53:02.000Z
2021-09-13T08:36:27.000Z
Qwen/Qwen-72B
1. YES
2. YES