<kaggle_start><data_title>Skin Cancer MNIST: HAM10000<data_description># Overview
Another more interesting than digit classification dataset to use to get biology and medicine students more excited about machine learning and image processing. 


## Original Data  Source
- Original Challenge: https://challenge2018.isic-archive.com
- https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T
[1] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, Allan Halpern: “Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)”, 2018; https://arxiv.org/abs/1902.03368

[2] Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 doi:10.1038/sdata.2018.161 (2018).

## From Authors

Training of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available dataset of dermatoscopic images. We tackle this problem by releasing the HAM10000 ("Human Against Machine with 10000 training images") dataset. We collected dermatoscopic images from different populations, acquired and stored by different modalities. The final dataset consists of 10015 dermatoscopic images which can serve as a training set for academic machine learning purposes. Cases include a representative collection of all important diagnostic categories in the realm of pigmented lesions: Actinic keratoses and intraepithelial carcinoma / Bowen's disease (akiec), basal cell carcinoma (bcc), benign keratosis-like lesions (solar lentigines / seborrheic keratoses and lichen-planus like keratoses, bkl), dermatofibroma (df), melanoma (mel), melanocytic nevi (nv) and vascular lesions (angiomas, angiokeratomas, pyogenic granulomas and hemorrhage, vasc).

More than 50% of lesions are confirmed through histopathology (histo), the ground truth for the rest of the cases is either follow-up examination (follow_up), expert consensus (consensus), or confirmation by in-vivo confocal microscopy (confocal). The dataset includes lesions with multiple images, which can be tracked by the lesion_id-column within the HAM10000_metadata file.

The test set is not public, but the evaluation server remains running (see the challenge website). Any publications written using the HAM10000 data should be evaluated on the official test set hosted there, so that methods can be fairly compared.<data_name>skin-cancer-mnist-ham10000
<code>import numpy as np
import pandas as pd
import os

print(os.listdir("../input"))
import pandas as pd

df = pd.read_csv("../input/skin-cancer-mnist-ham10000/HAM10000_metadata.csv")
df.head()
from os.path import isfile
from PIL import Image as pil_image

df["num_images"] = df.groupby("lesion_id")["image_id"].transform("count")
classes = df["dx"].unique()
labeldict = {}
for num, name in enumerate(classes):
    labeldict[name] = num
df["dx_id"] = df["dx"].map(lambda x: labeldict[x])


def expand_path(p):
    if isfile(
        "../input/skin-cancer-mnist-ham10000/ham10000_images_part_1/" + p + ".jpg"
    ):
        return (
            "../input/skin-cancer-mnist-ham10000/ham10000_images_part_1/" + p + ".jpg"
        )
    if isfile(
        "../input/skin-cancer-mnist-ham10000/ham10000_images_part_2/" + p + ".jpg"
    ):
        return (
            "../input/skin-cancer-mnist-ham10000/ham10000_images_part_2/" + p + ".jpg"
        )
    return p


df["image_path"] = df["image_id"]
df["image_path"] = df["image_path"].apply(expand_path)
df["images"] = df["image_path"].map(
    lambda x: np.asarray(pil_image.open(x).resize((150, 112)))
)
df.head()
from sklearn.model_selection import train_test_split

df_single = df[df["num_images"] == 1]
trainset1, testset = train_test_split(df_single, test_size=0.2, random_state=80)
trainset2, validationset = train_test_split(trainset1, test_size=0.2, random_state=600)
trainset3 = df[df["num_images"] != 1]
frames = [trainset2, trainset3]
trainset = pd.concat(frames)


def prepareimages(images):
    # images is a list of images
    images = np.asarray(images).astype(np.float64)
    images = images[:, :, :, ::-1]
    m0 = np.mean(images[:, :, :, 0])
    m1 = np.mean(images[:, :, :, 1])
    m2 = np.mean(images[:, :, :, 2])
    images[:, :, :, 0] -= m0
    images[:, :, :, 1] -= m1
    images[:, :, :, 2] -= m2
    return images


trainimages = prepareimages(list(trainset["images"]))
testimages = prepareimages(list(testset["images"]))
validationimages = prepareimages(list(validationset["images"]))
trainlabels = np.asarray(trainset["dx_id"])
testlabels = np.asarray(testset["dx_id"])
validationlabels = np.asarray(validationset["dx_id"])
import matplotlib.pyplot as plt

plt.hist(trainlabels, bins=7, density=True)
plt.show()
plt.hist(validationlabels, bins=7, density=True)
plt.show()
plt.hist(testlabels, bins=7, density=True)
plt.show()
from keras.preprocessing.image import ImageDataGenerator

trainimages = trainimages.reshape(trainimages.shape[0], *(112, 150, 3))
data_gen = ImageDataGenerator(
    rotation_range=90, zoom_range=0.1, width_shift_range=0.1, height_shift_range=0.1
)
data_gen.fit(trainimages)
from keras.models import Sequential
from keras import optimizers
from keras.layers import Dense, Conv2D, Dropout, Flatten, MaxPooling2D
from keras.applications import ResNet50
from keras import regularizers
import numpy as np

model = Sequential()
num_labels = 7
base_model = ResNet50(
    include_top=False, input_shape=(224, 224, 3), pooling="avg", weights="imagenet"
)
model = Sequential()
model.add(base_model)
model.add(Dropout(0.5))
model.add(Dense(128, activation="relu", kernel_regularizer=regularizers.l2(0.02)))
model.add(Dropout(0.5))
model.add(
    Dense(num_labels, activation="softmax", kernel_regularizer=regularizers.l2(0.02))
)
for layer in base_model.layers:
    layer.trainable = False
for layer in base_model.layers[-22:]:
    layer.trainable = True
model.summary()
from tensorflow.keras.optimizers import Adam

optimizer = Adam(
    lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=1e-6, amsgrad=False
)
model.compile(
    optimizer=optimizer, loss="sparse_categorical_crossentropy", metrics=["accuracy"]
)
# Fit the model
import keras
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint


class CustomModelCheckPoint(keras.callbacks.Callback):
    def __init__(self, **kargs):
        super(CustomModelCheckPoint, self).__init__(**kargs)
        self.epoch_accuracy = {}  # loss at given epoch
        self.epoch_loss = {}  # accuracy at given epoch

    def on_epoch_begin(self, epoch, logs={}):
        # Things done on beginning of epoch.
        return

    def on_epoch_end(self, epoch, logs={}):
        # things done on end of the epoch
        self.epoch_accuracy[epoch] = logs.get("acc")
        self.epoch_loss[epoch] = logs.get("loss")
        self.model.save_weights("../output/resnet50/name-of-model-%d.h5" % epoch)


cb_early_stopper = EarlyStopping(monitor="val_loss", patience=4)
cb_checkpointer = ModelCheckpoint(
    filepath="../working/best.hdf5",
    monitor="val_loss",
    save_best_only=True,
    mode="auto",
)
epochs = 30
batch_size = 20
trainhistory = model.fit_generator(
    data_gen.flow(trainimages, trainlabels, batch_size=batch_size),
    epochs=epochs,
    validation_data=(validationimages, validationlabels),
    verbose=1,
    steps_per_epoch=trainimages.shape[0] // batch_size,
    callbacks=[cb_checkpointer, cb_early_stopper],
)
import matplotlib.pyplot as plt

acc = trainhistory.history["accuracy"]
val_acc = trainhistory.history["val_accuracy"]
loss = trainhistory.history["loss"]
val_loss = trainhistory.history["val_loss"]
epochs = range(1, len(acc) + 1)
plt.plot(epochs, loss, "", label="Training loss")
plt.plot(epochs, val_loss, "", label="Validation loss")
plt.title("Training and validation loss")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.legend()
plt.figure()
plt.plot(epochs, acc, "", label="Training accuracy")
plt.plot(epochs, val_acc, "", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.xlabel("Epoch")
plt.ylabel("Accuracy")
plt.legend()
model.load_weights("../working/best.hdf5")
test_loss, test_acc = model.evaluate(testimages, testlabels, verbose=1)
print("test_accuracy = %f  ;  test_loss = %f" % (test_acc, test_loss))
from sklearn.metrics import confusion_matrix

train_pred = model.predict(trainimages)
train_pred_classes = np.argmax(train_pred, axis=1)
test_pred = model.predict(testimages)
# Convert predictions classes to one hot vectors
test_pred_classes = np.argmax(test_pred, axis=1)
confusionmatrix = confusion_matrix(testlabels, test_pred_classes)
confusionmatrix
from sklearn.metrics import classification_report

labels = labeldict.keys()
# Generate a classification report
trainreport = classification_report(
    trainlabels, train_pred_classes, target_names=list(labels)
)
testreport = classification_report(
    testlabels, test_pred_classes, target_names=list(labels)
)
print(trainreport)
print(testreport)
<|endoftext|>
[{"Id": 51821409, "ScriptId": 14216919, "ParentScriptVersionId": NaN, "ScriptLanguageId": 9, "AuthorUserId": 2669586, "CreationDate": "01/14/2021 02:49:07", "VersionNumber": 2.0, "Title": "skin_cancer-resnet50", "EvaluationDate": "01/14/2021", "IsChange": false, "TotalLines": 182.0, "LinesInsertedFromPrevious": 0.0, "LinesChangedFromPrevious": 0.0, "LinesUnchangedFromPrevious": 182.0, "LinesInsertedFromFork": NaN, "LinesDeletedFromFork": NaN, "LinesChangedFromFork": NaN, "LinesUnchangedFromFork": NaN, "TotalVotes": 0}]
[{"Id": 64608613, "KernelVersionId": 51821409, "SourceDatasetVersionId": 104884}]
[{"Id": 104884, "DatasetId": 54339, "DatasourceVersionId": 111874, "CreatorUserId": 67483, "LicenseName": "CC BY-NC-SA 4.0", "CreationDate": "09/20/2018 20:36:13", "VersionNumber": 2.0, "Title": "Skin Cancer MNIST: HAM10000", "Slug": "skin-cancer-mnist-ham10000", "Subtitle": "a large collection of multi-source dermatoscopic images of pigmented lesions", "Description": "# Overview\nAnother more interesting than digit classification dataset to use to get biology and medicine students more excited about machine learning and image processing. \n\n\n## Original Data  Source\n- Original Challenge: https://challenge2018.isic-archive.com\n- https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T\n[1] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, Allan Halpern: \u201cSkin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)\u201d, 2018; https://arxiv.org/abs/1902.03368\n\n[2] Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 doi:10.1038/sdata.2018.161 (2018).\n\n## From Authors\n\nTraining of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available dataset of dermatoscopic images. We tackle this problem by releasing the HAM10000 (\"Human Against Machine with 10000 training images\") dataset. We collected dermatoscopic images from different populations, acquired and stored by different modalities. The final dataset consists of 10015 dermatoscopic images which can serve as a training set for academic machine learning purposes. Cases include a representative collection of all important diagnostic categories in the realm of pigmented lesions: Actinic keratoses and intraepithelial carcinoma / Bowen's disease (akiec), basal cell carcinoma (bcc), benign keratosis-like lesions (solar lentigines / seborrheic keratoses and lichen-planus like keratoses, bkl), dermatofibroma (df), melanoma (mel), melanocytic nevi (nv) and vascular lesions (angiomas, angiokeratomas, pyogenic granulomas and hemorrhage, vasc).\n\nMore than 50% of lesions are confirmed through histopathology (histo), the ground truth for the rest of the cases is either follow-up examination (follow_up), expert consensus (consensus), or confirmation by in-vivo confocal microscopy (confocal). The dataset includes lesions with multiple images, which can be tracked by the lesion_id-column within the HAM10000_metadata file.\n\nThe test set is not public, but the evaluation server remains running (see the challenge website). Any publications written using the HAM10000 data should be evaluated on the official test set hosted there, so that methods can be fairly compared.", "VersionNotes": "addding mnist-style csv files", "TotalCompressedBytes": 2903343040.0, "TotalUncompressedBytes": 2808586677.0}]
[{"Id": 54339, "CreatorUserId": 67483, "OwnerUserId": 67483.0, "OwnerOrganizationId": NaN, "CurrentDatasetVersionId": 104884.0, "CurrentDatasourceVersionId": 111874.0, "ForumId": 63066, "Type": 2, "CreationDate": "09/19/2018 13:42:20", "LastActivityDate": "09/19/2018", "TotalViews": 703251, "TotalDownloads": 90643, "TotalVotes": 1616, "TotalKernels": 430}]
import numpy as np
import pandas as pd
import os

print(os.listdir("../input"))
import pandas as pd

df = pd.read_csv("../input/skin-cancer-mnist-ham10000/HAM10000_metadata.csv")
df.head()
from os.path import isfile
from PIL import Image as pil_image

df["num_images"] = df.groupby("lesion_id")["image_id"].transform("count")
classes = df["dx"].unique()
labeldict = {}
for num, name in enumerate(classes):
    labeldict[name] = num
df["dx_id"] = df["dx"].map(lambda x: labeldict[x])


def expand_path(p):
    if isfile(
        "../input/skin-cancer-mnist-ham10000/ham10000_images_part_1/" + p + ".jpg"
    ):
        return (
            "../input/skin-cancer-mnist-ham10000/ham10000_images_part_1/" + p + ".jpg"
        )
    if isfile(
        "../input/skin-cancer-mnist-ham10000/ham10000_images_part_2/" + p + ".jpg"
    ):
        return (
            "../input/skin-cancer-mnist-ham10000/ham10000_images_part_2/" + p + ".jpg"
        )
    return p


df["image_path"] = df["image_id"]
df["image_path"] = df["image_path"].apply(expand_path)
df["images"] = df["image_path"].map(
    lambda x: np.asarray(pil_image.open(x).resize((150, 112)))
)
df.head()
from sklearn.model_selection import train_test_split

df_single = df[df["num_images"] == 1]
trainset1, testset = train_test_split(df_single, test_size=0.2, random_state=80)
trainset2, validationset = train_test_split(trainset1, test_size=0.2, random_state=600)
trainset3 = df[df["num_images"] != 1]
frames = [trainset2, trainset3]
trainset = pd.concat(frames)


def prepareimages(images):
    # images is a list of images
    images = np.asarray(images).astype(np.float64)
    images = images[:, :, :, ::-1]
    m0 = np.mean(images[:, :, :, 0])
    m1 = np.mean(images[:, :, :, 1])
    m2 = np.mean(images[:, :, :, 2])
    images[:, :, :, 0] -= m0
    images[:, :, :, 1] -= m1
    images[:, :, :, 2] -= m2
    return images


trainimages = prepareimages(list(trainset["images"]))
testimages = prepareimages(list(testset["images"]))
validationimages = prepareimages(list(validationset["images"]))
trainlabels = np.asarray(trainset["dx_id"])
testlabels = np.asarray(testset["dx_id"])
validationlabels = np.asarray(validationset["dx_id"])
import matplotlib.pyplot as plt

plt.hist(trainlabels, bins=7, density=True)
plt.show()
plt.hist(validationlabels, bins=7, density=True)
plt.show()
plt.hist(testlabels, bins=7, density=True)
plt.show()
from keras.preprocessing.image import ImageDataGenerator

trainimages = trainimages.reshape(trainimages.shape[0], *(112, 150, 3))
data_gen = ImageDataGenerator(
    rotation_range=90, zoom_range=0.1, width_shift_range=0.1, height_shift_range=0.1
)
data_gen.fit(trainimages)
from keras.models import Sequential
from keras import optimizers
from keras.layers import Dense, Conv2D, Dropout, Flatten, MaxPooling2D
from keras.applications import ResNet50
from keras import regularizers
import numpy as np

model = Sequential()
num_labels = 7
base_model = ResNet50(
    include_top=False, input_shape=(224, 224, 3), pooling="avg", weights="imagenet"
)
model = Sequential()
model.add(base_model)
model.add(Dropout(0.5))
model.add(Dense(128, activation="relu", kernel_regularizer=regularizers.l2(0.02)))
model.add(Dropout(0.5))
model.add(
    Dense(num_labels, activation="softmax", kernel_regularizer=regularizers.l2(0.02))
)
for layer in base_model.layers:
    layer.trainable = False
for layer in base_model.layers[-22:]:
    layer.trainable = True
model.summary()
from tensorflow.keras.optimizers import Adam

optimizer = Adam(
    lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=1e-6, amsgrad=False
)
model.compile(
    optimizer=optimizer, loss="sparse_categorical_crossentropy", metrics=["accuracy"]
)
# Fit the model
import keras
from tensorflow.python.keras.callbacks import EarlyStopping, ModelCheckpoint


class CustomModelCheckPoint(keras.callbacks.Callback):
    def __init__(self, **kargs):
        super(CustomModelCheckPoint, self).__init__(**kargs)
        self.epoch_accuracy = {}  # loss at given epoch
        self.epoch_loss = {}  # accuracy at given epoch

    def on_epoch_begin(self, epoch, logs={}):
        # Things done on beginning of epoch.
        return

    def on_epoch_end(self, epoch, logs={}):
        # things done on end of the epoch
        self.epoch_accuracy[epoch] = logs.get("acc")
        self.epoch_loss[epoch] = logs.get("loss")
        self.model.save_weights("../output/resnet50/name-of-model-%d.h5" % epoch)


cb_early_stopper = EarlyStopping(monitor="val_loss", patience=4)
cb_checkpointer = ModelCheckpoint(
    filepath="../working/best.hdf5",
    monitor="val_loss",
    save_best_only=True,
    mode="auto",
)
epochs = 30
batch_size = 20
trainhistory = model.fit_generator(
    data_gen.flow(trainimages, trainlabels, batch_size=batch_size),
    epochs=epochs,
    validation_data=(validationimages, validationlabels),
    verbose=1,
    steps_per_epoch=trainimages.shape[0] // batch_size,
    callbacks=[cb_checkpointer, cb_early_stopper],
)
import matplotlib.pyplot as plt

acc = trainhistory.history["accuracy"]
val_acc = trainhistory.history["val_accuracy"]
loss = trainhistory.history["loss"]
val_loss = trainhistory.history["val_loss"]
epochs = range(1, len(acc) + 1)
plt.plot(epochs, loss, "", label="Training loss")
plt.plot(epochs, val_loss, "", label="Validation loss")
plt.title("Training and validation loss")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.legend()
plt.figure()
plt.plot(epochs, acc, "", label="Training accuracy")
plt.plot(epochs, val_acc, "", label="Validation accuracy")
plt.title("Training and validation accuracy")
plt.xlabel("Epoch")
plt.ylabel("Accuracy")
plt.legend()
model.load_weights("../working/best.hdf5")
test_loss, test_acc = model.evaluate(testimages, testlabels, verbose=1)
print("test_accuracy = %f  ;  test_loss = %f" % (test_acc, test_loss))
from sklearn.metrics import confusion_matrix

train_pred = model.predict(trainimages)
train_pred_classes = np.argmax(train_pred, axis=1)
test_pred = model.predict(testimages)
# Convert predictions classes to one hot vectors
test_pred_classes = np.argmax(test_pred, axis=1)
confusionmatrix = confusion_matrix(testlabels, test_pred_classes)
confusionmatrix
from sklearn.metrics import classification_report

labels = labeldict.keys()
# Generate a classification report
trainreport = classification_report(
    trainlabels, train_pred_classes, target_names=list(labels)
)
testreport = classification_report(
    testlabels, test_pred_classes, target_names=list(labels)
)
print(trainreport)
print(testreport)
