application

.gitignore

@@ -0,0 +1,9 @@
+dataset/
+*/__pycache__/
+__pycache__
+src/__pycache__
+logs/
+src/__pycache__/*
+**/__pycache__/
+notebook/__pycache__/*
+

README.md

@@ -10,4 +10,132 @@ pinned: false
 license: openrail
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Amazon review sentiment analysis
+![GitHub Repo stars](https://img.shields.io/github/stars/tikendraw/Amazon-review-sentiment-analysis?style=flat&logo=github&logoColor=white&label=Github%20Stars)
+
+Welcome to the Amazon Review Sentiment Analysis project! This repository contains code for training a sentiment analysis model on a large dataset of Amazon reviews using Long Short-Term Memory (LSTM) neural networks. The trained model can predict the sentiment (positive or negative) of Amazon reviews. The dataset used for training consists of over 2 million reviews, totaling 2.6 GB of data.
+
+<img src='https://img.shields.io/badge/TensorFlow-FF6F00?style=for-the-badge&logo=tensorflow&logoColor=white'>
+
+<img src='https://img.shields.io/badge/scikit--learn-%23F7931E.svg?style=for-the-badge&logo=scikit-learn&logoColor=white'>
+
+<img src='https://img.shields.io/badge/Polars-CD792C.svg?style=for-the-badge&logo=Polars&logoColor=white'>
+
+
+## Table of Contents
+* Introduction
+* Dataset
+* Model
+* Getting Started
+    * Prerequisites
+    * Training
+    * Prediction
+    * Running the Streamlit App
+* Contributing
+* Acknowledgements
+
+## Introduction
+Sentiment analysis is the process of determining the sentiment or emotion expressed in a piece of text. In this project, we focus on predicting whether Amazon reviews are positive or negative based on their text content. We use LSTM neural networks, a type of recurrent neural network (RNN), to capture the sequential patterns in the text data and make accurate sentiment predictions.
+
+## Dataset
+The dataset used for this project is a massive collection of Amazon reviews, comprising more than 2 million reviews with a total size of 2.6 GB. The dataset is [ here](https://www.kaggle.com/datasets/kritanjalijain/amazon-reviews). It contains both positive and negative reviews, making it suitable for training a sentiment analysis model.
+
+### Challenges 
+* Dataset is very larget (2.6 GB) with 2.6 Million Reviews
+* Machine's resources are limiting as loading multiple variables with processed data is eating up RAM
+
+### Work Arounds
+* Used polars for data manipulation and Preprocessings ( Uses Parallel computation, Doesn't load data on memory)
+
+## Model
+The sentiment analysis model is built using TensorFlow and Keras libraries. We employ LSTM layers to effectively capture the sequential nature of text data. The model is trained on the labeled Amazon reviews dataset, and its performance is evaluated using various metrics such as accuracy, precision, recall, and F1-score.
+
+## Model architectures
+```
+Model: "model_lstm"
+_________________________________________________________________
+ Layer (type)                Output Shape              Param #   
+=================================================================
+ input_3 (InputLayer)        [(None, 175)]             0         
+                                                                 
+ embedding_layer (Embedding)  (None, 175, 8)           2400000   
+                                                                 
+ lstm_layer_1 (LSTM)         (None, 175, 16)           1600      
+                                                                 
+ lstm_layer_2 (LSTM)         (None, 16)                2112      
+                                                                 
+ dropout_layer (Dropout)     (None, 16)                0         
+                                                                 
+ dense_layer_1 (Dense)       (None, 64)                1088      
+                                                                 
+ dense_layer_2_final (Dense)  (None, 1)                65        
+                                                                 
+=================================================================
+Total params: 2,404,865
+Trainable params: 2,404,865
+Non-trainable params: 0
+_________________________________________________________________
+```
+## Model Performance
+
+| Model             | Accuracy           | Precision          | Recall             | F1                 | Description                                      |
+|-------------------|--------------------|--------------------|--------------------|--------------------|--------------------------------------------------|
+| model0: Naive Bayes | 84.79% | 84.82% | 84.79% | 84.79% |  |
+| model1: **LSTM**(in use)      | 94.06%   | 94.06% | 94.06% | 94.06% | small lstm model with vectorizer and embedding layer |
+
+## Getting Started
+Follow these steps to get started with the project:
+
+### Prerequisites
+* Python 3.x
+* TensorFlow
+* Keras
+* Polars
+* Streamlit
+
+You can install the required dependencies using the following command:
+
+```
+pip install -r requirements.txt
+```
+
+### Training
+To train the LSTM model, run the train.py script:
+
+```
+python3 train.py
+```
+This script will preprocess the dataset, train the model, and save the trained weights to disk.
+
+### Prediction
+
+To use the trained model for making predictions on new reviews, run the predict.py script:
+
+```
+python3 predict.py
+```
+### Running the Streamlit App
+We've also provided a user-friendly Streamlit app to interact with the trained
+Model. Run the app using the following command:
+```
+streamlit run app.py
+```
+This will launch a local web app where you can input your own Amazon review and see the model's sentiment prediction.
+
+## Contributing
+Contributions are welcome! If you find any issues or have suggestions for improvements, please feel free to open an issue or create a pull request.
+
+
+## Acknowledgements
+We would like to express our gratitude to the open-source community for providing invaluable resources and tools that made this project possible.
+
+Don't Forget to Star!
+If you find this project interesting or useful, please consider starring the repository. Your support is greatly appreciated!
+
+Star
+
+Happy coding!
+
+Your Name
+Your Contact Info
+Date

app.py

@@ -0,0 +1,90 @@
+import os
+import webbrowser
+from pathlib import Path
+
+import streamlit as st
+import tensorflow as tf
+
+import config
+from src import data_preprocessing, utils
+
+MODEL_PATH = Path(config.MODEL_DIR) / config.MODEL_FILENAME
+VECTORIZER_PATH = Path(config.MODEL_DIR) / config.TEXT_VECTOR_FILENAME
+
+
+
+    
+
+def load_model_and_vectorizer(vectorizer_path, model_path):
+    try:
+        text_vectorizer = utils.load_text_vectorizer(vectorizer_path)
+        lstm_model = tf.keras.models.load_model(model_path)
+        return text_vectorizer, lstm_model
+    except Exception as e:
+        return None, None
+
+
+def predict_sentiment(title, text, text_vectorizer, lstm_model):
+    review = f'{title} {text}' # concatenate the title and text
+    clean_review = data_preprocessing.clean_text(review)
+    review_sequence = text_vectorizer([clean_review])
+    prediction = lstm_model.predict(review_sequence)
+    sentiment_score = prediction[0][0]
+    sentiment_label = 'Positive' if sentiment_score >= 0.5 else 'Negative'
+    return sentiment_label, sentiment_score
+
+# Introduction and Project Information
+st.title("Amazon Review Sentiment Analysis")
+st.write("This is a Streamlit app for performing sentiment analysis on Amazon reviews.")
+st.write("Enter the title and text of the review to analyze its sentiment.")
+
+# User Inputs
+review_title = st.text_input("Enter the review title:")
+review_text = st.text_area("Enter the review text:(required)")
+
+submit = st.button("Analyze Sentiment")
+
+
+
+text_vectorizer, lstm_model = load_model_and_vectorizer( VECTORIZER_PATH, MODEL_PATH)
+if text_vectorizer is None or lstm_model is None:
+    st.error('Could not load text vectorizer and model. Aborting prediction.')
+
+# Perform Sentiment Analysis
+if submit:
+    with st.spinner():
+        sentiment_label, sentiment_score = predict_sentiment(review_title, review_text, text_vectorizer, lstm_model)
+        new_sentiment_score= abs(0.5 - sentiment_score)*2
+
+        if sentiment_score >=0.5:
+            st.success(f"Sentiment: {sentiment_label} (Score: {new_sentiment_score:.2f})")
+        else:
+            st.error(f"Sentiment: {sentiment_label} (Score: {new_sentiment_score:.2f})")
+                
+                
+# Project Usage and Links
+st.sidebar.write("## Project Usage")
+st.sidebar.write("This project performs sentiment analysis on Amazon reviews to determine whether a review's sentiment is positive or negative.")
+st.sidebar.write("## GitHub Repository")
+st.sidebar.write("Source Code here [GitHub repository](https://github.com/tikendraw/Amazon-review-sentiment-analysis).")
+st.sidebar.write("If you have any feedback or suggestions, feel free to open an issue or a pull request.")
+st.sidebar.write("## Like the Project?")
+st.sidebar.write("If you find this project interesting or useful, don't forget to give it a star on GitHub!")
+st.sidebar.markdown('![GitHub Repo stars](https://img.shields.io/github/stars/tikendraw/Amazon-review-sentiment-analysis?style=flat&logo=github&logoColor=white&label=Github%20Stars)', unsafe_allow_html=True)
+
+
+st.sidebar.write('### Created by:')
+c1, c2 = st.sidebar.columns([4,4])
+c1.image('./src/me.jpg', width=150)
+c2.write('### Tikendra Kumar Sahu')
+st.sidebar.write('Data Science Enthusiast')
+
+if st.sidebar.button('Github'):
+    webbrowser.open('https://github.com/tikendraw')
+
+if st.sidebar.button('LinkdIn'):
+    webbrowser.open('https://www.linkedin.com/in/tikendraw/')
+        
+if st.sidebar.button('Instagram'):
+    webbrowser.open('https://www.instagram.com/tikendraw/')
+    

config.py

@@ -0,0 +1,33 @@
+import os
+from pathlib import Path
+
+cur_dir= Path(os.getcwd())
+
+# Data paths
+DATA_DIR = cur_dir / "dataset"
+PREPROCESSED_DATA_PATH = DATA_DIR / "preprocessed_df.csv"
+
+# Paths
+MODEL_DIR = cur_dir / "model"
+LOG_DIR = cur_dir / "logs"
+VECTORIZE_PATH = cur_dir / 'model' / 'text_vectorizer.pkl'
+
+TEXT_VECTOR_FILENAME = "text_vectorizer.pkl"
+MODEL_FILENAME = "full_model.h5"
+COUNTER_NAME  = "counter.pkl"
+
+# Text Vectorizer hyperparameters
+MAX_TOKEN = 100_000  # don't change this
+OUTPUT_SEQUENCE_LENGTH = 175 # don't change this
+
+# Model hyperparameters
+BATCH_SIZE = 32
+DIM = 8
+EPOCHS = 10
+TRAIN_SIZE = 0.05
+TEST_SIZE = 0.01
+LEARNING_RATE = 0.002
+RANDOM_STATE = 42
+SEED = 42
+#callback
+EARLY_STOPPING_PATIENCE = 2

model/counter.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dafbc639b1e553ae24200370ca9f49e1d6779ba72c22573ae9b78ae15aff772e
+size 12171041

model/full_model.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:560d2c40c82de35d9238a85dedae6b85a5df85617212c12df67bef32825ff83f
+size 9714960

model/model_weights.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b5912bd4d93874c16f8eb4269dd86d1226c8c5d20ddec48aa8d6ce326b3da027
+size 3245400

model/text_vectorizer.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:40f64980ca826dde2aeb0d092e7e47ed633ec13eb0a079be4b280c7ad0c7ea3f
+size 1074029

predict.py

@@ -0,0 +1,45 @@
+import os
+import logging
+import tensorflow as tf
+from pathlib import Path
+from src.utils import configure_logging, load_model_and_vectorizer
+
+from src.data_preprocessing import clean_text
+import config
+from tensorflow.keras.layers import TextVectorization
+
+    
+# constants
+DATA_DIR = Path(os.getcwd()) / 'dataset'
+DATA_PATH = DATA_DIR / 'preprocessed_df.csv'
+MODEL_PATH = Path(config.MODEL_DIR) / config.MODEL_FILENAME
+VECTORIZER_PATH = Path(config.MODEL_DIR) / config.TEXT_VECTOR_FILENAME
+COUNTER_PATH = Path(config.MODEL_DIR) / config.COUNTER_NAME
+
+
+def predict_sentiment(title, text, text_vectorizer, lstm_model):
+    review = f'{title} {text}' # concatenate the title and text
+    clean_review = clean_text(review)
+    review_sequence = text_vectorizer([clean_review])
+    prediction = lstm_model.predict(review_sequence)
+    sentiment_score = prediction[0][0]
+    sentiment_label = 'Positive' if sentiment_score >= 0.5 else 'Negative'
+    return sentiment_label, sentiment_score
+
+def main():
+    configure_logging(config.LOG_DIR, "prediction_log.txt", logging.INFO)
+    text_vectorizer, lstm_model = load_model_and_vectorizer(VECTORIZER_PATH, MODEL_PATH)
+    
+    if text_vectorizer is None or lstm_model is None:
+        logging.error('Could not load text vectorizer and model. Aborting prediction.')
+        return
+    
+    title = input("Enter the title of the review: ")
+    text = input("Enter the text of the review: ")
+    
+    sentiment_label, sentiment_score = predict_sentiment(title, text, text_vectorizer, lstm_model)
+    logging.debug(f'\nReview title: {title} \nReview text: {text}')
+    logging.info(f'Review Sentiment: {sentiment_label} (Score: {sentiment_score:.4f})')
+
+if __name__ == "__main__":
+    main()

requirements.txt

@@ -0,0 +1,6 @@
+contractions
+tensorflow
+sklearn
+funcyou
+polars
+scikit-learn

src/__init__.py

@@ -0,0 +1 @@
+__all__ = ['data_preprocessing','make_dataset','model', 'utils']

src/data_preprocessing.py

@@ -0,0 +1,68 @@
+
+import re
+from pathlib import Path
+
+import polars as pl
+from sklearn.model_selection import train_test_split
+
+
+def preprocess_data(data_dir:Path):
+    # Read the CSV file using Polars
+    df = pl.read_csv(data_dir / 'train.csv', new_columns=['polarity', 'title', 'text'])
+
+    assert df['polarity'].max()==2
+    assert df['polarity'].min()==1
+
+    # Drop rows with null values
+    df.drop_nulls()
+
+    # Map polarity to binary values (0 for negative, 1 for positive)
+    df = df.with_columns([
+        pl.col('polarity').apply(lambda x: 0 if x == 1 else 1)
+    ])
+
+    # Cast polarity column to Int16
+    df = df.with_columns([
+        pl.col('polarity').cast(pl.Int16, strict=False)
+    ])
+
+    # Combine title and text columns to create the review column
+    df = df.with_columns([
+        (pl.col('title') + ' ' + pl.col('text')).alias('review')
+    ])
+    
+    df = df.with_columns([
+	(pl.col('review').str().lower())
+	])
+	
+    # Select relevant columns
+    df = df.select(['review', 'polarity'])
+
+    # Perform text cleaning using a function
+    df = df.with_columns([
+        pl.col('review').apply(clean_text)
+    ])
+
+    df.write_csv(data_dir/'preprocessed_df.csv')
+    
+    
+
+import re
+
+import contractions
+
+# Compile the regular expressions outside the function for better performance
+PUNCTUATION_REGEX = re.compile(r'[^\w\s]')
+DIGIT_REGEX = re.compile(r'\d')
+SPECIAL_CHARACTERS_REGEX = re.compile(r'[#,@,&]')
+MULTIPLE_SPACES_REGEX = re.compile(r'\s+')
+
+def clean_text(x: str) -> str:
+    expanded_text = contractions.fix(x)  # Expand contractions
+    text = PUNCTUATION_REGEX.sub(' ', expanded_text.lower())  # Remove punctuation after lowering
+    text = DIGIT_REGEX.sub('', text)  # Remove digits
+    # Remove special characters (#,@,&)
+    text = SPECIAL_CHARACTERS_REGEX.sub('', text)
+    # Remove multiple spaces with single space
+    text = MULTIPLE_SPACES_REGEX.sub(' ', text)
+    return text.strip()

src/make_dataset.py

@@ -0,0 +1,60 @@
+# train.py
+import sys
+
+import tensorflow as tf
+
+# def create_datasets(x_train, y_train, text_vectorizer, batch_size):
+#     print('Building slices...')
+#     train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(batch_size)
+#     print('Mapping...')
+#     train_dataset = train_dataset.map(lambda x, y: (text_vectorizer(x), y), tf.data.AUTOTUNE)
+#     print('Prefetching...')    
+#     train_dataset = train_dataset.prefetch(tf.data.AUTOTUNE)
+#     return train_dataset
+
+def sizeof_fmt(num, suffix='B'):
+    ''' by Fred Cirera,  https://stackoverflow.com/a/1094933/1870254, modified'''
+    for unit in ['','Ki','Mi','Gi','Ti','Pi','Ei','Zi']:
+        if abs(num) < 1024.0:
+            return "%3.1f %s%s" % (num, unit, suffix)
+        num /= 1024.0
+    return "%.1f %s%s" % (num, 'Yi', suffix)
+
+for name, size in sorted(((name, sys.getsizeof(value)) for name, value in list(
+                          locals().items())), key= lambda x: -x[1], reverse = False)[:10]:
+    print("{:>30}: {:>8}".format(name, sizeof_fmt(size)))
+
+
+
+def data_generator(x, y):
+    num_samples = len(x)
+    for i in range(num_samples):
+        yield x[i], y[i]
+
+
+def create_datasets(x, y, text_vectorizer, batch_size:int = 32, shuffle:bool=False, n_repeat:int = 0, buffer_size:int=1_000_000):
+
+    generator = data_generator(x, y)
+    print('Generating...')
+    train_dataset = tf.data.Dataset.from_generator(
+        lambda: generator,
+        output_signature=(
+            tf.TensorSpec(shape=(None, x.shape[1]), dtype=tf.string),
+            tf.TensorSpec(shape=(None, y.shape[1]), dtype=tf.int32)
+        )
+    )
+    print('Mapping...')
+    train_dataset = train_dataset.map(lambda x, y: (tf.cast(text_vectorizer(x), tf.int32)[0], y[0]), tf.data.AUTOTUNE)
+    train_dataset = train_dataset.batch(batch_size)
+
+    if shuffle:
+        train_dataset = train_dataset.shuffle(buffer_size)
+
+
+    if n_repeat > 0:
+        return train_dataset.cache().repeat(n_repeat).prefetch(tf.data.AUTOTUNE)
+    elif n_repeat == -1:
+        return train_dataset.cache().repeat().prefetch(tf.data.AUTOTUNE)
+    elif n_repeat == 0:
+        return train_dataset.cache().prefetch(tf.data.AUTOTUNE)
+    

src/model.py

@@ -0,0 +1,16 @@
+# model.py
+from tensorflow import keras
+from tensorflow.keras.layers import (LSTM, Dense, Dropout, Embedding,
+                                     TextVectorization)
+
+
+def create_lstm_model(input_shape, max_tokens, dim):
+    inputs = keras.Input(shape=(input_shape))
+    embedding_layer = Embedding(input_dim=max_tokens, output_dim=dim, mask_zero=True, input_length=input_shape, name='embedding_layer')(inputs)
+    x = LSTM(16, return_sequences=True, name = 'lstm_layer_1')(embedding_layer)
+    x = LSTM(16, name = 'lstm_layer_2')(x)
+    x = Dropout(0.4, name ='dropout_layer')(x)
+    x = Dense(64, activation='relu', name = 'dense_layer_1')(x)
+    outputs = Dense(1, activation='sigmoid', name = 'dense_layer_2_final')(x)
+    return keras.Model(inputs=inputs, outputs=outputs, name='model_lstm')
+

src/utils.py

@@ -0,0 +1,77 @@
+import logging
+import os
+import pickle
+import re
+from pathlib import Path
+
+import tensorflow as tf
+from tensorflow.keras.layers import TextVectorization
+
+from .data_preprocessing import clean_text
+
+
+# Configure logging
+def configure_logging(log_dir, log_filename, log_level=logging.INFO):
+    
+    log_dir = Path(log_dir)
+    log_dir.mkdir(exist_ok=True)
+    log_file = log_dir / log_filename
+    
+    # Configure logging to both console and file
+    logging.basicConfig(level=log_level,
+                        format='%(asctime)s - %(levelname)s - %(message)s',
+                        handlers=[
+                            logging.StreamHandler(),
+                            logging.FileHandler(log_file)
+                        ])
+    return
+    
+
+def save_text_vectorizer(text_vectorizer, filename):
+    config = text_vectorizer.get_config()
+    with open(filename, 'wb') as f:
+        pickle.dump({'config': config}, f)
+
+
+def load_counter(filename):
+    with open (filename,'rb')  as counter :
+        return pickle.load(counter)
+    
+
+def load_model(model, model_dir):
+    """Load the model from disk."""
+    # Load the Keras model
+    return model.load_weights(model_dir)
+
+
+# to load text vectorizer
+def load_text_vectorizer(vectorizer_path):
+    from_disk = pickle.load(open(vectorizer_path, "rb"))
+    return TextVectorization.from_config(from_disk['config'])
+
+# Pickle the config and weights
+def save_text_vectorizer(vectorizer_path):
+    pickle.dump({'config': text_vectorizer.get_config()}
+                , open(vectorizer_path, "wb"))
+
+
+def load_model_and_vectorizer(vectorizer_path, model_path):
+    try:
+        text_vectorizer = load_text_vectorizer(vectorizer_path)
+        lstm_model = tf.keras.models.load_model(model_path)
+        return text_vectorizer, lstm_model
+    except Exception as e:
+        logging.error(f'Error loading vectorizer and model: {e}')
+        return None, None
+
+
+
+def predict_sentiment(title, text, text_vectorizer, lstm_model):
+    review = f'{title} {text}' # concatenate the title and text
+    clean_review = clean_text(review)
+    review_sequence = text_vectorizer([clean_review])
+    prediction = lstm_model.predict(review_sequence)
+    sentiment_score = prediction[0][0]
+    sentiment_label = 'Positive' if sentiment_score >= 0.5 else 'Negative'
+    return sentiment_label, sentiment_score
+

train.py

@@ -0,0 +1,120 @@
+import polars as pl
+import tensorflow as tf
+from sklearn.model_selection import train_test_split
+from src import utils, make_dataset, model
+import config
+import pickle
+from tensorflow.keras.layers import TextVectorization
+import os
+import logging
+from pathlib import Path
+import numpy as np
+from sklearn.utils import check_random_state
+
+# constants
+DATA_DIR = Path(os.getcwd()) / 'dataset'
+DATA_PATH = DATA_DIR / 'preprocessed_df.csv'
+MODEL_PATH = Path(config.MODEL_DIR) / config.MODEL_FILENAME
+VECTORIZER_PATH = Path(config.MODEL_DIR) / config.TEXT_VECTOR_FILENAME
+COUNTER_PATH = Path(config.MODEL_DIR) / config.COUNTER_NAME
+
+def set_global_seed(seed):
+    np.random.seed(seed)
+    tf.random.set_seed(seed)
+    global random_state
+    random_state = check_random_state(seed)
+
+
+
+def read_data(DATA_PATH, train_size:float = 1.0):
+    logging.info('Reading data...')
+    df = pl.read_csv(DATA_PATH)
+    sample_rate = int(df.shape[0] * train_size)
+    df = df.sample(sample_rate, seed=config.SEED)
+    logging.info(f'Data shape after sampling: {df.shape}')
+    return df
+
+
+def main():
+    
+    # Call the function to set the seeds
+    set_global_seed(config.SEED)
+
+    utils.configure_logging(config.LOG_DIR, "training_log.txt", log_level=logging.INFO)  
+    
+    df = read_data(DATA_PATH, config.TRAIN_SIZE)
+
+    logging.info(f'GPU count: {len(tf.config.list_physical_devices("GPU"))}')
+    
+    counter = utils.load_counter(COUNTER_PATH)
+    
+    # Text vectorization
+    logging.info('Text Vectorizer loading  ...')
+    text_vectorizer = TextVectorization(max_tokens=config.MAX_TOKEN, standardize='lower_and_strip_punctuation',
+                                   split='whitespace',
+                                    ngrams= None ,
+                                    output_mode='int',
+                                    output_sequence_length=config.OUTPUT_SEQUENCE_LENGTH, 
+                                    pad_to_max_tokens=True,
+                                    vocabulary = list(counter.keys())[:config.MAX_TOKEN-2])
+
+    logging.info(f"text vectorizer vocab size: {text_vectorizer.vocabulary_size()}")
+
+    # Create datasets
+    logging.info('Preparing dataset...')
+    xtrain, xtest, ytrain, ytest = train_test_split(df.select('review'), df.select('polarity'), test_size=config.TEST_SIZE, random_state=config.SEED, stratify=df['polarity'])
+    del(df)
+    
+    train_len = xtrain.shape[0]//config.BATCH_SIZE
+    test_len = xtest.shape[0]//config.BATCH_SIZE
+    
+    logging.info('Preparing train dataset...')
+    train_dataset = make_dataset.create_datasets(xtrain, ytrain, text_vectorizer, batch_size=config.BATCH_SIZE, shuffle=False)
+    del(xtrain, ytrain)
+    
+    logging.info('Preparing test dataset...')
+    test_dataset = make_dataset.create_datasets(xtest, ytest, text_vectorizer, batch_size=config.BATCH_SIZE, shuffle=False)
+    del(xtest, ytest, counter, text_vectorizer )
+    
+    logging.info('Model loading...')
+    # Train LSTM model
+    lstm_model = model.create_lstm_model(input_shape=(config.OUTPUT_SEQUENCE_LENGTH,), max_tokens=config.MAX_TOKEN, dim=config.DIM)
+    lstm_model.compile(optimizer=tf.keras.optimizers.Nadam(learning_rate=config.LEARNING_RATE),
+                       loss = tf.keras.losses.BinaryCrossentropy(), 
+                       metrics=['Accuracy'])
+    
+    print(lstm_model.summary())
+    
+    # Callbacks
+    callbacks = [
+        tf.keras.callbacks.EarlyStopping(monitor='loss', patience=config.EARLY_STOPPING_PATIENCE, restore_best_weights=True),
+        tf.keras.callbacks.ModelCheckpoint(monitor='loss', filepath=MODEL_PATH, save_best_only=True)
+    ]
+    
+    # Load model weights if exists
+    try:
+        lstm_model.load_weights(MODEL_PATH) 
+        logging.info('Model weights loaded!') 
+    except Exception as e:
+        logging.error(f'Exception occured while loading model weights {e}')
+        
+     
+    # Training
+    logging.info('Model training...')
+    lstm_history = lstm_model.fit(train_dataset, validation_data=test_dataset, epochs=config.EPOCHS, 
+                                  steps_per_epoch=int(1.0*(train_len / config.EPOCHS)),
+                                  validation_steps=int(1.0*(test_len / config.EPOCHS)),
+                                  callbacks=callbacks)
+    logging.info('Training Complete!')
+    
+    logging.info('Training history:')
+    logging.info(lstm_history.history)
+    print(pl.DataFrame(lstm_history.history))
+    
+    # Save text vectorizer and LSTM model
+    logging.info('Saving Model')
+    lstm_model.save(MODEL_PATH, save_format='h5')
+    logging.info('Done')
+
+if __name__ == "__main__":
+    main()