Kaggle Cometitions: LLM - Detect AI Generated Text

Summary:

● Implemented an LTSM-based model for LLM text identification, including embedding layer, LSTM layer, fully connected layer, and other layers.

● The process included data preprocessing, feature extraction, clustering, dimension reduction, model optimization and other steps.

● Model optimization approaches including using pre-trained language models (BERT, RoBERTa, Deberta, etc.), data enhancement, optimizing network architecture and number of layers, learning rate scheduling, hyperparameter optimization, adding Dropout layers, trying different word embeddings, etc.

● This model has a large improvement compared to the baseline (based on LR, SGD and NB Bernoulli's voting).