add v2

LLM-Detector-V2-11w/README.md

@@ -0,0 +1,56 @@
+---
+license: other
+base_model: ./Qwen-7B-Chat
+tags:
+- llama-factory
+- lora
+- generated_from_trainer
+model-index:
+- name: qwen-7b
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# qwen-7b
+
+This model is a fine-tuned version of [./Qwen-7B-Chat](https://huggingface.co/./Qwen-7B-Chat) on the ta, the tb, the tc, the td, the te, the tf, the tg and the th datasets.
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- num_epochs: 3.0
+
+### Training results
+
+
+
+### Framework versions
+
+- Transformers 4.33.0
+- Pytorch 2.1.1+cu121
+- Datasets 2.14.7
+- Tokenizers 0.13.3

LLM-Detector-V2-11w/adapter_config.json

@@ -0,0 +1,22 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./Qwen-7B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16.0,
+  "lora_dropout": 0.1,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM"
+}

LLM-Detector-V2-11w/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa28fb27cb3e2ac4a0dcc009ae2625d376e7364c2e8d729cd3e3fe1fc668d427
+size 16800174

LLM-Detector-V2-11w/all_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.02652581251227368,
+    "train_runtime": 75119.2077,
+    "train_samples_per_second": 4.749,
+    "train_steps_per_second": 0.148
+}

LLM-Detector-V2-11w/checkpoint-10000/README.md

@@ -0,0 +1,219 @@
+---
+library_name: peft
+base_model: ./Qwen-7B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Data Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: QuantizationMethod.BITS_AND_BYTES
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+
+### Framework versions
+
+
+- PEFT 0.6.2

LLM-Detector-V2-11w/checkpoint-10000/adapter_config.json

@@ -0,0 +1,22 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./Qwen-7B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16.0,
+  "lora_dropout": 0.1,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM"
+}

LLM-Detector-V2-11w/checkpoint-10000/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:428b8088a29cff7c5c089fc070676ce5d5533d090968bef405cecf60563d7d89
+size 16800174

LLM-Detector-V2-11w/checkpoint-10000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:761aa59808bd5f017a6eda15a41348687530b88f7cd62c4d7b3c8b23e1b5ed04
+size 33608634

LLM-Detector-V2-11w/checkpoint-10000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c035419991828d3dd21375923850ee5fac9f2d49f2961f78cb1c9b4e67dafdc
+size 14244

LLM-Detector-V2-11w/checkpoint-10000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b2a8d700988cc2f86488cac04f06e51e20e5906b4dba4ca183109282717cb7e
+size 1064

LLM-Detector-V2-11w/checkpoint-10000/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "additional_special_tokens": [
+    "<|im_end|>"
+  ],
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>"
+}

LLM-Detector-V2-11w/checkpoint-10000/tokenization_qwen.py

@@ -0,0 +1,276 @@
+# Copyright (c) Alibaba Cloud.
+#
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Tokenization classes for QWen."""
+
+import base64
+import logging
+import os
+import unicodedata
+from typing import Collection, Dict, List, Set, Tuple, Union
+
+import tiktoken
+from transformers import PreTrainedTokenizer, AddedToken
+
+logger = logging.getLogger(__name__)
+
+
+VOCAB_FILES_NAMES = {"vocab_file": "qwen.tiktoken"}
+
+PAT_STR = r"""(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+"""
+ENDOFTEXT = "<|endoftext|>"
+IMSTART = "<|im_start|>"
+IMEND = "<|im_end|>"
+# as the default behavior is changed to allow special tokens in
+# regular texts, the surface forms of special tokens need to be
+# as different as possible to minimize the impact
+EXTRAS = tuple((f"<|extra_{i}|>" for i in range(205)))
+# changed to use actual index to avoid misconfiguration with vocabulary expansion
+SPECIAL_START_ID = 151643
+SPECIAL_TOKENS = tuple(
+    enumerate(
+        (
+            (
+                ENDOFTEXT,
+                IMSTART,
+                IMEND,
+            )
+            + EXTRAS
+        ),
+        start=SPECIAL_START_ID,
+    )
+)
+SPECIAL_TOKENS_SET = set(t for i, t in SPECIAL_TOKENS)
+
+
+def _load_tiktoken_bpe(tiktoken_bpe_file: str) -> Dict[bytes, int]:
+    with open(tiktoken_bpe_file, "rb") as f:
+        contents = f.read()
+    return {
+        base64.b64decode(token): int(rank)
+        for token, rank in (line.split() for line in contents.splitlines() if line)
+    }
+
+
+class QWenTokenizer(PreTrainedTokenizer):
+    """QWen tokenizer."""
+
+    vocab_files_names = VOCAB_FILES_NAMES
+
+    def __init__(
+        self,
+        vocab_file,
+        errors="replace",
+        extra_vocab_file=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        # how to handle errors in decoding UTF-8 byte sequences
+        # use ignore if you are in streaming inference
+        self.errors = errors  
+
+        self.mergeable_ranks = _load_tiktoken_bpe(vocab_file)  # type: Dict[bytes, int]
+        self.special_tokens = {
+            token: index
+            for index, token in SPECIAL_TOKENS
+        }
+
+        # try load extra vocab from file
+        if extra_vocab_file is not None:
+            used_ids = set(self.mergeable_ranks.values()) | set(self.special_tokens.values())
+            extra_mergeable_ranks = _load_tiktoken_bpe(extra_vocab_file)
+            for token, index in extra_mergeable_ranks.items():
+                if token in self.mergeable_ranks:
+                    logger.info(f"extra token {token} exists, skipping")
+                    continue
+                if index in used_ids:
+                    logger.info(f'the index {index} for extra token {token} exists, skipping')
+                    continue
+                self.mergeable_ranks[token] = index
+            # the index may be sparse after this, but don't worry tiktoken.Encoding will handle this
+
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        assert (
+            len(self.mergeable_ranks) + len(self.special_tokens) == enc.n_vocab
+        ), f"{len(self.mergeable_ranks) + len(self.special_tokens)} != {enc.n_vocab} in encoding"
+
+        self.decoder = {
+            v: k for k, v in self.mergeable_ranks.items()
+        }  # type: dict[int, bytes|str]
+        self.decoder.update({v: k for k, v in self.special_tokens.items()})
+
+        self.tokenizer = enc  # type: tiktoken.Encoding
+
+        self.eod_id = self.tokenizer.eot_token
+        self.im_start_id = self.special_tokens[IMSTART]
+        self.im_end_id = self.special_tokens[IMEND]
+
+    def __getstate__(self):
+        # for pickle lovers
+        state = self.__dict__.copy()
+        del state["tokenizer"]
+        return state
+
+    def __setstate__(self, state):
+        # tokenizer is not python native; don't pass it; rebuild it
+        self.__dict__.update(state)
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        self.tokenizer = enc
+
+    def __len__(self) -> int:
+        return self.tokenizer.n_vocab
+
+    def get_vocab(self) -> Dict[bytes, int]:
+        return self.mergeable_ranks
+
+    def convert_tokens_to_ids(
+        self, tokens: Union[bytes, str, List[Union[bytes, str]]]
+    ) -> List[int]:
+        ids = []
+        if isinstance(tokens, (str, bytes)):
+            if tokens in self.special_tokens:
+                return self.special_tokens[tokens]
+            else:
+                return self.mergeable_ranks.get(tokens)
+        for token in tokens:
+            if token in self.special_tokens:
+                ids.append(self.special_tokens[token])
+            else:
+                ids.append(self.mergeable_ranks.get(token))
+        return ids
+
+    def _add_tokens(
+        self,
+        new_tokens: Union[List[str], List[AddedToken]],
+        special_tokens: bool = False,
+    ) -> int:
+        if not special_tokens and new_tokens:
+            raise ValueError("Adding regular tokens is not supported")
+        for token in new_tokens:
+            surface_form = token.content if isinstance(token, AddedToken) else token
+            if surface_form not in SPECIAL_TOKENS_SET:
+                raise ValueError("Adding unknown special tokens is not supported")
+        return 0
+
+    def save_vocabulary(self, save_directory: str, **kwargs) -> Tuple[str]:
+        """
+        Save only the vocabulary of the tokenizer (vocabulary).
+
+        Returns:
+            `Tuple(str)`: Paths to the files saved.
+        """
+        file_path = os.path.join(save_directory, "qwen.tiktoken")
+        with open(file_path, "w", encoding="utf8") as w:
+            for k, v in self.mergeable_ranks.items():
+                line = base64.b64encode(k).decode("utf8") + " " + str(v) + "\n"
+                w.write(line)
+        return (file_path,)
+
+    def tokenize(
+        self,
+        text: str,
+        allowed_special: Union[Set, str] = "all",
+        disallowed_special: Union[Collection, str] = (),
+        **kwargs,
+    ) -> List[Union[bytes, str]]:
+        """
+        Converts a string in a sequence of tokens.
+
+        Args:
+            text (`str`):
+                The sequence to be encoded.
+            allowed_special (`Literal["all"]` or `set`):
+                The surface forms of the tokens to be encoded as special tokens in regular texts.
+                Default to "all".
+            disallowed_special (`Literal["all"]` or `Collection`):
+                The surface forms of the tokens that should not be in regular texts and trigger errors.
+                Default to an empty tuple.
+
+            kwargs (additional keyword arguments, *optional*):
+                Will be passed to the underlying model specific encode method.
+
+        Returns:
+            `List[bytes|str]`: The list of tokens.
+        """
+        tokens = []
+        text = unicodedata.normalize("NFC", text)
+
+        # this implementation takes a detour: text -> token id -> token surface forms
+        for t in self.tokenizer.encode(
+            text, allowed_special=allowed_special, disallowed_special=disallowed_special
+        ):
+            tokens.append(self.decoder[t])
+        return tokens
+
+    def convert_tokens_to_string(self, tokens: List[Union[bytes, str]]) -> str:
+        """
+        Converts a sequence of tokens in a single string.
+        """
+        text = ""
+        temp = b""
+        for t in tokens:
+            if isinstance(t, str):
+                if temp:
+                    text += temp.decode("utf-8", errors=self.errors)
+                    temp = b""
+                text += t
+            elif isinstance(t, bytes):
+                temp += t
+            else:
+                raise TypeError("token should only be of type types or str")
+        if temp:
+            text += temp.decode("utf-8", errors=self.errors)
+        return text
+
+    @property
+    def vocab_size(self):
+        return self.tokenizer.n_vocab
+
+    def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
+        """Converts an id to a token, special tokens included"""
+        if index in self.decoder:
+            return self.decoder[index]
+        raise ValueError("unknown ids")
+
+    def _convert_token_to_id(self, token: Union[bytes, str]) -> int:
+        """Converts a token to an id using the vocab, special tokens included"""
+        if token in self.special_tokens:
+            return self.special_tokens[token]
+        if token in self.mergeable_ranks:
+            return self.mergeable_ranks[token]
+        raise ValueError("unknown token")
+
+    def _tokenize(self, text: str, **kwargs):
+        """
+        Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based
+        vocabulary or sub-words for sub-word-based vocabularies (BPE/SentencePieces/WordPieces).
+
+        Do NOT take care of added tokens.
+        """
+        raise NotImplementedError
+
+    def _decode(
+        self,
+        token_ids: Union[int, List[int]],
+        skip_special_tokens: bool = False,
+        errors: str = None,
+        **kwargs,
+    ) -> str:
+        if isinstance(token_ids, int):
+            token_ids = [token_ids]
+        if skip_special_tokens:
+            token_ids = [i for i in token_ids if i < self.eod_id]
+        return self.tokenizer.decode(token_ids, errors=errors or self.errors)

LLM-Detector-V2-11w/checkpoint-10000/tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_qwen.QWenTokenizer",
+      null
+    ]
+  },
+  "clean_up_tokenization_spaces": true,
+  "model_max_length": 8192,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "QWenTokenizer"
+}

LLM-Detector-V2-11w/checkpoint-10000/trainer_state.json

@@ -0,0 +1,619 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.6907036189963676,
+  "eval_steps": 500,
+  "global_step": 10000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03,
+      "learning_rate": 4.9990073706856164e-05,
+      "loss": 2.2244,
+      "step": 100
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 4.99603027099283e-05,
+      "loss": 0.0286,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 4.991071065046783e-05,
+      "loss": 0.0144,
+      "step": 300
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 4.984133690970033e-05,
+      "loss": 0.0223,
+      "step": 400
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 4.975223657755279e-05,
+      "loss": 0.0179,
+      "step": 500
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 4.9643480408906496e-05,
+      "loss": 0.0246,
+      "step": 600
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 4.951515476741036e-05,
+      "loss": 0.0181,
+      "step": 700
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 4.936736155689918e-05,
+      "loss": 0.0155,
+      "step": 800
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 4.920021814047156e-05,
+      "loss": 0.0122,
+      "step": 900
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 4.901385724729137e-05,
+      "loss": 0.0128,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 4.880842686718711e-05,
+      "loss": 0.016,
+      "step": 1100
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 4.858409013313266e-05,
+      "loss": 0.0134,
+      "step": 1200
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 4.8341025191702847e-05,
+      "loss": 0.0099,
+      "step": 1300
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 4.80794250616066e-05,
+      "loss": 0.0102,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 4.7799497480410125e-05,
+      "loss": 0.0109,
+      "step": 1500
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 4.7501464739571836e-05,
+      "loss": 0.0117,
+      "step": 1600
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 4.7185563507919895e-05,
+      "loss": 0.0154,
+      "step": 1700
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 4.685204464371269e-05,
+      "loss": 0.0107,
+      "step": 1800
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 4.6501172995431424e-05,
+      "loss": 0.0133,
+      "step": 1900
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 4.6133227191463044e-05,
+      "loss": 0.0157,
+      "step": 2000
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 4.574849941884044e-05,
+      "loss": 0.0115,
+      "step": 2100
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 4.534729519121574e-05,
+      "loss": 0.0135,
+      "step": 2200
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 4.492993310625088e-05,
+      "loss": 0.0104,
+      "step": 2300
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 4.449674459261804e-05,
+      "loss": 0.0113,
+      "step": 2400
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 4.404807364681105e-05,
+      "loss": 0.0075,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 4.358427655997657e-05,
+      "loss": 0.011,
+      "step": 2600
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 4.310572163498205e-05,
+      "loss": 0.0103,
+      "step": 2700
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 4.2612788893945176e-05,
+      "loss": 0.0126,
+      "step": 2800
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 4.2105869776456944e-05,
+      "loss": 0.0084,
+      "step": 2900
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 4.158536682873821e-05,
+      "loss": 0.0069,
+      "step": 3000
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 4.1051693383976264e-05,
+      "loss": 0.0089,
+      "step": 3100
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 4.050527323409557e-05,
+      "loss": 0.0103,
+      "step": 3200
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 3.994654029322313e-05,
+      "loss": 0.011,
+      "step": 3300
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 3.937593825311575e-05,
+      "loss": 0.0063,
+      "step": 3400
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 3.8793920230822925e-05,
+      "loss": 0.0091,
+      "step": 3500
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 3.8200948408864986e-05,
+      "loss": 0.0072,
+      "step": 3600
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 3.759749366821241e-05,
+      "loss": 0.007,
+      "step": 3700
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 3.698403521435756e-05,
+      "loss": 0.0067,
+      "step": 3800
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 3.636106019677602e-05,
+      "loss": 0.0055,
+      "step": 3900
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 3.572906332207943e-05,
+      "loss": 0.0071,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 3.5088546461167346e-05,
+      "loss": 0.0056,
+      "step": 4100
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 3.4440018250689767e-05,
+      "loss": 0.0074,
+      "step": 4200
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 3.3783993689137015e-05,
+      "loss": 0.0061,
+      "step": 4300
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 3.312099372787772e-05,
+      "loss": 0.0051,
+      "step": 4400
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 3.2451544857469436e-05,
+      "loss": 0.0078,
+      "step": 4500
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 3.177617868957078e-05,
+      "loss": 0.0064,
+      "step": 4600
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 3.109543153478671e-05,
+      "loss": 0.0064,
+      "step": 4700
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 3.040984397678245e-05,
+      "loss": 0.0064,
+      "step": 4800
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 2.9719960443004107e-05,
+      "loss": 0.0062,
+      "step": 4900
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 2.9026328772347e-05,
+      "loss": 0.0056,
+      "step": 5000
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 2.8329499780114865e-05,
+      "loss": 0.0096,
+      "step": 5100
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 2.763002682061555e-05,
+      "loss": 0.0054,
+      "step": 5200
+    },
+    {
+      "epoch": 1.43,
+      "learning_rate": 2.6928465347740434e-05,
+      "loss": 0.0049,
+      "step": 5300
+    },
+    {
+      "epoch": 1.45,
+      "learning_rate": 2.6225372473876565e-05,
+      "loss": 0.0023,
+      "step": 5400
+    },
+    {
+      "epoch": 1.48,
+      "learning_rate": 2.552835016828629e-05,
+      "loss": 0.0065,
+      "step": 5500
+    },
+    {
+      "epoch": 1.51,
+      "learning_rate": 2.4823871621313255e-05,
+      "loss": 0.0032,
+      "step": 5600
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 2.411953293849365e-05,
+      "loss": 0.0043,
+      "step": 5700
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 2.341589343760652e-05,
+      "loss": 0.0038,
+      "step": 5800
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 2.271351188120813e-05,
+      "loss": 0.0048,
+      "step": 5900
+    },
+    {
+      "epoch": 1.61,
+      "learning_rate": 2.2012946032916717e-05,
+      "loss": 0.0082,
+      "step": 6000
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 2.1314752214490396e-05,
+      "loss": 0.0046,
+      "step": 6100
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 2.061948486405021e-05,
+      "loss": 0.0063,
+      "step": 6200
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 1.992769609579885e-05,
+      "loss": 0.0052,
+      "step": 6300
+    },
+    {
+      "epoch": 1.72,
+      "learning_rate": 1.9239935261584917e-05,
+      "loss": 0.0041,
+      "step": 6400
+    },
+    {
+      "epoch": 1.75,
+      "learning_rate": 1.8556748514660664e-05,
+      "loss": 0.004,
+      "step": 6500
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 1.7878678375979845e-05,
+      "loss": 0.0035,
+      "step": 6600
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 1.7206263303379948e-05,
+      "loss": 0.0066,
+      "step": 6700
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 1.654003726399097e-05,
+      "loss": 0.0047,
+      "step": 6800
+    },
+    {
+      "epoch": 1.86,
+      "learning_rate": 1.5880529310210283e-05,
+      "loss": 0.005,
+      "step": 6900
+    },
+    {
+      "epoch": 1.88,
+      "learning_rate": 1.5228263159580275e-05,
+      "loss": 0.0033,
+      "step": 7000
+    },
+    {
+      "epoch": 1.91,
+      "learning_rate": 1.4583756778902463e-05,
+      "loss": 0.0059,
+      "step": 7100
+    },
+    {
+      "epoch": 1.94,
+      "learning_rate": 1.3947521972918251e-05,
+      "loss": 0.0051,
+      "step": 7200
+    },
+    {
+      "epoch": 1.96,
+      "learning_rate": 1.3320063977883046e-05,
+      "loss": 0.0041,
+      "step": 7300
+    },
+    {
+      "epoch": 1.99,
+      "learning_rate": 1.2701881060356396e-05,
+      "loss": 0.0049,
+      "step": 7400
+    },
+    {
+      "epoch": 2.02,
+      "learning_rate": 1.209346412152676e-05,
+      "loss": 0.0024,
+      "step": 7500
+    },
+    {
+      "epoch": 2.04,
+      "learning_rate": 1.149529630738521e-05,
+      "loss": 0.0035,
+      "step": 7600
+    },
+    {
+      "epoch": 2.07,
+      "learning_rate": 1.0907852625057538e-05,
+      "loss": 0.0013,
+      "step": 7700
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 1.0331599565599464e-05,
+      "loss": 0.0014,
+      "step": 7800
+    },
+    {
+      "epoch": 2.13,
+      "learning_rate": 9.766994733554432e-06,
+      "loss": 0.004,
+      "step": 7900
+    },
+    {
+      "epoch": 2.15,
+      "learning_rate": 9.214486483568307e-06,
+      "loss": 0.0021,
+      "step": 8000
+    },
+    {
+      "epoch": 2.18,
+      "learning_rate": 8.674513564349376e-06,
+      "loss": 0.0025,
+      "step": 8100
+    },
+    {
+      "epoch": 2.21,
+      "learning_rate": 8.147504770256536e-06,
+      "loss": 0.0026,
+      "step": 8200
+    },
+    {
+      "epoch": 2.23,
+      "learning_rate": 7.633878600792143e-06,
+      "loss": 0.0018,
+      "step": 8300
+    },
+    {
+      "epoch": 2.26,
+      "learning_rate": 7.138971709507527e-06,
+      "loss": 0.0016,
+      "step": 8400
+    },
+    {
+      "epoch": 2.29,
+      "learning_rate": 6.653179652954706e-06,
+      "loss": 0.003,
+      "step": 8500
+    },
+    {
+      "epoch": 2.31,
+      "learning_rate": 6.181956869763645e-06,
+      "loss": 0.0011,
+      "step": 8600
+    },
+    {
+      "epoch": 2.34,
+      "learning_rate": 5.725677559572909e-06,
+      "loss": 0.0024,
+      "step": 8700
+    },
+    {
+      "epoch": 2.37,
+      "learning_rate": 5.284704055357578e-06,
+      "loss": 0.0007,
+      "step": 8800
+    },
+    {
+      "epoch": 2.39,
+      "learning_rate": 4.859386535699373e-06,
+      "loss": 0.0017,
+      "step": 8900
+    },
+    {
+      "epoch": 2.42,
+      "learning_rate": 4.4500627467086245e-06,
+      "loss": 0.0021,
+      "step": 9000
+    },
+    {
+      "epoch": 2.45,
+      "learning_rate": 4.057057733818967e-06,
+      "loss": 0.0016,
+      "step": 9100
+    },
+    {
+      "epoch": 2.48,
+      "learning_rate": 3.680683583667599e-06,
+      "loss": 0.0013,
+      "step": 9200
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 3.321239176266211e-06,
+      "loss": 0.0028,
+      "step": 9300
+    },
+    {
+      "epoch": 2.53,
+      "learning_rate": 2.9790099476593393e-06,
+      "loss": 0.0007,
+      "step": 9400
+    },
+    {
+      "epoch": 2.56,
+      "learning_rate": 2.654267663258628e-06,
+      "loss": 0.0026,
+      "step": 9500
+    },
+    {
+      "epoch": 2.58,
+      "learning_rate": 2.3472702020329696e-06,
+      "loss": 0.0013,
+      "step": 9600
+    },
+    {
+      "epoch": 2.61,
+      "learning_rate": 2.0582613517259377e-06,
+      "loss": 0.0014,
+      "step": 9700
+    },
+    {
+      "epoch": 2.64,
+      "learning_rate": 1.7874706152630705e-06,
+      "loss": 0.0011,
+      "step": 9800
+    },
+    {
+      "epoch": 2.66,
+      "learning_rate": 1.5351130285028214e-06,
+      "loss": 0.0022,
+      "step": 9900
+    },
+    {
+      "epoch": 2.69,
+      "learning_rate": 1.3036333720407912e-06,
+      "loss": 0.0012,
+      "step": 10000
+    }
+  ],
+  "logging_steps": 100,
+  "max_steps": 11148,
+  "num_train_epochs": 3,
+  "save_steps": 5000,
+  "total_flos": 2.0893490293152154e+18,
+  "trial_name": null,
+  "trial_params": null
+}

LLM-Detector-V2-11w/checkpoint-10000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2501d0db9b5d010a4a25759f877d55c11a9cd335425e4c2cc6f2683c0ebb1612
+size 4600

LLM-Detector-V2-11w/checkpoint-5000/README.md

@@ -0,0 +1,219 @@
+---
+library_name: peft
+base_model: ./Qwen-7B-Chat
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure 
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Data Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+
+
+## Training procedure
+
+
+The following `bitsandbytes` quantization config was used during training:
+- quant_method: QuantizationMethod.BITS_AND_BYTES
+- load_in_8bit: False
+- load_in_4bit: True
+- llm_int8_threshold: 6.0
+- llm_int8_skip_modules: None
+- llm_int8_enable_fp32_cpu_offload: False
+- llm_int8_has_fp16_weight: False
+- bnb_4bit_quant_type: nf4
+- bnb_4bit_use_double_quant: True
+- bnb_4bit_compute_dtype: float16
+
+### Framework versions
+
+
+- PEFT 0.6.2

LLM-Detector-V2-11w/checkpoint-5000/adapter_config.json

@@ -0,0 +1,22 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "./Qwen-7B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "lora_alpha": 16.0,
+  "lora_dropout": 0.1,
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM"
+}

LLM-Detector-V2-11w/checkpoint-5000/adapter_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:73cf151fd4a6873a2c4a037defe6bd2cb0cf499be5d89967c9f023286b4258a6
+size 16800174

LLM-Detector-V2-11w/checkpoint-5000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4a59bdfff13b7ea192b5b06e51c9a56fd45cb8aaa3b592a6ae50215c7c1d6b86
+size 33608634

LLM-Detector-V2-11w/checkpoint-5000/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0873e58da7017c4ae0ea0a268eaa53956bc8ffd92e4697645c97f4f607410322
+size 14244

LLM-Detector-V2-11w/checkpoint-5000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf48e457843d094d3761669f67579e960ebc0206509d4ada3fbacc891a92bf86
+size 1064

LLM-Detector-V2-11w/checkpoint-5000/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "additional_special_tokens": [
+    "<|im_end|>"
+  ],
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>"
+}

LLM-Detector-V2-11w/checkpoint-5000/tokenization_qwen.py

@@ -0,0 +1,276 @@
+# Copyright (c) Alibaba Cloud.
+#
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Tokenization classes for QWen."""
+
+import base64
+import logging
+import os
+import unicodedata
+from typing import Collection, Dict, List, Set, Tuple, Union
+
+import tiktoken
+from transformers import PreTrainedTokenizer, AddedToken
+
+logger = logging.getLogger(__name__)
+
+
+VOCAB_FILES_NAMES = {"vocab_file": "qwen.tiktoken"}
+
+PAT_STR = r"""(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+"""
+ENDOFTEXT = "<|endoftext|>"
+IMSTART = "<|im_start|>"
+IMEND = "<|im_end|>"
+# as the default behavior is changed to allow special tokens in
+# regular texts, the surface forms of special tokens need to be
+# as different as possible to minimize the impact
+EXTRAS = tuple((f"<|extra_{i}|>" for i in range(205)))
+# changed to use actual index to avoid misconfiguration with vocabulary expansion
+SPECIAL_START_ID = 151643
+SPECIAL_TOKENS = tuple(
+    enumerate(
+        (
+            (
+                ENDOFTEXT,
+                IMSTART,
+                IMEND,
+            )
+            + EXTRAS
+        ),
+        start=SPECIAL_START_ID,
+    )
+)
+SPECIAL_TOKENS_SET = set(t for i, t in SPECIAL_TOKENS)
+
+
+def _load_tiktoken_bpe(tiktoken_bpe_file: str) -> Dict[bytes, int]:
+    with open(tiktoken_bpe_file, "rb") as f:
+        contents = f.read()
+    return {
+        base64.b64decode(token): int(rank)
+        for token, rank in (line.split() for line in contents.splitlines() if line)
+    }
+
+
+class QWenTokenizer(PreTrainedTokenizer):
+    """QWen tokenizer."""
+
+    vocab_files_names = VOCAB_FILES_NAMES
+
+    def __init__(
+        self,
+        vocab_file,
+        errors="replace",
+        extra_vocab_file=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        # how to handle errors in decoding UTF-8 byte sequences
+        # use ignore if you are in streaming inference
+        self.errors = errors  
+
+        self.mergeable_ranks = _load_tiktoken_bpe(vocab_file)  # type: Dict[bytes, int]
+        self.special_tokens = {
+            token: index
+            for index, token in SPECIAL_TOKENS
+        }
+
+        # try load extra vocab from file
+        if extra_vocab_file is not None:
+            used_ids = set(self.mergeable_ranks.values()) | set(self.special_tokens.values())
+            extra_mergeable_ranks = _load_tiktoken_bpe(extra_vocab_file)
+            for token, index in extra_mergeable_ranks.items():
+                if token in self.mergeable_ranks:
+                    logger.info(f"extra token {token} exists, skipping")
+                    continue
+                if index in used_ids:
+                    logger.info(f'the index {index} for extra token {token} exists, skipping')
+                    continue
+                self.mergeable_ranks[token] = index
+            # the index may be sparse after this, but don't worry tiktoken.Encoding will handle this
+
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        assert (
+            len(self.mergeable_ranks) + len(self.special_tokens) == enc.n_vocab
+        ), f"{len(self.mergeable_ranks) + len(self.special_tokens)} != {enc.n_vocab} in encoding"
+
+        self.decoder = {
+            v: k for k, v in self.mergeable_ranks.items()
+        }  # type: dict[int, bytes|str]
+        self.decoder.update({v: k for k, v in self.special_tokens.items()})
+
+        self.tokenizer = enc  # type: tiktoken.Encoding
+
+        self.eod_id = self.tokenizer.eot_token
+        self.im_start_id = self.special_tokens[IMSTART]
+        self.im_end_id = self.special_tokens[IMEND]
+
+    def __getstate__(self):
+        # for pickle lovers
+        state = self.__dict__.copy()
+        del state["tokenizer"]
+        return state
+
+    def __setstate__(self, state):
+        # tokenizer is not python native; don't pass it; rebuild it
+        self.__dict__.update(state)
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        self.tokenizer = enc
+
+    def __len__(self) -> int:
+        return self.tokenizer.n_vocab
+
+    def get_vocab(self) -> Dict[bytes, int]:
+        return self.mergeable_ranks
+
+    def convert_tokens_to_ids(
+        self, tokens: Union[bytes, str, List[Union[bytes, str]]]
+    ) -> List[int]:
+        ids = []
+        if isinstance(tokens, (str, bytes)):
+            if tokens in self.special_tokens:
+                return self.special_tokens[tokens]
+            else:
+                return self.mergeable_ranks.get(tokens)
+        for token in tokens:
+            if token in self.special_tokens:
+                ids.append(self.special_tokens[token])
+            else:
+                ids.append(self.mergeable_ranks.get(token))
+        return ids
+
+    def _add_tokens(
+        self,
+        new_tokens: Union[List[str], List[AddedToken]],
+        special_tokens: bool = False,
+    ) -> int:
+        if not special_tokens and new_tokens:
+            raise ValueError("Adding regular tokens is not supported")
+        for token in new_tokens:
+            surface_form = token.content if isinstance(token, AddedToken) else token
+            if surface_form not in SPECIAL_TOKENS_SET:
+                raise ValueError("Adding unknown special tokens is not supported")
+        return 0
+
+    def save_vocabulary(self, save_directory: str, **kwargs) -> Tuple[str]:
+        """
+        Save only the vocabulary of the tokenizer (vocabulary).
+
+        Returns:
+            `Tuple(str)`: Paths to the files saved.
+        """
+        file_path = os.path.join(save_directory, "qwen.tiktoken")
+        with open(file_path, "w", encoding="utf8") as w:
+            for k, v in self.mergeable_ranks.items():
+                line = base64.b64encode(k).decode("utf8") + " " + str(v) + "\n"
+                w.write(line)
+        return (file_path,)
+
+    def tokenize(
+        self,
+        text: str,
+        allowed_special: Union[Set, str] = "all",
+        disallowed_special: Union[Collection, str] = (),
+        **kwargs,
+    ) -> List[Union[bytes, str]]:
+        """
+        Converts a string in a sequence of tokens.
+
+        Args:
+            text (`str`):
+                The sequence to be encoded.
+            allowed_special (`Literal["all"]` or `set`):
+                The surface forms of the tokens to be encoded as special tokens in regular texts.
+                Default to "all".
+            disallowed_special (`Literal["all"]` or `Collection`):
+                The surface forms of the tokens that should not be in regular texts and trigger errors.
+                Default to an empty tuple.
+
+            kwargs (additional keyword arguments, *optional*):
+                Will be passed to the underlying model specific encode method.
+
+        Returns:
+            `List[bytes|str]`: The list of tokens.
+        """
+        tokens = []
+        text = unicodedata.normalize("NFC", text)
+
+        # this implementation takes a detour: text -> token id -> token surface forms
+        for t in self.tokenizer.encode(
+            text, allowed_special=allowed_special, disallowed_special=disallowed_special
+        ):
+            tokens.append(self.decoder[t])
+        return tokens
+
+    def convert_tokens_to_string(self, tokens: List[Union[bytes, str]]) -> str:
+        """
+        Converts a sequence of tokens in a single string.
+        """
+        text = ""
+        temp = b""
+        for t in tokens:
+            if isinstance(t, str):
+                if temp:
+                    text += temp.decode("utf-8", errors=self.errors)
+                    temp = b""
+                text += t
+            elif isinstance(t, bytes):
+                temp += t
+            else:
+                raise TypeError("token should only be of type types or str")
+        if temp:
+            text += temp.decode("utf-8", errors=self.errors)
+        return text
+
+    @property
+    def vocab_size(self):
+        return self.tokenizer.n_vocab
+
+    def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
+        """Converts an id to a token, special tokens included"""
+        if index in self.decoder:
+            return self.decoder[index]
+        raise ValueError("unknown ids")
+
+    def _convert_token_to_id(self, token: Union[bytes, str]) -> int:
+        """Converts a token to an id using the vocab, special tokens included"""
+        if token in self.special_tokens:
+            return self.special_tokens[token]
+        if token in self.mergeable_ranks:
+            return self.mergeable_ranks[token]
+        raise ValueError("unknown token")
+
+    def _tokenize(self, text: str, **kwargs):
+        """
+        Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based
+        vocabulary or sub-words for sub-word-based vocabularies (BPE/SentencePieces/WordPieces).
+
+        Do NOT take care of added tokens.
+        """
+        raise NotImplementedError
+
+    def _decode(
+        self,
+        token_ids: Union[int, List[int]],
+        skip_special_tokens: bool = False,
+        errors: str = None,
+        **kwargs,
+    ) -> str:
+        if isinstance(token_ids, int):
+            token_ids = [token_ids]
+        if skip_special_tokens:
+            token_ids = [i for i in token_ids if i < self.eod_id]
+        return self.tokenizer.decode(token_ids, errors=errors or self.errors)

LLM-Detector-V2-11w/checkpoint-5000/tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_qwen.QWenTokenizer",
+      null
+    ]
+  },
+  "clean_up_tokenization_spaces": true,
+  "model_max_length": 8192,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "QWenTokenizer"
+}

LLM-Detector-V2-11w/checkpoint-5000/trainer_state.json

@@ -0,0 +1,319 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.3453518094981838,
+  "eval_steps": 500,
+  "global_step": 5000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03,
+      "learning_rate": 4.9990073706856164e-05,
+      "loss": 2.2244,
+      "step": 100
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 4.99603027099283e-05,
+      "loss": 0.0286,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 4.991071065046783e-05,
+      "loss": 0.0144,
+      "step": 300
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 4.984133690970033e-05,
+      "loss": 0.0223,
+      "step": 400
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 4.975223657755279e-05,
+      "loss": 0.0179,
+      "step": 500
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 4.9643480408906496e-05,
+      "loss": 0.0246,
+      "step": 600
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 4.951515476741036e-05,
+      "loss": 0.0181,
+      "step": 700
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 4.936736155689918e-05,
+      "loss": 0.0155,
+      "step": 800
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 4.920021814047156e-05,
+      "loss": 0.0122,
+      "step": 900
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 4.901385724729137e-05,
+      "loss": 0.0128,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 4.880842686718711e-05,
+      "loss": 0.016,
+      "step": 1100
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 4.858409013313266e-05,
+      "loss": 0.0134,
+      "step": 1200
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 4.8341025191702847e-05,
+      "loss": 0.0099,
+      "step": 1300
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 4.80794250616066e-05,
+      "loss": 0.0102,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 4.7799497480410125e-05,
+      "loss": 0.0109,
+      "step": 1500
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 4.7501464739571836e-05,
+      "loss": 0.0117,
+      "step": 1600
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 4.7185563507919895e-05,
+      "loss": 0.0154,
+      "step": 1700
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 4.685204464371269e-05,
+      "loss": 0.0107,
+      "step": 1800
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 4.6501172995431424e-05,
+      "loss": 0.0133,
+      "step": 1900
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 4.6133227191463044e-05,
+      "loss": 0.0157,
+      "step": 2000
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 4.574849941884044e-05,
+      "loss": 0.0115,
+      "step": 2100
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 4.534729519121574e-05,
+      "loss": 0.0135,
+      "step": 2200
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 4.492993310625088e-05,
+      "loss": 0.0104,
+      "step": 2300
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 4.449674459261804e-05,
+      "loss": 0.0113,
+      "step": 2400
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 4.404807364681105e-05,
+      "loss": 0.0075,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 4.358427655997657e-05,
+      "loss": 0.011,
+      "step": 2600
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 4.310572163498205e-05,
+      "loss": 0.0103,
+      "step": 2700
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 4.2612788893945176e-05,
+      "loss": 0.0126,
+      "step": 2800
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 4.2105869776456944e-05,
+      "loss": 0.0084,
+      "step": 2900
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 4.158536682873821e-05,
+      "loss": 0.0069,
+      "step": 3000
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 4.1051693383976264e-05,
+      "loss": 0.0089,
+      "step": 3100
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 4.050527323409557e-05,
+      "loss": 0.0103,
+      "step": 3200
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 3.994654029322313e-05,
+      "loss": 0.011,
+      "step": 3300
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 3.937593825311575e-05,
+      "loss": 0.0063,
+      "step": 3400
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 3.8793920230822925e-05,
+      "loss": 0.0091,
+      "step": 3500
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 3.8200948408864986e-05,
+      "loss": 0.0072,
+      "step": 3600
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 3.759749366821241e-05,
+      "loss": 0.007,
+      "step": 3700
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 3.698403521435756e-05,
+      "loss": 0.0067,
+      "step": 3800
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 3.636106019677602e-05,
+      "loss": 0.0055,
+      "step": 3900
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 3.572906332207943e-05,
+      "loss": 0.0071,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 3.5088546461167346e-05,
+      "loss": 0.0056,
+      "step": 4100
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 3.4440018250689767e-05,
+      "loss": 0.0074,
+      "step": 4200
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 3.3783993689137015e-05,
+      "loss": 0.0061,
+      "step": 4300
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 3.312099372787772e-05,
+      "loss": 0.0051,
+      "step": 4400
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 3.2451544857469436e-05,
+      "loss": 0.0078,
+      "step": 4500
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 3.177617868957078e-05,
+      "loss": 0.0064,
+      "step": 4600
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 3.109543153478671e-05,
+      "loss": 0.0064,
+      "step": 4700
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 3.040984397678245e-05,
+      "loss": 0.0064,
+      "step": 4800
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 2.9719960443004107e-05,
+      "loss": 0.0062,
+      "step": 4900
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 2.9026328772347e-05,
+      "loss": 0.0056,
+      "step": 5000
+    }
+  ],
+  "logging_steps": 100,
+  "max_steps": 11148,
+  "num_train_epochs": 3,
+  "save_steps": 5000,
+  "total_flos": 1.044197334217261e+18,
+  "trial_name": null,
+  "trial_params": null
+}

LLM-Detector-V2-11w/checkpoint-5000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2501d0db9b5d010a4a25759f877d55c11a9cd335425e4c2cc6f2683c0ebb1612
+size 4600

LLM-Detector-V2-11w/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "additional_special_tokens": [
+    "<|im_end|>"
+  ],
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>"
+}

LLM-Detector-V2-11w/tokenization_qwen.py

@@ -0,0 +1,276 @@
+# Copyright (c) Alibaba Cloud.
+#
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Tokenization classes for QWen."""
+
+import base64
+import logging
+import os
+import unicodedata
+from typing import Collection, Dict, List, Set, Tuple, Union
+
+import tiktoken
+from transformers import PreTrainedTokenizer, AddedToken
+
+logger = logging.getLogger(__name__)
+
+
+VOCAB_FILES_NAMES = {"vocab_file": "qwen.tiktoken"}
+
+PAT_STR = r"""(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+"""
+ENDOFTEXT = "<|endoftext|>"
+IMSTART = "<|im_start|>"
+IMEND = "<|im_end|>"
+# as the default behavior is changed to allow special tokens in
+# regular texts, the surface forms of special tokens need to be
+# as different as possible to minimize the impact
+EXTRAS = tuple((f"<|extra_{i}|>" for i in range(205)))
+# changed to use actual index to avoid misconfiguration with vocabulary expansion
+SPECIAL_START_ID = 151643
+SPECIAL_TOKENS = tuple(
+    enumerate(
+        (
+            (
+                ENDOFTEXT,
+                IMSTART,
+                IMEND,
+            )
+            + EXTRAS
+        ),
+        start=SPECIAL_START_ID,
+    )
+)
+SPECIAL_TOKENS_SET = set(t for i, t in SPECIAL_TOKENS)
+
+
+def _load_tiktoken_bpe(tiktoken_bpe_file: str) -> Dict[bytes, int]:
+    with open(tiktoken_bpe_file, "rb") as f:
+        contents = f.read()
+    return {
+        base64.b64decode(token): int(rank)
+        for token, rank in (line.split() for line in contents.splitlines() if line)
+    }
+
+
+class QWenTokenizer(PreTrainedTokenizer):
+    """QWen tokenizer."""
+
+    vocab_files_names = VOCAB_FILES_NAMES
+
+    def __init__(
+        self,
+        vocab_file,
+        errors="replace",
+        extra_vocab_file=None,
+        **kwargs,
+    ):
+        super().__init__(**kwargs)
+
+        # how to handle errors in decoding UTF-8 byte sequences
+        # use ignore if you are in streaming inference
+        self.errors = errors  
+
+        self.mergeable_ranks = _load_tiktoken_bpe(vocab_file)  # type: Dict[bytes, int]
+        self.special_tokens = {
+            token: index
+            for index, token in SPECIAL_TOKENS
+        }
+
+        # try load extra vocab from file
+        if extra_vocab_file is not None:
+            used_ids = set(self.mergeable_ranks.values()) | set(self.special_tokens.values())
+            extra_mergeable_ranks = _load_tiktoken_bpe(extra_vocab_file)
+            for token, index in extra_mergeable_ranks.items():
+                if token in self.mergeable_ranks:
+                    logger.info(f"extra token {token} exists, skipping")
+                    continue
+                if index in used_ids:
+                    logger.info(f'the index {index} for extra token {token} exists, skipping')
+                    continue
+                self.mergeable_ranks[token] = index
+            # the index may be sparse after this, but don't worry tiktoken.Encoding will handle this
+
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        assert (
+            len(self.mergeable_ranks) + len(self.special_tokens) == enc.n_vocab
+        ), f"{len(self.mergeable_ranks) + len(self.special_tokens)} != {enc.n_vocab} in encoding"
+
+        self.decoder = {
+            v: k for k, v in self.mergeable_ranks.items()
+        }  # type: dict[int, bytes|str]
+        self.decoder.update({v: k for k, v in self.special_tokens.items()})
+
+        self.tokenizer = enc  # type: tiktoken.Encoding
+
+        self.eod_id = self.tokenizer.eot_token
+        self.im_start_id = self.special_tokens[IMSTART]
+        self.im_end_id = self.special_tokens[IMEND]
+
+    def __getstate__(self):
+        # for pickle lovers
+        state = self.__dict__.copy()
+        del state["tokenizer"]
+        return state
+
+    def __setstate__(self, state):
+        # tokenizer is not python native; don't pass it; rebuild it
+        self.__dict__.update(state)
+        enc = tiktoken.Encoding(
+            "Qwen",
+            pat_str=PAT_STR,
+            mergeable_ranks=self.mergeable_ranks,
+            special_tokens=self.special_tokens,
+        )
+        self.tokenizer = enc
+
+    def __len__(self) -> int:
+        return self.tokenizer.n_vocab
+
+    def get_vocab(self) -> Dict[bytes, int]:
+        return self.mergeable_ranks
+
+    def convert_tokens_to_ids(
+        self, tokens: Union[bytes, str, List[Union[bytes, str]]]
+    ) -> List[int]:
+        ids = []
+        if isinstance(tokens, (str, bytes)):
+            if tokens in self.special_tokens:
+                return self.special_tokens[tokens]
+            else:
+                return self.mergeable_ranks.get(tokens)
+        for token in tokens:
+            if token in self.special_tokens:
+                ids.append(self.special_tokens[token])
+            else:
+                ids.append(self.mergeable_ranks.get(token))
+        return ids
+
+    def _add_tokens(
+        self,
+        new_tokens: Union[List[str], List[AddedToken]],
+        special_tokens: bool = False,
+    ) -> int:
+        if not special_tokens and new_tokens:
+            raise ValueError("Adding regular tokens is not supported")
+        for token in new_tokens:
+            surface_form = token.content if isinstance(token, AddedToken) else token
+            if surface_form not in SPECIAL_TOKENS_SET:
+                raise ValueError("Adding unknown special tokens is not supported")
+        return 0
+
+    def save_vocabulary(self, save_directory: str, **kwargs) -> Tuple[str]:
+        """
+        Save only the vocabulary of the tokenizer (vocabulary).
+
+        Returns:
+            `Tuple(str)`: Paths to the files saved.
+        """
+        file_path = os.path.join(save_directory, "qwen.tiktoken")
+        with open(file_path, "w", encoding="utf8") as w:
+            for k, v in self.mergeable_ranks.items():
+                line = base64.b64encode(k).decode("utf8") + " " + str(v) + "\n"
+                w.write(line)
+        return (file_path,)
+
+    def tokenize(
+        self,
+        text: str,
+        allowed_special: Union[Set, str] = "all",
+        disallowed_special: Union[Collection, str] = (),
+        **kwargs,
+    ) -> List[Union[bytes, str]]:
+        """
+        Converts a string in a sequence of tokens.
+
+        Args:
+            text (`str`):
+                The sequence to be encoded.
+            allowed_special (`Literal["all"]` or `set`):
+                The surface forms of the tokens to be encoded as special tokens in regular texts.
+                Default to "all".
+            disallowed_special (`Literal["all"]` or `Collection`):
+                The surface forms of the tokens that should not be in regular texts and trigger errors.
+                Default to an empty tuple.
+
+            kwargs (additional keyword arguments, *optional*):
+                Will be passed to the underlying model specific encode method.
+
+        Returns:
+            `List[bytes|str]`: The list of tokens.
+        """
+        tokens = []
+        text = unicodedata.normalize("NFC", text)
+
+        # this implementation takes a detour: text -> token id -> token surface forms
+        for t in self.tokenizer.encode(
+            text, allowed_special=allowed_special, disallowed_special=disallowed_special
+        ):
+            tokens.append(self.decoder[t])
+        return tokens
+
+    def convert_tokens_to_string(self, tokens: List[Union[bytes, str]]) -> str:
+        """
+        Converts a sequence of tokens in a single string.
+        """
+        text = ""
+        temp = b""
+        for t in tokens:
+            if isinstance(t, str):
+                if temp:
+                    text += temp.decode("utf-8", errors=self.errors)
+                    temp = b""
+                text += t
+            elif isinstance(t, bytes):
+                temp += t
+            else:
+                raise TypeError("token should only be of type types or str")
+        if temp:
+            text += temp.decode("utf-8", errors=self.errors)
+        return text
+
+    @property
+    def vocab_size(self):
+        return self.tokenizer.n_vocab
+
+    def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
+        """Converts an id to a token, special tokens included"""
+        if index in self.decoder:
+            return self.decoder[index]
+        raise ValueError("unknown ids")
+
+    def _convert_token_to_id(self, token: Union[bytes, str]) -> int:
+        """Converts a token to an id using the vocab, special tokens included"""
+        if token in self.special_tokens:
+            return self.special_tokens[token]
+        if token in self.mergeable_ranks:
+            return self.mergeable_ranks[token]
+        raise ValueError("unknown token")
+
+    def _tokenize(self, text: str, **kwargs):
+        """
+        Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based
+        vocabulary or sub-words for sub-word-based vocabularies (BPE/SentencePieces/WordPieces).
+
+        Do NOT take care of added tokens.
+        """
+        raise NotImplementedError
+
+    def _decode(
+        self,
+        token_ids: Union[int, List[int]],
+        skip_special_tokens: bool = False,
+        errors: str = None,
+        **kwargs,
+    ) -> str:
+        if isinstance(token_ids, int):
+            token_ids = [token_ids]
+        if skip_special_tokens:
+            token_ids = [i for i in token_ids if i < self.eod_id]
+        return self.tokenizer.decode(token_ids, errors=errors or self.errors)

LLM-Detector-V2-11w/tokenizer_config.json

@@ -0,0 +1,13 @@
+{
+  "auto_map": {
+    "AutoTokenizer": [
+      "tokenization_qwen.QWenTokenizer",
+      null
+    ]
+  },
+  "clean_up_tokenization_spaces": true,
+  "model_max_length": 8192,
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "QWenTokenizer"
+}

LLM-Detector-V2-11w/train_results.json

@@ -0,0 +1,7 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.02652581251227368,
+    "train_runtime": 75119.2077,
+    "train_samples_per_second": 4.749,
+    "train_steps_per_second": 0.148
+}

LLM-Detector-V2-11w/trainer_log.jsonl

@@ -0,0 +1,112 @@
+{"current_steps": 100, "total_steps": 11148, "loss": 2.2244, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.9990073706856164e-05, "epoch": 0.03, "percentage": 0.9, "elapsed_time": "0:11:12", "remaining_time": "20:37:47"}
+{"current_steps": 200, "total_steps": 11148, "loss": 0.0286, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.99603027099283e-05, "epoch": 0.05, "percentage": 1.79, "elapsed_time": "0:22:37", "remaining_time": "20:38:29"}
+{"current_steps": 300, "total_steps": 11148, "loss": 0.0144, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.991071065046783e-05, "epoch": 0.08, "percentage": 2.69, "elapsed_time": "0:33:36", "remaining_time": "20:15:19"}
+{"current_steps": 400, "total_steps": 11148, "loss": 0.0223, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.984133690970033e-05, "epoch": 0.11, "percentage": 3.59, "elapsed_time": "0:44:40", "remaining_time": "20:00:36"}
+{"current_steps": 500, "total_steps": 11148, "loss": 0.0179, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.975223657755279e-05, "epoch": 0.13, "percentage": 4.49, "elapsed_time": "0:56:02", "remaining_time": "19:53:35"}
+{"current_steps": 600, "total_steps": 11148, "loss": 0.0246, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.9643480408906496e-05, "epoch": 0.16, "percentage": 5.38, "elapsed_time": "1:07:23", "remaining_time": "19:44:49"}
+{"current_steps": 700, "total_steps": 11148, "loss": 0.0181, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.951515476741036e-05, "epoch": 0.19, "percentage": 6.28, "elapsed_time": "1:18:37", "remaining_time": "19:33:33"}
+{"current_steps": 800, "total_steps": 11148, "loss": 0.0155, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.936736155689918e-05, "epoch": 0.22, "percentage": 7.18, "elapsed_time": "1:29:44", "remaining_time": "19:20:42"}
+{"current_steps": 900, "total_steps": 11148, "loss": 0.0122, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.920021814047156e-05, "epoch": 0.24, "percentage": 8.07, "elapsed_time": "1:41:04", "remaining_time": "19:10:52"}
+{"current_steps": 1000, "total_steps": 11148, "loss": 0.0128, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.901385724729137e-05, "epoch": 0.27, "percentage": 8.97, "elapsed_time": "1:52:26", "remaining_time": "19:01:04"}
+{"current_steps": 1100, "total_steps": 11148, "loss": 0.016, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.880842686718711e-05, "epoch": 0.3, "percentage": 9.87, "elapsed_time": "2:03:33", "remaining_time": "18:48:42"}
+{"current_steps": 1200, "total_steps": 11148, "loss": 0.0134, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.858409013313266e-05, "epoch": 0.32, "percentage": 10.76, "elapsed_time": "2:15:16", "remaining_time": "18:41:23"}
+{"current_steps": 1300, "total_steps": 11148, "loss": 0.0099, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.8341025191702847e-05, "epoch": 0.35, "percentage": 11.66, "elapsed_time": "2:26:27", "remaining_time": "18:29:25"}
+{"current_steps": 1400, "total_steps": 11148, "loss": 0.0102, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.80794250616066e-05, "epoch": 0.38, "percentage": 12.56, "elapsed_time": "2:37:43", "remaining_time": "18:18:11"}
+{"current_steps": 1500, "total_steps": 11148, "loss": 0.0109, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.7799497480410125e-05, "epoch": 0.4, "percentage": 13.46, "elapsed_time": "2:48:51", "remaining_time": "18:06:03"}
+{"current_steps": 1600, "total_steps": 11148, "loss": 0.0117, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.7501464739571836e-05, "epoch": 0.43, "percentage": 14.35, "elapsed_time": "3:00:10", "remaining_time": "17:55:10"}
+{"current_steps": 1700, "total_steps": 11148, "loss": 0.0154, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.7185563507919895e-05, "epoch": 0.46, "percentage": 15.25, "elapsed_time": "3:11:16", "remaining_time": "17:43:00"}
+{"current_steps": 1800, "total_steps": 11148, "loss": 0.0107, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.685204464371269e-05, "epoch": 0.48, "percentage": 16.15, "elapsed_time": "3:22:35", "remaining_time": "17:32:08"}
+{"current_steps": 1900, "total_steps": 11148, "loss": 0.0133, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.6501172995431424e-05, "epoch": 0.51, "percentage": 17.04, "elapsed_time": "3:33:46", "remaining_time": "17:20:28"}
+{"current_steps": 2000, "total_steps": 11148, "loss": 0.0157, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.6133227191463044e-05, "epoch": 0.54, "percentage": 17.94, "elapsed_time": "3:45:13", "remaining_time": "17:10:12"}
+{"current_steps": 2100, "total_steps": 11148, "loss": 0.0115, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.574849941884044e-05, "epoch": 0.57, "percentage": 18.84, "elapsed_time": "3:56:12", "remaining_time": "16:57:42"}
+{"current_steps": 2200, "total_steps": 11148, "loss": 0.0135, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.534729519121574e-05, "epoch": 0.59, "percentage": 19.73, "elapsed_time": "4:07:35", "remaining_time": "16:47:00"}
+{"current_steps": 2300, "total_steps": 11148, "loss": 0.0104, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.492993310625088e-05, "epoch": 0.62, "percentage": 20.63, "elapsed_time": "4:18:55", "remaining_time": "16:36:05"}
+{"current_steps": 2400, "total_steps": 11148, "loss": 0.0113, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.449674459261804e-05, "epoch": 0.65, "percentage": 21.53, "elapsed_time": "4:30:14", "remaining_time": "16:25:00"}
+{"current_steps": 2500, "total_steps": 11148, "loss": 0.0075, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.404807364681105e-05, "epoch": 0.67, "percentage": 22.43, "elapsed_time": "4:41:29", "remaining_time": "16:13:45"}
+{"current_steps": 2600, "total_steps": 11148, "loss": 0.011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.358427655997657e-05, "epoch": 0.7, "percentage": 23.32, "elapsed_time": "4:52:40", "remaining_time": "16:02:13"}
+{"current_steps": 2700, "total_steps": 11148, "loss": 0.0103, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.310572163498205e-05, "epoch": 0.73, "percentage": 24.22, "elapsed_time": "5:04:00", "remaining_time": "15:51:11"}
+{"current_steps": 2800, "total_steps": 11148, "loss": 0.0126, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.2612788893945176e-05, "epoch": 0.75, "percentage": 25.12, "elapsed_time": "5:15:26", "remaining_time": "15:40:28"}
+{"current_steps": 2900, "total_steps": 11148, "loss": 0.0084, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.2105869776456944e-05, "epoch": 0.78, "percentage": 26.01, "elapsed_time": "5:26:33", "remaining_time": "15:28:47"}
+{"current_steps": 3000, "total_steps": 11148, "loss": 0.0069, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.158536682873821e-05, "epoch": 0.81, "percentage": 26.91, "elapsed_time": "5:37:50", "remaining_time": "15:17:34"}
+{"current_steps": 3100, "total_steps": 11148, "loss": 0.0089, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.1051693383976264e-05, "epoch": 0.83, "percentage": 27.81, "elapsed_time": "5:49:28", "remaining_time": "15:07:17"}
+{"current_steps": 3200, "total_steps": 11148, "loss": 0.0103, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.050527323409557e-05, "epoch": 0.86, "percentage": 28.7, "elapsed_time": "6:00:48", "remaining_time": "14:56:08"}
+{"current_steps": 3300, "total_steps": 11148, "loss": 0.011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.994654029322313e-05, "epoch": 0.89, "percentage": 29.6, "elapsed_time": "6:12:20", "remaining_time": "14:45:29"}
+{"current_steps": 3400, "total_steps": 11148, "loss": 0.0063, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.937593825311575e-05, "epoch": 0.91, "percentage": 30.5, "elapsed_time": "6:23:47", "remaining_time": "14:34:36"}
+{"current_steps": 3500, "total_steps": 11148, "loss": 0.0091, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.8793920230822925e-05, "epoch": 0.94, "percentage": 31.4, "elapsed_time": "6:34:46", "remaining_time": "14:22:39"}
+{"current_steps": 3600, "total_steps": 11148, "loss": 0.0072, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.8200948408864986e-05, "epoch": 0.97, "percentage": 32.29, "elapsed_time": "6:46:02", "remaining_time": "14:11:19"}
+{"current_steps": 3700, "total_steps": 11148, "loss": 0.007, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.759749366821241e-05, "epoch": 1.0, "percentage": 33.19, "elapsed_time": "6:57:15", "remaining_time": "13:59:55"}
+{"current_steps": 3800, "total_steps": 11148, "loss": 0.0067, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.698403521435756e-05, "epoch": 1.02, "percentage": 34.09, "elapsed_time": "7:08:23", "remaining_time": "13:48:22"}
+{"current_steps": 3900, "total_steps": 11148, "loss": 0.0055, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.636106019677602e-05, "epoch": 1.05, "percentage": 34.98, "elapsed_time": "7:19:37", "remaining_time": "13:37:01"}
+{"current_steps": 4000, "total_steps": 11148, "loss": 0.0071, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.572906332207943e-05, "epoch": 1.08, "percentage": 35.88, "elapsed_time": "7:30:50", "remaining_time": "13:25:39"}
+{"current_steps": 4100, "total_steps": 11148, "loss": 0.0056, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.5088546461167346e-05, "epoch": 1.1, "percentage": 36.78, "elapsed_time": "7:42:28", "remaining_time": "13:15:01"}
+{"current_steps": 4200, "total_steps": 11148, "loss": 0.0074, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.4440018250689767e-05, "epoch": 1.13, "percentage": 37.67, "elapsed_time": "7:53:22", "remaining_time": "13:03:06"}
+{"current_steps": 4300, "total_steps": 11148, "loss": 0.0061, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.3783993689137015e-05, "epoch": 1.16, "percentage": 38.57, "elapsed_time": "8:04:39", "remaining_time": "12:51:51"}
+{"current_steps": 4400, "total_steps": 11148, "loss": 0.0051, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.312099372787772e-05, "epoch": 1.18, "percentage": 39.47, "elapsed_time": "8:15:38", "remaining_time": "12:40:08"}
+{"current_steps": 4500, "total_steps": 11148, "loss": 0.0078, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.2451544857469436e-05, "epoch": 1.21, "percentage": 40.37, "elapsed_time": "8:27:19", "remaining_time": "12:29:28"}
+{"current_steps": 4600, "total_steps": 11148, "loss": 0.0064, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.177617868957078e-05, "epoch": 1.24, "percentage": 41.26, "elapsed_time": "8:38:24", "remaining_time": "12:17:56"}
+{"current_steps": 4700, "total_steps": 11148, "loss": 0.0064, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.109543153478671e-05, "epoch": 1.26, "percentage": 42.16, "elapsed_time": "8:49:17", "remaining_time": "12:06:08"}
+{"current_steps": 4800, "total_steps": 11148, "loss": 0.0064, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.040984397678245e-05, "epoch": 1.29, "percentage": 43.06, "elapsed_time": "9:00:24", "remaining_time": "11:54:41"}
+{"current_steps": 4900, "total_steps": 11148, "loss": 0.0062, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.9719960443004107e-05, "epoch": 1.32, "percentage": 43.95, "elapsed_time": "9:11:44", "remaining_time": "11:43:31"}
+{"current_steps": 5000, "total_steps": 11148, "loss": 0.0056, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.9026328772347e-05, "epoch": 1.35, "percentage": 44.85, "elapsed_time": "9:22:47", "remaining_time": "11:32:01"}
+{"current_steps": 5100, "total_steps": 11148, "loss": 0.0096, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.8329499780114865e-05, "epoch": 1.37, "percentage": 45.75, "elapsed_time": "9:33:55", "remaining_time": "11:20:36"}
+{"current_steps": 5200, "total_steps": 11148, "loss": 0.0054, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.763002682061555e-05, "epoch": 1.4, "percentage": 46.65, "elapsed_time": "9:44:58", "remaining_time": "11:09:07"}
+{"current_steps": 5300, "total_steps": 11148, "loss": 0.0049, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.6928465347740434e-05, "epoch": 1.43, "percentage": 47.54, "elapsed_time": "9:56:00", "remaining_time": "10:57:37"}
+{"current_steps": 5400, "total_steps": 11148, "loss": 0.0023, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.6225372473876565e-05, "epoch": 1.45, "percentage": 48.44, "elapsed_time": "10:07:17", "remaining_time": "10:46:25"}
+{"current_steps": 5500, "total_steps": 11148, "loss": 0.0065, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.552835016828629e-05, "epoch": 1.48, "percentage": 49.34, "elapsed_time": "10:18:24", "remaining_time": "10:35:03"}
+{"current_steps": 5600, "total_steps": 11148, "loss": 0.0032, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.4823871621313255e-05, "epoch": 1.51, "percentage": 50.23, "elapsed_time": "10:29:47", "remaining_time": "10:23:56"}
+{"current_steps": 5700, "total_steps": 11148, "loss": 0.0043, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.411953293849365e-05, "epoch": 1.53, "percentage": 51.13, "elapsed_time": "10:40:54", "remaining_time": "10:12:34"}
+{"current_steps": 5800, "total_steps": 11148, "loss": 0.0038, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.341589343760652e-05, "epoch": 1.56, "percentage": 52.03, "elapsed_time": "10:51:35", "remaining_time": "10:00:48"}
+{"current_steps": 5900, "total_steps": 11148, "loss": 0.0048, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.271351188120813e-05, "epoch": 1.59, "percentage": 52.92, "elapsed_time": "11:03:01", "remaining_time": "9:49:45"}
+{"current_steps": 6000, "total_steps": 11148, "loss": 0.0082, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.2012946032916717e-05, "epoch": 1.61, "percentage": 53.82, "elapsed_time": "11:14:20", "remaining_time": "9:38:34"}
+{"current_steps": 6100, "total_steps": 11148, "loss": 0.0046, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.1314752214490396e-05, "epoch": 1.64, "percentage": 54.72, "elapsed_time": "11:25:45", "remaining_time": "9:27:29"}
+{"current_steps": 6200, "total_steps": 11148, "loss": 0.0063, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.061948486405021e-05, "epoch": 1.67, "percentage": 55.62, "elapsed_time": "11:36:47", "remaining_time": "9:16:05"}
+{"current_steps": 6300, "total_steps": 11148, "loss": 0.0052, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.992769609579885e-05, "epoch": 1.7, "percentage": 56.51, "elapsed_time": "11:47:49", "remaining_time": "9:04:41"}
+{"current_steps": 6400, "total_steps": 11148, "loss": 0.0041, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.9239935261584917e-05, "epoch": 1.72, "percentage": 57.41, "elapsed_time": "11:59:07", "remaining_time": "8:53:29"}
+{"current_steps": 6500, "total_steps": 11148, "loss": 0.004, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.8556748514660664e-05, "epoch": 1.75, "percentage": 58.31, "elapsed_time": "12:10:29", "remaining_time": "8:42:21"}
+{"current_steps": 6600, "total_steps": 11148, "loss": 0.0035, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.7878678375979845e-05, "epoch": 1.78, "percentage": 59.2, "elapsed_time": "12:22:04", "remaining_time": "8:31:21"}
+{"current_steps": 6700, "total_steps": 11148, "loss": 0.0066, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.7206263303379948e-05, "epoch": 1.8, "percentage": 60.1, "elapsed_time": "12:33:49", "remaining_time": "8:20:27"}
+{"current_steps": 6800, "total_steps": 11148, "loss": 0.0047, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.654003726399097e-05, "epoch": 1.83, "percentage": 61.0, "elapsed_time": "12:44:59", "remaining_time": "8:09:08"}
+{"current_steps": 6900, "total_steps": 11148, "loss": 0.005, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.5880529310210283e-05, "epoch": 1.86, "percentage": 61.89, "elapsed_time": "12:56:13", "remaining_time": "7:57:53"}
+{"current_steps": 7000, "total_steps": 11148, "loss": 0.0033, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.5228263159580275e-05, "epoch": 1.88, "percentage": 62.79, "elapsed_time": "13:07:30", "remaining_time": "7:46:39"}
+{"current_steps": 7100, "total_steps": 11148, "loss": 0.0059, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.4583756778902463e-05, "epoch": 1.91, "percentage": 63.69, "elapsed_time": "13:18:37", "remaining_time": "7:35:19"}
+{"current_steps": 7200, "total_steps": 11148, "loss": 0.0051, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.3947521972918251e-05, "epoch": 1.94, "percentage": 64.59, "elapsed_time": "13:29:49", "remaining_time": "7:24:03"}
+{"current_steps": 7300, "total_steps": 11148, "loss": 0.0041, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.3320063977883046e-05, "epoch": 1.96, "percentage": 65.48, "elapsed_time": "13:41:06", "remaining_time": "7:12:49"}
+{"current_steps": 7400, "total_steps": 11148, "loss": 0.0049, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.2701881060356396e-05, "epoch": 1.99, "percentage": 66.38, "elapsed_time": "13:52:06", "remaining_time": "7:01:27"}
+{"current_steps": 7500, "total_steps": 11148, "loss": 0.0024, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.209346412152676e-05, "epoch": 2.02, "percentage": 67.28, "elapsed_time": "14:03:11", "remaining_time": "6:50:07"}
+{"current_steps": 7600, "total_steps": 11148, "loss": 0.0035, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.149529630738521e-05, "epoch": 2.04, "percentage": 68.17, "elapsed_time": "14:14:31", "remaining_time": "6:38:55"}
+{"current_steps": 7700, "total_steps": 11148, "loss": 0.0013, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.0907852625057538e-05, "epoch": 2.07, "percentage": 69.07, "elapsed_time": "14:25:35", "remaining_time": "6:27:36"}
+{"current_steps": 7800, "total_steps": 11148, "loss": 0.0014, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.0331599565599464e-05, "epoch": 2.1, "percentage": 69.97, "elapsed_time": "14:36:28", "remaining_time": "6:16:12"}
+{"current_steps": 7900, "total_steps": 11148, "loss": 0.004, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 9.766994733554432e-06, "epoch": 2.13, "percentage": 70.86, "elapsed_time": "14:47:19", "remaining_time": "6:04:48"}
+{"current_steps": 8000, "total_steps": 11148, "loss": 0.0021, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 9.214486483568307e-06, "epoch": 2.15, "percentage": 71.76, "elapsed_time": "14:58:44", "remaining_time": "5:53:39"}
+{"current_steps": 8100, "total_steps": 11148, "loss": 0.0025, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 8.674513564349376e-06, "epoch": 2.18, "percentage": 72.66, "elapsed_time": "15:09:39", "remaining_time": "5:42:17"}
+{"current_steps": 8200, "total_steps": 11148, "loss": 0.0026, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 8.147504770256536e-06, "epoch": 2.21, "percentage": 73.56, "elapsed_time": "15:20:57", "remaining_time": "5:31:05"}
+{"current_steps": 8300, "total_steps": 11148, "loss": 0.0018, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 7.633878600792143e-06, "epoch": 2.23, "percentage": 74.45, "elapsed_time": "15:32:04", "remaining_time": "5:19:49"}
+{"current_steps": 8400, "total_steps": 11148, "loss": 0.0016, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 7.138971709507527e-06, "epoch": 2.26, "percentage": 75.35, "elapsed_time": "15:43:11", "remaining_time": "5:08:33"}
+{"current_steps": 8500, "total_steps": 11148, "loss": 0.003, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 6.653179652954706e-06, "epoch": 2.29, "percentage": 76.25, "elapsed_time": "15:54:31", "remaining_time": "4:57:21"}
+{"current_steps": 8600, "total_steps": 11148, "loss": 0.0011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 6.181956869763645e-06, "epoch": 2.31, "percentage": 77.14, "elapsed_time": "16:05:52", "remaining_time": "4:46:10"}
+{"current_steps": 8700, "total_steps": 11148, "loss": 0.0024, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 5.725677559572909e-06, "epoch": 2.34, "percentage": 78.04, "elapsed_time": "16:17:01", "remaining_time": "4:34:54"}
+{"current_steps": 8800, "total_steps": 11148, "loss": 0.0007, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 5.284704055357578e-06, "epoch": 2.37, "percentage": 78.94, "elapsed_time": "16:28:18", "remaining_time": "4:23:41"}
+{"current_steps": 8900, "total_steps": 11148, "loss": 0.0017, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.859386535699373e-06, "epoch": 2.39, "percentage": 79.83, "elapsed_time": "16:39:37", "remaining_time": "4:12:29"}
+{"current_steps": 9000, "total_steps": 11148, "loss": 0.0021, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.4500627467086245e-06, "epoch": 2.42, "percentage": 80.73, "elapsed_time": "16:51:12", "remaining_time": "4:01:20"}
+{"current_steps": 9100, "total_steps": 11148, "loss": 0.0016, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.057057733818967e-06, "epoch": 2.45, "percentage": 81.63, "elapsed_time": "17:02:20", "remaining_time": "3:50:04"}
+{"current_steps": 9200, "total_steps": 11148, "loss": 0.0013, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.680683583667599e-06, "epoch": 2.48, "percentage": 82.53, "elapsed_time": "17:13:20", "remaining_time": "3:38:47"}
+{"current_steps": 9300, "total_steps": 11148, "loss": 0.0028, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.321239176266211e-06, "epoch": 2.5, "percentage": 83.42, "elapsed_time": "17:24:23", "remaining_time": "3:27:31"}
+{"current_steps": 9400, "total_steps": 11148, "loss": 0.0007, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.9790099476593393e-06, "epoch": 2.53, "percentage": 84.32, "elapsed_time": "17:35:33", "remaining_time": "3:16:17"}
+{"current_steps": 9500, "total_steps": 11148, "loss": 0.0026, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.654267663258628e-06, "epoch": 2.56, "percentage": 85.22, "elapsed_time": "17:47:00", "remaining_time": "3:05:05"}
+{"current_steps": 9600, "total_steps": 11148, "loss": 0.0013, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.3472702020329696e-06, "epoch": 2.58, "percentage": 86.11, "elapsed_time": "17:58:19", "remaining_time": "2:53:52"}
+{"current_steps": 9700, "total_steps": 11148, "loss": 0.0014, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.0582613517259377e-06, "epoch": 2.61, "percentage": 87.01, "elapsed_time": "18:09:34", "remaining_time": "2:42:39"}
+{"current_steps": 9800, "total_steps": 11148, "loss": 0.0011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.7874706152630705e-06, "epoch": 2.64, "percentage": 87.91, "elapsed_time": "18:20:45", "remaining_time": "2:31:24"}
+{"current_steps": 9900, "total_steps": 11148, "loss": 0.0022, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.5351130285028214e-06, "epoch": 2.66, "percentage": 88.81, "elapsed_time": "18:31:59", "remaining_time": "2:20:10"}
+{"current_steps": 10000, "total_steps": 11148, "loss": 0.0012, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.3036333720407912e-06, "epoch": 2.69, "percentage": 89.7, "elapsed_time": "18:42:49", "remaining_time": "2:08:54"}
+{"current_steps": 10100, "total_steps": 11148, "loss": 0.0018, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.0885394205533805e-06, "epoch": 2.72, "percentage": 90.6, "elapsed_time": "18:54:12", "remaining_time": "1:57:41"}
+{"current_steps": 10200, "total_steps": 11148, "loss": 0.0021, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 8.924336424426832e-07, "epoch": 2.74, "percentage": 91.5, "elapsed_time": "19:05:08", "remaining_time": "1:46:25"}
+{"current_steps": 10300, "total_steps": 11148, "loss": 0.0022, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 7.154717659839549e-07, "epoch": 2.77, "percentage": 92.39, "elapsed_time": "19:16:30", "remaining_time": "1:35:12"}
+{"current_steps": 10400, "total_steps": 11148, "loss": 0.002, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 5.57794317214072e-07, "epoch": 2.8, "percentage": 93.29, "elapsed_time": "19:27:31", "remaining_time": "1:23:58"}
+{"current_steps": 10500, "total_steps": 11148, "loss": 0.0017, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 4.1952650833933515e-07, "epoch": 2.83, "percentage": 94.19, "elapsed_time": "19:38:44", "remaining_time": "1:12:44"}
+{"current_steps": 10600, "total_steps": 11148, "loss": 0.0024, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 3.007781383039976e-07, "epoch": 2.85, "percentage": 95.08, "elapsed_time": "19:50:06", "remaining_time": "1:01:31"}
+{"current_steps": 10700, "total_steps": 11148, "loss": 0.0011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.0164350559856714e-07, "epoch": 2.88, "percentage": 95.98, "elapsed_time": "20:01:33", "remaining_time": "0:50:18"}
+{"current_steps": 10800, "total_steps": 11148, "loss": 0.0013, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 1.222013333770422e-07, "epoch": 2.91, "percentage": 96.88, "elapsed_time": "20:12:33", "remaining_time": "0:39:04"}
+{"current_steps": 10900, "total_steps": 11148, "loss": 0.0011, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 6.251470694257356e-08, "epoch": 2.93, "percentage": 97.78, "elapsed_time": "20:24:02", "remaining_time": "0:27:50"}
+{"current_steps": 11000, "total_steps": 11148, "loss": 0.003, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.2631023651226157e-08, "epoch": 2.96, "percentage": 98.67, "elapsed_time": "20:35:15", "remaining_time": "0:16:37"}
+{"current_steps": 11100, "total_steps": 11148, "loss": 0.0026, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": 2.581955273558978e-09, "epoch": 2.99, "percentage": 99.57, "elapsed_time": "20:46:31", "remaining_time": "0:05:23"}
+{"current_steps": 11148, "total_steps": 11148, "loss": null, "eval_loss": null, "predict_loss": null, "reward": null, "learning_rate": null, "epoch": 3.0, "percentage": 100.0, "elapsed_time": "20:51:59", "remaining_time": "0:00:00"}

LLM-Detector-V2-11w/trainer_state.json

@@ -0,0 +1,694 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.9995963944571504,
+  "eval_steps": 500,
+  "global_step": 11148,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03,
+      "learning_rate": 4.9990073706856164e-05,
+      "loss": 2.2244,
+      "step": 100
+    },
+    {
+      "epoch": 0.05,
+      "learning_rate": 4.99603027099283e-05,
+      "loss": 0.0286,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "learning_rate": 4.991071065046783e-05,
+      "loss": 0.0144,
+      "step": 300
+    },
+    {
+      "epoch": 0.11,
+      "learning_rate": 4.984133690970033e-05,
+      "loss": 0.0223,
+      "step": 400
+    },
+    {
+      "epoch": 0.13,
+      "learning_rate": 4.975223657755279e-05,
+      "loss": 0.0179,
+      "step": 500
+    },
+    {
+      "epoch": 0.16,
+      "learning_rate": 4.9643480408906496e-05,
+      "loss": 0.0246,
+      "step": 600
+    },
+    {
+      "epoch": 0.19,
+      "learning_rate": 4.951515476741036e-05,
+      "loss": 0.0181,
+      "step": 700
+    },
+    {
+      "epoch": 0.22,
+      "learning_rate": 4.936736155689918e-05,
+      "loss": 0.0155,
+      "step": 800
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 4.920021814047156e-05,
+      "loss": 0.0122,
+      "step": 900
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 4.901385724729137e-05,
+      "loss": 0.0128,
+      "step": 1000
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 4.880842686718711e-05,
+      "loss": 0.016,
+      "step": 1100
+    },
+    {
+      "epoch": 0.32,
+      "learning_rate": 4.858409013313266e-05,
+      "loss": 0.0134,
+      "step": 1200
+    },
+    {
+      "epoch": 0.35,
+      "learning_rate": 4.8341025191702847e-05,
+      "loss": 0.0099,
+      "step": 1300
+    },
+    {
+      "epoch": 0.38,
+      "learning_rate": 4.80794250616066e-05,
+      "loss": 0.0102,
+      "step": 1400
+    },
+    {
+      "epoch": 0.4,
+      "learning_rate": 4.7799497480410125e-05,
+      "loss": 0.0109,
+      "step": 1500
+    },
+    {
+      "epoch": 0.43,
+      "learning_rate": 4.7501464739571836e-05,
+      "loss": 0.0117,
+      "step": 1600
+    },
+    {
+      "epoch": 0.46,
+      "learning_rate": 4.7185563507919895e-05,
+      "loss": 0.0154,
+      "step": 1700
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 4.685204464371269e-05,
+      "loss": 0.0107,
+      "step": 1800
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 4.6501172995431424e-05,
+      "loss": 0.0133,
+      "step": 1900
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 4.6133227191463044e-05,
+      "loss": 0.0157,
+      "step": 2000
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 4.574849941884044e-05,
+      "loss": 0.0115,
+      "step": 2100
+    },
+    {
+      "epoch": 0.59,
+      "learning_rate": 4.534729519121574e-05,
+      "loss": 0.0135,
+      "step": 2200
+    },
+    {
+      "epoch": 0.62,
+      "learning_rate": 4.492993310625088e-05,
+      "loss": 0.0104,
+      "step": 2300
+    },
+    {
+      "epoch": 0.65,
+      "learning_rate": 4.449674459261804e-05,
+      "loss": 0.0113,
+      "step": 2400
+    },
+    {
+      "epoch": 0.67,
+      "learning_rate": 4.404807364681105e-05,
+      "loss": 0.0075,
+      "step": 2500
+    },
+    {
+      "epoch": 0.7,
+      "learning_rate": 4.358427655997657e-05,
+      "loss": 0.011,
+      "step": 2600
+    },
+    {
+      "epoch": 0.73,
+      "learning_rate": 4.310572163498205e-05,
+      "loss": 0.0103,
+      "step": 2700
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 4.2612788893945176e-05,
+      "loss": 0.0126,
+      "step": 2800
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 4.2105869776456944e-05,
+      "loss": 0.0084,
+      "step": 2900
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 4.158536682873821e-05,
+      "loss": 0.0069,
+      "step": 3000
+    },
+    {
+      "epoch": 0.83,
+      "learning_rate": 4.1051693383976264e-05,
+      "loss": 0.0089,
+      "step": 3100
+    },
+    {
+      "epoch": 0.86,
+      "learning_rate": 4.050527323409557e-05,
+      "loss": 0.0103,
+      "step": 3200
+    },
+    {
+      "epoch": 0.89,
+      "learning_rate": 3.994654029322313e-05,
+      "loss": 0.011,
+      "step": 3300
+    },
+    {
+      "epoch": 0.91,
+      "learning_rate": 3.937593825311575e-05,
+      "loss": 0.0063,
+      "step": 3400
+    },
+    {
+      "epoch": 0.94,
+      "learning_rate": 3.8793920230822925e-05,
+      "loss": 0.0091,
+      "step": 3500
+    },
+    {
+      "epoch": 0.97,
+      "learning_rate": 3.8200948408864986e-05,
+      "loss": 0.0072,
+      "step": 3600
+    },
+    {
+      "epoch": 1.0,
+      "learning_rate": 3.759749366821241e-05,
+      "loss": 0.007,
+      "step": 3700
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 3.698403521435756e-05,
+      "loss": 0.0067,
+      "step": 3800
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 3.636106019677602e-05,
+      "loss": 0.0055,
+      "step": 3900
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 3.572906332207943e-05,
+      "loss": 0.0071,
+      "step": 4000
+    },
+    {
+      "epoch": 1.1,
+      "learning_rate": 3.5088546461167346e-05,
+      "loss": 0.0056,
+      "step": 4100
+    },
+    {
+      "epoch": 1.13,
+      "learning_rate": 3.4440018250689767e-05,
+      "loss": 0.0074,
+      "step": 4200
+    },
+    {
+      "epoch": 1.16,
+      "learning_rate": 3.3783993689137015e-05,
+      "loss": 0.0061,
+      "step": 4300
+    },
+    {
+      "epoch": 1.18,
+      "learning_rate": 3.312099372787772e-05,
+      "loss": 0.0051,
+      "step": 4400
+    },
+    {
+      "epoch": 1.21,
+      "learning_rate": 3.2451544857469436e-05,
+      "loss": 0.0078,
+      "step": 4500
+    },
+    {
+      "epoch": 1.24,
+      "learning_rate": 3.177617868957078e-05,
+      "loss": 0.0064,
+      "step": 4600
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 3.109543153478671e-05,
+      "loss": 0.0064,
+      "step": 4700
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 3.040984397678245e-05,
+      "loss": 0.0064,
+      "step": 4800
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 2.9719960443004107e-05,
+      "loss": 0.0062,
+      "step": 4900
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 2.9026328772347e-05,
+      "loss": 0.0056,
+      "step": 5000
+    },
+    {
+      "epoch": 1.37,
+      "learning_rate": 2.8329499780114865e-05,
+      "loss": 0.0096,
+      "step": 5100
+    },
+    {
+      "epoch": 1.4,
+      "learning_rate": 2.763002682061555e-05,
+      "loss": 0.0054,
+      "step": 5200
+    },
+    {
+      "epoch": 1.43,
+      "learning_rate": 2.6928465347740434e-05,
+      "loss": 0.0049,
+      "step": 5300
+    },
+    {
+      "epoch": 1.45,
+      "learning_rate": 2.6225372473876565e-05,
+      "loss": 0.0023,
+      "step": 5400
+    },
+    {
+      "epoch": 1.48,
+      "learning_rate": 2.552835016828629e-05,
+      "loss": 0.0065,
+      "step": 5500
+    },
+    {
+      "epoch": 1.51,
+      "learning_rate": 2.4823871621313255e-05,
+      "loss": 0.0032,
+      "step": 5600
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 2.411953293849365e-05,
+      "loss": 0.0043,
+      "step": 5700
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 2.341589343760652e-05,
+      "loss": 0.0038,
+      "step": 5800
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 2.271351188120813e-05,
+      "loss": 0.0048,
+      "step": 5900
+    },
+    {
+      "epoch": 1.61,
+      "learning_rate": 2.2012946032916717e-05,
+      "loss": 0.0082,
+      "step": 6000
+    },
+    {
+      "epoch": 1.64,
+      "learning_rate": 2.1314752214490396e-05,
+      "loss": 0.0046,
+      "step": 6100
+    },
+    {
+      "epoch": 1.67,
+      "learning_rate": 2.061948486405021e-05,
+      "loss": 0.0063,
+      "step": 6200
+    },
+    {
+      "epoch": 1.7,
+      "learning_rate": 1.992769609579885e-05,
+      "loss": 0.0052,
+      "step": 6300
+    },
+    {
+      "epoch": 1.72,
+      "learning_rate": 1.9239935261584917e-05,
+      "loss": 0.0041,
+      "step": 6400
+    },
+    {
+      "epoch": 1.75,
+      "learning_rate": 1.8556748514660664e-05,
+      "loss": 0.004,
+      "step": 6500
+    },
+    {
+      "epoch": 1.78,
+      "learning_rate": 1.7878678375979845e-05,
+      "loss": 0.0035,
+      "step": 6600
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 1.7206263303379948e-05,
+      "loss": 0.0066,
+      "step": 6700
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 1.654003726399097e-05,
+      "loss": 0.0047,
+      "step": 6800
+    },
+    {
+      "epoch": 1.86,
+      "learning_rate": 1.5880529310210283e-05,
+      "loss": 0.005,
+      "step": 6900
+    },
+    {
+      "epoch": 1.88,
+      "learning_rate": 1.5228263159580275e-05,
+      "loss": 0.0033,
+      "step": 7000
+    },
+    {
+      "epoch": 1.91,
+      "learning_rate": 1.4583756778902463e-05,
+      "loss": 0.0059,
+      "step": 7100
+    },
+    {
+      "epoch": 1.94,
+      "learning_rate": 1.3947521972918251e-05,
+      "loss": 0.0051,
+      "step": 7200
+    },
+    {
+      "epoch": 1.96,
+      "learning_rate": 1.3320063977883046e-05,
+      "loss": 0.0041,
+      "step": 7300
+    },
+    {
+      "epoch": 1.99,
+      "learning_rate": 1.2701881060356396e-05,
+      "loss": 0.0049,
+      "step": 7400
+    },
+    {
+      "epoch": 2.02,
+      "learning_rate": 1.209346412152676e-05,
+      "loss": 0.0024,
+      "step": 7500
+    },
+    {
+      "epoch": 2.04,
+      "learning_rate": 1.149529630738521e-05,
+      "loss": 0.0035,
+      "step": 7600
+    },
+    {
+      "epoch": 2.07,
+      "learning_rate": 1.0907852625057538e-05,
+      "loss": 0.0013,
+      "step": 7700
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 1.0331599565599464e-05,
+      "loss": 0.0014,
+      "step": 7800
+    },
+    {
+      "epoch": 2.13,
+      "learning_rate": 9.766994733554432e-06,
+      "loss": 0.004,
+      "step": 7900
+    },
+    {
+      "epoch": 2.15,
+      "learning_rate": 9.214486483568307e-06,
+      "loss": 0.0021,
+      "step": 8000
+    },
+    {
+      "epoch": 2.18,
+      "learning_rate": 8.674513564349376e-06,
+      "loss": 0.0025,
+      "step": 8100
+    },
+    {
+      "epoch": 2.21,
+      "learning_rate": 8.147504770256536e-06,
+      "loss": 0.0026,
+      "step": 8200
+    },
+    {
+      "epoch": 2.23,
+      "learning_rate": 7.633878600792143e-06,
+      "loss": 0.0018,
+      "step": 8300
+    },
+    {
+      "epoch": 2.26,
+      "learning_rate": 7.138971709507527e-06,
+      "loss": 0.0016,
+      "step": 8400
+    },
+    {
+      "epoch": 2.29,
+      "learning_rate": 6.653179652954706e-06,
+      "loss": 0.003,
+      "step": 8500
+    },
+    {
+      "epoch": 2.31,
+      "learning_rate": 6.181956869763645e-06,
+      "loss": 0.0011,
+      "step": 8600
+    },
+    {
+      "epoch": 2.34,
+      "learning_rate": 5.725677559572909e-06,
+      "loss": 0.0024,
+      "step": 8700
+    },
+    {
+      "epoch": 2.37,
+      "learning_rate": 5.284704055357578e-06,
+      "loss": 0.0007,
+      "step": 8800
+    },
+    {
+      "epoch": 2.39,
+      "learning_rate": 4.859386535699373e-06,
+      "loss": 0.0017,
+      "step": 8900
+    },
+    {
+      "epoch": 2.42,
+      "learning_rate": 4.4500627467086245e-06,
+      "loss": 0.0021,
+      "step": 9000
+    },
+    {
+      "epoch": 2.45,
+      "learning_rate": 4.057057733818967e-06,
+      "loss": 0.0016,
+      "step": 9100
+    },
+    {
+      "epoch": 2.48,
+      "learning_rate": 3.680683583667599e-06,
+      "loss": 0.0013,
+      "step": 9200
+    },
+    {
+      "epoch": 2.5,
+      "learning_rate": 3.321239176266211e-06,
+      "loss": 0.0028,
+      "step": 9300
+    },
+    {
+      "epoch": 2.53,
+      "learning_rate": 2.9790099476593393e-06,
+      "loss": 0.0007,
+      "step": 9400
+    },
+    {
+      "epoch": 2.56,
+      "learning_rate": 2.654267663258628e-06,
+      "loss": 0.0026,
+      "step": 9500
+    },
+    {
+      "epoch": 2.58,
+      "learning_rate": 2.3472702020329696e-06,
+      "loss": 0.0013,
+      "step": 9600
+    },
+    {
+      "epoch": 2.61,
+      "learning_rate": 2.0582613517259377e-06,
+      "loss": 0.0014,
+      "step": 9700
+    },
+    {
+      "epoch": 2.64,
+      "learning_rate": 1.7874706152630705e-06,
+      "loss": 0.0011,
+      "step": 9800
+    },
+    {
+      "epoch": 2.66,
+      "learning_rate": 1.5351130285028214e-06,
+      "loss": 0.0022,
+      "step": 9900
+    },
+    {
+      "epoch": 2.69,
+      "learning_rate": 1.3036333720407912e-06,
+      "loss": 0.0012,
+      "step": 10000
+    },
+    {
+      "epoch": 2.72,
+      "learning_rate": 1.0885394205533805e-06,
+      "loss": 0.0018,
+      "step": 10100
+    },
+    {
+      "epoch": 2.74,
+      "learning_rate": 8.924336424426832e-07,
+      "loss": 0.0021,
+      "step": 10200
+    },
+    {
+      "epoch": 2.77,
+      "learning_rate": 7.154717659839549e-07,
+      "loss": 0.0022,
+      "step": 10300
+    },
+    {
+      "epoch": 2.8,
+      "learning_rate": 5.57794317214072e-07,
+      "loss": 0.002,
+      "step": 10400
+    },
+    {
+      "epoch": 2.83,
+      "learning_rate": 4.1952650833933515e-07,
+      "loss": 0.0017,
+      "step": 10500
+    },
+    {
+      "epoch": 2.85,
+      "learning_rate": 3.007781383039976e-07,
+      "loss": 0.0024,
+      "step": 10600
+    },
+    {
+      "epoch": 2.88,
+      "learning_rate": 2.0164350559856714e-07,
+      "loss": 0.0011,
+      "step": 10700
+    },
+    {
+      "epoch": 2.91,
+      "learning_rate": 1.222013333770422e-07,
+      "loss": 0.0013,
+      "step": 10800
+    },
+    {
+      "epoch": 2.93,
+      "learning_rate": 6.251470694257356e-08,
+      "loss": 0.0011,
+      "step": 10900
+    },
+    {
+      "epoch": 2.96,
+      "learning_rate": 2.2631023651226157e-08,
+      "loss": 0.003,
+      "step": 11000
+    },
+    {
+      "epoch": 2.99,
+      "learning_rate": 2.581955273558978e-09,
+      "loss": 0.0026,
+      "step": 11100
+    },
+    {
+      "epoch": 3.0,
+      "step": 11148,
+      "total_flos": 2.3299941680566764e+18,
+      "train_loss": 0.02652581251227368,
+      "train_runtime": 75119.2077,
+      "train_samples_per_second": 4.749,
+      "train_steps_per_second": 0.148
+    }
+  ],
+  "logging_steps": 100,
+  "max_steps": 11148,
+  "num_train_epochs": 3,
+  "save_steps": 5000,
+  "total_flos": 2.3299941680566764e+18,
+  "trial_name": null,
+  "trial_params": null
+}

LLM-Detector-V2-11w/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2501d0db9b5d010a4a25759f877d55c11a9cd335425e4c2cc6f2683c0ebb1612
+size 4600