--- base_model: Qwen/Qwen2-1.5B datasets: - macadeliccc/opus_samantha - teknium/OpenHermes-2.5 - cognitivecomputations/samantha-data - cognitivecomputations/samantha-1.5 - HuggingfaceH4/ultrachat_200k - jondurbin/airoboros-3.2 - microsoft/orca-math-word-problems-200k - Sao10K/Claude-3-Opus-Instruct-15K - Locutusque/function-calling-chatml - Migtissera/Hitchhikers --- # Samantha Qwen2 1.5B This model was trained on 2xL40S using FSDP and QLoRa. Adapter is available [here](https://huggingface.co/macadeliccc/Samantha-Qwen2-1.5B-QLoRa). Character based RP that is mostly SFW. Trained on a mix of rp, uncensored data, samantha, function calling, etc.. ## Prompt Template ``` <|im_start|>system You are a helpful AI assistant<|im_end|> <|im_start|>user What is the capital of France?<|im_end|> <|im_start|>assistant ``` ## Example output "In a hypothetical case, a social media company implements an AI-powered content moderation system. This system mistakenly flags and removes a user's post criticizing a political candidate, citing it as 'hate speech.' The user sues the company, claiming violation of their First Amendment rights. How would you analyze this case, considering: 1. The First Amendment's application to private companies 2. Section 230 of the Communications Decency Act 3. The growing debate around AI decision-making in content moderation 4. Potential liability for AI-driven mistakes" Response: Analyzing this case, we must consider the following aspects: 1. First Amendment application to private companies: The First Amendment is a federal constitutional right that protects free speech. While public companies must follow antitrust laws, private companies, including social media companies, can still be held accountable for their actions under the First Amendment. 2. Section 230 of the Communications Decency Act: Section 230 of the Communications Decency Act is a federal law that shields internet companies from liability for certain types of content that they host or facilitate, such as indecent or defamatory content. This legislation aims to protect internet companies from being burdened by the responsibility of moderating content on their platforms. 3. The growing debate around AI decision-making in content moderation: The role of AI in content moderation continues to evolve, with some arguing that it could improve accuracy and reduce bias. However, others see the potential for AI to make mistakes and argue that companies must be held accountable for their actions. There is a growing debate within the tech industry about the ethics and liability of AI-powered content moderation systems. 4. Liability for AI-driven mistakes: In the hypothetical case, the social media company could be held liable for the AI's mistakes if the mistake was deliberate or caused harm. The company would need to demonstrate that they acted negligently, failed to take reasonable steps to prevent the AI from making the mistake, or ignored available information that would have prevented the mistake. In conclusion, the case raises complex legal and ethical questions about the First Amendment, Section 230, and the liability of AI-driven content moderation systems. While the company could argue that it did not cause any harm and that the AI acted within its own rights, the company would need to demonstrate that its actions were reasonable and that it took appropriate measures to prevent mistakes from occurring. ## Launch Using VLLM ```bash python -m vllm.entrypoints.openai.api_server \ --model macadeliccc/Samantha-Qwen2-1.5B \ --chat-template ./examples/template_chatml.jinja \ ``` ```python from openai import OpenAI # Set OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) chat_response = client.chat.completions.create( model="macadeliccc/Samantha-Qwen2-1.5B", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Tell me a joke."}, ] ) print("Chat response:", chat_response) ``` ## Quants TODO ## Config [Built with Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl)
See axolotl config axolotl version: `0.4.0` ```yaml base_model: Qwen/Qwen2-1.5B trust_remote_code: true load_in_8bit: false load_in_4bit: true strict: false datasets: - path: macadeliccc/opus_samantha type: sharegpt field: conversations conversation: chatml - path: json data_files: uncensored_ultrachat_20k_sharegpt.json type: sharegpt field: conversations conversation: chatml - path: json data_files: flattened_openhermes_200k.json type: sharegpt field: conversations conversation: chatml - path: json data_files: opus_instruct.json type: sharegpt field: conversations conversation: chatml - path: json data_files: airoboros_uncensored.json type: sharegpt field: conversations conversation: chatml - path: json data_files: orca_math_word_problems_sharegpt.json type: sharegpt field: conversations conversation: chatml - path: json data_files: sharegpt_starcoder.json type: sharegpt field: conversations conversation: chatml - path: json data_files: samantha_1.1_uncensored.json type: sharegpt field: conversations conversation: chatml - path: json data_files: samantha_1.5.json type: sharegpt field: conversations conversation: chatml - path: json data_files: sharegpt_hitchhikers_v1.json type: sharegpt field: conversations conversation: chatml chat_template: chatml dataset_prepared_path: val_set_size: 0.05 output_dir: ./outputs/out sequence_len: 4096 sample_packing: true eval_sample_packing: true pad_to_sequence_len: true adapter: qlora lora_model_dir: lora_r: 32 lora_alpha: 64 lora_dropout: 0.05 lora_target_linear: true lora_fan_in_fan_out: wandb_project: wandb_entity: wandb_watch: wandb_name: wandb_log_model: gradient_accumulation_steps: 4 micro_batch_size: 1 num_epochs: 3 optimizer: adamw_torch lr_scheduler: cosine learning_rate: 0.0002 train_on_inputs: false group_by_length: false bf16: auto fp16: tf32: true gradient_checkpointing: true gradient_checkpointing_kwargs: use_reentrant: false early_stopping_patience: resume_from_checkpoint: local_rank: logging_steps: 1 xformers_attention: flash_attention: true warmup_steps: 10 evals_per_epoch: 4 saves_per_epoch: 1 debug: deepspeed: weight_decay: 0.0 fsdp: - full_shard - auto_wrap fsdp_config: fsdp_limit_all_gathers: true fsdp_sync_module_states: true fsdp_offload_params: true fsdp_use_orig_params: false fsdp_cpu_ram_efficient_loading: true fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP fsdp_transformer_layer_cls_to_wrap: Qwen2DecoderLayer fsdp_state_dict_type: FULL_STATE_DICT special_tokens: ```