Text Web UI Generation Blanks (goes Black) on a Character
After some conversation, about five minutes worth maybe more. This is what I am getting as a result. I have searched for this and have not found a solution. I am guessing it deals with the Character due to the fact if I go to another character. I do not have this issue until I speak to it for 5 minutes. I have messed with the “max_new_tokens” and it doesn’t clear the issue. I have to restart the server to conversate again but this will happen again with another 5 minutes of conversation.I have done other experiments with the Llama.cpp loader as well as other playing around. I have changed the offloading to the GPU, batch and other parameters.Please note that this does occur with 13B models as well.I have not found someone speak specifically on this issue.Any help with this?Screenshot 2023-09-17 084643864×205 2.8 KBCMD Prompt2023-09-17 08:49:06 INFO:Saved C:\Users\natha\AI\text-generation-webui\presets\My Preset.yaml.Llama.generate: prefix-match hitOutput generated in 15.69 seconds (2.42 tokens/s, 38 tokens, context 465, seed 193558016)Llama.generate: prefix-match hitOutput generated in 11.16 seconds (2.33 tokens/s, 26 tokens, context 520, seed 683430275)llama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensOutput generated in 0.27 seconds (0.00 tokens/s, 0 tokens, context 2058, seed 27346203)2023-09-17 09:20:18 INFO:Loading TheBloke_Wizard-Vicuna-30B-Uncensored-GGML…2023-09-17 09:20:18 INFO:llama.cpp weights detected: models\TheBloke_Wizard-Vicuna-30B-Uncensored-GGML\Wizard-Vicuna-30B-Uncensored.ggmlv3.q2_K.bin2023-09-17 09:20:18 INFO:Cache capacity is 0 bytesllama.cpp: loading model from models\TheBloke_Wizard-Vicuna-30B-Uncensored-GGML\Wizard-Vicuna-30B-Uncensored.ggmlv3.q2_K.binllama_model_load_internal: format     = ggjt v3 (latest)llama_model_load_internal: n_vocab    = 32000llama_model_load_internal: n_ctx      = 2048llama_model_load_internal: n_embd     = 6656llama_model_load_internal: n_mult     = 256llama_model_load_internal: n_head     = 52llama_model_load_internal: n_head_kv  = 52llama_model_load_internal: n_layer    = 60llama_model_load_internal: n_rot      = 128llama_model_load_internal: n_gqa      = 1llama_model_load_internal: rnorm_eps  = 5.0e-06llama_model_load_internal: n_ff       = 17920llama_model_load_internal: freq_base  = 10000.0llama_model_load_internal: freq_scale = 1llama_model_load_internal: ftype      = 10 (mostly Q2_K)llama_model_load_internal: model size = 30Bllama_model_load_internal: ggml ctx size =    0.16 MBllama_model_load_internal: using CUDA for GPU accelerationllama_model_load_internal: mem required  = 13628.98 MB (+ 3120.00 MB per state)llama_model_load_internal: offloading 0 repeating layers to GPUllama_model_load_internal: offloaded 0/63 layers to GPUllama_model_load_internal: total VRAM used: 592 MBllama_new_context_with_model: kv self size  = 3120.00 MBAVX = 1 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | VSX = 0 |2023-09-17 09:20:19 INFO:Loaded the model in 1.36 seconds.llama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensllama_tokenize_with_model: too many tokensOutput generated in 0.25 seconds (0.00 tokens/s, 0 tokens, context 2066, seed 576701495)