Add BERTopic model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+ctfidf_config.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,101 @@
+
+---
+tags:
+- bertopic
+library_name: bertopic
+pipeline_tag: text-classification
+---
+
+# ISSR_Dark_Web_31Topics_White_Nation
+
+This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model. 
+BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets. 
+
+## Usage 
+
+To use this model, please install BERTopic:
+
+```
+pip install -U bertopic
+```
+
+You can use the model as follows:
+
+```python
+from bertopic import BERTopic
+topic_model = BERTopic.load("D0men1c0/ISSR_Dark_Web_31Topics_White_Nation")
+
+topic_model.get_topic_info()
+```
+
+## Topic overview
+
+* Number of topics: 32
+* Number of training documents: 52310
+
+<details>
+  <summary>Click here for an overview of all topics.</summary>
+  
+  | Topic ID | Topic Keywords | Topic Frequency | Label | 
+|----------|----------------|-----------------|-------| 
+| -1 | the - trump - to - of - in | 340 | outliers | 
+| 0 | socialism - lesson - applied socialism - practical - practical lesson applied | 16801 | Applied Socialism | 
+| 1 | trump - democrats - pelosi - biden - election | 10136 | 2020 Election Fraud Impeachment | 
+| 2 | border - illegal - wall - trump - mexico | 2606 | Border Wall Debate | 
+| 3 | israel - iran - syria - us - israeli | 1802 | Middle East Tensions Wars | 
+| 4 | brexit - eu - farage - europe - yellow | 1740 | EU Elections and Brexit Leaders | 
+| 5 | thread - re - you - pictures - pictures thread | 1184 | Funny Pictures Threads | 
+| 6 | climate - climate change - change - warming - global warming | 1067 | Climate Change Funding | 
+| 7 | the - fed - market - bank - banks | 915 | Global Central Banks | 
+| 8 | sgt - sgt report - report - appeared first - appeared first sgt | 1227 | SGT Report Articles | 
+| 9 | mueller - fbi - trump - clinton - obama | 832 | Trump Deep State | 
+| 10 | gun - guns - gun control - shooting - control | 3596 | Gun control and police shootings | 
+| 11 | facebook - google - tech - twitter - social media | 828 | Big Tech Censorship | 
+| 12 | china - trade - chinese - trump - tariffs | 818 | US Trade War | 
+| 13 | gold - silver - report - the post - sgt report | 642 | Gold Silver Ratio | 
+| 14 | epstein - jeffrey epstein - jeffrey - sex - maxwell | 750 | Epstein Maxwell Sex Scandal | 
+| 15 | women - men - transgender - gender - feminism | 569 | Transgender Rights and Feminism | 
+| 16 | jews - jewish - jew - holocaust - the jews | 485 | 20th Century Jewish History | 
+| 17 | kavanaugh - ford - christine - brett - brett kavanaugh | 590 | Kavanaugh Accuser | 
+| 18 | white - racist - white people - race - black | 442 | White Racism Follow | 
+| 19 | youtube - music - favorite - what favorite - what favorite music | 571 | Favorite Music Youtube | 
+| 20 | vaccine - vaccines - measles - vaccination - flu | 398 | Vaccine Lawsuit Losses | 
+| 21 | cancer - monsanto - pharma - drug - big pharma | 400 | Diabetes and Health | 
+| 22 | america - the - world - empire - globalists | 645 | Global Empire War | 
+| 23 | abortion - planned parenthood - parenthood - planned - babies | 332 | Planned Parenthood Abortion | 
+| 24 | christians - christianity - pope - christian - church | 281 | Christianity & Religion | 
+| 25 | media - news - cnn - fake news - fake | 551 | Mainstream Media and Fake News | 
+| 26 | antifa - portland - police - violence - protesters | 662 | Antifa Portland Attacks Journalist | 
+| 27 | college - school - students - schools - education | 337 | Education Politics | 
+| 28 | stormfront - stormfront sucks - re stormfront sucks - re stormfront - sucks | 374 | Stormfront Criticism | 
+| 29 | assange - julian - julian assange - wikileaks - us | 197 | Julian Assange Expulsion | 
+| 30 | coronavirus - virus - pandemic - outbreak - wuhan | 192 | Coronavirus Pandemic |
+  
+</details>
+
+## Training hyperparameters
+
+* calculate_probabilities: True
+* language: None
+* low_memory: False
+* min_topic_size: 10
+* n_gram_range: (1, 3)
+* nr_topics: 32
+* seed_topic_list: None
+* top_n_words: 10
+* verbose: True
+* zeroshot_min_similarity: 0.7
+* zeroshot_topic_list: None
+
+## Framework versions
+
+* Numpy: 1.26.4
+* HDBSCAN: 0.8.36
+* UMAP: 0.5.6
+* Pandas: 2.2.1
+* Scikit-Learn: 1.4.1.post1
+* Sentence-transformers: 3.0.1
+* Transformers: 4.39.3
+* Numba: 0.60.0
+* Plotly: 5.22.0
+* Python: 3.12.2

config.json

@@ -0,0 +1,17 @@
+{
+  "calculate_probabilities": true,
+  "language": null,
+  "low_memory": false,
+  "min_topic_size": 10,
+  "n_gram_range": [
+    1,
+    3
+  ],
+  "nr_topics": 32,
+  "seed_topic_list": null,
+  "top_n_words": 10,
+  "verbose": true,
+  "zeroshot_min_similarity": 0.7,
+  "zeroshot_topic_list": null,
+  "embedding_model": "all-MiniLM-L6-v2"
+}

ctfidf.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d887f7bda90422a03a8d12269fba005a5b47800a998a479b7e1de9794810ea3
+size 21386092

ctfidf_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:201181018c7ccb67b98b78b6c31fb96240fea9b3ac7555a6d178a6d741d2f477
+size 33880722

topic_embeddings.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a881f87103d7cd85ed1a8cfe9a5eca1776e8b4dc3ac67b1df4beb555053fdc10
+size 49240