Commit
·
9a647d3
1
Parent(s):
d976538
Add BERTopic model
Browse files- README.md +124 -0
- config.json +15 -0
- topic_embeddings.safetensors +3 -0
- topics.json +0 -0
README.md
ADDED
|
@@ -0,0 +1,124 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
---
|
| 3 |
+
tags:
|
| 4 |
+
- bertopic
|
| 5 |
+
library_name: bertopic
|
| 6 |
+
pipeline_tag: text-classification
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# blbooksgenre_topics
|
| 10 |
+
|
| 11 |
+
This is a [BERTopic](https://github.com/MaartenGr/BERTopic) model.
|
| 12 |
+
BERTopic is a flexible and modular topic modeling framework that allows for the generation of easily interpretable topics from large datasets.
|
| 13 |
+
|
| 14 |
+
## Usage
|
| 15 |
+
|
| 16 |
+
To use this model, please install BERTopic:
|
| 17 |
+
|
| 18 |
+
```
|
| 19 |
+
pip install -U bertopic
|
| 20 |
+
```
|
| 21 |
+
|
| 22 |
+
You can use the model as follows:
|
| 23 |
+
|
| 24 |
+
```python
|
| 25 |
+
from bertopic import BERTopic
|
| 26 |
+
topic_model = BERTopic.load("davanstrien/blbooksgenre_topics")
|
| 27 |
+
|
| 28 |
+
topic_model.get_topic_info()
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
## Topic overview
|
| 32 |
+
|
| 33 |
+
* Number of topics: 57
|
| 34 |
+
* Number of training documents: 43752
|
| 35 |
+
|
| 36 |
+
<details>
|
| 37 |
+
<summary>Click here for an overview of all topics.</summary>
|
| 38 |
+
|
| 39 |
+
| Topic ID | Topic Keywords | Topic Frequency | Label |
|
| 40 |
+
|----------|----------------|-----------------|-------|
|
| 41 |
+
| -1 | poems - novel - poem - prose - book | 11 | -1_poems_novel_poem_prose |
|
| 42 |
+
| 0 | poems - poem - poetry - poets - poetical | 18624 | 0_poems_poem_poetry_poets |
|
| 43 |
+
| 1 | novel - author - poem - heir - tales | 4698 | 1_novel_author_poem_heir |
|
| 44 |
+
| 2 | ireland - dublin - scotland - irish - edinburgh | 3576 | 2_ireland_dublin_scotland_irish |
|
| 45 |
+
| 3 | geography - geographical - maps - map - history | 3104 | 3_geography_geographical_maps_map |
|
| 46 |
+
| 4 | shakespeare - acts - prose - comedy - theatre | 1377 | 4_shakespeare_acts_prose_comedy |
|
| 47 |
+
| 5 | county - counties - pennsylvania - hampshire - history | 1089 | 5_county_counties_pennsylvania_hampshire |
|
| 48 |
+
| 6 | france - spain - europe - pyrenees - paris | 990 | 6_france_spain_europe_pyrenees |
|
| 49 |
+
| 7 | sailing - nautical - maritime - boat - voyages | 986 | 7_sailing_nautical_maritime_boat |
|
| 50 |
+
| 8 | antiquity - greeks - rome - romans - greece | 744 | 8_antiquity_greeks_rome_romans |
|
| 51 |
+
| 9 | illustrations - drawings - pencil - drawn - sketches | 631 | 9_illustrations_drawings_pencil_drawn |
|
| 52 |
+
| 10 | africa - transvaal - cape - zululand - african | 610 | 10_africa_transvaal_cape_zululand |
|
| 53 |
+
| 11 | egypt - egyptians - cairo - sinai - egyptian | 610 | 11_egypt_egyptians_cairo_sinai |
|
| 54 |
+
| 12 | england - britain - british - george - english | 570 | 12_england_britain_british_george |
|
| 55 |
+
| 13 | california - alaska - regions - tour - states | 546 | 13_california_alaska_regions_tour |
|
| 56 |
+
| 14 | italia - italy - sicily - italian - italians | 491 | 14_italia_italy_sicily_italian |
|
| 57 |
+
| 15 | crimean - crimea - turkey - turks - russia | 481 | 15_crimean_crimea_turkey_turks |
|
| 58 |
+
| 16 | mexico - rio - honduras - colombia - panama | 433 | 16_mexico_rio_honduras_colombia |
|
| 59 |
+
| 17 | wales - maoriland - otago - zealand - auckland | 423 | 17_wales_maoriland_otago_zealand |
|
| 60 |
+
| 18 | waterloo - poem - battle - napoleon - battles | 405 | 18_waterloo_poem_battle_napoleon |
|
| 61 |
+
| 19 | mining - mineralogy - minerals - metallurgy - metals | 396 | 19_mining_mineralogy_minerals_metallurgy |
|
| 62 |
+
| 20 | history - america - states - historical - american | 377 | 20_history_america_states_historical |
|
| 63 |
+
| 21 | geology - geological - geologists - cambrian - fossils | 305 | 21_geology_geological_geologists_cambrian |
|
| 64 |
+
| 22 | quebec - scotia - canadas - ontario - province | 204 | 22_quebec_scotia_canadas_ontario |
|
| 65 |
+
| 23 | rambles - ramble - south - lands - scrambles | 194 | 23_rambles_ramble_south_lands |
|
| 66 |
+
| 24 | edition - second - series - third - revised | 159 | 24_edition_second_series_third |
|
| 67 |
+
| 25 | rudge - barnaby - hutton - rivers - osborne | 149 | 25_rudge_barnaby_hutton_rivers |
|
| 68 |
+
| 26 | memorials - anniversary - memorial - london - address | 134 | 26_memorials_anniversary_memorial_london |
|
| 69 |
+
| 27 | railway - railways - railroad - railroads - railroadiana | 115 | 27_railway_railways_railroad_railroads |
|
| 70 |
+
| 28 | forest - foresters - woods - trees - forestalled | 112 | 28_forest_foresters_woods_trees |
|
| 71 |
+
| 29 | philosophy - humanity - philosophie - moralities - conscience | 97 | 29_philosophy_humanity_philosophie_moralities |
|
| 72 |
+
| 30 | gazetteer - geography - geographical - dictionary - topographical | 96 | 30_gazetteer_geography_geographical_dictionary |
|
| 73 |
+
| 31 | goldsmith - goldsmiths - novel - writings - epistle | 93 | 31_goldsmith_goldsmiths_novel_writings |
|
| 74 |
+
| 32 | regulations - members - committees - rules - committee | 89 | 32_regulations_members_committees_rules |
|
| 75 |
+
| 33 | odes - poems - poem - ode - hymno | 87 | 33_odes_poems_poem_ode |
|
| 76 |
+
| 34 | doctor - doctors - physician - patients - physicians | 79 | 34_doctor_doctors_physician_patients |
|
| 77 |
+
| 35 | geography - schools - longmans - colleges - school | 77 | 35_geography_schools_longmans_colleges |
|
| 78 |
+
| 36 | juan - juana - sequel - carlos - genista | 63 | 36_juan_juana_sequel_carlos |
|
| 79 |
+
| 37 | sporting - sports - sport - sportsmans - rugby | 56 | 37_sporting_sports_sport_sportsmans |
|
| 80 |
+
| 38 | detective - detectives - crime - policeman - city | 52 | 38_detective_detectives_crime_policeman |
|
| 81 |
+
| 39 | blanc - mont - blanche - montserrat - montacute | 47 | 39_blanc_mont_blanche_montserrat |
|
| 82 |
+
| 40 | jack - jacks - jackdaw - house - author | 46 | 40_jack_jacks_jackdaw_house |
|
| 83 |
+
| 41 | dutch - netherlands - holland - dutchman - dutchesse | 43 | 41_dutch_netherlands_holland_dutchman |
|
| 84 |
+
| 42 | spider - spiders - adventure - web - webs | 35 | 42_spider_spiders_adventure_web |
|
| 85 |
+
| 43 | madrasiana - madras - malabar - mysore - district | 31 | 43_madrasiana_madras_malabar_mysore |
|
| 86 |
+
| 44 | doncaster - 1835 - gazette - 1862 - 1868 | 31 | 44_doncaster_1835_gazette_1862 |
|
| 87 |
+
| 45 | lays - lay - land - empire - sea | 28 | 45_lays_lay_land_empire |
|
| 88 |
+
| 46 | cyprus - syria - palestine - island - asia | 28 | 46_cyprus_syria_palestine_island |
|
| 89 |
+
| 47 | gipsies - gipsy - snakes - encyclopaedia - bunyan | 20 | 47_gipsies_gipsy_snakes_encyclopaedia |
|
| 90 |
+
| 48 | abydos - bride - turkish - marriage - euphrosyne | 18 | 48_abydos_bride_turkish_marriage |
|
| 91 |
+
| 49 | derby - castleton - buxton - matlock - nottingham | 16 | 49_derby_castleton_buxton_matlock |
|
| 92 |
+
| 50 | corsair - tale - carlo - mystery - monte | 16 | 50_corsair_tale_carlo_mystery |
|
| 93 |
+
| 51 | bushman - bushranger - bushrangers - australian - novel | 13 | 51_bushman_bushranger_bushrangers_australian |
|
| 94 |
+
| 52 | months - italy - weeks - six - france | 12 | 52_months_italy_weeks_six |
|
| 95 |
+
| 53 | kitty - kittys - catspaw - catriona - father | 12 | 53_kitty_kittys_catspaw_catriona |
|
| 96 |
+
| 54 | lighthouses - lighthouse - beacons - lights - lighting | 12 | 54_lighthouses_lighthouse_beacons_lights |
|
| 97 |
+
| 55 | balfour - kidnapped - balfouriana - memoirs - adventures | 11 | 55_balfour_kidnapped_balfouriana_memoirs |
|
| 98 |
+
|
| 99 |
+
</details>
|
| 100 |
+
|
| 101 |
+
## Training hyperparameters
|
| 102 |
+
|
| 103 |
+
* calculate_probabilities: False
|
| 104 |
+
* language: english
|
| 105 |
+
* low_memory: False
|
| 106 |
+
* min_topic_size: 10
|
| 107 |
+
* n_gram_range: (1, 1)
|
| 108 |
+
* nr_topics: 57
|
| 109 |
+
* seed_topic_list: None
|
| 110 |
+
* top_n_words: 10
|
| 111 |
+
* verbose: True
|
| 112 |
+
|
| 113 |
+
## Framework versions
|
| 114 |
+
|
| 115 |
+
* Numpy: 1.22.4
|
| 116 |
+
* HDBSCAN: 0.8.29
|
| 117 |
+
* UMAP: 0.5.3
|
| 118 |
+
* Pandas: 1.5.3
|
| 119 |
+
* Scikit-Learn: 1.2.2
|
| 120 |
+
* Sentence-transformers: 2.2.2
|
| 121 |
+
* Transformers: 4.29.2
|
| 122 |
+
* Numba: 0.56.4
|
| 123 |
+
* Plotly: 5.13.1
|
| 124 |
+
* Python: 3.10.11
|
config.json
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"calculate_probabilities": false,
|
| 3 |
+
"language": "english",
|
| 4 |
+
"low_memory": false,
|
| 5 |
+
"min_topic_size": 10,
|
| 6 |
+
"n_gram_range": [
|
| 7 |
+
1,
|
| 8 |
+
1
|
| 9 |
+
],
|
| 10 |
+
"nr_topics": 57,
|
| 11 |
+
"seed_topic_list": null,
|
| 12 |
+
"top_n_words": 10,
|
| 13 |
+
"verbose": true,
|
| 14 |
+
"embedding_model": "sentence-transformers/all-MiniLM-L6-v2"
|
| 15 |
+
}
|
topic_embeddings.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:966f8ad1d2bedc104793a454677d782849a9fbdc3967b20e00e0b3faf6ca5bff
|
| 3 |
+
size 87640
|
topics.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|