
# sharegpt-hyperfiltered-3k

90k sharegpt convos brought down to ~3k (3243) via language filtering, keyword detection, deduping, and regex. Following things were done:

- Deduplication on first message from human
- Remove non-English convos
- Remove censorship, refusals, and alignment
- Remove incorrect/low-quality answers
- Remove creative tasks
  - ChatGPT's creative outputs are very censored and robotic; I think the base model can do better.
- Remove URLs
- Remove cutoffs
- Remove math/reasoning questions
  - It sucks without CoT prompting, so this data should be mixed with better reasoning examples like OpenOrca or Dolphin.

license:apache-2.0 region:us
{"license": "apache-2.0"}
TAGS
#license-apache-2.0 #region-us 


# sharegpt-hyperfiltered-3k

90k sharegpt convos brought down to ~3k (3243) via language filtering, keyword detection, deduping, and regex. Following things were done:

- Deduplication on first message from human
- Remove non-English convos
- Remove censorship, refusals, and alignment
- Remove incorrect/low-quality answers
- Remove creative tasks
  - ChatGPT's creative outputs are very censored and robotic; I think the base model can do better.
- Remove URLs
- Remove cutoffs
- Remove math/reasoning questions
  - It sucks without CoT prompting, so this data should be mixed with better reasoning examples like OpenOrca or Dolphin.

# sharegpt-hyperfiltered-3k

90k sharegpt convos brought down to ~3k (3243) via language filtering, keyword detection, deduping, and regex. Following things were done:

- Deduplication on first message from human
- Remove non-English convos
- Remove censorship, refusals, and alignment
- Remove incorrect/low-quality answers
- Remove creative tasks
  - ChatGPT's creative outputs are very censored and robotic; I think the base model can do better.
- Remove URLs
- Remove cutoffs
- Remove math/reasoning questions
  - It sucks without CoT prompting, so this data should be mixed with better reasoning examples like OpenOrca or Dolphin.
TAGS
#license-apache-2.0 #region-us 
 # sharegpt-hyperfiltered-3k

90k sharegpt convos brought down to ~3k (3243) via language filtering, keyword detection, deduping, and regex. Following things were done:

- Deduplication on first message from human
- Remove non-English convos
- Remove censorship, refusals, and alignment
- Remove incorrect/low-quality answers
- Remove creative tasks
  - ChatGPT's creative outputs are very censored and robotic; I think the base model can do better.
- Remove URLs
- Remove cutoffs
- Remove math/reasoning questions
  - It sucks without CoT prompting, so this data should be mixed with better reasoning examples like OpenOrca or Dolphin.
passage: TAGS
#license-apache-2.0 #region-us 
# sharegpt-hyperfiltered-3k

90k sharegpt convos brought down to ~3k (3243) via language filtering, keyword detection, deduping, and regex. Following things were done:

- Deduplication on first message from human
- Remove non-English convos
- Remove censorship, refusals, and alignment
- Remove incorrect/low-quality answers
- Remove creative tasks
  - ChatGPT's creative outputs are very censored and robotic; I think the base model can do better.
- Remove URLs
- Remove cutoffs
- Remove math/reasoning questions
  - It sucks without CoT prompting, so this data should be mixed with better reasoning examples like OpenOrca or Dolphin.