what does this mean:
Aligning large language models (LLMs) with human preferences has proven to
drastically improve usability and has driven rapid adoption as demonstrated by

ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and rein-
forcement learning from human feedback (RLHF) greatly reduce the required skill

and domain knowledge to effectively harness the capabilities of LLMs, increasing
their accessibility and utility across various domains. However, state-of-the-art
alignment techniques like RLHF rely on high-quality human feedback data, which
is expensive to create and often remains proprietary. In an effort to democratize
research on large-scale alignment, we release OpenAssistant Conversations, a
human-generated, human-annotated assistant-style conversation corpus consisting

of 161,443 messages distributed across 66,497 conversation trees, in 35 differ-
ent languages, annotated with 461,292 quality ratings. The corpus is a product

of a worldwide crowd-sourcing effort involving over 13,500 volunteers.2 To
demonstrate the OpenAssistant Conversations dataset’s effectiveness, we present
OpenAssistant, the first fully open-source large-scale instruction-tuned model to be
trained on human data. A preference study revealed that OpenAssistant replies are
comparably preferred to GPT-3.5-turbo (ChatGPT) with a relative winrate of 48.3%
vs. 51.7% respectively. We release our code3 and data4 under fully permissive licenses.