In condensed matter physics, a spin glass is a  magnetic state characterized by randomness, besides cooperative behavior in freezing of spins at a temperature called "freezing temperature" T.  In ferromagnetic solids, component atoms' magnetic spins all align in the same direction.  Spin glass when contrasted with a ferromagnet is defined as "disordered" magnetic state in which spins are aligned randomly or without a regular pattern and the couplings too are random.
The term "glass" comes from an analogy between the magnetic disorder in a spin glass and the positional disorder of a conventional, chemical glass, e.g., a window glass.  In window glass or any amorphous solid the atomic bond structure is highly irregular; in contrast, a crystal has a uniform pattern of atomic bonds.  In ferromagnetic solids, magnetic spins all align in the same direction; this is analogous to a crystal's lattice-based structure.
The individual atomic bonds in a spin glass are a mixture of roughly equal numbers of ferromagnetic bonds (where neighbors have the same orientation) and antiferromagnetic bonds (where neighbors have exactly the opposite orientation: north and south poles are flipped 180 degrees).  These patterns of aligned and misaligned atomic magnets create what are known as frustrated interactions distortions in the geometry of atomic bonds compared to what would be seen in a regular, fully aligned solid.  They may also create situations where more than one geometric arrangement of atoms is stable.
Spin glasses and the complex internal structures that arise within them are termed "metastable" because they are "stuck" in stable configurations other than the lowest-energy configuration (which would be aligned and ferromagnetic).  The mathematical complexity of these structures is difficult but fruitful to study experimentally or in simulations; with applications to physics, chemistry, materials science and artificial neural networks in computer science.