Kilbey's lyrics often quote historical and mythological events, and his interest in Eastern culture and religion frequently informs his music—this also applies to his painting (he often paints Hindu gods and goddesses).