He withdrew from the partnership in 1932, returned to Perth and worked as a branch manager for Bruce Small Pty Ltd. Daffen then moved to Geraldton in 1936, where he worked for Young Motors Limited as manager of their radio and sports department until his election to parliament.