Histone modifications play an important role in the formation of an epigenetic memory system that maintains cellular identity. Their complex patterns have been suggested to constitute a histone code, which encodes for specific forms of chromatin. According to the histone code hypothesis these specific patterns are passed on from one cell generation to the next. This enables cells to keep a specific gene expression pattern even in absence of the specific transcription factors that initiated the expression of lineage determining genes. The methylation of specific lysine residues within the histone tails plays a particularly important role in defining the histone modification pattern as mutations of the enzymes that catalyze the formation or the removal of methyl groups have severe effects on cellular physiology. Lysines can get mono-, di- or trimethylated, but the molecular function of the different modification states is still not fully understood. In the following review we will highlight recent data that try to tackle this question and discuss their potential impact for our understanding of the role of histone methylation in epigenetic inheritance.