The colors are the visual realization of Rand's thread pulling on lesser Ta'veren. For example, the colors that swirl up when Mat thinks about Rand are an exclamation point of some sort, stressing Rand's importance.