Linear algebra comes up all the time in advanced courses. Certainly in graphics, but also if you do any machine learning, optimization, probabilistic algorithms, any sort of scientific computation. I never took a linear algebra course, convincing myself that I could just pick it up as I went along and getting into grad level computer science courses was ultimately a rather painful experience because of this. If I were you, I'd try to take it seriously and really try to develop an intuitive feel for linear algebra because (depending on what courses you want to take), it can really save you time and headaches in the future.
