Is the idea that when analyzing some natural phenomenon there's a bunch of complexity, but in certain cases (where we can use symmetry groups) the complexity follows a pattern. If we recognize the pattern, then in order to get at the heart of the matter we only need to look at one element of the pattern for the next stages of our analysis (rather than considering all of its repetitions).
And then maybe the next thing is that often times the patterns occur in abstract mathematical spaces rather than being literal physical patterns.
Can anyone comment on the accuracy of that interpretation?
So in physics, the best examples take the form of a continuous symmetry. A very simple example is the special orthogonal group SO(3) - the group that describes all length preserving rotations.
Another example in relativity are Lorentz and Poincaré groups. The Poincaré group effectively describes affine transformations, a transformation of space that preserves parallel lines and ratios of distances.
More abstract examples occur in field theory. The unitary group of one dimension U(1), the special unitary groups SU(2) and SU(3) can be used to "describe" electrodynamics, the weak and strong force respectively.
In these cases the solutions to the equations that describe how particles interact under these forces have these symmetries. These theories all have some conserved quantity that is unchanged by the action of these groups (charge, color etc).
Perhaps the one of the important thing to take away is; if the equations that describes your system has some symmetry, then there is a conserved quantity (something that doesnt change in time). This is Noether's Theorem.
Another take away is, if your equations have some symmetry, you can use it to describe all the possible solutions. Once you have one solution you can find others by applying the transformation. So perhaps these are a bit more abstract to think about initially, but there is something physical there.
Interesting. That sounds like roughly the inverse of what I was trying to describe: using the transformations to generate rather than analyze.
That's pretty cool though. If I understand correctly, the process would be something like: prove that an equation has some symmetries; then find one solution; then, since we know the equation's symmetries, we can use them to gain access to a formal expression of all the solutions, which could then be used (for example) to assert that such and such must be in the solution set or not in the solution set, etc. (actually, bad example there since our original equation already allows us to do that...)
If that's right, then there's probably some way of making reference to the symmetry groups algebraically—maybe that's part of what the U(1) etc. notations are for? And then some way of getting a symbol meaning essentially, "group X applied to entity Y", where entity Y is something like a solution to an equation with the appropriate symmetries, and group X is a symbol for the symmetry group like U(1)? Or maybe "entity Y" is the symmetric equation rather than a solution.
My initial thinking was more like that you'd start with something symmetric like the full set of solutions, then use knowledge of the symmetry to eliminate repetitions, so that you're only left with what's unique and can analyze that in isolation. But I guess that would make more sense to apply to observational data rather than to a set of solutions.
Using the electrodynamics example: maybe you'd start with a bunch of recorded data related to electric and magnetic phenomena interacting, which in some sense is like a sampling of the solutions to (ideally not yet discovered) equations describing electrodynamics. So if you could determine that that data has some symmetries, you can use it to eliminate everything that's repeated, and what's left (a single solution I suppose...) might given you some insight about an equation which would have it as a solution.
 When it's said that "an equation has a symmetry" is it just shorthand for "the solutions to an equation have a symmetry"?
In the field theory example, you have something called a Lagrangian which you can use to derive the equations which describe the dynamics of your system. In those cases it will have a symmetry of one of those groups, you can transform the Lagrangian by one of those groups.
Knowledge of the symmetry does help you simplify your problem - its basically used to remove redundancy.