A five star rating or equivalent can be earned by extensive contracting out rather than developing internal talent, and accounting choices like using operating leases rather than ever purchasing equipment.
An extreme focus on general and administrative expenses can mean that what counts as direct expenditures on recipients is poorly targeted, gets wasted, or even ends up in the wrong hands. For example there are many stories of how food aid ends up in warlords hands, then used as a tool of influence.
When valuing a regular company you look at the expenses and the revenues, but out of necessity these ratings only focus on expenses because it is to difficult to quantify the qualitative public good being performed. However, the justification for the donation is the public good (the revenue of a nonprofit), and the expenses out of that context is as meaningless, or even misleading, as it is to look at a for profit company's expenses without the context of their revenues.
Even in the case of a nonprofit that is passing through 100% of its funds to people, that means that 100% of the expenses are payouts to recipients. It is worth recognizing that the pass through is an expense, while the revenue equivalent is the actual improvement of the public good achieved by those funds.
Are the nonprofit rating agencies better than nothing? I'm not sure they are. I've had experiences, and heard similar experiences from others, where nonprofits with questionable operating management practices and poor focus on the mission were able to achieve perfect ratings. I think the ratings are a useful veil for questionable operators that choose to focus on their ratings. Furthermore, achieving the ratings can become an important goal in itself, and undermine the mission by discouraging some otherwise productive choices.
Presumably, for a donor the most important factors are how much they care about the mission, how effective the nonprofit's approach is in serving that mission, and how much of it is done per dollar they contribute to the organization. While it is impossible to put such considerations into a star ratings system, it is much more important to you as a donor interested in achieving good than anything the ratings can tell you.
Anyway, there aren't any easy ways to choose, but I think it is worth looking at the annual reports, considering whether the mission statement is clear and focused, that the mission is one you care about, that their approach makes sense, and that they are transparent, and report their dollar expenditures in enough detail that you can clearly understand how they function.