You do need to secure the rights to information and the celebrity status.
Answer Applies to: Michigan
The big legal issue here is Right of Publicity. Right of Publicity laws differ greatly from state to state. Therefore, to create such a show and have it be marketable state to state, and to be able to put parts of it on the internet, you will need to get permission from the person you are depicting. You will need to work on this very carefully with a lawyer and have a great contract written. You will also need to make sure you are working with the music artist, so she approves of the script, casting, and production. If she does not, lawsuits will surely ensue.
You will also need to get clearance to use each of the songs. That clearance will come from the publishers who represent the songwriters on each song. If you are using existing recorded music, you will also need clearance from the record labels or whoever owns those rights. Because of this, most such shows create fully new recordings.
Answer Applies to: Illinois
Just as Kitty Kelly writes unauthorized biographies of celebrities, you can write a musical about one. However, you should also be prepared to defend against a law suit from the celebrity on a variety of charges unless you can prove that every single bit of the play is factual. It would be much safer to get the celebrity's permission and pay whatever percentage of the profits he or she demands.
Answer Applies to: Nebraska