According to the New Catholic Encyclopedia, "Rosarium (a rose garden) [was] a common term to designate a collection of similar material. In preempting this term, Mary's clients applied the rose, the symbol of joy, to Mary.
The name was later transferred to the recitation of 50 Aves [Hail Marys] commemorating Mary's joys. As devotion to Mary's dolors [sorrows] arose during the 14th century, the second chaplet was dedicated to them. Logically, the third chaplet was set aside for her heavenly [glorious] joys."
Another explanation is that the 150 psalms were matched with 150 Our Fathers and then later Hail Marys.
Information about Mary's association with various flowers can be found in Mary's Flowers: Gardens, Legends and Meditations, by Vincenzina Krymow (St. Anthony Messenger Press).