Hi @akdev,
I think this might be much easier if you create courses for the activity groups, rather than using groups. If you created individual courses for each activity group, you could use the self-enroll link options (found under course settings -> more options, as long as you do not give the course an sis_id). Students would be able to join and leave the courses as they please, and you could also enroll specific users through the SIS processes just like a regular course, if you wanted to.
Using groups, I don't know of a way to provide links to join groups. You may also be seeing issues with the enrollment CSV because for groups, the student would have to be enrolled both in the home in the course and then a group in that course. if you try to put a student in a group without them being enrolled in the course, I believe that will fail.
Hope this info helps a bit and gives you an idea for a possible alternate approach.
-Chris