I don't think you have to ask him to meet some friends. Maybe it would be nice if you decided on a day and then spend e.g. every thursday evening together. I think as long as you do make some time to spend with him, it should be fine.
You should tell him that you don't always want to ask him first. You want to be able to make your own decisions about how you spend your time