Free Time

I'm starting to have free time, and don’t know how to deal with it. I was thinking the other day, about how busy I keep myself.

When I moved to Toronto, I was getting used to a new city. I eventually got a midnight-shift job which took most of my time and energy. Any spare time I had was spent partying.
In 1999, I worked the day shift at a gas station and played in a band. Those both ended at the same time, so I took a corporate day job and spent my evenings apprenticing as a tattooist and volunteering at a youth-support phone line.
When the tattooing ended, I joined a band that took every spare minute I had. Days, evenings, weekends, whatever.
When the music ended, I was dating polyamorously. Often going out on many dates a week, I always kept Wednesday aside for laundry. That lasted for a couple of years.  Then i realized that I wanted to be monogamous and found a long-term girlfriend.  That time involved lots of drinking, arguements, and generally causing trouble.  That went on until I met my first wife, and I moved to Georgetown to live with her.
Upon my return from Georgetown a couple of years later, I immediately started night school. That just ended.

There has been VERY LITTLE spare time in my life. And I think I'm scared of it.
I grew up in a very small town, quite isolated. I was bored and became very self-destructive because of that. I was suicidally depressed, with no visible opportunities in my life. My hand, arms, and shoulders still bear the scars of growing up lonely. 
When I moved to Georgetown, the same feeling set in. Isolated, alone, bored, with no visible opportunity. To (quite literally) save my life, I left my wife and moved home to the city.
Now I have a 9-5 job, and no formal responsibilities for my evenings. My singing lessons vary, happening on random evenings once a week. I've got my second weekend of sailing, and then hopefully crewing once or twice a week.

God only knows what will happen with downtime, and I'm apprehensive to find out...