My (20F) boyfriend (24M) flinches when I move too close to him.

So, first of all, I've never hit my boyfriend before. I grew up in an abusive household, and would never do that to anyone else.
My boyfriend has never really opened up about his family and I've only ever gotten snippets of stuff that has happened when he was in high school, but he doesn't tell me much about them and definitely doesn't want me to meet them, even though we've been together for close to a year now.
Recently, I realized that I was the only person my boyfriend flinches around. It's also not just little flinches, he'll curl up on himself and close his eyes for a second before realizing I'm not going to do anything to him. This breaks my heart, seeing this big 6'2 muscly guy look terrified of ME, a girl he could literally squish with one hand.
I guess I'm here to ask for advice. I don't want to force him to open up, but I definitely would like it if he felt safer around me. How do I let him know he's safe and I wouldn't do anything like that to him, without forcing a conversation he doesn't want/isn't ready to have? What would you guys do?
TLDR;; My boyfriend flinches a lot around me, as if expecting a hit, and I just want to help him feel safe.


Source.