Can the word 'petite' be used for boys/men?

Because normally I've seen the word 'petite' is used for girls/women and when I googled the word it also said the word is used for girls/women. Yesterday, my online friend (not a native English speaker) called me petite (I'm almost 20 bit I'm kinda short and skinny) but I was kinda confused and thought it was funny because I thought 'petite' is only use for girls/women?

Do native speakers also use the word petite for men? Or is it rude/offensive to call a man petite?

Thank you in advance!