Japan is unique among first world nations in its status as a monoculture. It is not a western melting pot democracy that receives newcomers openly. I suspect that your experiences came with a realization that you are more French and European than you initially accounted for.
Culture is not something we can uninstall like a software program. Western pluralist societies tend to dismiss their own cultures. We are led to believe that western culture is crass materialism and commercialism and therefore no culture at all. But this attitude belies the fact that our own cultures are deeply formative for those who grew up in them.
I admire your courage for diving deeply into a culturally immersive experience, even if it left you with regrets.