Tolerance is a stick with two ends
Majority of population in Western countries are disillusioned by the fact that Muslim population living in their country are mostly secular, peaceful people who do not harm others. Being a minority minding their business, they gain natural sympathy of the majority. This is acceptable and nothing is wrong with that, however it reflects a partial truth.
When numbers are reversed, and muslims become majority, they become part of political power granted by their religion. One good example to support this argument is Turkey. In the past decade or so the Islamist political party in power started to challenge and erode secularism, established a police state with biased judiciary and began to jail or intimidate seculars. Islam is a political ideology as well as a religion, therefore this shouldn't surprise us.
However this view also explains why non-religious populations, who were trapped in those countries are worried. Lets be clear, we cannot call this Islamophobia, as this is not an irrational fear we can mock.
If you are a non-religious person, living in a religious country is demoralising and degrading to say the least, because you will be oppressed one way or another if you choose to express yourself.
If you express your disinterest in religion and don’t abide by its restrictions in ways to draw attention, you can be intimidated or punished by mobs, you could even be prosecuted for that. This is true for countries such as Sauidi Arabia, Iran and Turkey. Penalties are different but principles are the same.
Ultimately this is the bit Western liberals are missing, they have partial perspective because either they don't have first hand experience of living in a religious society or they lack deeper knowledge on history and on the nature of religions, or they don't push their intellectual capacity hard enough to see the big picture, because they focus on their lives in the West.