Two suspected Muslim extremists have been arrested at a mosque in Sydney's south after threats were allegedly made against worshippers.

Police say two men are in custody after an operation at the mosque in Arncliffe, in Sydney's south, on Sunday afternoon.

They said one man was removed from the mosque around 1.15pm by "members of the congregation" before being arrested by police stationed outside.

Another man was arrested outside the mosque, police told AAP.

They did not specify the nature of the arrests.

A witness at the mosque who does not want to be named, said he saw members of the congregation drag a man wearing Islamic clothing from the building around 1.00pm (AEDT).

The witness said worshippers hauled the man out of the mosque and took him "down the street", before police arrived and apprehended him.

He said the man appeared to be an Anglo-Saxon Australian and said he thought he may have been a Wahabi extremist.