A bid by Islamic State to expand its territory in Libya has been hampered by a lack of fighters and the militant group is struggling to win local support because it is viewed as an "outsider", according to a report by United Nations experts.

Islamic State in Libya has between 2,000 and 3,000 fighters and is the only affiliate known to have received support and guidance from the extremist group's stronghold in Syria and Iraq, said the UN experts, who monitor Islamic State (ISIL) and al Qaeda-linked groups for the U.N. Security Council.

In a 24-page report circulated to reporters on Tuesday, they said most Islamic State fighters are in the city of Sirte and while the group has "clearly demonstrated" its intention to control more territory in Libya, it seems "limited in its ability" to expand quickly.

"According to several (U.N.) member states, while ISIL is able to perpetrate terror attacks in any part of Libya, its limited number of fighters does not allow for rapid territorial expansion," the report said.

"In contrast to Iraq and the Syrian Arab Republic, the relative sectarian homogeneity in Libya prevents ISIL from taking advantage of sectarian rifts and societal discord to quickly increase its domestic recruitment base," it said.

Libya is caught up in a conflict between rival governments and their armed factions, leaving a security vacuum that has allowed Islamic State to gain a foothold. The group controls swathes of territory in Syria and Iraq and sees Libya as its "best opportunity" to expands its caliphate, the UN experts said.

They said around 800 Libyans fighting with Islamic State in Libya had previously fought with the group in Syria and Iraq.

"ISIL in Iraq and the Syrian Arab Republic continues to send emissaries with instructions, albeit infrequently, to ISIL in Libya," the report said. "The travel of these emissaries distinguishes the ISIL affiliate in Libya from other ISIL affiliates where travel of emissaries has not been reported."

The group is viewed as an outsider in Libya and "is not embedded in local communities and has not succeeded in gaining the population's support," but it has attracted foreign fighters, mainly from elsewhere in North Africa.

"ISIL is only one player among multiple warring factions in Libya and faces strong resistance from the population, as well as difficulties in building and maintaining local alliances," the UN experts said.

The Islamic State in Libya has massacred Christian Egyptians on a local beach, publicly flogged criminals in Sirte, stormed oilfields, and attacked a five-star hotel in Tripoli.