Adding a favicon in Muse is quite simple.
On your masterpage go to:
Then, in the Layout tab, at the bottom of the window, you will find the place to add your favicon. This should be an image (jpg, png, etc.) with 32×32 pixels in size.