This isn't really a category, but a specific instance of a protocol. MOSPF is the multicast extension to OSPF (Open Shortest Path First) which is a unicast link-state routing protocol.
Link-state routing protocols work by having each router send a routing message periodically listing its neighbours and how far away they are. These routing messages are flooded throughout the entire network, and so every router can build up a map of the network which it can then use to build forwarding tables (using a Dijkstra algorithm) to decide quickly which is the correct next hop for send a particular packet.
Extending this to multicast is achieved simply by having each router also list in a routing message the groups for which it has local receivers. Thus given the map and the locations of the receivers, a router can also build a multicast forwarding table for each group.
MOSPF also suffers from poor scaling. With flood-and-prune protocols, data traffic is an implicit message about where there are senders, and so routers need to store unwanted state where there are no receivers. With MOSPF there are explicit messages about where all the receivers are, and so routers need to store unwanted state where there are no senders. However, both types of protocol build very efficient distribution trees.