How do we know that black holes are spinning? How is it possible to know if a black hole is spinning or not?

If a planet is spinning, you can see it clearly but you can't really see a black hole.

Next thing would be that matter interacts with adjacent matter and we could see in which direction the matter surrounding the BH spins (like if you spin a ball on water, the water around would spin too in the same direction) but matter can't interact from inside the event horizon to the outside, so matter right at the event horizon would just be interacting with gravity (like the BH has no friction).

Now gravity. I would think that you could measure the differences in gravity if a large object is not perfectly uniform but I think a BH has the same gravitation pull on all sides.

What am I missing here? How can one even detect or determine by observation that a black hole is spinning, or better yet, measure how fast?
The gravitational field of spinning matter, or a spinning black hole, causes matter around it to start spinning. This is called "frame dragging" or "gravitomagnetism", the latter name coming from the fact that it's closely analogous to the magnetic effect of moving electric charges. The existence of gravitomagnetism is tied to the finite speed of gravity, so it doesn't exist in Newtonian gravity where that speed is infinite, but it's present in general relativity, and for black holes it's large enough to be detectable.

Also, for purely theoretical reasons we expect that all black holes are spinning because a non-spinning black hole is the same as a spinning black hole with an angular velocity of exactly zero, and there's no reason why a black hole's angular velocity would be exactly zero. On the contrary, because they are so much smaller than the matter that collapses to produce them, even a small, random net angular momentum of the collapsing matter should lead to a rapidly spinning black hole. (The classic analogy for this is an ice skater spinning faster when they pull their arms in.)