I suggest using this information for installing it: https://www.replaymod.com/docs/#installation-ffmpeg-mac
Basically all you need to do is type this in the terminal:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
and then when prompted:
brew install ffmpeg
If that does not work for you then there are alternative instructions also found at that link. You can also ask me further questions but my accuracy is limited as I am not running Mac OSX.
I did that before I followed the tutorial and I thought it worked but I couldn't use any ffmpeg commands, and I kept getting a warning from homebrew that since I was running OS X 10.11 that it might not work properly. When I followed the tutorial on YouTube, I had ffmpeg running perfectly in Terminal but the mod still didn't let me create renders.