Smoke Shadows & Advanced Blending are special Windows specific features written by Intel that are not exposed on Mac OS X.
I don't recall exactly why Advanced Ambient Occlusion isn't available right now but it's either also an Intel feature like above or the it could have been causing GPU restarts / kernel panics and would have been removed for stability reasons.
While the game runs fine (and it should, as I basically have the fastest Mac), performance under boot camp is 30-50% better.
Our application takes advantage of running in "detached mode" so that the window compositor is bypassed which should result in a smoother experience. A requirement on running in detached mode is that the application is fullscreen. As soon as you enable Windowed mode you can't use detached mode.
With regards to performance, I would be interested as to which GPU you are testing on, for AMD GPUs anything below 20% performance disparity is very unusual for us to see on any of our Metal games. Depending on the setting, if framerate is above 120fps (which is quite possible on DiRT Rally with it's engine), we are capped at 120 by the OS, irrespective of the Vsync interval or cap set by the application, which would massively skew comparisons to Windows at lower settings.
With recommended mode we run a stable and consistent capped framerate in all scenarios, and we have done this for every single supported Mac in the range, which we see as far more important that maximum framerate, when running on a 60Hz screen. By making the frame rate stable and not rendering frames you'll never see on a 60Hz screen it allows us to thermal temperatures down which in turn means you'll get consistent performance over extended gameplay. This means a cooler Mac and your batteries will last longer too if you're a laptop.
If you think you have a bug with DiRT Rally then please do contact our support with a support report attached and we'll investigate your issue.