For all current MSB DACs (Select, Reference, Premier and Discrete):
"Native DSD" is the native DSD data-stream at the native sample rate (eg. 2.8224 MHz, etc...) sent directly to a large number of parallel DSD single bit DACs without manipulation (the number DSD DACs available depends on the model). The converted result is then low pass filtered in the analog domain to remove the large amount of DSD quantization noise. Analog volume control is required because there is no manipulation of the digital data.
"Optimized DSD" removes the majority of the DSD quantization noise using digital filtering without changing the sample rate (eg. 2.8224 MHz, etc...) and then plays back the multi-bit super high sample rate data using the multi-bit capabilities of the DACs. The converted result is then low pass filtered in the analog domain to remove the very small amount of DSD quantization noise remaining. It would be accurate to call this PCM playback, but that is also an oversimplification since the "PCM" DACs are running at the native DSD sample rate and there is no sample rate conversion, just quantization noise removal. On models without analog volume control this mode also allows digital volume control.
DOP is just a way of transporting DSD on a link such as S/PDIF that would not normally be compatible with DSD. DOP is native DSD that has been packed into a PCM container and must be unpacked by a compatible decoder at the end of the link (such as in a DAC that supports DOP). All current MSB DACs support DOP unpacking if DOP data is detected on any of the inputs.