The audio is handled the same way on the graphics chip as it is in a chipset southbridge. The signals from the source are passed over the PC bus (in this case, PCI Express), and we have an audio controller in the GPU and a virtual codec is used to pass the output via HDMI.
With regards to setup, when you install the graphics board Windows will detect the GPU and another audio controller. For XP when you install the Catalyst drivers the HD audio drivers will install. For Vista, presently Vista will natively support 5.1 audio (as per the HD 2000/3000 series) but a high def codec that supports the HD 4800 series is available from RealTek here