Could it be a setting on the tv to enable the optical out?
If everything is going to the AV then normally the AV would process the sound and sent it to your speakers connected to the AV. However, you say that the sound does come out of the TV speakers. Because of this, it is strange that the optical out of your tv does not work in this case.
Just for the sake of it, have you tried the 3.5mm audio jack?
Another thing you could test is, if your cable box or one of the other devices has an optical out, to connect the transmitter directly to that device.
Just a few things to try 