I don't know how Flash or its player works from a developer integration perspective, but looking at it from purely a blackbox viewpoint, it seems to me that you should be able to implement it as an HTTP proxy. You can set up an embedded web server on some port at localhost, which services requests from Flash. So, you point flash at your localhost URL. Then your server program contacts the URL on the remote system that is encrypted. As it downloads data it decrypts it and forwards it to the local flash client. If you do it that way you can play files of unlimited size (because you don't have to buffer the entire file), and the client can start to play the file before it is all downloaded.
Ultimately, though, this sounds like a waste of time to do what I said, because this is what SSL (https) does for you. The SSL sends the video encrypted over the network and then decrypts it. Would it just be possible to have the remote URL serve the encrypted data simply as https?