Normal asterisk servers use RTP for deliver audio over the internet. Any one can capture this RTP streams by using Packet Sniffer tools, and can listen to the call content. So VoIP calls are not secure for confidential communications until using a secure VoIP system.
Therefore we can use advance protocol called SRTP (Secure Real-time Transport Protocol) which provide encryption for RTP stream. So no one can listen to the VoIP conversations while using SRTP. If someone try to capture the SRTP stream, it will play only a noise (even it was a empty conversation).
But it will not complete the secure VoIP system. You MUST secure the signalling (SIP) because SRTP keys are exchanged in plaintext with SDES(http://en.wikipedia.org/wiki/SDES). To do that we have to use Asterisk TLS (Transport Layer Security). It provides encryption for call signaling.
Asterisk 1.8 has native support for SRTP. So you can easily try this with a Asterisk 1.8.x server. The normal SIP phones are not support for this system. You need to find SRTP supported and TLS supported IP/soft phones for setup this. Use blink soft-phone for this (http://www.icanblink.com). I followed following article to setup secure VoIP system.
For SRTP : http://www.remiphilippe.fr/2011/01/16/asterisk-srtp-with-1-8/
For TLS : https://wiki.asterisk.org/wiki/display/AST/Secure+Calling+Tutorial, ( You can find ast_tls_cert script from here : https://reviewboard.asterisk.org/r/979/diff/)