In a variety of settings, the performance of  a system can be assured or characterized by relating it to a distribution that closely approximates its behavior.  The approximating distribution may be simpler to analyze than the exact behavior or ideal in some sense.  In this talk we show several different settings where we have developed and used approximation techniques to obtain new  results. First we will discuss a wiretap channel setting (with channel state at the encoder) for which we have recently obtained a secrecy rate that improves on the literature.  Our encoder is both simple and unusual.  In these settings of information theoretic security, the analysis of even simple encoders can be a challenge.  For this result, we greatly benefited from our recently established technique of distribution approximation.  We construct a proxy distribution for which it is simple to prove the security of the communication, and we show that this proxy distribution is close in total variation to the true system behavior. Next we will discuss differential privacy, which has become a popular notion of database privacy.  This privacy definition involves distributions being close to each other under a particular metric.  We develop new bounds to establish that differential privacy is equivalent to a mutual information constraint.  In fact, it is conditional mutual information that must be used to establish this equivalence, which is the feature that was overlooked in previous attempts in the literature to make this sort of connection.  This new viewpoint brings simple intuition and understanding to the concept. Finally, a classic statistics question that has gathered a great deal of interest and activity in recent years is that of estimating key features about the distribution from which data was generated without making any assumptions---in particular, estimating how many possible values the data can take (e.g. how many species were not sampled?) or estimating the entropy.  Results of the past few years have shown that you can accurately estimate these quantities from the data quicker than you might expect (even logarithmically undersampling the space); however, unfortunately, you have no way of knowing that you’ve taken sufficient samples for an accurate estimate.  We show that by modifying the objective, allowing the estimate to be accurate for an approximating distribution, you can overcome this obstacle and know from the data alone that you have an accurate estimate.