I get asked a lot about how to get started studying church history. These are the books I recommend. Also see How to Study Christian History, which gives a plan for learning church history.
To get a quick introduction, here's an excellent starter package:
If you use the links on this page to purchase the books from Amazon, I get a small commission, which helps support this site.
The Ante-Nicene Fathers and specifically volume one and volume two. I cannot overemphasize how important these writings are. If you are going to read any Christian books outside the Bible, you ought to be reading these! These are the writings of those closest in time to the apostles. If you don't read these, you're just guessing at what the apostles taught the churches, and you're just hoping that you've joined the right church and interpreted the Bible correctly.
I have the entire Ante-Nicene Fathers set in paper, on Kindle, and on my computer as .pdf. The Kindle volumes are the most pleasant to read and inexpensive. (Get the "optimized" version so you can read the footnotes). However, I use the .pdf's the most because I can search them if I don't remember where I read something. I had to pay for the .pdf's, but they were very inexpensive. I got them from the Christian Classics Ethereal Library (linked in the previous paragraph), but as of July 11, 2017, I can't find them there anymore. I suppose the Kindle volumes can be searched and highlighted as well.
Also, as of 2017, the online sources for reading the early Christian writings or quotes from the early Christian writings keep increasing, so make use of your favorite search engine!
An excellent beginning is to read the writings of the earliest Christians, several of whom were hearers of the apostles, right here on this site. We have taken a number of very early Christian writings, updated the language, and have them available for free reading online.