I get asked a lot about how to get started studying church history. These are the books I recommend. Also see How to Study Christian History, which gives a plan for learning church history.
To get a quick introduction, here's an excellent starter package:
If you use the links on this page to purchase the books from Amazon, I get a small commission, which helps support this site.
The Ante-Nicene Fathers and specifically volume one and volume two. I cannot overemphasize how important these writings are. If you are going to read any Christian books outside the Bible, you ought to be reading these! These are the writings of those closest in time to the apostles. If you don't read these, you're just guessing at what the apostles taught the churches, and you're just hoping that you've joined the right church and interpreted the Bible correctly.
I have the entire Ante-Nicene Fathers set in paper, on Kindle, and on my computer as .pdf. The Kindle volumes are the most pleasant to read and inexpensive. (Get the "optimized" version so you can read the footnotes). However, I use the .pdf's the most because I can search them if I don't remember where I read something. I had to pay for the .pdf's. You can get them free now.
An excellent beginning is to read the writings of the earliest Christians, several of whom were hearers of the apostles, right here on this site. We have taken a number of very early Christian writings, updated the language, and have them available for free reading online.