A terrific, short collection of important early Christian writings in modern English. This is therefore a great, rapid introduction to the pre-Nicene church fathers for those that want a feel for the early apostolic churches.
I started with writings that gave me my best quick introduction to early Christianiaty. These are writings that I have updated myself and host right here on Christian History for Everyman.
You might want to read my suggestions for how to read the early church fathers before going through the writings that follow.
Also, I have an overview of early Christianity with links to individual early Christians if you'd like a general introduction before you read their writings.
Finally, here is the best explanation of the importance of the early Christian writings that I have ever heard. It's long, but interesting. I skipped all the introduction by the host. You may want to listen to it if you don't know who David Bercot is. Otherwise, I suggest you skip to 5:00 to begin. (If you have any problems playing the video, click here to watch on Youtube.)
You can also find translations of early Christian writings (three or four pre-Nicene and the rest post-Nicene) that are not included in the Edinburgh translations (The Ante-Nicene Fathers and the Nicene and Post-Nicene Fathers series one and two) at earlychristianwritings.com. That site also carries other translations than those in the Ante-Nicene Fathers and Nicene and Post-Nicene Fathers series. They don't carry new translations, and there are a few made over the last 40 or 50 years, especially by Roman Catholics. You would have to research those and buy those, as they are copyrighted. EarlyChristianWritings.com carries only those translations that are no longer under copyright.
Note that the early Christian writings listed in the bullet points above are all updated to at least somewhat modern English by myself or others that I have hired. (Yes, sometimes this site costs me money.)
I did, with permission, copy their translation of The Proof of the Apostolic Preaching by Irenaeus (c. A.D. 180) because as a comprehensive summation of the apostolic faith it combines authority, directness, an early date, and brevity. By this I mean that Irenaeus is one of the most trustworthy early Christian sources, the Proof of the Apostolic Preaching is directly on the subject of what the faith of the apostles was, and compared to many of the early Christian writings, it is not long.
I am going to use that page to read through Proof of the Apostolic Preaching a second time, and I will update it to modern English as I go, so that will be a work in progress. It's introduction is now on this site, too.