These quotes all contain the phrase "remission of sins." The Greek word underlying the word remission, aphesis, is critically important to the doctrine of the atonement (see "Jesus Died for Aphesis: A Reminder" on my blog). I do not have access to the Greek text of any of the following quotes except the ones from the letter of Barnabas, and Barnabas does use aphesis where the English has "remission."
In my book, Rebuilding the Foundations, I argue that the aphesis, or "remission," of sins is actually a reference to being delivered from slavery to sin, especially since verses like Ephesians 1:7 say that it required apolutrosis, "a release effected by payment of ransom", to obtain the "remission" of our sins. The forgiveness of sin has always been offered by God to all who repent, even without sacrifice, even in the Old Testament. Deliverance from the slavery to sin described in Ephesians 2:1-3, however, required Jesus to ransom us out of slavery. Again, Ephesians 1:7, translated correctly, is a perfect reference, and the verse is repeated almost verbatim in Colossians 1:14. "Ransom," lutron or antilutron in Greek, and redemption, meaning "release upon receipt of ransom," are common New Testament words for what Jesus did for us on the cross. Lutron and antilutron are used just 3 times, but apolutrosis is used 10 times.
I thought these quotes, from early Christians, many of them native Greek speakers (Barnabas, Justin, Irenaeus), would give interesting context to my argument. The source for all of them is the translation by Robertson and Donaldson found in The Ante-Nicene Fathers. I have given links so you can read all the quotes in context.
Since, therefore, having renewed us by the remission of our sins, He hath made us after another pattern, that we should possess the soul of children, inasmuch as He has created us anew by His Spirit. (Letter of Barnabas 6)
In the quote that follows, no one knows where the author (probably not the Barnabas that was Paul's companion) got the following description of the sacrifice of the red heifer (cf. Num. 19).
Now what do you suppose this to be a type of, that a command was given to Israel, that men of the greatest wickedness should offer a heifer, and slay and burn it, and, that then boys should take the ashes, and put these into vessels, and bind round a stick purple wool along with hyssop, and that thus the boys should sprinkle the people, one by one, in order that they might be purified from their sins? Consider how He speaks to you with simplicity. The calf is Jesus: the sinful men offering it are those who led Him to the slaughter. But now the men are no longer guilty, are no longer regarded as sinners. And the boys that sprinkle are those that have proclaimed to us the remission of sins and purification of heart. To these He gave authority to preach the Gospel, being twelve in number, corresponding to the twelve tribes of Israel. But why are there three boys that sprinkle? To correspond to Abraham, and Isaac, and Jacob, because these were great with God. And why was the wool upon the wood? Because by wood [the cross] Jesus holds His kingdom, so that those believing on Him shall live for ever. (Letter of Barnabas 8)
Let us further inquire whether the Lord took any care to foreshadow the water [of baptism] and the cross. Concerning the water, indeed, it is written, in reference to the Israelites, that they should not receive that baptism which leads to the remission of sins, but should procure another for themselves. The prophet therefore declares, "Be astonished, O heaven, and let the earth tremble at this, because this people hath committed two great evils: they have forsaken Me, a living fountain, and have hewn out for themselves broken cisterns" [Jer. 2:12-13]. ... "The man who doeth these things shall be like a tree planted by the courses of waters, which shall yield its fruit in due season; and his leaf shall not fade, and all that he doeth shall prosper. Not so are the ungodly, not so, but even as chaff, which the wind sweeps away from the face of the earth. Therefore the ungodly shall not stand in judgment, nor sinners in the counsel of the just; for the Lord knoweth the way of the righteous, but the way of the ungodly shall perish" [Ps. 1:3-6]. Mark how He has described at once both the water and the cross. (Letter of Barnabas 11; brackets mine)
As many as are persuaded and believe that what we teach and say is true, and undertake to be able to live accordingly, are instructed to pray and to entreat God with fasting, for the remission of their sins that are past, we praying and fasting with them. Then they are brought by us where there is water, and are regenerated in the same manner in which we were ourselves regenerated. For, in the name of God, the Father and Lord of the universe, and of our Saviour Jesus Christ, and of the Holy Spirit, they then receive the washing with water. For Christ also said, "Except ye be born again, ye shall not enter into the kingdom of heaven" [Jn. 3:5].
And for this [i.e., water baptism] we have learned from the apostles this reason. Since at our birth we were born without our own knowledge or choice, by our parents coming together, and were brought up in bad habits and wicked training; in order that we may not remain the children of necessity and of ignorance, but may become the children of choice and knowledge, and may obtain in the water the remission of sins formerly committed, there is pronounced over him who chooses to be born again, and has repented of his sins, the name of God the Father and Lord of the universe ... And in the name of Jesus Christ, who was crucified under Pontius Pilate, and in the name of the Holy Ghost, who through the prophets foretold all things about Jesus, he who is illuminated is washed. (First Apology 61, brackets mine)
And this food is called among us Εὐχαριστία [the Eucharist], of which no one is allowed to partake but the man who believes that the things which we teach are true, and who has been washed with the washing that is for the remission of sins, and unto regeneration, and who is so living as Christ has enjoined. For not as common bread and common drink do we receive these; but in like manner as Jesus Christ our Saviour, having been made flesh by the Word of God, had both flesh and blood for our salvation, so likewise have we been taught that the food which is blessed by the prayer of His word, and from which our blood and flesh by transmutation are nourished, is the flesh and blood of that Jesus who was made flesh. (First Apology 66)
Note that "remission of sins" is tied to living sinless lives in the following quote. I would remind you that just as 1 John 3:9 cannot mean that Christians never commit a single sin, or otherwise he would contradict what he said in 1 John 1:7-2:2, so Justin is speaking of a pattern of avoiding sin, not sinless perfection.
It becomes you to ... hasten to know in what way forgiveness of sins, and a hope of inheriting the promised good things, shall be yours. But there is no other (way) than this: to become acquainted with this Christ, to be washed in the fountain spoken of by Isaiah for the remission of sins; and for the rest, to live sinless lives. (Dialogue with Trypho 44; paretheses in original)
And that expression which was committed to writing by Moses, and prophesied by the patriarch Jacob, namely, "He shall wash His garments with wine, and His vesture with the blood of the grape" [Gen. 49:11], signified that He would wash those that believe in Him with His own blood. For the Holy Spirit called those who receive remission of sins through Him, His garments; amongst whom He is always present in power, but will be manifestly present at His second coming. That the Scripture mentions the blood of the grape has been evidently designed, because Christ derives blood not from the seed of man, but from the power of God. For as God, and not man, has produced the blood of the vine, so also [the Scripture] has predicted that the blood of Christ would be not of the seed of man, but of the power of God. (Dialogue with Trypho 54; brackets mine)
If, indeed, you repent of your sins, and recognise Him to be Christ, and observe His commandments, then you may assert this; for, as I have said before, remission of sins shall be yours. (Dialogue with Trypho 95)
Note that "remission of sins" is tied to no longer continuing in sin in the following quote.
For the sign of the scarlet thread, which the spies, sent to Jericho by Joshua, son of Nave (Nun), gave to Rahab the harlot, telling her to bind it to the window through which she let them down to escape from their enemies, also manifested the symbol of the blood of Christ, by which those who were at one time harlots and unrighteous persons out of all nations are saved, receiving remission of sins, and continuing no longer in sin. (Dialogue with Trypho 111; parentheses in original)
In the following quote, the translators use "Jesus" for Joshua the son of Nun. Jesus' name in Hebrew is the same as Joshua's; therefore, Joshua's name in Greek is, of course, "Jesus." The footnote in The Ante-Nicene Fathers, vol. I, for the following quote explains the reference to "a harlot for a wife" with "Justin either confuses Joshua son of Josedech with Hosea the prophet, or he refers to the Jewish tradition that 'filthy garments' signified either an illicit marriage, or sins of the people, or the squalor of captivity."
For just as that Jesus (Joshua), called by the prophet a priest, evidently had on filthy garments because he is said to have taken a harlot for a wife, and is called a brand plucked out of the fire, because he had received remission of sins when the devil that resisted him was rebuked [Zech 3]; even so we, who through the name of Jesus have believed as one man in God the Maker of all, have been stripped, through the name of His first-begotten Son, of the filthy garments, i.e., of our sins. (Dialogue with Trypho 116; parentheses in original)
If they repent, all who wish for it can obtain mercy from God: and the Scripture foretells that they shall be blessed, saying, "Blessed is the man to whom the Lord imputeth not sin" [Ps. 32:2]; that is, having repented of his sins, that he may receive remission of them from God; and not as you deceive yourselves, and some others who resemble you in this, who say, that even though they be sinners, but know God, the Lord will not impute sin to them. We have as proof of this the one fall of David, which happened through his boasting, which was forgiven then when he so mourned and wept, as it is written. But if even to such a man no remission was granted before repentance, and only when this great king, and anointed one, and prophet, mourned and conducted himself so, how can the impure and utterly abandoned, if they weep not, and mourn not, and repent not, entertain the hope that the Lord will not impute to them sin? (Dialogue with Trypho 141; brackets mine)
In the following quote, Irenaeus is describing the beliefs of gnostic heretics, not the churches of his time.
[The gnostics] maintain that those who have attained to perfect knowledge must of necessity be regenerated into that power which is above all. For it is otherwise impossible to find admittance within the Pleroma, since this it is which leads them down into the depths of Bythus. For the baptism instituted by the visible Jesus was for the remission of sins, but the redemption brought in by that Christ who descended upon Him, was for perfection; and they allege that the former is animal, but the latter spiritual. (Against Heresies, Bk. I, ch. 21, par. 2)
For whom, then, did [John the Baptist] prepare the people, and in the sight of what Lord was he made great? Truly of Him who said that John had something even "more than a prophet," and that "among those born of women none is greater than John the Baptist" [Luke 7:28], who did also make the people ready for the Lord’s advent, warning his fellow-servants, and preaching to them repentance, that they might receive remission from the Lord when He should be present, having been converted to Him, from whom they had been alienated because of sins and transgressions. (Against Heresies, Bk. III, ch. 10, par. 1; brackets mine)
Then [Zachariah] says to John: "And thou, child, shalt be called the prophet of the Highest: for thou shalt go before the face of the Lord to prepare His ways; to give knowledge of salvation to His people, for the remission of their sins" [Luke 1:76]. For this is the knowledge of salvation which was wanting to them, that of the Son of God, which John made known, saying, "Behold the Lamb of God, who taketh away the sin of the world. This is He of whom I said, After me cometh a man who was before me; because He was prior to me: and of His fulness have all we received" [Jn. 1:29; Jn. 1:15-16] This, therefore, was the knowledge of salvation. (Against Heresies, Bk. III, ch. 10, par. 2; brackets mine)