Have you checked your reputation log extensively enough? You get -2 when someone downvotes you, so that SF&F post got 2 upvotes and 1 downvote, resulting in +20-2 = +18 for you and 1 upvote for the answer.
The second case of getting -2 for changing the accepted answer is probably due to the fact that you get +2 for accepting an answer, but not if it's your own. But since you changed the accepted answer from JohnSmithOptional's to yours, you lost the 2 that you once got for accepting his answer (marked as "unaccept" in your reputation log).
In general the reputation scores are explained here (though, I don't see a rule for not getting +2 if the accepted answer is yours right away). Wait, I found it:
Now, there are some special rules around owner-accepted answers, to
prevent gaming:
- Wait 48 hours. You must wait 2 days from the time you originally asked your question before you can accept your own answer. This gives
other users a chance to answer the question in good faith, and earn
the accepted answer.
- No change in sort order. Normally, accepted answers are “docked” under the question. This is not true for owner accepted answers; they
stay in standard sort order like any other answer.
- No reputation is earned. Normally, accepted answers confer +15 rep to the answerer and +2 to the owner. Owner accepted answers do not
earn rep (or badges) for anyone.