so why was he arrested, after all?
Comments about your question:
1. As far as I understand, Mr. Swartz' access of JSTOR was legal in the sense that it is OK to download individual (one, two, three) articles, but it was not legal to download "too many" articles. Was there any document where this "too much" defined? Was Mr. Swartz aware of this document? Was there a history of Mr Swartz access which resulted in MIT issuing a warning of sorts?
2. Why penalties came from the "downloading" activity, not from "distributing" activity? I can imagine a researcher who is interested, for example, in calculating of the frequency of certain words across many articles, which would involve downloading articles for that purpose. I see no criminal activity here. Usually criminal activity is to distribute something for free which is a copyright infringement.