Police say the Oak Street resident arrived home on Tuesday, to find a man in his back yard.
When the home owner went inside, he discovered his house had been broken into, and a coin collection worth several thousand dollars was missing.
Police tracked 27-year-old Eric Becker of Bangor to a nearby store where they say he spent some of the coins.
Becker was arrested a short time later.
Police have recovered some of the stolen coins.
Becker is being charged with burglary and theft.
A home owner on Sterling Street in Ellsworth also reported an attempted burglary.
Authorities believe the two cases are connected.