I see, it looks like your results are pretty good.
Regarding your comment about training the classifier, I agree that you probably need more training material. 1500 annotations per class isn't bad to start with, but 6 images for training are too few. I imagine that if those 1500 annotations/class were spread among more images (keeping the training volume the same while...