They're usually quite easy. However depending at what time of day you go, there can be lots of people.
I advise brewing iron clad and slow falling potions and carrying around 3 iron clad and 6 slow falling, you can splash slow falling on anyone who tries to attack you and if there is somebody...