I'm back and forth on this review, because the Victoria House and the staff were top notch. It is a great facility, very nice courtyard, great staff, beautiful views.
The area around the hotel I would consider sketchy. Strand street is relatively safe feeling at least the north end where the street lights mostly work. The south end around the Fred hotel is dark and we had an incident one night after eating at the BBQ joint next door as we were walking back in the dark, turned out ok - but solidified my feelings of the area. Frederiksted is a rough town and it feels that way. There were some great bars to stop and eat in, a few nice shops, it just didn't feel safe when the sun went down.
After traveling around the island if I was to go back I'd be looking at the north shore away from the bigger towns. The Cane bay area is where we would stay if we visit again sometime.
Frederiksted & Christansted are the two main bigger towns and felt safe going through them during the day and would be fine places to visit during the day, night time just has a different feel.
With us staying in Frederiksted we learned the area pretty well over a week. Polly's at the Pier is great for breakfast. Nep2une diving is great, my son and I dove for the first time with our instructor Shawn who was superb and would recommend if your staying in the area or visiting for the day. Rainbow beach is a nice beach in the area with jet skis, snorkel equipment, and other fun beach stuff.