I liked Arsenio in the beginning. but as time went on he started booking mostly rappers as musical guests and fewer country, rock and white guests. The tone of the show began to change and was taking itself too serious. The constant Bush bashing and "white folks" jokes began to chase away viewers. He even mentioned on air that stations executives complained that the show had become "too black" he continued on the same path which eventually led to the shows demise. Giving Louis Farrakahn an entire hour didn't help either. In the early episodes his monologes were ok, but his interview and hosting skills were never as good as Carson
I think you miss (or oddly, hit) the point: Being 'the anti-Carson' was
precisely the appeal of Hall's show.
For all of his accolades, Johnny Carson was not universally loved by the entire TV watching audience. At the very least, there was a definite section of ongoing culture his show definitely ignored during the last years of his tenure. Arsenio Hall's show definitely catered to younger tastes of the day, and healthy helpings of hip-hop culture (found in music, film, comedy, even pro sports and politics) were what people were consuming.
(That's not to say that he or his audience didn't/couldn't appreciate older legends; YouTube search 'Sammy Davis Jr. on Arsenio' and enjoy one of the best moments in the show's history.)
I'd argue that part of Conan O'Brien's success (and Jimmy Kimmel's approach) can be traced to Arsenio proving that you can thrive catering to young viewers. For that matter, Arsenio's unflinching embrace of hip-hop was the right move, now that hip-hop is entrenched as part of the mainstream. Neither Leno and Letterman's bookers can ignore Wale, Nicki Minaj, or Pitbull.