So what I would say is this....not to 'justify' but more in the vein of "Hey Johnny (and Jenny), here is why Jazz can be an interesting and exciting genre to play and listen to !"
1) Jazz, unlike man other contemporary genres...involves itself heavily with improvisation. So a recording, or live performance, will have the exciting dynamic of you watching/listening to a group of players who are literally, on the fly, creating a solo (or even an entire ensemble piece) which does not adhere to what they did last time, necessarily. Literally you are witness to that creative process, or you are (if a player) creating and participating in it.
*I understand some may say "well Jazz isn't the ONLY genre that exhibits this"...and that would be true...improv exists in a lotta genres " and that's true but in 'western' music it is one of the first which we associate with improv.
So if you like that idea...the idea of either listening to people create on the fly, or create notes and phrases on the fly...the element of spontaneity....then you may appreciate Jazz.
2) Jazz requires, and a fairly high level of musical competency. It is challenging to play, and I recall Wayne Shorter once saying in an interview "It is challenging to listen to"...but like other challenging sectors, be they visual arts, sports, or even academia...there is a fulfilling payout at the end of those achievements..from a spiritual and personal standpoint.
3) IT is a genre that HAS HAD the ability to evolve and morph from something which became for a while, conventional or standardized, into something ELSE (although it has NOT done this for around 50 years now, remember it morphed from Dixie to Big Band/Swing, to Bop with some branches off of that including "West Coast", to "Outside" and Jazz Fusion...those were all exciting artistic leaps ). Perhaps there are more leaps in store in the future.
4) You can say stuff like "that was hip", or "I know that cat !" or "c'mon, guys, get it togetha'....we're swingin' like a lead toilet seat !" or "yeah he (she) likes to take it a little outside"...and it'd be completely normal ....and cool.....(you try that without being a jazz musician and it'd just be ...totally....beat).
There...off the top of my head......
(yeah,I was improvising this response...)