FWIW, I moved from the X3L to the HD500x. I had the same reservations--starting over building patches is a big investment in time. So that's a given no matter which option you choose. But, it's also a good excuse to do some spring cleaning. I completely reorganized and simplified my patch structure as it had become kind of a mess.
For me, the transition was not that difficult. HD Edit felt familiar, having used GearBox--it actually felt like a better version of GearBox. You still had dual amp paths, you can assign foot switches to anything you want (don't remember X3L, but it seems like it was more limited there). Most of the amp sounds are better--I'm not opening the debate here. I never really A-B'd it cuz once I laid out the $500 for the HD, it wasn't like I was going back.
The effects that are available are greatly improved, comes from the M9-M13 multi effect unit I believe. You have a lot more options there.
For me, it was a more natural transition than Axe FX or something else...also the price point makes it doable. I sold my X3L on eBay, so I ended up only being out a few hundred bucks, net.