I just pulled mine out and am picking up the new part from the dealer tomorrow. They wanted to charge me $220 to swap out the hubs so I had to at least try and do it myself.
I started by looking at this video
it does a real good job of showing you what pieces to remove. Once you get the silver and side moldings off, remove the curved piece that covers the pass though (where the USB hub back would be). That gives you access to the harness and you can simply release it and push it though....much easier than trying to pry and damaging the trim.