8 of the Shards of the Ultimate Orb are the "Prime" Shards, which have inherited the full power of the Eight Elemental Orbs and serve the purpose they did, or will.
The rest of the Ultimate Orb's power is divided up amongst the other shards, though for the sake of attuning to the Ultimate Orb they'll do, but I think you'd need at bare mimimum the eight prime shards and the majority of the other pieces to come anywhere near close to the full power of the Ultimate Orb--the Shards Caitiff found are probably broken too small to be retrived without powerful magic unless it didn't have them on it when we destroyed it, part of one is canonically forged into a weapon the hero wields, and Theano has a pretty sizable and powerful chunk that we need to retrieve... I think Xan found one and uses it to restore his powers, if that's what the cutscene at the end of book one was meant to imply...
...Seriously, we need to gather up as many pieces of this thing as we can find and lock them up somewhere safe. Give them to Warlic, maybe? The Avatars respect him and I doubt anything is getting into his tower.