The Instability Orb is, as expected, a very unstable and unpredictable material. When exposed (thrown on the ground), it will start slowly decomposing and after a short and random amount of time, one of the effects described below will happen.
There are 3 types of effects that can happen once the Orb is disintegrated. The most likely effect is turning into 1-3 pieces of a random block or item, these include biome blocks, anything that's smeltable and its outcome, and basic recipe ingredients and results (you cannot get an Enhanced Ender Pearl or a Firework, because they require special conditions).
Another effect that can happen is turning into one randomly selected non-boss vanilla mob. Modded mobs may spawn if they spawn naturally in any biome.
The last and least likely effect is a random explosion, with radius varying between 3.4 and 4.4 (radius of TNT is equal to 4), and a 1 in 3 chance of creating fire. However if a TNT is exploded near one or more decomposing orbs, it will prevent the orb from exploding, which is useful for decomposing large amount of orbs at once, as explosions may destroy items created from other orbs.
- Explosions don't destroy the orb, they just slightly slow down the decomposition process (and make the orb not explode, as explained above)