Sometimes an invading culture assimilates those they conquer and sometimes they get assimilated, what would you say differentiates these circumstances?

Good question!

Historically, it seems to be a mix of two factors:

• Demographics would be a major one – in the case of an invasion that is also a mass migration, you’re more likely to become the dominant culture that people assimilate into than if you have the conquest of a region by a relatively small army. So for example, the Anglo-Saxon invasion of Britain had a way bigger influence on the culture going forward than say, the less numerous Danes. 
• Policy is another factor – in some cases, invaders actively sought to force their new subjects to assimilate into the invader’s culture, and in other cases the invaders eagerly adopted the culture of the conquered. The Goths who sacked Rome learned to speak Latin, wore togas, kept the Senate around, all of which puts a different light on Justinian’s reconquest. Likewise, the Norman Kings of Sicily really quickly assimilated into the mixed Byzantine/Arab culture – if for no other reason than silks are a lot more comfortable in the Med than heavy furs…

But then again, you can also find historical counter-examples: the Normans who invaded England in 1066 were a numerical minority, especially in comparison to the Anglo-Saxon invasion, but they still had a tremendous impact on the culture and the language, even if they did later assimilate.