Polysemantic neurons (neurons that activate for a set of unrelated features) have been seen as a significant obstacle towards interpretability of task-optimized deep networks, with implications for AI safety. The classic origin story of polysemanticity is that the data contains more "features" than neurons, such that learning to perform a task forces the network to co-allocate multiple unrelated features to the same neuron, endangering our ability to understand the network’s internal processing. In this work, we present a second and non-mutually exclusive origin story of polysemanticity. We show that polysemanticity can arise incidentally, even when there are ample neurons to represent all features in the data, using a combination of theory and experiments. This second type of polysemanticity occurs because random initialization can, by chance alone, initially assign multiple features to the same neuron, and the training dynamics then strengthen such overlap. Due to its origin, we term this incidental polysemanticity.

What types of syllables are possible in a language? What types of syllables are favored? These two questions may initially seem independent but we show that they are deeply connected. Answers to both questions follow from the same phonological constraints given an appropriate theory of phonology. Building on a standard set of syllable structure constraints, we show that Optimality Theory (OT) predicts universals that are empirically supported whereas Maximum Entropy (ME) is so unrestrictive that no syllable is predicted to be universally worse than any other syllable, suggesting that in this respect ME fails as a theory of natural language phonology. Empirical support for our argument comes from a quantitative study of syllable structure in two genetically unrelated languages: Finnish and Dagaare.

The purpose of this paper is to develop a model for orbital resonance, both when it leads to unstable chaotic orbits and when it leads to stable configurations. The study of resonance is a great interest in astronomy as it can give clues to the formation of the early solar system by analyzing where certain objects are, and where there is a characteristic lack of objects. This paper applies elementary techniques from Newtonian mechanics to accurately and reliably predict the relative strengths of Kirkwood gaps in the main asteroid belt. It also builds off of prior work to create a more updated and complete approximation of the liberation period in stable resonant systems such as Saturn’s moons, Titan and Hyperion.