The brain forms and stores distributed representations from sparse external input that compete for neuronal resources with already stored memory traces. It is unclear what dynamical properties of neural systems allow formation and subsequent consolidation of new, distributed memory representations under these conditions. Here we use analytical, computational, and experimental approaches to show that a dynamical regime near a phase-transition in neuronal network activity (i.e. criticality) may play an important role in this process. Our results reveal that near-critical dynamics are necessary to stabilize and store new sparsely driven representations when they compete with native network states.